Backups Run VPS Low on Space While Transferring to AWS

[Hatsumi]

Hi, imagine this scenario:

- SSD VPS with 30 GB space
- Around 13 GB used for /home in multiple accounts, 7GB free in the VPS
- Max user account size around 4GB~, let's call this site A
- Backup set to upload to AWS S3 and remove the local copy

What happens here:
- Backup starts copying and packing accounts to /Backup
- The first few smaller sites upload pretty fast
- site A comes up.. it gets packed and upload to S3 starts..
- While site A is uploading to S3, backup continues in parallel with some more accounts
- S3 upload is slow~, so the queue for the next packaged accounts starts piling up..
- site A still uploading and using considerable space under /Backup
- Other accounts are packed and ready to upload after site A will finish

In this situation the used space under /Backup keeps going up and up.. so while we provisioned sufficient space for the largest account keeping in mind that once the sites are uploaded to S3 they are removed from our server.. this fails, because the process keeps working on other accounts in parallel with the risk to fill the disk (this is very bad on a vps, services will stop, mysql tables might get corrupted etc) if one large upload is taking too long.

tl;dr: Backup needs an option to not continue on the next account until the current S3/w/e upload has finished.

 

[cPanelMichael]

Hello,

There's an existing feature request that's relevant to this topic at:

Temporary pause backup script

I encourage you to vote and add feedback to this request.

Thanks!

 

[Hatsumi]

Hi, this is still a major issue, every time I need to backup it's a hassle. Please add an option to backup and upload accounts 1 by 1.

Will this ever be fixed?

 

[cPanelMichael]

Hello,

The "Minimum Free Disk Space Check" option is available in "WHM >> Backup Configuration", but it's only going to run at the start of the backup process. There's also another feature request you can vote for at:

Remote FTP in the new backup system should pack and upload one file at a time

Thank you.

 

[Hatsumi]

Hello,

The "Minimum Free Disk Space Check" option is available in "WHM >> Backup Configuration", but it's only going to run at the start of the backup process. There's also another feature request you can vote for at:

Remote FTP in the new backup system should pack and upload one file at a time

Thank you.

That option does not help, I have around 4GB free space, then a few accounts that use 2-3GB each, so it's useless in this case, I would need an option to disable running the backups/uploads in parallel and instead do the backup+upload process 1 by 1. I've had to disable backups for now.