Hi, imagine this scenario:
- SSD VPS with 30 GB space
- Around 13 GB used for /home in multiple accounts, 7GB free in the VPS
- Max user account size around 4GB~, let's call this site A
- Backup set to upload to AWS S3 and remove the local copy
What happens here:
- Backup starts copying and packing accounts to /Backup
- The first few smaller sites upload pretty fast
- site A comes up.. it gets packed and upload to S3 starts..
- While site A is uploading to S3, backup continues in parallel with some more accounts
- S3 upload is slow~, so the queue for the next packaged accounts starts piling up..
- site A still uploading and using considerable space under /Backup
- Other accounts are packed and ready to upload after site A will finish
In this situation the used space under /Backup keeps going up and up.. so while we provisioned sufficient space for the largest account keeping in mind that once the sites are uploaded to S3 they are removed from our server.. this fails, because the process keeps working on other accounts in parallel with the risk to fill the disk (this is very bad on a vps, services will stop, mysql tables might get corrupted etc) if one large upload is taking too long.
tl;dr: Backup needs an option to not continue on the next account until the current S3/w/e upload has finished.
- SSD VPS with 30 GB space
- Around 13 GB used for /home in multiple accounts, 7GB free in the VPS
- Max user account size around 4GB~, let's call this site A
- Backup set to upload to AWS S3 and remove the local copy
What happens here:
- Backup starts copying and packing accounts to /Backup
- The first few smaller sites upload pretty fast
- site A comes up.. it gets packed and upload to S3 starts..
- While site A is uploading to S3, backup continues in parallel with some more accounts
- S3 upload is slow~, so the queue for the next packaged accounts starts piling up..
- site A still uploading and using considerable space under /Backup
- Other accounts are packed and ready to upload after site A will finish
In this situation the used space under /Backup keeps going up and up.. so while we provisioned sufficient space for the largest account keeping in mind that once the sites are uploaded to S3 they are removed from our server.. this fails, because the process keeps working on other accounts in parallel with the risk to fill the disk (this is very bad on a vps, services will stop, mysql tables might get corrupted etc) if one large upload is taking too long.
tl;dr: Backup needs an option to not continue on the next account until the current S3/w/e upload has finished.