How to dimension a server for successful backups

[coolcom]

Hi all,
I have a few servers where backups systematically fail with "Insufficient disk space" during the backup process. Backups are offloaded to AWS S3. The "Check Available disk space is set to 10,000 and usual disk space remaining is some 40%. Yet every monthly backup fails.

What are the "right" settings for this to have them successfully finish?

The server has 350 accounts, 77% used disk space (backup transport still in progress, despite being marked as incomplete)

Thanks

Henk

 

[cPRex]

Hey there! The backups have to get created and compressed on the local system before they are moved over to the remote destination, so that does take up some space, but in general, we like to see 2 times the space of the largest account, plus 1G for some breathing room. So, if your largest account is50G, we'd want to see 101G of space to ensure there is room for that to compress and copy.

 

[Duplika]

Just to confirm, does cPanel create, compress, transfer and remove 1 backup at a time right?

We plan to use remote Wasabi compressed backups.
In your example. even if we have hundreds of accounts to backup, if "Retain Backups in the Default Backup Directory" is disabled, we would only need 101 GB of space on the local server for all backups to be successfully transfered to remote destination right?

 

[cPRex]

Yes - if you watch one of the backup logs in /usr/local/cpanel/logs/cpbackup, you'll see something similar to this, which is taken from the most recent log on my personal server:

[2022-08-24 02:00:28 -0400] info [backup] Calling pkgacct under cpuwatch to backup user “cptest2”
[2022-08-24 02:00:29 -0400] pkgacct started.
[...several lines later...]
[2022-08-24 02:04:06 -0400] info [backup] Calling pkgacct under cpuwatch to backup user “domaintest”

so we can see they are sequential.

Your disk space thought is also correct - we always tell people "two times the space of the largest account, plug 1G for some breathing room" for the tool to actually function.