How to dimension a server for successful backups

Operating System & Version
centos 7
cPanel & WHM Version
11.104.0.7

coolcom

Active Member
Mar 3, 2005
30
10
158
Hi all,
I have a few servers where backups systematically fail with "Insufficient disk space" during the backup process. Backups are offloaded to AWS S3. The "Check Available disk space is set to 10,000 and usual disk space remaining is some 40%. Yet every monthly backup fails.

What are the "right" settings for this to have them successfully finish?

The server has 350 accounts, 77% used disk space (backup transport still in progress, despite being marked as incomplete)

Thanks

Henk
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,497
1,967
363
cPanel Access Level
Root Administrator
Hey there! The backups have to get created and compressed on the local system before they are moved over to the remote destination, so that does take up some space, but in general, we like to see 2 times the space of the largest account, plus 1G for some breathing room. So, if your largest account is50G, we'd want to see 101G of space to ensure there is room for that to compress and copy.
 

Duplika

Well-Known Member
Feb 26, 2005
87
12
158
Buenos Aires, Argentina
cPanel Access Level
Root Administrator
Twitter
Just to confirm, does cPanel create, compress, transfer and remove 1 backup at a time right?

We plan to use remote Wasabi compressed backups.
In your example. even if we have hundreds of accounts to backup, if "Retain Backups in the Default Backup Directory" is disabled, we would only need 101 GB of space on the local server for all backups to be successfully transfered to remote destination right?
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,497
1,967
363
cPanel Access Level
Root Administrator
Yes - if you watch one of the backup logs in /usr/local/cpanel/logs/cpbackup, you'll see something similar to this, which is taken from the most recent log on my personal server:

Code:
[2022-08-24 02:00:28 -0400] info [backup] Calling pkgacct under cpuwatch to backup user “cptest2”
[2022-08-24 02:00:29 -0400] pkgacct started.
[...several lines later...]
[2022-08-24 02:04:06 -0400] info [backup] Calling pkgacct under cpuwatch to backup user “domaintest”
so we can see they are sequential.

Your disk space thought is also correct - we always tell people "two times the space of the largest account, plug 1G for some breathing room" for the tool to actually function.
 
  • Like
Reactions: Duplika