cPanel backups are very slow

[johnburk]

The backups over FTP are very slow. The backups start at 01:00 and finish around 13:00 if there are only daily backups. Add weekly backups and the finish around 15:00, add month backups and it is around 17:00. During this time the load is between the 3 and 10 depending on the size of the accounts.

The causes problems on days when the server is busy due to traffic. It can add another 5 to 20 load because of traffic.

I don’t mind the load being 8 to 10 between 01:00 and 07:30 as long as the backup is finished by 08:00 when traffic starts.

Size
Total size of the files are 61GB uncompressed and 28GB compressed.

Server
Server is a Xeon E3-1240 with 16GB of ram. 1Gbit uplink to the backup server. Upload speed is not the issue.

My settings
Use pigz = Yes
gzip compression level = 1
Number of pigz processes =16
Number of kilobyte chunks per compression work unit = 256

I have tried putting the last 2 setting on default, lower or high. But it does not seem to make any significant difference.

61GB is not that much. At the speeds it currently backups, I think that the backup might take more than 24h if the total data is more than 100GB.

I am using legacy backup. Will the new backup system have any speed benefits? Are there other things I can do to speed up the backups?

 

[cPanelMichael]

Server is a Xeon E3-1240 with 16GB of ram. 1Gbit uplink to the backup server. Upload speed is not the issue.

Have you tried manually uploading a large file to the remote FTP server from the cPanel server to verify the data transfer speed? Also, to answer your other question, I do recommend trying out the new backup system. There are several improvements compared to the legacy system so it's worth an attempt.

Thank you.

 

[kbisignani]

One of the nice aspects of the new backup system is it slightly decouples the FTP uploads with the backup creation. So the system continues to package up accounts and doesn't hang up making more backups while it uploads them.

Another benefit I find is that you can diagnose the time it takes to create the backups vs the time it take to upload the backups. I had a problem with my backups but was able to dig in to the logs and find it was really a problem with FTP, not the backup engine.

I'd say switch if there's nothing specific to the old backup system that's holding you back.

 

[johnburk]

Have you tried manually uploading a large file to the remote FTP server from the cPanel server to verify the data transfer speed? Also, to answer your other question, I do recommend trying out the new backup system. There are several improvements compared to the legacy system so it's worth an attempt.

Thank you.

Yes I have tried it. Server to server I get an average of 10.8 MiB/s with 5x3GB of data. At 10.8 MiB/s the transfer of 28GB should take about 45 minutes. The connection speed does not seem to be an issue.

I switched to the new backup and will see if the speed improves.

 

[johnburk]

The new backup system made a big difference.

The old one started at 01:00 and finish around 13:00 for daily backups.

The new one starts at 01:00 and finish around 07:30 for daily backups. It is nearly 2x faster and a less resource intensive.

 

[Snehaa]

I can suggest a couple of things:

1) Looking at what you posted from top since it shows you not using CPU but still having a high CPU load there is someone else on the server using it. This would of course slow down backups and everything for everyone on that particular machine.

2) You could try a reboot. Although if the cause is indeed due to #1 this will not help much.

What I would do is a reboot. And then after rebooting just give it some time, very often these things tend to work themselves out. Then after some time or even if you do not want to wait shoot support a ticket and ask them to look at it since there is load on the server that is effecting your VPS (and others' of course).