getting crazy with backups

what method are you using to store the backup files ?

2nd drive,ftp,nfs ??

 

It is TAR and Gzip causing high IOwait so the servern load gets really high when backup
pretty big accounts. Also if theres big databases it will take it's time and load will go up.

I also get very high load when pckacct is running and thought you could control
how/when backup should stop becuase of the load but that only applies to the
system and not the load it's causing itself it seems. Or it just have to finish what
it's doing at the moment maybe and then it will wait until load goes down.

Anyway - it's a huge issue for me and my servers and clients.
I really need FTP backup cause I'm running RAID and can't do incre.... backups.
I wonder if it's possible at all to do it differently, my guess is not.

But how do the majority handle this question - it must be a problem many of us face ?
My backup starts friday 10PM and can go on until late saturday night.
Maybe 30GB of files and a few gigs of mysql.

 

Yes, the main cause is GZIP rather than tar i think. We have dual cpu's but it still takes a couple of hours to run.
I guess another way would be to get a huge backup drive ( 300GB ?? ) and then you could disable compression for backups.

 

Yup, that's the only way you're going to relieve the system load of backups. They're intensive by their very nature. They'll consume resources based on the tightest bottlneck, be that IO or CPU. Using incrementals or modifying the backup script to tarball and not compress are the main two ways of reducing CPU overheads from gzipping - that and throwing hardware at it. If CPU loads are low and IOWAIT is high, then a faster disk subsystem would also help.

 

Can you add a third disk as backup on a RAID system ?

Taking away gzip from the backup script would be really nice.
Also - just backup /var/lib/mysql - instead of doing dump would probably reduce
the load of huge databases.

I think I will start to look att the script and try to remove compression.
Maybe running a simple script to backup /var/lib/mysql seperatly and
the remove mysql from the regularly backup.

hm..

 

Very interesting idea, I have tons of HD space on my backup drive and even with a dual opteron machine backups will take 16 hours to complete, even after disabling the largest accounts on the server. Any hints on how I could disable compression in the backup script? I had a look through /scripts/cpbackup but I am not the best shell coder so it did not seem obvious as to where I could make the changes? Maybe I should just be using the Incremental backup method.

 

that's what I do. My backups have gone from 2 hours to 10 minutes. I can't see any reason to do the compressed backups as long as you have the space for the uncompressed. Just seems stupid, gzipping the same files every backup even when they haven't changed.

 

Yeah I ran my first INC backup last night, it took around 2/3rds as much time. Now that the incremental is in place though I am hoping for faster processing in the future. To combat the old files in the backup, I think I will set cpanel to only run daily backups, and then I will ncftp my daily directory to an offsite FTP backup to create a weekly backup as I did with the compressed backups. After that I will completely remove the inc backups on the server once a month in order to refresh them to make sure I do not get alot of old files in the backup. I wish I had thought of this sooner.

 

Since incemental backups is not possible with FTP backup I wonder if following would be possible and atleast reduce a little load as it compress less.

In cpbackup it say: cpusystem("gzip","-f"

What if I change this to cpusystem("gzip","-f1"

-1 is faster compression
-9 is best compression

so I guess when there's no -1 or -9 it's a deafult in between value.
So adding -1 will speed up the compression and reduce load.

I will check how much faster but if anyone know modifing the cpbackup script this way
will work please let me know, so I don't break anything.

 

Like a lot of people in this forum, we were having load issues. We setup NFS to another server in a nearby cabinet (10 Mbit link between them - if I put in a 100 Mbit switch between them instead of our old 10 Mbit core performance would obviously be much better)...

We are now using the incremental non-compressed to the NFS. The system uses virtually NO load. At the moment, I'm running the backups and load is under 1. Try that with compression!

It's also much easier to pull a file now - we're saved the uncompression which on a big site takes a LONG time.

Drew

 

One our new dual opterons backing up 247 websites at 1am is not a problem. The load is barely .30. Granted we have dual SCSI's too but even on our P4 2.6's its never higher than 1.2.

 

IS the dual Opteron as good as they claim?