Backup Script

[kris1351]

I am looking for a backup script I can install on our servers that will tar the weekly backups and then scp/ftp them to another server. I want to have one server on a different network than our hosting servers and it do nothing but hold these backups.

Currently I am doing this by hand as I haven't found a good script I like yet. It would need to send the entire cpbackup/weekly file preferably tar'd and zip'd. I would want it to overwrite the one on the remote server each week as we keep do daily, weekly and monthly backups on each servers spare cold drive.

Any help would be appreciated.

[Setheronie12]

If you look in the backup functions of cPanel there is an option to backup to a Remote Server

[chirpy]

Kris,

We do something similar. I'd make the following suggestions:

1. Don't tar up the backup - the max size you can have is (generally) limited to 2GB and you risk losing the lot if you have a corrupt archive. Also, since the files are already gzip compressed, you won't be gaining much by compressing them again.

2. With 1 in mind, I'd suggest that you look into using rsync over SSH with key authentication. You can then pull the backup files over from the other server.

This is the command I use (I actually copy a daily backup once a week off server) that uses key authentication:

rsync --delete --stats -vzae ssh 11.22.33.44:/backup/cpbackup/daily /backup2/other/

[kris1351]

Seth - The remote functions of Cpanel are slow and cumbersome. Traditionally over the last year and a half we have used Cpanel it has not written weekly/monthly backups when we selected remote dailies either. We want to keep the cold backups on the server for quick customer account restores and we can have daily, weekly and monthly. In addition it is much quicker and less server intensive for them to just write to a secondary or tertiary disk.

Chirp - That is pretty much what we are looking to do. I didn't think of the potential loss though with an additional tar/gz. Forgot rsync would send over the entire directories.

The goal is just to setup a cron job for each server to send its weekly backups to a basic little celly box with a large drive that we can add to. We ran into a problem early in the year where the DC went to restore someones box and they restored our box and formatted the second drive. It wasn't pretty so since then I have just been making manual copies of the weeklies over a partial T-1 box I own. Pushing 10 GB of backups over a small pipe though is not fun.

[chirpy]

Kris, if you need any help with setting it up let me know. I just have a cron job running on the backup server sucking each cPanel backup in using the one line command for rsync for each server.

[kris1351]

Yea, I have it testing on my server here now. Going to let it run for awhile on a local box. Setup a nice little Fedora box with raided 0+1 160gb drives. Uploading will be a little slow until the Covad connection gets installed, but these are only for DRP backups anyways.

[oderland]

any idea how to rsync to another /backup server if all our server has root login not allowed?
btw could that be run in some "nice" mode? I have seen in the past that the load of the servers have been high during those remote backups.

BR

[chirpy]

Server loads will always be high during such periods because of the amount of I/O you're doing on your disks/network. Depending on how you transfer it, it'll also use high memory and CPU (if sending over SSH for example and/or compressed) - there's nothing much you can do about that than accept the hit. Running it at the quietest time on your server is probably, obviously, best.

When I configure for remote rsync I use a cPanel account with normal shell enabled. On the source server I have a line in /scripts/postupcp to chown -R the backup directory to the ownership of a cPanel account on that server. I then rsync from cPanel account to cPanel account. This means you don't need to use root at all. The only tricky bit is setting up a secure key pair for SSH, but a search on the web can give you information on how to do that.

[DSD]

this sounds exactly like what I want. Just so that I'm on the same page, I only need to run this on my destination box via cron:

 --delete --stats -vzae ssh 11.22.33.44:/backup/cpbackup/daily /backup2/other/

and the server's backup/cpbackup/daily folder will be copied to /backup2/other/ on my destination box.

My question is:
Is it better to copy the cpbackups folder (this would allow me to use cpanel to restore), or could I simply copy over /home/ thus giving me all the html files. I'm assuming that if I did that, I'd miss out on email / mysql, and probably some other stuff, but at the same time, I also think that the data transfer would be alot less (as each archive will change every day).


Here is a good tutorial I found:

http://www.tux.org/~tbr/rsync/rsynchowto.html

[chirpy]

Hi,

You should copy over the /backup/cpbackup/daily/ directory and its contents. Doing just /home will not copy enough to enable you to restore accounts should you have a disk failure.

[jdonoso]

Hello Chirpy,

I have 2 vps accounts, I want to use one of them (the cheapy and unstable one ), as a backup for my other stable and more expensive backup.

I have all my client's accounts setup in both server, but I'm looking for a way to maintain both in sync, in case any of these fail, the other kicks in. Both servers are clustered.

Any ideas if this is possible? Can you help? If I would be able to pay, how much would you charge?

I'm pretty much able to follow instructions (I meam, I can install, uninstall antiviruses, firewalls, etc, in linux), so if you can point me in the right direction, would be best.

Thanks and my regards,

[chirpy]

Clustering in that sense, is no mean feat and would be probably be impossible to achieve with any reliability. Someone recently posted how to cluster servers running cPanel and it involves a lot of hardware investment.

If you're looking for cheap and cheerful solution, then you simply need to copy over your full backup and then restore each account, but change it's IP address to the one on the "backup" server as you do it. I don't know how easy it would be to script that.

[Roy@ENHOST]

if you can provide me the location of the file containing the IP, I will gladly slap up a script real quick for that purpose.

[jamesbond]

Kris,
1. Don't tar up the backup - the max size you can have is (generally) limited to 2GB and you risk losing the lot if you have a corrupt archive. Also, since the files are already gzip compressed, you won't be gaining much by compressing them again.

What I do weekly is tar -zcvf the /backup/cpbackup/daily directory and ftp it to my home computer and do a md5sum check.

Filesize is around 1 GB now.
I tried tar -cvf and it resulted in a 3GB file, that's quite a bit larger.

If I would ftp the individual files over is it then still possible to easily verify (md5sum?) that all the files transferred correctly?

[jdonoso]

if you can provide me the location of the file containing the IP, I will gladly slap up a script real quick for that purpose.

Was that for me??