Transfer Incremental Backups to Remote FTP

[jasons]

I perform incremental backups of my server on a secondary hard drive, as WHM will not allow incremental backups to a remote FTP. For added security, I recently purchased a remote FTP to store copies of the backups. My problem is, how am I to go about transfering the files stored on my server to the remote FTP (I am running a Linux based system)? I know that programs exist to transfer files, but I cannot find one to transfer directory structures as well. The size of the backups is about 50-60GB. Additionally, is there a way to make sure that as the transfer is taking place, it is not overloading the server?

[linux-image]

if it is just to move a file check for the command "scp"

[brianoz]

Moving a 50-60Gb backup over the network isn't a fantastic idea, it will take ages, unless you're local to the FTP box you've bought.

The best way to do it is probably to tar the directory hierarchy up into a compressed tar image. (Scp will take forever on a backup of that size.) Once you have the backup, then you scan scp or ftp it to your remote box in one hit. This isn't exactly a trivial exercise as far as writing shell or PHP code is concerned, so good luck!

(The good news is that compression will probably make the backup a lot smaller, maybe even 80% smaller if it is mainly web pages and databases and logs).

[dgbaker]

Try this for using rsync (much better than SFTP or SCP)

#/bin/bash

# Backup to backup server 

DATE=`/bin/date +%Y.%m.%d::%H:%M`
HOST=`hostname`
BCKUSER="REMOTE USER ID"
BCKSERV="REMOTE SERVER NAME"
SSH="/usr/bin/ssh -l $BCKUSER $BCKSERV "
list="/backup/cpbackup/daily"

rsync -av --delete -e ssh $list $BCKUSER@$BCKSERV:/backup/$HOST

Best is to also setup a public/private key between the two so that you do not have to deal with supplying or hardcoding a password.

But I must agree, unless the ftp server is local to your server, it will take a painstakingly long time and will be using your bandwidth. Keep also in mind that by using rsync, future copy overs will only copy what has changed, so it will/can be less time.

[brianoz]

Actually, I think the best rsync tool is rsnapshot - http://www.rsnapshot.org - check it out, it keeps several days worth of backups online using minimal disk space. Very nice stuff

But definitely, I should have mentioned rsync too, it's advantage is that it will only send the files that have changed each time you do a backup.