The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Transfer Incremental Backups to Remote FTP

Discussion in 'General Discussion' started by jasons, Nov 14, 2004.

  1. jasons

    jasons Registered

    Joined:
    Mar 24, 2004
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    I perform incremental backups of my server on a secondary hard drive, as WHM will not allow incremental backups to a remote FTP. For added security, I recently purchased a remote FTP to store copies of the backups. My problem is, how am I to go about transfering the files stored on my server to the remote FTP (I am running a Linux based system)? I know that programs exist to transfer files, but I cannot find one to transfer directory structures as well. The size of the backups is about 50-60GB. Additionally, is there a way to make sure that as the transfer is taking place, it is not overloading the server?
     
  2. linux-image

    linux-image Well-Known Member

    Joined:
    Jun 8, 2004
    Messages:
    1,192
    Likes Received:
    1
    Trophy Points:
    38
    Location:
    India
    cPanel Access Level:
    Root Administrator
    if it is just to move a file check for the command "scp"
     
  3. brianoz

    brianoz Well-Known Member

    Joined:
    Mar 13, 2004
    Messages:
    1,146
    Likes Received:
    6
    Trophy Points:
    38
    Location:
    Melbourne, Australia
    cPanel Access Level:
    Root Administrator
    Moving a 50-60Gb backup over the network isn't a fantastic idea, it will take ages, unless you're local to the FTP box you've bought.

    The best way to do it is probably to tar the directory hierarchy up into a compressed tar image. (Scp will take forever on a backup of that size.) Once you have the backup, then you scan scp or ftp it to your remote box in one hit. This isn't exactly a trivial exercise as far as writing shell or PHP code is concerned, so good luck! :)

    (The good news is that compression will probably make the backup a lot smaller, maybe even 80% smaller if it is mainly web pages and databases and logs).
     
  4. dgbaker

    dgbaker Well-Known Member
    PartnerNOC

    Joined:
    Sep 20, 2002
    Messages:
    2,578
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Toronto, Ontario Canada
    cPanel Access Level:
    DataCenter Provider
    Try this for using rsync (much better than SFTP or SCP)

    Code:
    #/bin/bash
    
    # Backup to backup server 
    
    DATE=`/bin/date +%Y.%m.%d::%H:%M`
    HOST=`hostname`
    BCKUSER="REMOTE USER ID"
    BCKSERV="REMOTE SERVER NAME"
    SSH="/usr/bin/ssh -l $BCKUSER $BCKSERV "
    list="/backup/cpbackup/daily"
    
    rsync -av --delete -e ssh $list $BCKUSER@$BCKSERV:/backup/$HOST
     
    Best is to also setup a public/private key between the two so that you do not have to deal with supplying or hardcoding a password.

    But I must agree, unless the ftp server is local to your server, it will take a painstakingly long time and will be using your bandwidth. Keep also in mind that by using rsync, future copy overs will only copy what has changed, so it will/can be less time.
     
    #4 dgbaker, Nov 16, 2004
    Last edited: Nov 16, 2004
  5. brianoz

    brianoz Well-Known Member

    Joined:
    Mar 13, 2004
    Messages:
    1,146
    Likes Received:
    6
    Trophy Points:
    38
    Location:
    Melbourne, Australia
    cPanel Access Level:
    Root Administrator
    Actually, I think the best rsync tool is rsnapshot - http://www.rsnapshot.org - check it out, it keeps several days worth of backups online using minimal disk space. Very nice stuff :)

    But definitely, I should have mentioned rsync too, it's advantage is that it will only send the files that have changed each time you do a backup.
     
Loading...

Share This Page