The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Backup Script

Discussion in 'General Discussion' started by kris1351, May 22, 2004.

  1. kris1351

    kris1351 Well-Known Member

    Joined:
    Apr 18, 2003
    Messages:
    963
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Lewisville, Tx
    I am looking for a backup script I can install on our servers that will tar the weekly backups and then scp/ftp them to another server. I want to have one server on a different network than our hosting servers and it do nothing but hold these backups.

    Currently I am doing this by hand as I haven't found a good script I like yet. It would need to send the entire cpbackup/weekly file preferably tar'd and zip'd. I would want it to overwrite the one on the remote server each week as we keep do daily, weekly and monthly backups on each servers spare cold drive.

    Any help would be appreciated.
     
  2. Setheronie12

    Setheronie12 Well-Known Member

    Joined:
    Feb 3, 2004
    Messages:
    46
    Likes Received:
    0
    Trophy Points:
    6
    If you look in the backup functions of cPanel there is an option to backup to a Remote Server
     
  3. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Kris,

    We do something similar. I'd make the following suggestions:

    1. Don't tar up the backup - the max size you can have is (generally) limited to 2GB and you risk losing the lot if you have a corrupt archive. Also, since the files are already gzip compressed, you won't be gaining much by compressing them again.

    2. With 1 in mind, I'd suggest that you look into using rsync over SSH with key authentication. You can then pull the backup files over from the other server.

    This is the command I use (I actually copy a daily backup once a week off server) that uses key authentication:
    Code:
    rsync --delete --stats -vzae ssh 11.22.33.44:/backup/cpbackup/daily /backup2/other/
     
  4. kris1351

    kris1351 Well-Known Member

    Joined:
    Apr 18, 2003
    Messages:
    963
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Lewisville, Tx
    Seth - The remote functions of Cpanel are slow and cumbersome. Traditionally over the last year and a half we have used Cpanel it has not written weekly/monthly backups when we selected remote dailies either. We want to keep the cold backups on the server for quick customer account restores and we can have daily, weekly and monthly. In addition it is much quicker and less server intensive for them to just write to a secondary or tertiary disk.

    Chirp - That is pretty much what we are looking to do. I didn't think of the potential loss though with an additional tar/gz. Forgot rsync would send over the entire directories.

    The goal is just to setup a cron job for each server to send its weekly backups to a basic little celly box with a large drive that we can add to. We ran into a problem early in the year where the DC went to restore someones box and they restored our box and formatted the second drive. It wasn't pretty so since then I have just been making manual copies of the weeklies over a partial T-1 box I own. Pushing 10 GB of backups over a small pipe though is not fun.
     
  5. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Kris, if you need any help with setting it up let me know. I just have a cron job running on the backup server sucking each cPanel backup in using the one line command for rsync for each server.
     
  6. kris1351

    kris1351 Well-Known Member

    Joined:
    Apr 18, 2003
    Messages:
    963
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Lewisville, Tx
    Yea, I have it testing on my server here now. Going to let it run for awhile on a local box. Setup a nice little Fedora box with raided 0+1 160gb drives. Uploading will be a little slow until the Covad connection gets installed, but these are only for DRP backups anyways.
     
  7. oderland

    oderland Well-Known Member
    PartnerNOC

    Joined:
    Dec 30, 2002
    Messages:
    103
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Kungsbacka, Sweden
    any idea how to rsync to another /backup server if all our server has root login not allowed?
    btw could that be run in some "nice" mode? I have seen in the past that the load of the servers have been high during those remote backups.

    BR
     
  8. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Server loads will always be high during such periods because of the amount of I/O you're doing on your disks/network. Depending on how you transfer it, it'll also use high memory and CPU (if sending over SSH for example and/or compressed) - there's nothing much you can do about that than accept the hit. Running it at the quietest time on your server is probably, obviously, best.

    When I configure for remote rsync I use a cPanel account with normal shell enabled. On the source server I have a line in /scripts/postupcp to chown -R the backup directory to the ownership of a cPanel account on that server. I then rsync from cPanel account to cPanel account. This means you don't need to use root at all. The only tricky bit is setting up a secure key pair for SSH, but a search on the web can give you information on how to do that.
     
  9. DSD

    DSD Member

    Joined:
    Mar 14, 2003
    Messages:
    20
    Likes Received:
    0
    Trophy Points:
    1
    this sounds exactly like what I want. Just so that I'm on the same page, I only need to run this on my destination box via cron:
    Code:
     --delete --stats -vzae ssh 11.22.33.44:/backup/cpbackup/daily /backup2/other/
    and the server's backup/cpbackup/daily folder will be copied to /backup2/other/ on my destination box.

    My question is:
    Is it better to copy the cpbackups folder (this would allow me to use cpanel to restore), or could I simply copy over /home/ thus giving me all the html files. I'm assuming that if I did that, I'd miss out on email / mysql, and probably some other stuff, but at the same time, I also think that the data transfer would be alot less (as each archive will change every day).


    Here is a good tutorial I found:
    HTML:
    http://www.tux.org/~tbr/rsync/rsynchowto.html
     
  10. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Hi,

    You should copy over the /backup/cpbackup/daily/ directory and its contents. Doing just /home will not copy enough to enable you to restore accounts should you have a disk failure.
     
  11. jdonoso

    jdonoso Well-Known Member

    Joined:
    Nov 15, 2004
    Messages:
    61
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    The Third Rock from The Sun!
    Hello Chirpy,

    I have 2 vps accounts, I want to use one of them (the cheapy and unstable one :p), as a backup for my other stable and more expensive backup.

    I have all my client's accounts setup in both server, but I'm looking for a way to maintain both in sync, in case any of these fail, the other kicks in. Both servers are clustered.

    Any ideas if this is possible? Can you help? If I would be able to pay, how much would you charge?

    I'm pretty much able to follow instructions (I meam, I can install, uninstall antiviruses, firewalls, etc, in linux), so if you can point me in the right direction, would be best.

    Thanks and my regards,
     
  12. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Clustering in that sense, is no mean feat and would be probably be impossible to achieve with any reliability. Someone recently posted how to cluster servers running cPanel and it involves a lot of hardware investment.

    If you're looking for cheap and cheerful solution, then you simply need to copy over your full backup and then restore each account, but change it's IP address to the one on the "backup" server as you do it. I don't know how easy it would be to script that.
     
  13. Roy@ENHOST

    Roy@ENHOST Well-Known Member

    Joined:
    Mar 5, 2002
    Messages:
    495
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Los Angeles California
    if you can provide me the location of the file containing the IP, I will gladly slap up a script real quick for that purpose.
     
  14. jamesbond

    jamesbond Well-Known Member

    Joined:
    Oct 9, 2002
    Messages:
    738
    Likes Received:
    1
    Trophy Points:
    18
    What I do weekly is tar -zcvf the /backup/cpbackup/daily directory and ftp it to my home computer and do a md5sum check.

    Filesize is around 1 GB now.
    I tried tar -cvf and it resulted in a 3GB file, that's quite a bit larger.

    If I would ftp the individual files over is it then still possible to easily verify (md5sum?) that all the files transferred correctly?
     
  15. jdonoso

    jdonoso Well-Known Member

    Joined:
    Nov 15, 2004
    Messages:
    61
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    The Third Rock from The Sun!
    Was that for me?? :p
     
  16. jamesbond

    jamesbond Well-Known Member

    Joined:
    Oct 9, 2002
    Messages:
    738
    Likes Received:
    1
    Trophy Points:
    18
    Is it possible with rsync to preserve user and group settings somehow if the target user is not root?

    I signed up for a remote backup plan and use rsync for the backups which works well.

    On top of that I would like to rsync certain directories, but every file I rsync is automatically set to the user/group on my backup location. I know this is caused because I'm not root on the target server, but I was wondering if it at all possible to preserve owner id in this case (without tarring everything)

    I've tried using --numeric-ids but that doesn't seem to work.
     
  17. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    What rsync qualifiers are you using? I would suggest that you use -a which should preserve UID/GID and permission settings.
     
  18. jamesbond

    jamesbond Well-Known Member

    Joined:
    Oct 9, 2002
    Messages:
    738
    Likes Received:
    1
    Trophy Points:
    18
    I am using -a:
    sync --delete --stats -avz --numeric-ids -e ssh

    When I login (SCP) to the backup account all files have the username and groupname of the backup account user. The permissions are correct though.

    Does the following mean root on source or root on target?

    -o, --owner preserve owner (root only)
     
  19. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Ah! I guess it's root on target, which you don't have. Looks like you're going to have to tar the files before rsync'ing them.
     
  20. bjdea1

    bjdea1 Well-Known Member

    Joined:
    Mar 6, 2003
    Messages:
    83
    Likes Received:
    1
    Trophy Points:
    8
    Response

    I've written my own C program that runs on a weekly basis and backups all accounts to another server independantly of cPanel - although it uses the "pkgacct" script from cpanel. It uses sftp and also has a load sensitive feature - if it senses server load above 4 it waits until it goes down again before proceeding with the next backup. Its been remote backing up accounts for over a year now and runs perfectly.

    I could modify this script and make it usable by anyone if there is demand for it. Is there demand for this? We would charge a small fee (since I'd have to mofidy and create a config file), say a few dollars. People interested in this? If so then please email me at sales@deasoft.com. I don't check these forums much so I probably won't be back here for a while - better to email me.
     
Loading...

Share This Page