Backup Script

[krystalx]

Heya,

is it possible to use rsync over ftp ? or does anyone have a script to upload to a remote ftp server?

I already have a backup in cpanel going to a /backup on the server but i would like to send it over ftp to another datacenter hosting box.

I just want to copy the backup files to a seperate location

Thanks
kx

[abubin]

argghhh...I can't get this to work!!!

I went through a lot of howtos which somehow a lot of them differs. But none works. It keep on asking for root password when I try to ssh to remote server.

If I configure PermitRootLogin on sshd_config to :

yes - can login with root password
no - cannot login despite typing correct password (gave error saying wrong password)
forced-commands-only - cannot login despite typing correct password (gave error saying wrong password)

There are a few other howtos that I tried that also didn't work.

Anyone have a link to one that work? Or should I be using root for the rsync at all? BTW, I would prefer to have Root login disabled.

Thanks in advance.

[chirpy]

Heya,

is it possible to use rsync over ftp ? or does anyone have a script to upload to a remote ftp server?

I already have a backup in cpanel going to a /backup on the server but i would like to send it over ftp to another datacenter hosting box.

I just want to copy the backup files to a seperate location

Thanks
kx

This works well:
http://www.webhostgear.com/174.html

[chirpy]

argghhh...I can't get this to work!!!

You really need to use key authentication instead of password authentication when using rsync over SSH. I don't rsync SSH using my root account, I use a non-priv account to connect to for the backups and so avoid the root login issue.

[krystalx]

Jonathan,

I have run that script that you pointed me to .. however i get a "no proxy connection"
However if i go through the script and run each command individually it works fine.

Here is an output.

root@hostname [~/v1]# ./ftpbackup.sh
Starting FTP Backup on HOSTNAME Thu Oct 13 10:19:36 EST 2005
Verbose mode on.
Connected to 202.X.X.X.
No proxy connection.
Mode: stream; Type: binary; Form: non-print; Structure: file
Verbose: on; Bell: off; Prompting: off; Globbing: on
Store unique: off; Receive unique: off
Case: off; CR stripping: on
Ntrans: off
Nmap: off
Hash mark printing: off; Use of PORT cmds: on
Tick counter printing: off
221 Goodbye.
root@hostname [~/v1]#


I have replaced the values above to protect my privacy on this.

Any ideas?

Thanks
Kx

[chirpy]

Not a problem that I've seen before.

[DWHS.net]

What we do is back up daily only changed files to a local drive.

Then use this script to ssh the files to a the remote server:

 You may want to change "scp -r" to "scp -rpq".. That will preserve permissions/timestamps and also prevent output of the progress meter.

Follow the steps as given below:
1.SSH the server on which you want to setup the cron job.
2.cd /usr/bin
3.Create a file scpe.sh
4.Copy the contents between the below lines to scpe.sh
==========
#!/usr/bin/expect -f

set root_pass "xxxxxxxxxxx"
set hostname "123.45.678.910"

set folder "/backup/cpbackup/daily"

spawn /usr/bin/scp -r $folder root@$hostname:/backup/
sleep 3
expect "password: "
sleep 3
send "$root_pass\r"
interact
===========
6.chmod 755 scpe.sh.
5.Setup a daily cronjob which calls the script. (ie. a file in cron.daily named backup.cron and containing the command:
/usr/bin/scpe.sh )

Note : REPLACE THE IP AND PASSWD IN THE SCRIPT WITH THE IP AND PASSWD AS THAT OF THE REMOTE SERVER.

The script will copy the entire folder daily to the remote server's /backup/ 
partition, and put it into a directory named daily.

The problem is this overwrites files that have not changed.

If it would just do a rcync method and only change new files it will be near perfect.

Any thought on how to add to this code to only have changed files be transfered to the remote folder?

The the ssh is good because it encrypts the data when being transfered in case of sniffers.

[Lyttek]

I'm having good luck (finally) with rsync pulling backups (daily and weekly) down from the cpanel server onto a local box setup just for this purpose.

While looking into Windows rsync clients, I came across this:

"Using rsync to download updates to compressed files doesn't save much, due to the nature of compression - the entire compressed file changes at any minor change in the content. If you want to pack a lot of files to one file for easy download, do it without compression. So, tar without gzip, zip with speed set to "no compression" etc."

So, does that mean that using rsync to download daily backups (*.tar.gz) is essentially going to download the entire backup again, instead of just changes? My largest client thus far has about .75gb of IMAP mail, which translates into about 300 megs on the backup file, so downloading this to my local machine daily is going to eat a lot of bandwidth.

In this instance, would it be prudent to prevent the gzipping of backup files, previous posts about problems in restoring notwithstanding?

[Lyttek]

argghhh...I can't get this to work!!!

I went through a lot of howtos which somehow a lot of them differs. But none works. It keep on asking for root password when I try to ssh to remote server.

[snip]

Anyone have a link to one that work? Or should I be using root for the rsync at all? BTW, I would prefer to have Root login disabled.

Thanks in advance.

http://www.jdmz.net/ssh/

This is the How-To that I used, and it works flawlessly.

[chirpy]

So, does that mean that using rsync to download daily backups (*.tar.gz) is essentially going to download the entire backup again, instead of just changes?

That's correct, you are going to be downloading the whole backup again, regardless of whether there are changes or not.

It's for that reason that I always use a cheap server somewhere (at a different datacenter) to store the backups from disperate servers. Most come with large bandwidth allowances these days and in the few times when I have had to use remote backups, I believe the expenditure pays for itself, especially if you're using it to store remote backups for more than one server.

I'm not sure how beneficial disabling gzip'ing the backups in /scripts/cpbackup really would be. Bear in mind:

a. The initial size of the backup that you'd have to download since it won't be compressed

b. That you're going to have to have 2*original space to store data and backups of that data

[Lyttek]

Ah, the best laid plans....

[lloyd_tennison]

I want to use rsync with ssh to backup - how can I change the port that the info is sent to? I do not use ssh port 22 -easier to prevent hacks if it is firewalled. I use a different port.

And the same for scp, if anyone knows.


I see the --port command in rsync -but I think that is only for rsync - not ssh.

Thanks in advance.

[flash7]

Backup Script

[lloyd_tennison]

Thought I tried everything -but not the single quote. I do not see anything even on quotes
that in the man or help.

Thanks.

[flash7]

google :: rsync over different ssh port