HOW TO: make full backups use --rsyncable w/ gzip

isputra

Well-Known Member
May 3, 2003
574
0
166
Mbelitar
Lyttek said:
In the /etc/profile.d directory, create a script called 'gzipenv.sh' (or whatever you like, so long as the extension is .sh) with the following code:

This basically sets a global environment variable that tells gzip to use the rsyncable switch.
Hi Lyttek,

So i just create gzipenv.sh at /etc/profile.d directory ?

How about the script that already change like cpbackup and watchcpbackup ?

Is that i must change back cpbackup script and use the original file from cpanel and delete watchcpbackup ?
 

Lyttek

Well-Known Member
Jan 2, 2004
775
5
168
I am such a **freaking moron** !! :mad:

Ok, figured out the "fluke" because it happened again. All of my testing using the script in /etc/profile.d was while logged on... which means the GZIP env. var. existed, and all was well.

So I logoff. Now the GZIP variable doesn't exist, and so gzip reverts back to it's non-rsync-friendly algorithm.

Which means the whole "drop a script in /etc/profile.d" is bunk.

Chirpy? Anyone? How does one set a variable at system boot *without* logging on? Where does this script need to go? /etc/rc.d/rc.local ?
 
Last edited:

CoolMike

Well-Known Member
Sep 6, 2001
313
0
316
I saw that you can also use envoirement variables in the cronjob file. Would it maybe be possible to use this way?

Michael
 

Lyttek

Well-Known Member
Jan 2, 2004
775
5
168
Running a test now... this is driving me nuts! ;) Learned more in the past two weeks than in the previous two months!

Basically the current test does this:

1) script sets GZIP variable and exports it, then
2) calls /scripts/cpbackup then
3) calls /scripts/postbackup

modified crontab to remove the /cpbackup and /postbackup entries, replacing with the above script, /scripts/custom/backup

Will report results and full script if it works.

For all that are watching this thread, start a collection to send me to a therapist :D
 

Lyttek

Well-Known Member
Jan 2, 2004
775
5
168
Ok. I believe this to be true success finally. Step through this and verify.

1) modify crontab to remove refs to /scripts/cpbackup and replace with /scripts/custom/backup and save it.

2) create a script in /scripts/custom/ called 'backup'
This script is as follows:

#!/bin/bash
# Set GZIP variable, call /scripts/cpbackup (and others if needed)

GZIP="--rsyncable"
export GZIP

/scripts/cpbackup
/scripts/postbackup
3) chmod +x /scripts/custom/backup

To test, I did the following:
1) on source machine, renamed my 'daily' backup folder to 'daily.org'
2) set crontab to run the /scripts/custom/backup script
3) logged off source machine completely

4) on destination machine, copied my last good rsync'd 'daily' folder to 'daily.org' (This left a good rsync'd copy in 'daily', so I'm basically syncing 'daily' on both machines.
5) once it was apparent that the backup script on the source machine was finished, ran the script on my destination machine that actually pulls down the changes using rsync.

Transferred about 45 megs, which seems typical for my setup, as opposed to a 1.2 gig backup set.

Now then, does crontab ever get modified by cPanel? Is this going to break in the future? I've seen movement on the bugzilla entry for this, but who knows if that will make it and/or when.

If this works for others, and is all good and final, should I delete and recreate this HOWTO?

Please Chirpy, make the voices in my head stop! :p
 

Lyttek

Well-Known Member
Jan 2, 2004
775
5
168
Checked crontab this morning... lo and behold, /scripts/cpbackup was installed. I'm guessing upcp does that?

That causes a problem. I can't have a non-gzip friendly backup running in parallel with the one that is...

Next idea anyone? The purpose of a script calling cpbackup was the passing of the variable to the child process. I was unable to get the variable set any other way.
 

CoolMike

Well-Known Member
Sep 6, 2001
313
0
316
I still belive the best way would be, if cpanel would add this to the cpbackup script. So, please vote for it! :)

Michael
 

alwaysweb

Well-Known Member
Mar 8, 2002
97
0
306
Dallas, TX
cPanel Access Level
Root Administrator

CoolMike

Well-Known Member
Sep 6, 2001
313
0
316
I copy everyday the backup files to a NAS Server, each server has about 20GB Backup files. Last night I realized how much traffic I save with this solution, instead of 20GB I had to copy only 800MB. And this was only one of 17 servers... :)

Michael
 

Lyttek

Well-Known Member
Jan 2, 2004
775
5
168
Sounds about right. On my system, I was transfering about 4% instead of 100%, which the math seems to hold true on your figures as well.

Big difference!
 

Drew Nichols

Well-Known Member
May 5, 2003
96
0
156
SC
Lyttek said:
Checked crontab this morning... lo and behold, /scripts/cpbackup was installed. I'm guessing upcp does that?

That causes a problem. I can't have a non-gzip friendly backup running in parallel with the one that is...

Next idea anyone? The purpose of a script calling cpbackup was the passing of the variable to the child process. I was unable to get the variable set any other way.
So are you having to manually change your cpbackup script each night or have you locked it?
 

chirpy

Well-Known Member
Verifed Vendor
Jun 15, 2002
13,437
33
473
Go on, have a guess
I've got this working by simply changing the cpbackup crontab line from the normal:

0 1 * * * /scripts/cpbackup

to:

0 1 * * * export GZIP="--rsyncable" ; /scripts/cpbackup

upcp doesn't seem to mess with it and it works a treat. Thanks Lyttek :)
 

CoolMike

Well-Known Member
Sep 6, 2001
313
0
316
Chirpy, this sounds great. How important is the DOT?

0 1 * * * export GZIP="--rsyncable" ; /scripts/cpbackup

0 1 * * * export GZIP.="--rsyncable" ; /scripts/cpbackup

I think the DOT will just add the --rsyncable option to other GZIP options, and without the DOT, existing options would be overwritten, right?

Michael
 

chirpy

Well-Known Member
Verifed Vendor
Jun 15, 2002
13,437
33
473
Go on, have a guess
Yup, that's how I understand it. I didn't find any pre-existing instances of the environment variable GZIP being defined so didn't bother with the . (dot)
 

Lyttek

Well-Known Member
Jan 2, 2004
775
5
168
chirpy said:
I've got this working by simply changing the cpbackup crontab line from the normal:

0 1 * * * /scripts/cpbackup

to:

0 1 * * * export GZIP="--rsyncable" ; /scripts/cpbackup

upcp doesn't seem to mess with it and it works a treat. Thanks Lyttek :)
Ya know, I don't if I had tried it that way or not... that's certainly simple! Seems like I did and wasn't having luck... will give it another shot!