[Backup] Daily/Weekly/Monthly

[eagle]

Hi,

We have CPanel servers backing up externally on a daily basis.

When the scripts does the weekly or monthly backup, it still backups the daily. So 5 times a month it backups twice on a night.

The result is, that if a 'defect' is being backupped on a night that the daily and weekly are done, the nearest 'sane' backup may be allmost a month old. Is there a way to skip the daily backup if the weekly or monthly are done (not manually disabling one, which is not an option really)? Am I missing an option somewhere?

[chirpy]

That's the way it works. It doesn't repeat the backup it actually does the most frequent one and then simply rsyncs that backup to the less frequent iterations. No way around that as far as I know, other than implementing your own rotation for day of the week backups:
http://forums.cpanel.net/showthread.php?t=43683

[eagle]

That's the way it works. It doesn't repeat the backup it actually does the most frequent one and then simply rsyncs that backup to the less frequent iterations. No way around that as far as I know, other than implementing your own rotation for day of the week backups:
http://forums.cpanel.net/showthread.php?t=43683

Ok, thanks.

So the rotation scripts copies a backupfile on the day the script is run? To save all copies during a week, the script would have to be run daily?

[lloyd_tennison]

I have created my own rotation and backups for extended periods using rdiff-backup and/or rvm. Just set WHM to do daily only and then run whichever one you like better. Rdiff-backup uses far less space, but rvm is easier to restore.

I can fit a month's worth of backups (every day) in less space than I was just doing daily weekly monthly.

[webignition]

I can fit a month's worth of backups (every day) in less space than I was just doing daily weekly monthly.

Would you mind explaining a bit more about this?

It sounds like a decently efficient use of storage space!

[lloyd_tennison]

Differences between rvm and rdiff-backup.

rvm uses hardlinks and rsync to save backups. So if all but 5 files stay the same, only 5 files are backed up. It creates a directory by date - so you can do a simple copy to restore the files.

rdiff-backup just backs up the changed bytes, so its backups are generally much smaller. It also saves all permissions during each backup - even for different OS's.

Disadvantages of both:

rvm will backup until the partition is full and then delete from the oldest backup to allow it to finish. It does not currently have a feature to have it backup only so many days. etc. It also backs up the entire changed file, so its backups are generally larger than rdiff-backup.

rdiff-backup needs to rebuild the directory for restore and is a little less fault-tolerant if there was a prblem with the backup. It will fail and recreate from the last good one (usually the previous day) which means you can lose the differences for one day. It also takes much longer for the first backup as it builds checksums for all files when it starts that it uses to compare.

I actually use both, rdiff-backup on the same server on a second drive and rvm on a remote machine. I start rdiff-backup in the same cron job with cpbackup:

30 5 * * *  /scripts/cpbackup ; /usr/bin/rdiff-backup -v5 --print-statistics /backup/cpbackup/daily/  /backup/history

so I get the report at the same time I get the cpbackup report.

Rvm I run on the other machine and it pulls. I actually run rvm manually as I have it on a spare machine at my home office, and do not have it powered on all the time. (Actually run it during the evening news.)


Note: I have not put a day limit in rdiff-backup yet (they have that option), as I am only at 48% of the backup partition and have backups since last year. I used to be at 85%+ with just daily weekly monthly. Since I only use that drive for backup - I am going to let it go and see how much it can actually hold.

[webignition]

Differences between rvm and rdiff-backup.

rvm uses hardlinks and rsync to save backups. So if all but 5 files stay the same, only 5 files are backed up. It creates a directory by date - so you can do a simple copy to restore the files.

rdiff-backup just backs up the changed bytes, so its backups are generally much smaller. It also saves all permissions during each backup - even for different OS's.

Disadvantages of both:

rvm will backup until the partition is full and then delete from the oldest backup to allow it to finish. It does not currently have a feature to have it backup only so many days. etc. It also backs up the entire changed file, so its backups are generally larger than rdiff-backup.

rdiff-backup needs to rebuild the directory for restore and is a little less fault-tolerant if there was a prblem with the backup. It will fail and recreate from the last good one (usually the previous day) which means you can lose the differences for one day. It also takes much longer for the first backup as it builds checksums for all files when it starts that it uses to compare.

I actually use both, rdiff-backup on the same server on a second drive and rvm on a remote machine. I start rdiff-backup in the same cron job with cpbackup:

30 5 * * *  /scripts/cpbackup ; /usr/bin/rdiff-backup -v5 --print-statistics /backup/cpbackup/daily/  /backup/history

so I get the report at the same time I get the cpbackup report.

Rvm I run on the other machine and it pulls. I actually run rvm manually as I have it on a spare machine at my home office, and do not have it powered on all the time. (Actually run it during the evening news.)


Note: I have not put a day limit in rdiff-backup yet (they have that option), as I am only at 48% of the backup partition and have backups since last year. I used to be at 85%+ with just daily weekly monthly. Since I only use that drive for backup - I am going to let it go and see how much it can actually hold.

Great - thanks for all that info. I'll digest it and give things a go sometime.