Hi
About 8 months ago I took over two cpanel machines - my first experience of cpanel. Backups seemed to be happening as my predecessor intended them to - destination is a ReadyNAS sshfs mount - so I left things alone. Now I want to change the way our backups work but it looks lke {cp}backup can't be made to do exactly what I want. Or maybe it can but reading lots of threads about how it works has left me too confused to figure things out.
The first thing I want to do is to use rsync to get data to the NAS. Should be simple enough. Set target to a directory on the local drive and install a postcpbackup script to rsync between local drive and NAS. This will vastly cut the traffic between the two boxen. The second thing I want to do is use rsync's --backup and --backup-dir so each day I get the equivalent of a full backup and a directory of differences between that and the previous backup. All standard rsync stuff. Except the way cpanel backs things up seems to be inimical to an rsync solution.
The daily backups are incremental. So apparently I can't just rsync a daily backup over and get anything useful. The weekly backups are full backups but use tarball each account. I can make the gzip use --rsyncable or disable gzip completely (except for system configuration which uses tarballs anyway) and rsync will then transfer data efficiently but weekly backups still tar each account, which means a one-byte change in a small file of an account can cause a large tarball to be put into the --backup-dir.
It's not clear to me if daily backups are actually full backups if incremental is disabled. In fact, from contradictory information I've encountered it's not clear to me that daily backups are not full backups if incremental is enabled. Certainly the presence of cpbackup/weekly/daily implies they're always full backups, and the fact that we have incremental enabled and the daily inside weekly has the structure of incremental (no tarballs). If daily incrementals are actually full backups then I can happily rsync those and get what I want.
Confusing the issue is that our weekly backup is about twice the size of a daily backup. But in an older thread in the forums I've seen it said that nothing gets deleted from a weekly backup, even if something is deleted from a user account.
Another problem I have is MySQL. It would be nice to have account databases in with the appropriate account (which can be enabled) so that everything needed to restore an account is in one place and then the rest of the databases (particularly mysql) in the system configuration backup. But if I want the convenience of account database in the account backup and the safety net of mysql db in system backup, I have to have all the mysql dbs in the system backup too.
Oh, and it looks like you're using mysqldump for account dbs but when you backup all dbs to system area you're using mysqlhotcopy. Except perhaps you're doing something naughty because other threads here state that if a db is heavily accessed whilst being backed up it can be corrupted.
Note: I know the way cpanel backups work is designed to facilitate restoring accounts and moving accounts to another server. I understand that what I'd like to do would make restoring an account a little harder. I can live with that.
Note: I'm not making any type of feature request here. I want a solution that satisfies me fairly quickly. A feature request is likely to take to long and not fully satisfy me.
So can somebody explain how these things work and/or correct the errors in my understanding? That way I can figure out the best way to move forward.
--
Paul
About 8 months ago I took over two cpanel machines - my first experience of cpanel. Backups seemed to be happening as my predecessor intended them to - destination is a ReadyNAS sshfs mount - so I left things alone. Now I want to change the way our backups work but it looks lke {cp}backup can't be made to do exactly what I want. Or maybe it can but reading lots of threads about how it works has left me too confused to figure things out.
The first thing I want to do is to use rsync to get data to the NAS. Should be simple enough. Set target to a directory on the local drive and install a postcpbackup script to rsync between local drive and NAS. This will vastly cut the traffic between the two boxen. The second thing I want to do is use rsync's --backup and --backup-dir so each day I get the equivalent of a full backup and a directory of differences between that and the previous backup. All standard rsync stuff. Except the way cpanel backs things up seems to be inimical to an rsync solution.
The daily backups are incremental. So apparently I can't just rsync a daily backup over and get anything useful. The weekly backups are full backups but use tarball each account. I can make the gzip use --rsyncable or disable gzip completely (except for system configuration which uses tarballs anyway) and rsync will then transfer data efficiently but weekly backups still tar each account, which means a one-byte change in a small file of an account can cause a large tarball to be put into the --backup-dir.
It's not clear to me if daily backups are actually full backups if incremental is disabled. In fact, from contradictory information I've encountered it's not clear to me that daily backups are not full backups if incremental is enabled. Certainly the presence of cpbackup/weekly/daily implies they're always full backups, and the fact that we have incremental enabled and the daily inside weekly has the structure of incremental (no tarballs). If daily incrementals are actually full backups then I can happily rsync those and get what I want.
Confusing the issue is that our weekly backup is about twice the size of a daily backup. But in an older thread in the forums I've seen it said that nothing gets deleted from a weekly backup, even if something is deleted from a user account.
Another problem I have is MySQL. It would be nice to have account databases in with the appropriate account (which can be enabled) so that everything needed to restore an account is in one place and then the rest of the databases (particularly mysql) in the system configuration backup. But if I want the convenience of account database in the account backup and the safety net of mysql db in system backup, I have to have all the mysql dbs in the system backup too.
Oh, and it looks like you're using mysqldump for account dbs but when you backup all dbs to system area you're using mysqlhotcopy. Except perhaps you're doing something naughty because other threads here state that if a db is heavily accessed whilst being backed up it can be corrupted.
Note: I know the way cpanel backups work is designed to facilitate restoring accounts and moving accounts to another server. I understand that what I'd like to do would make restoring an account a little harder. I can live with that.
Note: I'm not making any type of feature request here. I want a solution that satisfies me fairly quickly. A feature request is likely to take to long and not fully satisfy me.
So can somebody explain how these things work and/or correct the errors in my understanding? That way I can figure out the best way to move forward.
--
Paul
