Full Backup Drive

[web12]

Hi,

Does anyone happen to know of a script that will search through a backup drive and check the backups against actual accounts still on the server.

My backup is at 78% full and still has many backups of accounts no longer on the server, would be nice to get rid of them.

thanks

cPanel.net Support Ticket Number:

[ciphervendor]

Here's something I've hacked together that will work for removing users home directories that are backed up but no longer active on the system.

First create a file called directory-integrity.sh in /backup with the following contents:

#!/bin/sh

mkdir -p /backup/cpbackup-modified
if [ ! -d /backup/cpbackup-modified ] 
then
   echo "Not a directory: /backup/cpbackup-modified"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/daily
if [ ! -d /backup/cpbackup-modified/daily ] 
then
   echo "Not a directory: /backup/cpbackup-modified/daily"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/daily/files
if [ ! -d /backup/cpbackup-modified/daily/files ] 
then
   echo "Not a directory: /backup/cpbackup-modified/daily/files"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/daily/dirs
if [ ! -d /backup/cpbackup-modified/daily/dirs ] 
then
   echo "Not a directory: /backup/cpbackup-modified/daily/dirs"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/weekly
if [ ! -d /backup/cpbackup-modified/weekly ] 
then
   echo "Not a directory: /backup/cpbackup-modified/weekly"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/weekly/files
if [ ! -d /backup/cpbackup-modified/weekly/files ] 
then
   echo "Not a directory: /backup/cpbackup-modified/weekly/files"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/weekly/dirs
if [ ! -d /backup/cpbackup-modified/weekly/dirs ] 
then
   echo "Not a directory: /backup/cpbackup-modified/weekly/dirs"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/monthly
if [ ! -d /backup/cpbackup-modified/monthly ] 
then
   echo "Not a directory: /backup/cpbackup-modified/monthly"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/monthly/files
if [ ! -d /backup/cpbackup-modified/monthly/files ] 
then
   echo "Not a directory: /backup/cpbackup-modified/monthly/files"
   exit 1
fi

mkdir -p /backup/cpbackup-modified/monthly/dirs
if [ ! -d /backup/cpbackup-modified/monthly/dirs ] 
then
   echo "Not a directory: /backup/cpbackup-modified/monthly/dirs"
   exit 1
fi

Now create another file in /backup called backup-verify.sh with the following contents:

#!/bin/sh

/bin/sh /backup/directory-integrity.sh

/bin/ls -1A /var/cpanel/users/ >> /backup/accounts.dat

for i in `cat /backup/accounts.dat`
do
   /bin/mv /backup/cpbackup/daily/$i.tar.gz /backup/cpbackup-modified/daily/
   /bin/mv /backup/cpbackup/weekly/$i.tar.gz /backup/cpbackup-modified/weekly/
   /bin/mv /backup/cpbackup/monthly/$i.tar.gz /backup/cpbackup-modified/monthly/
done

   /bin/mv /backup/cpbackup/daily/files/* /backup/cpbackup-modified/daily/files/
   /bin/mv /backup/cpbackup/weekly/files/* /backup/cpbackup-modified/weekly/files/
   /bin/mv /backup/cpbackup/monthly/files/* /backup/cpbackup-modified/monthly/files/
   
   /bin/mv /backup/cpbackup/daily/dirs/* /backup/cpbackup-modified/daily/dirs/
   /bin/mv /backup/cpbackup/weekly/dirs/* /backup/cpbackup-modified/weekly/dirs/
   /bin/mv /backup/cpbackup/monthly/dirs/* /backup/cpbackup-modified/monthly/dirs/
   
   /bin/rm -rf /backup/cpbackup
   /bin/mv /backup/cpbackup-modified /backup/cpbackup
   /bin/rm -f /backup/accounts.dat
   /bin/echo "Done & Done"

exit

Once you've completed the above and understand the bash script, execute as root: cd /backup; sh backup-verify.sh

Enjoy.

cPanel.net Support Ticket Number:

[jackal]

Ok followed your guidelines above. We have dual harddrives one used for backups only. We mounted the backdrive uploaded the 3 files to /backup and then ran the command from ssh.

This is what we see can you explain to us?

/bin/mv: cannot stat `/backup/cpbackup/daily/bdtc.tar.gz': No such file or directory
/bin/mv: cannot stat `/backup/cpbackup/weekly/bdtc.tar.gz': No such file or directory
/bin/mv: cannot stat `/backup/cpbackup/monthly/bdtc.tar.gz': No such file or directory

cPanel.net Support Ticket Number:

[ciphervendor]

This means that there is a cpanel users file (/var/cpanel/users/) for an account but no backup file (daily, weekly or monthly) or you have an erroneous cpanel users file.

cPanel.net Support Ticket Number:

[jackal]

looked in /var/cpanel/users and that user does not exist

cPanel.net Support Ticket Number:

[web12]

Originally posted by ciphervendor
This means that there is a cpanel users file (/var/cpanel/users/) for an account but no backup file (daily, weekly or monthly) or you have an erroneous cpanel users file.

cPanel.net Support Ticket Number:

Could it also be the same reason as mine, that i do incremental backups, so the backups are not stored in a tar, they are actual directories under the backup drive?

cPanel.net Support Ticket Number:

[jackal]

Well really not sure on one of our servers we have loads on backups but when we mount drive and run script it says there are no backup files there when they are

cPanel.net Support Ticket Number:

[ciphervendor]

Originally posted by jackal
looked in /var/cpanel/users and that user does not exist

cPanel.net Support Ticket Number:

That's odd because the only way a file in /backup/cpbackup/daily/ is known is because the mv command pulls from /var/cpanel/users/

cPanel.net Support Ticket Number:

[ciphervendor]

Originally posted by web12
Could it also be the same reason as mine, that i do incremental backups, so the backups are not stored in a tar, they are actual directories under the backup drive?

cPanel.net Support Ticket Number:

I haven't used the incremental backup feature yet, does cpanel simply create a directory in /backup/cpbackup/daily/ with all of the users files uncompressed? Kind of like an image of their home dir?

cPanel.net Support Ticket Number:

[web12]

It goes something like this:-

backup
|
cpbackup
|
daily
|
username
|
homedir | cp | mm | mysql | vf | fp | logs | mma | va
|
(same as in /home/user/ onwards)

Something like that anyway.

cPanel.net Support Ticket Number:

cPanel.net Support Ticket Number:

[ciphervendor]

So, something like this should work for incremental backups then:

#!/bin/sh

/bin/sh /backup/directory-integrity.sh

/bin/ls -1A /var/cpanel/users/ >> /backup/accounts.dat

for i in `cat /backup/accounts.dat`
do
   /bin/mv /backup/cpbackup/daily/$i/ /backup/cpbackup-modified/daily/
   /bin/mv /backup/cpbackup/weekly/$i/ /backup/cpbackup-modified/weekly/
   /bin/mv /backup/cpbackup/monthly/$i/ /backup/cpbackup-modified/monthly/
done

   /bin/mv /backup/cpbackup/daily/files/* /backup/cpbackup-modified/daily/files/
   /bin/mv /backup/cpbackup/weekly/files/* /backup/cpbackup-modified/weekly/files/
   /bin/mv /backup/cpbackup/monthly/files/* /backup/cpbackup-modified/monthly/files/
   
   /bin/mv /backup/cpbackup/daily/dirs/* /backup/cpbackup-modified/daily/dirs/
   /bin/mv /backup/cpbackup/weekly/dirs/* /backup/cpbackup-modified/weekly/dirs/
   /bin/mv /backup/cpbackup/monthly/dirs/* /backup/cpbackup-modified/monthly/dirs/
   
   /bin/rm -rf /backup/cpbackup
   /bin/mv /backup/cpbackup-modified /backup/cpbackup
  /bin/rm -f /backup/accounts.dat

   /bin/echo "Done & Done"

exit

cPanel.net Support Ticket Number:

[web12]

umm... tell me if Im wrong, but wont this line remove the whole /backup/cpbackup directory?

/bin/rm -rf /backup/cpbackup

cPanel.net Support Ticket Number:

[web12]

its ok.. im being thick.. I just went through the script and found out it moves it to the modified directory... Sorry.. its too early on a sunday for me

cPanel.net Support Ticket Number:

[web12]

so, just as an update... that script works great ciphervendor... thanks for posting that!

Can someone please clarify... I was always under the impression that it was less server intensive to use the incremental backup instead of the server having to tar up every account.. am I wrong in this assumption?

cPanel.net Support Ticket Number:

[ciphervendor]

Originally posted by web12
so, just as an update... that script works great ciphervendor... thanks for posting that!

Can someone please clarify... I was always under the impression that it was less server intensive to use the incremental backup instead of the server having to tar up every account.. am I wrong in this assumption?

cPanel.net Support Ticket Number:

Great, I'm glad it worked.

Incremental should be less server intense, however it does require a _lot_ more space since the files/directories aren't being gzipped.

cPanel.net Support Ticket Number: