The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Full Backup Drive

Discussion in 'General Discussion' started by web12, Aug 29, 2003.

  1. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    Hi,

    Does anyone happen to know of a script that will search through a backup drive and check the backups against actual accounts still on the server.

    My backup is at 78% full and still has many backups of accounts no longer on the server, would be nice to get rid of them.

    thanks

    cPanel.net Support Ticket Number:
     
  2. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    Here's something I've hacked together that will work for removing users home directories that are backed up but no longer active on the system.

    First create a file called directory-integrity.sh in /backup with the following contents:
    Code:
    #!/bin/sh
    
    mkdir -p /backup/cpbackup-modified
    if [ ! -d /backup/cpbackup-modified ] 
    then
       echo "Not a directory: /backup/cpbackup-modified"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/daily
    if [ ! -d /backup/cpbackup-modified/daily ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/daily"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/daily/files
    if [ ! -d /backup/cpbackup-modified/daily/files ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/daily/files"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/daily/dirs
    if [ ! -d /backup/cpbackup-modified/daily/dirs ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/daily/dirs"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/weekly
    if [ ! -d /backup/cpbackup-modified/weekly ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/weekly"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/weekly/files
    if [ ! -d /backup/cpbackup-modified/weekly/files ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/weekly/files"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/weekly/dirs
    if [ ! -d /backup/cpbackup-modified/weekly/dirs ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/weekly/dirs"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/monthly
    if [ ! -d /backup/cpbackup-modified/monthly ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/monthly"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/monthly/files
    if [ ! -d /backup/cpbackup-modified/monthly/files ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/monthly/files"
       exit 1
    fi
    
    mkdir -p /backup/cpbackup-modified/monthly/dirs
    if [ ! -d /backup/cpbackup-modified/monthly/dirs ] 
    then
       echo "Not a directory: /backup/cpbackup-modified/monthly/dirs"
       exit 1
    fi
    Now create another file in /backup called backup-verify.sh with the following contents:
    Code:
    #!/bin/sh
    
    /bin/sh /backup/directory-integrity.sh
    
    /bin/ls -1A /var/cpanel/users/ >> /backup/accounts.dat
    
    for i in `cat /backup/accounts.dat`
    do
       /bin/mv /backup/cpbackup/daily/$i.tar.gz /backup/cpbackup-modified/daily/
       /bin/mv /backup/cpbackup/weekly/$i.tar.gz /backup/cpbackup-modified/weekly/
       /bin/mv /backup/cpbackup/monthly/$i.tar.gz /backup/cpbackup-modified/monthly/
    done
    
       /bin/mv /backup/cpbackup/daily/files/* /backup/cpbackup-modified/daily/files/
       /bin/mv /backup/cpbackup/weekly/files/* /backup/cpbackup-modified/weekly/files/
       /bin/mv /backup/cpbackup/monthly/files/* /backup/cpbackup-modified/monthly/files/
       
       /bin/mv /backup/cpbackup/daily/dirs/* /backup/cpbackup-modified/daily/dirs/
       /bin/mv /backup/cpbackup/weekly/dirs/* /backup/cpbackup-modified/weekly/dirs/
       /bin/mv /backup/cpbackup/monthly/dirs/* /backup/cpbackup-modified/monthly/dirs/
       
       /bin/rm -rf /backup/cpbackup
       /bin/mv /backup/cpbackup-modified /backup/cpbackup
       /bin/rm -f /backup/accounts.dat
       /bin/echo "Done & Done"
    
    exit
    Once you've completed the above and understand the bash script, execute as root: cd /backup; sh backup-verify.sh

    Enjoy.

    cPanel.net Support Ticket Number:
     
    #2 ciphervendor, Aug 29, 2003
    Last edited: Aug 31, 2003
  3. jackal

    jackal Well-Known Member
    PartnerNOC

    Joined:
    Feb 23, 2002
    Messages:
    708
    Likes Received:
    0
    Trophy Points:
    16
    Ok followed your guidelines above. We have dual harddrives one used for backups only. We mounted the backdrive uploaded the 3 files to /backup and then ran the command from ssh.

    This is what we see can you explain to us?

    /bin/mv: cannot stat `/backup/cpbackup/daily/bdtc.tar.gz': No such file or directory
    /bin/mv: cannot stat `/backup/cpbackup/weekly/bdtc.tar.gz': No such file or directory
    /bin/mv: cannot stat `/backup/cpbackup/monthly/bdtc.tar.gz': No such file or directory

    cPanel.net Support Ticket Number:
     
  4. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    This means that there is a cpanel users file (/var/cpanel/users/) for an account but no backup file (daily, weekly or monthly) or you have an erroneous cpanel users file.

    cPanel.net Support Ticket Number:
     
  5. jackal

    jackal Well-Known Member
    PartnerNOC

    Joined:
    Feb 23, 2002
    Messages:
    708
    Likes Received:
    0
    Trophy Points:
    16
    looked in /var/cpanel/users and that user does not exist

    cPanel.net Support Ticket Number:
     
    #5 jackal, Aug 29, 2003
    Last edited: Aug 29, 2003
  6. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    Could it also be the same reason as mine, that i do incremental backups, so the backups are not stored in a tar, they are actual directories under the backup drive?

    cPanel.net Support Ticket Number:
     
  7. jackal

    jackal Well-Known Member
    PartnerNOC

    Joined:
    Feb 23, 2002
    Messages:
    708
    Likes Received:
    0
    Trophy Points:
    16
    Well really not sure on one of our servers we have loads on backups but when we mount drive and run script it says there are no backup files there when they are

    cPanel.net Support Ticket Number:
     
  8. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    That's odd because the only way a file in /backup/cpbackup/daily/ is known is because the mv command pulls from /var/cpanel/users/

    cPanel.net Support Ticket Number:
     
  9. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    I haven't used the incremental backup feature yet, does cpanel simply create a directory in /backup/cpbackup/daily/ with all of the users files uncompressed? Kind of like an image of their home dir?

    cPanel.net Support Ticket Number:
     
  10. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    It goes something like this:-

    Something like that anyway.

    cPanel.net Support Ticket Number:

    cPanel.net Support Ticket Number:
     
  11. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    So, something like this should work for incremental backups then:
    Code:
    #!/bin/sh
    
    /bin/sh /backup/directory-integrity.sh
    
    /bin/ls -1A /var/cpanel/users/ >> /backup/accounts.dat
    
    for i in `cat /backup/accounts.dat`
    do
       /bin/mv /backup/cpbackup/daily/$i/ /backup/cpbackup-modified/daily/
       /bin/mv /backup/cpbackup/weekly/$i/ /backup/cpbackup-modified/weekly/
       /bin/mv /backup/cpbackup/monthly/$i/ /backup/cpbackup-modified/monthly/
    done
    
       /bin/mv /backup/cpbackup/daily/files/* /backup/cpbackup-modified/daily/files/
       /bin/mv /backup/cpbackup/weekly/files/* /backup/cpbackup-modified/weekly/files/
       /bin/mv /backup/cpbackup/monthly/files/* /backup/cpbackup-modified/monthly/files/
       
       /bin/mv /backup/cpbackup/daily/dirs/* /backup/cpbackup-modified/daily/dirs/
       /bin/mv /backup/cpbackup/weekly/dirs/* /backup/cpbackup-modified/weekly/dirs/
       /bin/mv /backup/cpbackup/monthly/dirs/* /backup/cpbackup-modified/monthly/dirs/
       
       /bin/rm -rf /backup/cpbackup
       /bin/mv /backup/cpbackup-modified /backup/cpbackup
      /bin/rm -f /backup/accounts.dat
    
       /bin/echo "Done & Done"
    
    exit
    cPanel.net Support Ticket Number:
     
    #11 ciphervendor, Aug 29, 2003
    Last edited: Aug 31, 2003
  12. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    umm... tell me if Im wrong, but wont this line remove the whole /backup/cpbackup directory?

    cPanel.net Support Ticket Number:
     
  13. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    its ok.. im being thick.. I just went through the script and found out it moves it to the modified directory... Sorry.. its too early on a sunday for me ;)

    cPanel.net Support Ticket Number:
     
  14. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    so, just as an update... that script works great ciphervendor... thanks for posting that! :)

    Can someone please clarify... I was always under the impression that it was less server intensive to use the incremental backup instead of the server having to tar up every account.. am I wrong in this assumption?

    cPanel.net Support Ticket Number:
     
  15. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    Great, I'm glad it worked.

    Incremental should be less server intense, however it does require a _lot_ more space since the files/directories aren't being gzipped.

    cPanel.net Support Ticket Number:
     
  16. ciphervendor

    ciphervendor Well-Known Member

    Joined:
    Aug 26, 2002
    Messages:
    1,052
    Likes Received:
    0
    Trophy Points:
    36
    Just a note, whether you are going to use the above script regularly for incremental or gzipped backups, you should add the following line immediately below "/bin/mv /backup/cpbackup-modified /backup/cpbackup"

    /bin/rm -f /backup/accounts.dat

    I made an edit on both of the previous posts to reflect this change.
     
  17. web12

    web12 Well-Known Member

    Joined:
    Nov 20, 2002
    Messages:
    240
    Likes Received:
    0
    Trophy Points:
    16
    Yep, noticed that after I ran it on one server and made the changes before i ran it on anything else.

    thanks again :)

    cPanel.net Support Ticket Number:
     
Loading...

Share This Page