The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

cPanel Backup Experimental Feature

Discussion in 'E-mail Discussions' started by larryl, Mar 22, 2011.

  1. larryl

    larryl Active Member

    Joined:
    Feb 19, 2007
    Messages:
    42
    Likes Received:
    0
    Trophy Points:
    6
    We've been using the cPanel backup set up as follows:

    Schedule: daily, weekly
    Incremental backup: enabled
    Backup type: standard

    It has been running smoothly for several months.

    Last week we enabled this feature:

    EXPERIMENTAL: Use Hard Links for weekly and yearly backups to reduce disk usage and backup time.

    The weekly backup was copied over from the daily on Sunday night as expected, but the copy did not use hard links. Instead if created the weekly copy of all the files with separate inodes as it always has.

    Do we need to remove the weekly backup to get it to use hard links or should the backup program do this on its own?

    Thanks,
    Larry
     
  2. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    Hello Larry,

    Please do try moving the backups entirely out of the way, then force a re-run of /scripts/cpbackup --force on the machine to see the results. Of note, anything in weekly that is different (changed) from daily will not have a hard link.

    If this does not work, please open up a ticket, since we would want to see any issues with this new feature working. You can submit a ticket using the link in my signature or by choosing Contact cPanel in WHM's Support Center area.

    Thanks.
     
  3. larryl

    larryl Active Member

    Joined:
    Feb 19, 2007
    Messages:
    42
    Likes Received:
    0
    Trophy Points:
    6
    Thanks. It's scheduled to run this weekend. I'll update this thread with the results and open a ticket if necessary.
     
  4. alatar

    alatar Member

    Joined:
    Mar 15, 2011
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Sorry for updating thread that does belong to me, but we have almost similar problem.

    We had daily, weekly, monthly backups enabled. After that we decided to experiment with hardlinks to save some space:
    Code:
    root@ldz1 [~]# cat /etc/cpbackup.conf
    BACKUP2 yes
    BACKUPACCTS yes
    BACKUPCHECK yes
    BACKUPDAYS 0,1,2,3,4,5,6
    BACKUPDIR /backup
    BACKUPENABLE yes
    BACKUPFILES yes
    BACKUPFTPDIR
    BACKUPFTPHOST
    BACKUPFTPPASSIVE yes
    BACKUPFTPUSER root
    BACKUPINC yes
    BACKUPINT daily
    BACKUPLOGS no
    BACKUPMOUNT no
    BACKUPRETDAILY 1
    BACKUPRETMONTHLY 1
    BACKUPRETWEEKLY 1
    BACKUPTYPE normal
    COMPRESSACCTS no
    DIEIFNOTMOUNTED no
    GZIPRSYNCOPTS --rsyncable
    LINKDEST yes
    LOCALZONESONLY no
    MYSQLBACKUP accounts
    USEBINARYPKGACCT yes
    
    And now we have daily backup as usual. But weekly and monthly backups contain the same files, hardlinked to daily. For example:
    Code:
    root@server [~]# ls /backup/cpbackup/*/faisal/mysql -li
    /backup/cpbackup/daily/faisal/mysql:
    total 4152
    53576084 drwx------  2 root root      4096 Mar 30 05:15 ./
    53576079 drwxr-x--- 31 root faisal    4096 Mar 25 04:12 ../
    53584791 -rw-r--r--  3 root root   3840008 Mar 23 04:35 faisal_bagnetbiz.sql
    53584793 -rw-r--r--  3 root root    376722 Feb 16 18:06 faisal_wordpress.sql
    53584795 -rw-r--r--  3 root root     10014 Mar 30 05:15 horde.sql
    53584789 -rw-r--r--  3 root root      1680 Mar 30 05:15 roundcube.sql
    
    /backup/cpbackup/monthly/faisal/mysql:
    total 4152
    233374240 drwx------  2 root root    4096 Mar 13 06:24 ./
    201886075 drwxr-x--- 31 root root    4096 Feb 24 08:38 ../
     53584791 -rw-r--r--  3 root root 3840008 Mar 23 04:35 faisal_bagnetbiz.sql
     53584793 -rw-r--r--  3 root root  376722 Feb 16 18:06 faisal_wordpress.sql
     53584795 -rw-r--r--  3 root root   10014 Mar 30 05:15 horde.sql
     53584789 -rw-r--r--  3 root root    1680 Mar 30 05:15 roundcube.sql
    
    /backup/cpbackup/weekly/faisal/mysql:
    total 4152
    204179433 drwx------  2 root root    4096 Mar 24 05:26 ./
    204178039 drwxr-x--- 31 root root    4096 Mar 24 05:25 ../
     53584791 -rw-r--r--  3 root root 3840008 Mar 23 04:35 faisal_bagnetbiz.sql
     53584793 -rw-r--r--  3 root root  376722 Feb 16 18:06 faisal_wordpress.sql
     53584795 -rw-r--r--  3 root root   10014 Mar 30 05:15 horde.sql
     53584789 -rw-r--r--  3 root root    1680 Mar 30 05:15 roundcube.sql
    root@server [~]# date
    Wed Mar 30 08:42:45 CDT 2011
    They were enabled for two weeks now, and weekly should contain something different from daily. Or I'm wrong? Maybe we misunderstood something?
     
  5. alatar

    alatar Member

    Joined:
    Mar 15, 2011
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Does anyone have any ideas?
     
  6. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    I have Experimental Backups working perfectly on one of my servers. I did extensive testing and confirmed that everything is working exactly as it should. It's brilliant.

    I then deployed this on three more of my servers, but it is NOT working! I am getting the same results as alatar. Everyday, the weekly and monthly backups are "updated" to contain new hardlinks to the files in daily. Essentially, daily, weekly and monthly are all the same, so they provide no value at all.

    I opened a ticket on this, and received the following back:

    So, I guess I will leave Experimental Backups going on the server that is working, and remove it from the servers where it is NOT working. Maybe in 11.30 it will start working for all servers. It is a great idea, when it works.

    UPDATE: I asked for more information about this case, to try to help me understand why it's working fine on one server, but not others. Here is the response:

    Unfortunately, this doesn't help me understand the issue... so, for now, I'm removing Experimental Hard-Link Backups from the servers where it is not working.

    - Scott
     
    #6 sneader, Apr 13, 2011
    Last edited: Apr 13, 2011
  7. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,458
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    Scott,

    The problem is we are creating hard links for the entire contents of the directories. Much like a symlink, a hard link is the same as the file it is pointed at. The cpbackupstatus.cfg file contains the time stamps for the last time the backup was performed. Since the weekly file is a hardlink of the daily file (and the monthly a link of the weekly file) the backup appears to have just run (when in fact it did not).

    The process for creating the backups is (approximately):

    1. create the daily backup
    2. Record the backup time in daily/cpbackupstatus.cfg
    3. Create the weekly backup
    4. Record the backup time in weekly/cpbackupstatus.cfg
    5. create the monthly backup
    6. Record the backup time in monthly/cpbackupstatus.cfg

    The hard link feature changes the above to

    1. create the daily backup
    2. Record the backup time in daily/cpbackupstatus.cfg
    3. create hard links in weekly to daily
    4. create hard links in monthly to weekly

    Because of the different scenario, the cpbackupstatus.cfg file is only updated the one time. This means the weekly and montly cpbackupstatus.cfg files will always be evaluated as "no backup needed as the backup was just performed."
     
  8. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    Thanks, Kenneth! This helps greatly, although I think editing the process above to add some "evaluate" stuff would help me a little. (i.e. Check weekly backup, if older than 7 days... create weekly backup... record the weekly backup time...)

    Now, if I could only wrap my mind around the fact that I'm not having this problem on all servers. Does this make any sense?

    - Scott
     
  9. alatar

    alatar Member

    Joined:
    Mar 15, 2011
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Thanks for the info posted!
     
  10. Nick57

    Nick57 Well-Known Member

    Joined:
    Jul 19, 2005
    Messages:
    100
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    QTH, did you try to copy the files from /home/cpbackuptmp/cpbackup from the working server to the not working servers?
    I gues it might solve your issue... at least it is worth trying.
     
  11. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    Nick57, it turns out that it really was not working on any of the servers. I was fooled by the timestamps on the directories, but the actual data has proven that daily = weekly = monthly now.

    So, now we've removed experimental hardlinked backups, until cPanel makes it usable.

    - Scott
     
  12. alatar

    alatar Member

    Joined:
    Mar 15, 2011
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    cPanel 11.30 is ready. Has anybody tested backups there?
     
  13. alatar

    alatar Member

    Joined:
    Mar 15, 2011
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Anyone? I have not found this bug fixed in the changelog, so it would be great to receive some clarifications.
     
  14. cPanelKenneth

    cPanelKenneth cPanel Development
    Staff Member

    Joined:
    Apr 7, 2006
    Messages:
    4,458
    Likes Received:
    22
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    That was an oversight. It was included in the first 11.30.0 build. The change log is now updated with this information.
     
  15. jols

    jols Well-Known Member

    Joined:
    Mar 13, 2004
    Messages:
    1,111
    Likes Received:
    2
    Trophy Points:
    38
    We've never had neither of the experimental features enabled in the backup page of WHM, but nevertheless I am finding this file on the backup drive:

    /cpbackup/daily/cpbackupstatus.cfg

    Our backup drive clearner just removed this file. Will there be any problem with that?
     
  16. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    As Kenneth explained in his previous post, /cpbackup/daily/cpbackupstatus.cfg is created during "normal" backups (not experimental).

    - Scott
     
Loading...

Share This Page