The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Some doubts about incremental backups

Discussion in 'Data Protection' started by DrKnuth, Aug 19, 2015.

  1. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi all,

    This is my first message on the CPanel forums. I hope you can help me with my doubts.

    Since I was using full backup and this was greatly slowing down access to the sites, I decided to use incremental backup, to avoid data compression.

    Today the incremental backup was performed from scratch but I'm not sure if this has run successfully. For this I have compared:

    root@jeans [/home/harry2co]# du -sh public_html/
    32G public_html/

    root@jeans [/backup/cpbackup/weekly/harry2co/homedir]# du -sh public_html/
    58M public_html/

    root@jeans [/backup/cpbackup/monthly/harry2co/homedir]# du -sh public_html/
    58M public_html/

    Indeed, when checking the public_html directory on the backups (and another subdirectory of public_html) I see that there are no regular files that I would expect to have in the corresponding path of /home/harry2co. This makes me wonder whether the backup is running as expected or am I missing something.

    Moreover in the log I saw something like this:

    /bin/gtar: unrecognized option '--url'
    Try `gtar --help' or `gtar --usage' for more information.
    /bin/gtar: This does not look like a tar archive
    /bin/gtar: Exiting with failure status due to previous errors

    This error is "normal"? May it be related to what I asked before?

    On the other hand, I saw in the log that backup had finished based on the following lines:

    [cpbackup] Copying weekly backups to monthly backups for retention
    [cpbackup] Completed at Wed Aug 19 15:13:02 2015

    nevertheless I saw some processes running rsync seemingly linked to the backup:

    root 1000251 0.2 0.0 4060 504 ? SN 14:21 0:02 \_ /usr/local/cpanel/bin/cpuwatch 8.0 /usr/bin/rsync -rlptD --delete /backup/cpbackup/weekly/ /backup/cpbackup/monthly
    root 1000252 3.6 0.0 177564 4964 ? SNs 14:21 0:30 \_ /usr/bin/rsync -rlptD --delete /backup/cpbackup/weekly/ /backup/cpbackup/monthly
    root 1000253 0.6 0.0 176092 3836 ? SN 14:21 0:05 \_ /usr/bin/rsync -rlptD --delete /backup/cpbackup/weekly/ /backup/cpbackup/monthly
    root 1000254 29.9 0.0 174356 3320 ? RN 14:21 4:09 \_ /usr/bin/rsync -rlptD --delete /backup/cpbackup/weekly/ /backup/cpbackup/monthly


    If the backup has been completed, it is not expected that these processes have also finished?

    Another thing that is not clear to me is why there are three rsync processes to be launched with the same syntax:

    \_ /usr/local/cpanel/bin/cpuwatch 8.0 /usr/bin/rsync -rlptD --delete ...
    \_ /usr/bin/rsync -rlptD --delete ...
    \_ /usr/bin/rsync -rlptD --delete ...


    I hope you can help me clarify all these doubts.


    Thanks in advance.

    Best regards,
    Daniel
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    It's possible the backup process did not successfully complete. The full backup logs are found in the /usr/local/cpanel/logs/cpbackup directory. Review the backup log for this backup process and check if there were any errors during the entire process. Also, could you verify if there is sufficient disk space on your system to complete the backup process?

    Thank you.
     
  3. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hello, Michael.

    Thanks for your reply.

    > It's possible the transfer did not successfully complete.

    In this case the backup is local. Besides it strikes me not see regular files in multiple path but directories. That is, directories checked seem to contain the expected directories but not regular files. Example:

    root@jeans [/home/harry2co/public_html/track]# ll
    total 68K
    drwxr-xr-x 2 harry2co harry2co 4,0K abr 8 2010 ./
    drwxr-x--- 132 harry2co nobody 12K ago 18 21:03 ../
    -rw-r--r-- 1 harry2co harry2co 30K ago 16 03:55 error_log
    -rw-r--r-- 1 harry2co harry2co 3,2K abr 9 2010 harry_courier_page.php
    -rw-r--r-- 1 harry2co harry2co 2,7K oct 7 2006 harry_courier_page.php_bk
    -rw-r--r-- 1 harry2co harry2co 5 oct 7 2006 .htaccess
    -rw-r--r-- 1 harry2co harry2co 393 ene 19 2010 index.html

    root@jeans [/backup/cpbackup/monthly/harry2co/homedir/public_html/track]# ll
    total 8,0K
    drwxr-xr-x 2 root root 4,0K abr 8 2010 ./
    drwxr-x--- 132 root root 4,0K ago 19 13:07 ../

    root@jeans [/backup/cpbackup/weekly/harry2co/homedir/public_html/track]# ll
    total 8,0K
    drwxr-xr-x 2 root root 4,0K abr 8 2010 ./
    drwxr-x--- 132 root root 4,0K ago 19 13:07 ../

    > The full backup logs are found in the /usr/local/cpanel/logs/cpbackup directory. Review the backup log for this backup process
    > and check if there were any errors during the entire process.

    root@jeans [/usr/local/cpanel/logs/cpbackup]# grep -i error 1439969404.log
    /bin/gtar: Exiting with failure status due to previous errors
    [2015-08-19 14:21:43 +0530] (129545738 bytes) xjeans_plst1[2015-08-19 14:21:43 +0530] (927803 bytes) xjeans_skinnyx[2015-08-19 14:21:45 +0530] (3133308 bytes) xjeans_backuptemp/usr/bin/mysqlcheck: Got error: 1049: Unknown database 'xjeans_backuptemp' when selecting the database

    Gtar error is the one I mentioned in the previous message although I'm not sure if this has to do with the files I'm not finding.

    > Also, could you verify if there is sufficient disk space on your system to complete the backup process?

    Yes, there is enough space for the backup.

    Thanks,

    Best regards,
    Daniel
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    I suggest trying to backup an account manually to narrow down any particular issues in your system that could be preventing the backups from completing successfully. You can do this with the following command:

    Code:
    /scripts/pkgacct $username
    Also, to confirm, no files are excluded globally or for the individual account using the backup exclusion options?

    Thank you.
     
  5. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi, Michael.

    > I suggest trying to backup an account manually to narrow down any particular issues in your system that could be preventing the
    > backups from completing successfully. You can do this with the following command:

    Thank you for the suggestion. In this case I would have to see on the console the output of the operations that are made, right? Thus the output would be more verbose that when the backup is scheduled automatically?

    > Also, to confirm, no files are excluded globally or for the individual account using the backup exclusion options?

    I currently have the following:

    Code:
    root@jeans [/home/harry2co]# cat cpbackup-exclude.conf
    public_html/cache/*
    
    Code:
    root@jeans [/home/harry2co]# cat /etc/cpbackup-exclude.conf
    */.wysiwygPro_*
    */core.[0-9]*
    .MirrorSearch
    .cagefs*
    .cpan
    .cpanel/caches
    .cpanel/datastore
    .cpcpan
    .sqmailattach
    access-logs
    public_ftp/.ftpquota
    
    So, with this content, I think it should not have been excluded the files that I mentioned in the previous examples, right?

    Thanks for your reply.

    Best regards,
    Daniel
     
  6. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi again, Michael.

    I tried the command you suggested:

    Code:
    root@jeans [/backup/cpbackup/weekly]# /usr/local/cpanel/scripts/pkgacct harry2co
    [2015-08-20 00:33:00 +0530] pkgacct started.
    [2015-08-20 00:33:00 +0530] pkgacct version 10 - user : harry2co - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - archive version: 3 - running with uid 0
    
    But from what I see on the first lines, this generates a full backup and not incremental as I want. Should I use --incremental (with two dashes)?

    When I ran this command manually, hoping to find the backup on /backup/cpbackup/weekly and /backup/cpbackup/monthly, but do not see it there. What is the location used?

    Moreover, from I see on the console, the output is not more verbose than when the backup runs automatically (at least in the way I just ran it without any additional parameters).

    Best regards,
    Daniel
     
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    It's not more verbose, but you can determine if it was a temporary issue or an ongoing issue with packaging the account.

    Yes, that is correct. The account files you are referring to are not excluded based on the output.

    Thank you.
     
  8. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Right, the idea is to see is the account backup generates successfully as a full backup (the first incremental backup is always a full backup, and the rest are for the account changes).

    The backup is stored in /home by default when generated manually.

    Thank you.
     
  9. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi, Michael.

    Thanks for your replies.

    Good point :)

    Strangely I have seen that in the automatic backup, the backup for the other account has created the files as expected unlike of the one mentioned in this thread (harry2co) --- I have two accounts on this server.

    Good. But it would be a full backup without creating a tarball, right?Now I'm doing a test using the following syntax:

    Code:
    root@jeans [~]# /usr/local/cpanel/scripts/pkgacct --incremental harry2co /backup
    
    Apparently, this is creating the backup /backup/cpmove-harry2co. I have checked /backup/cpmove-harry2co/homedir/public_html but so far I only see directories again in the root directory and there is not regular files as I can find in the public_html of the home directory of harry2co.

    Thanks for the observation. I had to interrupt the full backup because in my experience this slows down quite the server. Then I deleted the files created in /home as you suggested.


    If it helps something, I forgot to say that I'm using the legacy backup system of CPanel.



    Best regards,
    Daniel
     
  10. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi again, Michael.

    The backup has finished but the result was as mentioned in the previous message, without regular files (in public_html, to mention one case). In the output I get an error as I mentioned in a previous message (but I don't know if this will be related to the absence of these regular files):

    Code:
    [2015-08-20 01:30:24 +0530] Copying homedir.............
    Skipping public_ftp/.ftpquota
    .........
    .........
    .........
    .........
    .........
    .........
    .........
    .........
    .........
    .........
    (...)
    .........
    .........
    /bin/gtar: unrecognized option '--url'
    Pruebe `gtar --help' o `gtar --usage' para más información.
    /bin/gtar: Esto no parece un archivo tar
    /bin/gtar: Exiting with failure status due to previous errors
    [2015-08-20 01:34:47 +0530] Done
    
    There are some messages in Spanish (I guess that's due to environment). It would say something like:

    Code:
    Try `gtar --help' o `gtar --usage' for more information.
    /bin/gtar: This does not look like a tar archive
    
    Best regards,
    Daniel
     
  11. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi, Michael.

    I tried manually run an incremental backup of the other user and here I see all files are in /backup/cpmove-xjeans. Interestingly, here I didn't get the error with gtar that I mentioned before for the other user.

    Any idea what might be the difference?

    Thanks.

    Best regards,
    Daniel
     
  12. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi, Michael.

    I read here [1] that there is a limitation of 4GB for the backup of accounts. This may have to do with the account that does not give the error has 2GB but the one that gives the error has over 10 GB.

    I tried the flag "--skip-limit-flag" but I'm still getting the gtar error. Do you know if has changed the name of this flag?

    Thanks.

    Best regards,
    Daniel

    [1] http://randomlinux.com/cpanel-hosting/create-a-cpanel-backup-from-the-command-line/
     
  13. DrKnuth

    DrKnuth Member

    Joined:
    Aug 19, 2015
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Argentina
    cPanel Access Level:
    Root Administrator
    Hi, Michael.

    I did a test with the following syntax:

    Code:
    root@jeans [~]# /usr/local/cpanel/scripts/pkgacct --incremental --skiphomedir harry2co /backup
    
    Thus, the backup does not give the gtar error. Apparently, the problem is to backup the home for this account.

    Code:
    root@jeans [/backup/cpbackup]# du -sh /home/harry2co/  <==== The account that generates the error
    81G  /home/harry2co/
    root@jeans [/backup/cpbackup]# du -sh /home/xjeans/      <==== The backup for this account no giving me errors
    4,0G  /home/xjeans/
    
    If there is no way to fix it, I think that the only viable alternative is not backing up the home directories and synchronize these using a script with rsync. But I have not found in the web interface for legacy backups how to avoid the backup of home directories for each account.

    I would appreciate any feedback on recently sent messages.

    Thank you.

    Best regards,
    Daniel
     
  14. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    There should be no 4G file size limit with incremental backups. It looks like the archive is simply failing to generate due to an issue with the files in the account's home directory. Is it possible that some files in that account's home directory are corrupted?

    You can not skip the home directory for accounts using the configuration options for backups in Web Host Manager. However, you can exclude the home directory with the exclude file as documented at:

    How to Exclude Files From Backups

    Thank you.
     
Loading...

Share This Page