The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

incremental backups creates dated directory

Discussion in 'Data Protection' started by JTZEnterprise, Apr 18, 2017.

  1. JTZEnterprise

    JTZEnterprise Registered

    Joined:
    Mar 24, 2009
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    51
    We have changed to incremental backups on two different servers. One of them still creates a new subdirectory in the /backups directory named by the date. The other creates a subdirectory named, "incremental". We have noticed that the one creating a dated subdirectory is doing a full backup every day and not an incremental backup. How can we get this server to store the backups in a subdirectory named, "/incremental" and not a dated file like "/2017-04-18"?
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,288
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    We added support for multiple incremental backups in cPanel 64. Since the backup directory structure changed as part of this new functionality, the most recent incremental backups before the update to cPanel 64 are automatically preserved to avoid data loss. As of cPanel 64.0.11, the "/backup/$weekly/incremental" directory is renamed to "/backup/$weekly/$date" when updating to cPanel 64. At this point, the backup system should follow your current retention values (e.g. if you configure the server to retain 1 weekly backup, then it should automatically remove the converted "/backup/$weekly/$date" directory once the new weekly incremental backup is generated.

    Thank you.
     
  3. JTZEnterprise

    JTZEnterprise Registered

    Joined:
    Mar 24, 2009
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    51
    I am referring specifically to the daily backup. We would prefer the daily backups not be stored in a $date subdirectory. We have a separate function that will rsync the daily backups to another server but because the directory date changes with each backup from cpanel, the rsync is not able to incrementally move the files to the other location.
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,288
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    The use of the date in the directory name is part of the new functionality designed to offer multiple incremental backups. You would need to update your script to have rsync consider the directory name based on the date. Here are a couple of StackOverflow threads you may find helpful:

    How to rsync a directory to a new directory with different name?
    Using rsync to maintain a copy of a directory with a name that changes

    Or, you may want to setup a hook that runs after each backup generation that runs a custom bash script to create a symbolic link to the previous incremental directory name. Note that support for remote incremental backups is planned for cPanel 66:

    Support for remote incremental backups

    Thank you.
     
  5. cesarlwh

    cesarlwh Well-Known Member

    Joined:
    Sep 23, 2008
    Messages:
    63
    Likes Received:
    0
    Trophy Points:
    56
    I am having the same problem! THe backup system are creating a new backup every day and not updating the current! Then is not a incremental and I need a incremental. My HDD of backup are full because of this!

    See:
    /backup/2017-04-11 - OK
    /backup/2017-04-12 - OK
    /backup/2017-04-13 - OK
    /backup/2017-04-14 - Error because the HDD are full
    /backup/2017-04-15 - Error because the HDD are full
    /backup/2017-04-16 - Error because the HDD are full

    The correct is move the old date dir to the new date and then make a incremental backup.

    I need a solution to this ASAP because my servers are without backups.
     
  6. cesarlwh

    cesarlwh Well-Known Member

    Joined:
    Sep 23, 2008
    Messages:
    63
    Likes Received:
    0
    Trophy Points:
    56
    I set the retention to 1 ONE

    Screenshot_2.png
     
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,288
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello @cesarlwh,

    You may want to review the backup logs in the /usr/local/cpanel/logs/cpbackup/ directory to see if the backups are partially failing (you can search for error or failed messages in these logs with the "grep" command). If so, you may want to adjust the retention settings for failed backups at:

    Backup Configuration - Version 64 Documentation - cPanel Documentation

    Or, free to open a support ticket using the link in my signature so we can take a closer look.

    Thank you.
     
  8. north captiva

    north captiva Registered

    Joined:
    May 4, 2017
    Messages:
    2
    Likes Received:
    3
    Trophy Points:
    3
    Location:
    Florida
    cPanel Access Level:
    Root Administrator
    With notice that the legacy backup will soon be retired, it would be a really nice option if cpanel would include an option to automatically create a /daily or /latest symlink for us to the latest dated 2017-xx-yy daily backup directory when the new backup script runs. This would make it much easier for those of us who have other backup routines rsyncing the /daily directory to remote backup locations seamless. Please... please...
     
    cPanelNick likes this.
  9. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,288
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    I encourage you to submit a feature request using the Submit A Feature Request link if you'd like to see this feature added.

    Thank you.
     
  10. north captiva

    north captiva Registered

    Joined:
    May 4, 2017
    Messages:
    2
    Likes Received:
    3
    Trophy Points:
    3
    Location:
    Florida
    cPanel Access Level:
    Root Administrator
    cPanelNick and cPanelMichael like this.
  11. mahinder

    mahinder Well-Known Member

    Joined:
    Jun 12, 2003
    Messages:
    69
    Likes Received:
    0
    Trophy Points:
    156
    Location:
    matrix
    How can we configure cPanel "not" to create different date directories with incremental backups every day. Some of the systems I manage, backup can only be taken for 3-4 hours in a day on particular days and it has to be monitored for server performance reasons. Some times backup has to be killed in middle and restarted when it pushes server load very high.

    In new backup system, backups are splitting in multiple date folders and I have to rename yesterday's backups folder before starting backup to prevent backup script from starting backup all over again.

    Old incremental backups system was simple and straight forward. /incremental/accounts folder was just updated with new files and there was no multiple date folders.

    In my opinion cPanel should provide a check box which gives user control to keep backups in just one folder instead of creating multiple folders every day.
     
  12. InterServed

    InterServed Well-Known Member

    Joined:
    Jul 10, 2007
    Messages:
    256
    Likes Received:
    3
    Trophy Points:
    68
    cPanel Access Level:
    DataCenter Provider
    Hello,

    I did "suffer" quite a lot by this change as well. Here is how i managed to work it out so i can still take rsync copies of the local generated incrementals (just 1 day retention locally as i take daily rsync of this and keep copies offsite):

    /usr/local/cpanel/scripts/postcpbackup (adjust based on your requirements, like BKP_PATH and BKP_LINK)
    Code:
    #!/bin/bash
    BKP_PATH=/backup
    BKP_LINK=/lastbackup
    
    BKP_DIR=$(find ${BKP_PATH} -maxdepth 1 -type d -not -name "*lost*" | sort -r | head -1)
    if [ "${BKP_DIR}" == "${BKP_PATH}" ]; then
      exit 0
    else
      BKP_DATE=$(basename ${BKP_DIR})
      [ ! -f ${BKP_DIR}/${BKP_DATE} ] && touch ${BKP_DIR}/${BKP_DATE}
      [ -L ${BKP_LINK} ] && rm -rf ${BKP_LINK}
      ln -s ${BKP_DIR} ${BKP_LINK} || exit 1
    fi
    
    1) .Make sure that the script has execution right
    Code:
    chmod +x /usr/local/cpanel/scripts/postcpbackup
    
    2) Make sure you have enabled the use of the post script (else edit the file and set it to yes) :
    Code:
    root@nlsrv1 [~]# grep -i POSTBACKUP /var/cpanel/backups/config
    POSTBACKUP: 'yes'
    
    Using this , i can sync the /lastbackup/ in my case as it will always be a symlink to the latest cpanel backup. I recommend to read the script and to understand it before using it. I'm not responsible for any damage it may cause if you do not understand it.
     
    #12 InterServed, May 15, 2017
    Last edited: May 15, 2017
    kdean likes this.
  13. holodyn

    holodyn Registered

    Joined:
    Feb 6, 2008
    Messages:
    3
    Likes Received:
    1
    Trophy Points:
    53
    This ruins incremental backups for us. We pair backup drives with usage 1-1, and the previous incremental allowed for that. This change requires us to consume large amounts of cache space with every backup, as all backups are created before cleanup is performed.

    Please restore the option to have the incremental update perform a sync instead of a whole new copy so that we aren't required to have TRIPLE the disk space for a backup drive and be forced to slam the system with an entire copy / delete cycle every day.

    For anyone who wants to re-create the previous incremental operation you can use a script like the following.

    ```
    #/usr/bin/bash
    cd /var/cpanel/users
    for i in `ls *`
    do
    if [ -d "/home/$i" ]; then
    /usr/local/cpanel/bin/pkgacct --incremental "$i" /backup/daily-incremental/accounts backup
    fi
    done
    ```
     
Loading...

Share This Page