Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New backup system: disk space

Discussion in 'Data Protection' started by mindnetcombr, Apr 25, 2017.

  1. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    156
    Hello,

    First, let me tell the behavior from legacy backup:

    - Server have 100Gb disk space
    - Data usage is 40Gb
    - Legacy backup run and generate a daily backup with 40Gb
    - Data 40Gb + Backup 40Gb = 80Gb, and left 20Gb of space on server
    - Next day: legacy backup run, overwrite the previous backup, 20Gb of space free, loop.

    Now with the new backup system:

    - Server have 100Gb disk space
    - Data usage is 40Gb
    - NEW backup run and generate a daily backup with 40Gb
    - Data 40Gb + Backup 40Gb = 80Gb, and left 20Gb of space on server
    - Next day: NEW backup run, and only will delete the previous backup (previous day) if the today backup finish without errors. We have 20Gb of space, new backup will fail because server dont have 40Gb free.

    So in the new backup system, the "1 day backup retention" need twice backup disk space than your data. If you have 1Tb of data you need at least 2Tb of disk space to save 1 backup:

    - "Retention behavior - When a complete backup finishes, the system deletes the oldest backup. When the system backup partially completes (fails), the system will not delete the oldest backup, and it will continue to retain the oldest backups. This procedure ensures that the system retains at least one retained complete backup." - source 11.64 manual.

    Good security measure, but now all servers will need more backup space.

    This is a problem on some servers, ex: VPS servers with one disk, need reserve 2/3 of disk space to save 1 backup. You have a VPS with 100Gb? Good, but if you want generate ONE backup you can only use 33Gb of your 100Gb disk space.

    Any way to the new backup system "overwrite" the previous backup, like the legacy do?
     
  2. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    15,774
    Likes Received:
    313
    Trophy Points:
    433
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    The docs should be of some use to you:
    Backup Configuration - Version 64 Documentation - cPanel Documentation
    cPanelTV has a new video explaining things as well:
     
  3. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    156
    I try this before open this thread.

    I enabled the option "Strictly enforce retention, regardless of backup success." but the new backup system only delete the previous backup after run the today backup.

    In any case the new backup system will keep the yesterday + today backup at same time until finish the today backup, because the option "Strictly enforce retention, regardless of backup success." only tell to backup system to delete previous day even if the today backup have errors.

    In any case the space used by previous backup will be used until the today backup finish.

    The new backup system dont delete the previous backup to start a new one, even with "Strictly enforce retention, regardless of backup success." enabled.

    So the server must have twice disk space to start a new backup job.

    Any way to force new backup system delete the previous backup before start a new one, like the legacy do?
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,659
    Likes Received:
    1,428
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello @mindnetcombr,

    Would you mind posting the contents of the /var/cpanel/backups/config file on your system so we can verify how backups are configured?

    Thank you.
     
  5. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    156
    Sure, this is the file:

    ---
    BACKUPACCTS: 'yes'
    BACKUPBWDATA: 'yes'
    BACKUPDAYS: 0,1,2,3,4,5,6
    BACKUPDIR: /backup
    BACKUPENABLE: 'yes'
    BACKUPFILES: 'no'
    BACKUPLOGS: 'no'
    BACKUPMOUNT: 'no'
    BACKUPSUSPENDEDACCTS: 'yes'
    BACKUPTYPE: compressed
    BACKUP_DAILY_ENABLE: 'yes'
    BACKUP_DAILY_RETENTION: 1
    BACKUP_MONTHLY_DATES: 1
    BACKUP_MONTHLY_ENABLE: 'no'
    BACKUP_MONTHLY_RETENTION: 1
    BACKUP_WEEKLY_DAY: 0
    BACKUP_WEEKLY_ENABLE: 'no'
    BACKUP_WEEKLY_RETENTION: 4
    CHECK_MIN_FREE_SPACE: 1
    ERRORTHRESHHOLD: 3
    FORCE_PRUNE_DAILY: 1
    FORCE_PRUNE_MONTHLY: 0
    FORCE_PRUNE_WEEKLY: 0
    GZIPRSYNCOPTS: --rsyncable
    KEEPLOCAL: 1
    LINKDEST: 0
    LOCALZONESONLY: 'no'
    MAXIMUM_RESTORE_TIMEOUT: 21600
    MAXIMUM_TIMEOUT: 7200
    MIN_FREE_SPACE: 5
    MIN_FREE_SPACE_UNIT: percent
    MYSQLBACKUP: accounts
    POSTBACKUP: 'no'
    PREBACKUP: -1
    PSQLBACKUP: 'no'


    The behavior of backup is:

    1) generate backup for "day 1", using 40Gb to save backup.
    2) on next day, generate a new backup for "day 2", using 40Gb to save backup (at this point backup use 80Gb)
    3) at the end of "day 2" backup the "day 1" is deleted (and now only 40Gb is used by backup)

    With or without the option "Strictly enforce retention, regardless of backup success." enabled ALWAYS the behavior of backup is this way described on these 3 steps above.

    So no matter what I do, I need 80Gb of disk space to save only 1 backup (retention = 1) of 40GB data size.

    I believe cpanel can make a option like "delete oldest backup on retention before start a new one" - DISABLED by default and with all warning about the risk.

    Thank
     
  6. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,659
    Likes Received:
    1,428
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    I believe the incremental backup type would better suite your needs if disk space is a concern. Here's a description of this backup type on our Backup Configuration document:

    Since it uses hard links, it won't make a full copy of all files under the accounts during the backup generation, leading to it taking up less space.

    Thank you.
     
  7. Wabun

    Wabun Well-Known Member

    Joined:
    Oct 6, 2012
    Messages:
    68
    Likes Received:
    3
    Trophy Points:
    8
    Location:
    Antwerpen
    cPanel Access Level:
    Root Administrator
    This is no good option for me either, I have one huge account and many small ones, VPS is only 50GB, using the legacy with FTP backup.
    I can't use for same space reason the new backup system, I wish you guys would not be so sure in deleting the legacy system when it is still used by many small VPS owners, in which I am sure I am not the only one!!!
     
  8. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,659
    Likes Received:
    1,428
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    Could you provide some more details about what in-particular uses more space with the new backup system? Here's a thread that details how hard links work:

    How do the backups work exactly?

    Thank you.
     
  9. Wabun

    Wabun Well-Known Member

    Joined:
    Oct 6, 2012
    Messages:
    68
    Likes Received:
    3
    Trophy Points:
    8
    Location:
    Antwerpen
    cPanel Access Level:
    Root Administrator
    Sure, it is very simple, I use the legacy with FTP, this works as follow, it copies one account to /backup, tars it and use FTP to send to off-site FTP server, but it WAITS, till the FTP session is done then it starts with the next account.

    The new system, doesn't care about what FTP is doing, it backups one after the other account in /backup, now if you have enough space, fine, but if your space is limited your server crashes as it runs out of space...

    This VPS is 50 GB, it has 36 accounts, and backing it up is no problem at all with the legacy ftp system, if we use the new system the server crashes!!!

    Filesystem Size Used Avail Use% Mounted on
    /dev/sda5 47G 22G 23G 49% /
    tmpfs 939M 0 939M 0% /dev/shm
    /dev/sda1 239M 101M 126M 45% /boot
    /dev/sda2 2.0G 3.3M 1.9G 1% /tmp
     
  10. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,659
    Likes Received:
    1,428
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    Thank you for clarifying. This functionality is not currently supported, but I encourage you to vote and add feedback to the existing feature request for this at:

    Remote FTP in the new backup system should pack and upload one file at a time

    Thank you.
     
Loading...

Share This Page