The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

backup process: temporary folder

Discussion in 'Data Protection' started by Eric St-Amand, Oct 18, 2016.

Tags:
  1. Eric St-Amand

    Eric St-Amand Registered

    Joined:
    May 6, 2016
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Canada
    cPanel Access Level:
    DataCenter Provider
    Hi

    I'm trying to figure out the best way to do my backup. I have alot of data in a bunch of account.

    Currently, I'm using AWS Storage gateway as a iSCI target in my ESXi (where my WHM boxes are hosted). I mount a partition in my linux box as follow:

    sda 8:0 0 900G 0 disk
    ├─sda1 8:1 0 500M 0 part /boot
    └─sda2 8:2 0 899.5G 0 part
    ├─centos_blsol-root 253:0 0 25G 0 lvm /
    ├─centos_blsol-swap 253:1 0 3.9G 0 lvm [SWAP]
    └─centos_blsol-home 253:2 0 870.6G 0 lvm /home
    sdb 8:16 0 2T 0 disk
    └─sdb1 8:17 0 2T 0 part /home/backup

    sda is a local drive
    sdb is a remote drive

    Everything works fine but it takes for ever to complete backup.

    The main reason is that WHM seems to do the follow:

    - Create a folder in /home/backup/DATE_OF_THE_DAY/accounts/ACCOUNT_NAME/
    - Copy everything from /home/ACCOUNT_NAME to /home/backup/DATE_OF_THE_DAY/accounts/ACCOUNT_NAME/
    - Create archive from /home/backup/DATE_OF_THE_DAY/accounts/ACCOUNT_NAME/ to /home/backup/DATE_OF_THE_DAY/accounts/ACCOUNT_NAME.tar

    Basically, transfering twice the data over my iSCSI target (sdb1 mounted in /home/backup)

    Question:

    Is there a way to configure the "temp" folder cpbackup is using ? For exemple if it could store the temporary data in /tmp then create the final archive in the destination that would probably cut the backup time in half.

    I know I could use S3 target but that don't allow me to do "live restore" and I don't have enough space on the local drive to keep 1-2 week of backup.

    Thanks alot!
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    649
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello @Eric St-Amand,

    You can use the "Additional Local Directory" option under the "Additional Destinations" section in "WHM >> Backup Configuration" for this purpose.

    1. Enter "/backup" or the local drive directory of your preference under "Default Backup Directory" in "WHM >> Backup Configuration".
    2. Uncheck "Retain backups in the default backup directory".
    3. Enable the "Additional Local Directory" option under the "Additional Destinations" section and enter the remote drive information.

    Let us know if this helps.

    Thank you.
     
  3. Eric St-Amand

    Eric St-Amand Registered

    Joined:
    May 6, 2016
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Canada
    cPanel Access Level:
    DataCenter Provider
    Hi Michael

    Thanks for your reply..

    I understand what you're explaining but i'm not sure this "fix" my problem as this setup would still require twice the space to execute the backup.. If there's 100gig in /home then I will need another 100gig of free space on the local drive for the backup to complete, then it will move it to the additionnal local directory and free up the space. I'm I correct ?

    Problem is, at the moment, one of my servers has 800gig of data to backup.. I don't have an extra 800gig (and more everyday as it grows really fast).

    Also, will this setup allow me to do "live restore" ? Will the restore scan that "Additionnal local directory" to see what are the available backup?

    Thanks!

    Eric
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    649
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    The backup archives are queued for transfer to the additional destination and then removed after the transfer completes for each account. Thus, you shouldn't encounter space issues unless all of the space comes from just a few accounts. However, note the backup process and the transfer process use separate queues. If each backup finishes much faster than each transfer, backup files can accumulate on the server and fill the hard drive. You can find additional information on this topic at:

    Backup Configuration - Documentation - cPanel Documentation

    Regarding restorations, you can define a custom path to the backup file if you restore via the command line:

    The restorepkg Script - Documentation - cPanel Documentation

    That said, I encourage you to submit a feature request if you'd like to see additional functionality or changes to the current system:

    Submit A Feature Request

    Thank you.
     
Loading...

Share This Page