The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Question about cpbackup-exclude.conf

Discussion in 'Data Protection' started by wtsdyo, Jan 27, 2014.

  1. wtsdyo

    wtsdyo Member

    Joined:
    Dec 15, 2010
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    I just need some clarification with regards to how cpbackup-exclude.conf.

    During nightly backups I notice that cPanel first copies the homdirectory to the cpbackup/daily/account/ folder. Does the folders and files listed in the cpbackup-exclude.conf file also gets copied in this phase? Does the file copy script immediately exclude those files?

    Or does the files/folders on cpbackup-exclude.conf only get excluded during the tar process?

    The reason I ask is that we have a temporary folder that will contain large number of files and subfolders during normal operation. I have included this folder into the cpbackup-exclude.conf file so it does not get included in the final archive. But the backup process is starting to take too much time. I looked at the backup logs and I see these lines:
    Copying homedir.............
    .........
    .........
    .........

    That is followed by 2000+ more ".........". The lines after the "Creating Archive..." line is much less.

    So this looks, at least to me, that the excluded folders still gets included in the homedir copy and only gets excluded in the archive creation process. Can anyone confirm if this is correct?

    If it is correct, can anyone suggest a workaround so that the excluded files/folders don't get copied to shorten the time it takes for the backup to complete? I am thinking of turning those temporary folders into symlinks and putting it into another location. Will the filecopy process still follow these symlinks?

    Thanks,
     
  2. wtsdyo

    wtsdyo Member

    Joined:
    Dec 15, 2010
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
    Question about backups, I see that during the nightly backups the homedir first gets copied to the cpbackup/daily/account/ folder before it gets tar'd.

    Are the files/folders in the cpbackup-exclude.conf file also gets copied or are they already ignored by the file copy script? Or are the excluded files only get ignored by the tar process?

    Two of our accounts have a temporary folder that contains a large number of files and folders. These temporary folder get changed a lot (subfolders and files get removed/created as their individual cache time expires). We already have listed this temporary folder in the cpbackup-exclude.conf for that account.

    The problem is that the nightly backup process is taking too much time to complete. I suspect that the culprit is the temporary folder. There are lots of "....." lines after the "Copying homedir" line in the backup log but only a few lines after the "Creating Archive" line. So this looks, at least to me, that most of the time is spent on copying the home folder.

    Is my assumption correct? If so, can anyone suggest how I can instruct cpbackup's homedir copy script to do not copy this temporary folder? Is there a setting for this?

    I am also thinking of moving the temporary folder into a separate location and then just creating a symlink so that our scripts can still locate the temporary folder. Will cpbackup's homedir copy script follow these symlinks?

    Thanks!
     
  3. cPanelPeter

    cPanelPeter Technical Analyst III
    Staff Member

    Joined:
    Sep 23, 2013
    Messages:
    569
    Likes Received:
    15
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    Files and directories in cpbackup-exclude.conf will get skipped from both tar and gzip.
    Here is the documentation on how to use cpbackup-exclude.conf
     
  4. luigidelgado

    luigidelgado Well-Known Member

    Joined:
    Nov 6, 2010
    Messages:
    109
    Likes Received:
    2
    Trophy Points:
    16
    Location:
    Mexico
    cPanel Access Level:
    Root Administrator
    Twitter:
    Peter,
    How would I set in /etc/cpbackup-exclude.conf to exclude any kind of */cache* files or directories? just like that?

    Code:
    */cache*
    ?
     
    #4 luigidelgado, Sep 17, 2014
    Last edited: Sep 17, 2014
  5. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Yes, however you should likely restrict it to your public_html directories. EX:

    Code:
    public_html/cache/*
    Thank you.
     
  6. luigidelgado

    luigidelgado Well-Known Member

    Joined:
    Nov 6, 2010
    Messages:
    109
    Likes Received:
    2
    Trophy Points:
    16
    Location:
    Mexico
    cPanel Access Level:
    Root Administrator
    Twitter:
    Ok... In the case caches are in a subdirectory inside public_html (for example: public_html/subdir/cache) would that work?
    Or should I use public_html/*/cache/* ?
     
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  8. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Acequias :: Granada :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Hi.

    How to Exclude Files From Backups not work for me with new system.

    On my /etc/cpbackup-exclude.conf

    ...
    */backup-*tar.gz
    */backup-*zip
    */backup*gz
    ...​

    But on backup of user after decompress--> <user>/homedir/backup-1.12.2014_11-47-34_clubf1e.tar.gz and others...
     
  9. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    If the files are stored in the account's home directory, then you should remove the initial wildcard from your entry. EX:

    backup-*tar.gz

    Thank you.
     
  10. speckados

    speckados Well-Known Member

    Joined:
    May 21, 2003
    Messages:
    291
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Acequias :: Granada :: España
    cPanel Access Level:
    DataCenter Provider
    Twitter:
  11. WCAndrew

    WCAndrew Registered

    Joined:
    Jan 19, 2015
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    So to exclude all files and directories in an account's home directory, the contents of /home/example_account/cpbackup-exclude.conf would just be:

    Code:
    *
    Is that right? That would be because I only want to backup the account's database(s).
     
  12. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    Yes, that would exclude all files from the account's home directory.

    Thank you.
     
  13. WCAndrew

    WCAndrew Registered

    Joined:
    Jan 19, 2015
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Thanks.

    Also I am wanting to exclude the e.g. /home/example_account/mail folder and everything inside the mail folder on every account in /home

    So I edited /etc/cpbackup-exclude.conf and added
    mail/*

    ... and I checked my backups this morning and the /home/example_account/mail folders have still been added to the accounts.

    What do I put in /etc/cpbackup-exclude.conf to exclude /home/example_account/mail and all files and sub-folders please?
     
  14. WCAndrew

    WCAndrew Registered

    Joined:
    Jan 19, 2015
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Re. my above post. That has actually worked as I intended, I tried to edit or delete the above post and the system wouldn't allow me to. Thanks.
     
  15. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    I am happy to see it worked as expected. Thank you for updating us with the outcome.
     
  16. hfinlay10

    hfinlay10 Registered

    Joined:
    Feb 21, 2016
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Australia
    cPanel Access Level:
    Website Owner
    Hi, i am using cpbackup-exclude.conf to exclude /mail directory and subdirectory for most backups, but would like to include it in a less frequent backup (say fortnightly). Is there a command line way to get cpanel backup to ignore cpbackup-exclude.conf? Thanks.
     
  17. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    You could temporarily remove the exclusions and force a backup generation via:

    Code:
    /usr/local/cpanel/bin/backup --force
    Thank you.
     
Loading...

Share This Page