Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Long lasting backup - Settings to make it faster ?

Discussion in 'Data Protection' started by SecondSight, Dec 28, 2015.

  1. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    70
    Likes Received:
    0
    Trophy Points:
    156
    Hello :)

    I looked at the backup log on my server, because backups are lasting too long.


    I found the archive creation for my biggest website is lasting more than 6 hours :

    [2015-12-28 05:33:07 +0100] Creating Archive
    .............................................
    [2015-12-28 11:57:55 +0100] Done
    [2015-12-28 11:57:55 +0100] pkgacctfile is: /backup/2015-12-28/accounts/mysite.tar.gz

    => When I do it myself, using a simple tar command, it doesn't take more than 10 minutes (I only backup public_html directory).



    Backing up the database is also taking more than 3 hours :

    [2015-12-28 02:17:05 +0100] Storing mysql dbs............
    mysite_forums.........
    .........
    .........
    [2015-12-28 05:33:06 +0100] ...Done

    => When I do it myself, using a mysqldump command, it lasts 5 minutes.


    Do you know the reasons why the WHM backup tool takes so long (more than 9 hours) to backup one account ? I do it myself in not more than 15 minutes and in the past, the backup lasted 45 minutes.

    I noticed these problems after I had CloudLinux installed but I was told it has nothing to do with it...

    What can I do to find the reasons why backup is so slow ?

    Can you tell me what are the settings I can change in WHM to make the backups faster ?

    Thank you :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    43,961
    Likes Received:
    1,821
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello :)

    Do you notice any additional entries in the backup log within the /usr/local/cpanel/logs/cpbackup directory during the backup process? What's the server load when the automatic backups are generating?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    70
    Likes Received:
    0
    Trophy Points:
    156
    Hello ! :)

    When I watched the server load, I saw it is between 1 and 4 when the backup is ran (that is to say from 2 am to 16 pm...).

    The only error message is about the database :
    ERROR: Failed to dump one or more databases

    In the error_log file I found :
    [2015-12-30 03:32:20 +0100] warn [pkgacct] mysite_forums: mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `ibf_posts` at row: 1658647 at bin/pkgacct.pl line 1953, <$fh> line 1.

    This table, ibf_posts is the biggest table.


    Here is the content of the log file sent to me by email : (I removed a lot of lines of dots and my site's/account's real name)
    Code:
    [2015-12-30 02:10:02 +0100] info [backup] Setting I/O priority to reduce system load: best-effort: prio 3
    [2015-12-30 02:10:02 +0100] info [backup] Hard Linking available on "/backup"
    [2015-12-30 02:10:02 +0100] info [backup] Running account backup with target : /backup/2015-12-30/accounts
    [2015-12-30 02:10:02 +0100] info [backup] Calling pkgacct under cpuwatch to backup user “mysite”
    [2015-12-30 02:10:02 +0100] pkgacct started.
    [2015-12-30 02:10:02 +0100] pkgacct version 10 - user : mysite - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - backup: 1 - archive version: 3 - running with uid 0
    [2015-12-30 02:10:02 +0100] pkgacct using '/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 12 --blocksize 128 --rsyncable' to compress archives
    [2015-12-30 02:10:02 +0100] pkgacct working dir : /backup/2015-12-30/accounts/mysite
    [2015-12-30 02:10:03 +0100] Copying Reseller Config...[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying Suspension Info (if needed)...[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying SSL certificates, CSRs, and keys...[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying Domain Keys....[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying Counter Data....[2015-12-30 02:10:03 +0100] Copying Bandwidth Data....Summary databases … done!
    [2015-12-30 02:10:10 +0100] Copying Dns Zones.......mysite.com...[2015-12-30 02:10:11 +0100] Done
    [2015-12-30 02:10:11 +0100] Copying Mail files....[2015-12-30 02:10:11 +0100] Done
    [2015-12-30 02:10:11 +0100] Copying proftpd file....[2015-12-30 02:10:11 +0100] Done
    [2015-12-30 02:10:11 +0100] Copying www logs.......log file sizes [369148419 byte(s)]............
    ...mysite.com-bytes_log......ftp.mysite.com-ftp_log.offsetftpbytes......mysite.com............
    .........
    [2015-12-30 02:18:20 +0100] Done
    [2015-12-30 02:18:20 +0100] Copy userdata...[2015-12-30 02:18:20 +0100] Done
    [2015-12-30 02:18:20 +0100] Copy custom virtualhost templates...[2015-12-30 02:18:20 +0100] Done
    [2015-12-30 02:18:20 +0100] Copying mailman lists and archives....
    [2015-12-30 02:18:20 +0100] Done copying mailman lists and archives.
    [2015-12-30 02:18:20 +0100] Copying homedir.............
    .........
    [2015-12-30 02:18:27 +0100] Done
    [2015-12-30 02:18:27 +0100] Calculating nobody files............
    .........
    [2015-12-30 02:18:34 +0100] Done
    [2015-12-30 02:18:34 +0100] Entering timeout safety mode for MySQL (suspending cpuwatch)
    Load watching suspended due to SIGUSR1
    [2015-12-30 02:18:34 +0100] Determining mysql dbs...[2015-12-30 02:18:34 +0100] ...mysqldump version: 5.5.47...[2015-12-30 02:18:34 +0100] ...mysql version: 5.5...[2015-12-30 02:18:34 +0100] Saving mysql privs...[2015-12-30 02:18:34 +0100] Done
    [2015-12-30 02:18:34 +0100] ...Done
    [2015-12-30 02:18:34 +0100] Storing mysql dbs............
    mysite_forums.........
    
    .........
    Here, you have the Database tables list. I removed it.
    .........
    
    [2015-12-30 05:05:48 +0100] (2638923665 bytes)
    ERROR: Failed to dump one or more databases
    [2015-12-30 05:05:48 +0100] ...Done
    [2015-12-30 05:05:48 +0100] Leaving timeout safety mode for MySQL (unsuspending cpuwatch)
    Load watching resumed due to SIGUSR2
    [2015-12-30 05:05:48 +0100] Copying cpuser file.......[2015-12-30 05:05:48 +0100] Done
    [2015-12-30 05:05:48 +0100] Copying crontab file.......[2015-12-30 05:05:48 +0100] Done
    [2015-12-30 05:05:48 +0100] Performing “Quota” component....[2015-12-30 05:05:48 +0100] Completed “Quota” component.
    [2015-12-30 05:05:48 +0100] Storing Subdomains....
    [2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Storing Parked Domains....
    [2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Storing Addon Domains....
    [2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Storing SSL domain......[2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Copying password.......[2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Copying shell.......[2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Creating Archive ...................................
    
    ....................
    [2015-12-30 10:50:09 +0100] Done
    [2015-12-30 10:50:09 +0100] pkgacctfile is: /backup/2015-12-30/accounts/mysite.tar.gz
    [2015-12-30 10:55:11 +0100] md5sum is: 53d94b548ee3106666fd36f56108fb3b
    [2015-12-30 10:55:11 +0100]
    size is: 14666831266
    [2015-12-30 10:55:11 +0100]
    homesize is: 35683127296
    [2015-12-30 10:55:11 +0100] pkgacct completed
    [2015-12-30 10:55:12 +0100] info [backup] Queuing daily backup copy for transport
    [2015-12-30 10:55:12 +0100] info [backup] Queuing transport of file: /backup/2015-12-30/accounts/mysite.tar.gz
    [2015-12-30 10:55:46 +0100] info [backup] Calling pkgacct under cpuwatch to backup user “anothersite”
    [2015-12-30 10:55:47 +0100] pkgacct started.
    
    ... (I removed information about other sites backup here)
    
    [2015-12-30 12:33:26 +0100] info [backup] Completed at Wed Dec 30 12:33:26 2015
    [2015-12-30 12:33:26 +0100] info [backup] Final state is Backup::Success (0)
    [2015-12-30 12:33:26 +0100] info [backup] Sent Backup::Success notification.
    
    
    Thank you for your help. :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    #3 SecondSight, Dec 30, 2015
    Last edited by a moderator: Jan 5, 2016
  4. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    70
    Likes Received:
    0
    Trophy Points:
    156
    Hello ! :)

    I've found that when I run /scripts/pkgacct mysite, the backup is done in 18 minutes... (while it's lasting more than 9 hours with the automatic backup system) and I don't get the "ERROR: Failed to dump one or more databases" error message.

    When I compare the two logs I see that I don't have this kind of lines when I use /scripts/pkgacct :
    Setting I/O priority to reduce system load: best-effort: prio 3
    Calling pkgacct under cpuwatch to backup user “mysite”


    Do you think it's cpuwatch which makes the backup last long ?

    Thank you ! :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    70
    Likes Received:
    0
    Trophy Points:
    156
    Hello ! :)

    My server management company techs told me :
    - the account is 36 Gb in size.
    - the backup lasts long because it is halted due tot the server load,
    - mysql times out when performing the backup of my biggest database's table.

    Anyway, /scripts/pkgacct works... (but I have not tested the backup file to see if it's ok).

    What can I do now to backup this 36 Gb account ? Can you give me pieces of advice ?

    Thank you ! :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    70
    Likes Received:
    0
    Trophy Points:
    156
    Hello ! :)

    Still trying to find a solution.

    1) Running backups as root works.
    2) The automatic WHM backup system doesn't work (slow and error messages).

    What are the differences between the two ? Can you tell me which user the automatic WHM backup task runs ?

    Thank you for your help. :)
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    43,961
    Likes Received:
    1,821
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice