The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Long lasting backup - Settings to make it faster ?

Discussion in 'Data Protection' started by SecondSight, Dec 28, 2015.

  1. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    66
    Likes Received:
    0
    Trophy Points:
    6
    Hello :)

    I looked at the backup log on my server, because backups are lasting too long.


    I found the archive creation for my biggest website is lasting more than 6 hours :

    [2015-12-28 05:33:07 +0100] Creating Archive
    .............................................
    [2015-12-28 11:57:55 +0100] Done
    [2015-12-28 11:57:55 +0100] pkgacctfile is: /backup/2015-12-28/accounts/mysite.tar.gz

    => When I do it myself, using a simple tar command, it doesn't take more than 10 minutes (I only backup public_html directory).



    Backing up the database is also taking more than 3 hours :

    [2015-12-28 02:17:05 +0100] Storing mysql dbs............
    mysite_forums.........
    .........
    .........
    [2015-12-28 05:33:06 +0100] ...Done

    => When I do it myself, using a mysqldump command, it lasts 5 minutes.


    Do you know the reasons why the WHM backup tool takes so long (more than 9 hours) to backup one account ? I do it myself in not more than 15 minutes and in the past, the backup lasted 45 minutes.

    I noticed these problems after I had CloudLinux installed but I was told it has nothing to do with it...

    What can I do to find the reasons why backup is so slow ?

    Can you tell me what are the settings I can change in WHM to make the backups faster ?

    Thank you :)
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    Do you notice any additional entries in the backup log within the /usr/local/cpanel/logs/cpbackup directory during the backup process? What's the server load when the automatic backups are generating?

    Thank you.
     
  3. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    66
    Likes Received:
    0
    Trophy Points:
    6
    Hello ! :)

    When I watched the server load, I saw it is between 1 and 4 when the backup is ran (that is to say from 2 am to 16 pm...).

    The only error message is about the database :
    ERROR: Failed to dump one or more databases

    In the error_log file I found :
    [2015-12-30 03:32:20 +0100] warn [pkgacct] mysite_forums: mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `ibf_posts` at row: 1658647 at bin/pkgacct.pl line 1953, <$fh> line 1.

    This table, ibf_posts is the biggest table.


    Here is the content of the log file sent to me by email : (I removed a lot of lines of dots and my site's/account's real name)
    Code:
    [2015-12-30 02:10:02 +0100] info [backup] Setting I/O priority to reduce system load: best-effort: prio 3
    [2015-12-30 02:10:02 +0100] info [backup] Hard Linking available on "/backup"
    [2015-12-30 02:10:02 +0100] info [backup] Running account backup with target : /backup/2015-12-30/accounts
    [2015-12-30 02:10:02 +0100] info [backup] Calling pkgacct under cpuwatch to backup user “mysite”
    [2015-12-30 02:10:02 +0100] pkgacct started.
    [2015-12-30 02:10:02 +0100] pkgacct version 10 - user : mysite - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - backup: 1 - archive version: 3 - running with uid 0
    [2015-12-30 02:10:02 +0100] pkgacct using '/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 12 --blocksize 128 --rsyncable' to compress archives
    [2015-12-30 02:10:02 +0100] pkgacct working dir : /backup/2015-12-30/accounts/mysite
    [2015-12-30 02:10:03 +0100] Copying Reseller Config...[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying Suspension Info (if needed)...[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying SSL certificates, CSRs, and keys...[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying Domain Keys....[2015-12-30 02:10:03 +0100] Done
    [2015-12-30 02:10:03 +0100] Copying Counter Data....[2015-12-30 02:10:03 +0100] Copying Bandwidth Data....Summary databases … done!
    [2015-12-30 02:10:10 +0100] Copying Dns Zones.......mysite.com...[2015-12-30 02:10:11 +0100] Done
    [2015-12-30 02:10:11 +0100] Copying Mail files....[2015-12-30 02:10:11 +0100] Done
    [2015-12-30 02:10:11 +0100] Copying proftpd file....[2015-12-30 02:10:11 +0100] Done
    [2015-12-30 02:10:11 +0100] Copying www logs.......log file sizes [369148419 byte(s)]............
    ...mysite.com-bytes_log......ftp.mysite.com-ftp_log.offsetftpbytes......mysite.com............
    .........
    [2015-12-30 02:18:20 +0100] Done
    [2015-12-30 02:18:20 +0100] Copy userdata...[2015-12-30 02:18:20 +0100] Done
    [2015-12-30 02:18:20 +0100] Copy custom virtualhost templates...[2015-12-30 02:18:20 +0100] Done
    [2015-12-30 02:18:20 +0100] Copying mailman lists and archives....
    [2015-12-30 02:18:20 +0100] Done copying mailman lists and archives.
    [2015-12-30 02:18:20 +0100] Copying homedir.............
    .........
    [2015-12-30 02:18:27 +0100] Done
    [2015-12-30 02:18:27 +0100] Calculating nobody files............
    .........
    [2015-12-30 02:18:34 +0100] Done
    [2015-12-30 02:18:34 +0100] Entering timeout safety mode for MySQL (suspending cpuwatch)
    Load watching suspended due to SIGUSR1
    [2015-12-30 02:18:34 +0100] Determining mysql dbs...[2015-12-30 02:18:34 +0100] ...mysqldump version: 5.5.47...[2015-12-30 02:18:34 +0100] ...mysql version: 5.5...[2015-12-30 02:18:34 +0100] Saving mysql privs...[2015-12-30 02:18:34 +0100] Done
    [2015-12-30 02:18:34 +0100] ...Done
    [2015-12-30 02:18:34 +0100] Storing mysql dbs............
    mysite_forums.........
    
    .........
    Here, you have the Database tables list. I removed it.
    .........
    
    [2015-12-30 05:05:48 +0100] (2638923665 bytes)
    ERROR: Failed to dump one or more databases
    [2015-12-30 05:05:48 +0100] ...Done
    [2015-12-30 05:05:48 +0100] Leaving timeout safety mode for MySQL (unsuspending cpuwatch)
    Load watching resumed due to SIGUSR2
    [2015-12-30 05:05:48 +0100] Copying cpuser file.......[2015-12-30 05:05:48 +0100] Done
    [2015-12-30 05:05:48 +0100] Copying crontab file.......[2015-12-30 05:05:48 +0100] Done
    [2015-12-30 05:05:48 +0100] Performing “Quota” component....[2015-12-30 05:05:48 +0100] Completed “Quota” component.
    [2015-12-30 05:05:48 +0100] Storing Subdomains....
    [2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Storing Parked Domains....
    [2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Storing Addon Domains....
    [2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Storing SSL domain......[2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Copying password.......[2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Copying shell.......[2015-12-30 05:05:50 +0100] Done
    [2015-12-30 05:05:50 +0100] Creating Archive ...................................
    
    ....................
    [2015-12-30 10:50:09 +0100] Done
    [2015-12-30 10:50:09 +0100] pkgacctfile is: /backup/2015-12-30/accounts/mysite.tar.gz
    [2015-12-30 10:55:11 +0100] md5sum is: 53d94b548ee3106666fd36f56108fb3b
    [2015-12-30 10:55:11 +0100]
    size is: 14666831266
    [2015-12-30 10:55:11 +0100]
    homesize is: 35683127296
    [2015-12-30 10:55:11 +0100] pkgacct completed
    [2015-12-30 10:55:12 +0100] info [backup] Queuing daily backup copy for transport
    [2015-12-30 10:55:12 +0100] info [backup] Queuing transport of file: /backup/2015-12-30/accounts/mysite.tar.gz
    [2015-12-30 10:55:46 +0100] info [backup] Calling pkgacct under cpuwatch to backup user “anothersite”
    [2015-12-30 10:55:47 +0100] pkgacct started.
    
    ... (I removed information about other sites backup here)
    
    [2015-12-30 12:33:26 +0100] info [backup] Completed at Wed Dec 30 12:33:26 2015
    [2015-12-30 12:33:26 +0100] info [backup] Final state is Backup::Success (0)
    [2015-12-30 12:33:26 +0100] info [backup] Sent Backup::Success notification.
    
    
    Thank you for your help. :)
     
    #3 SecondSight, Dec 30, 2015
    Last edited by a moderator: Jan 5, 2016
  4. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    66
    Likes Received:
    0
    Trophy Points:
    6
    Hello ! :)

    I've found that when I run /scripts/pkgacct mysite, the backup is done in 18 minutes... (while it's lasting more than 9 hours with the automatic backup system) and I don't get the "ERROR: Failed to dump one or more databases" error message.

    When I compare the two logs I see that I don't have this kind of lines when I use /scripts/pkgacct :
    Setting I/O priority to reduce system load: best-effort: prio 3
    Calling pkgacct under cpuwatch to backup user “mysite”


    Do you think it's cpuwatch which makes the backup last long ?

    Thank you ! :)
     
  5. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    66
    Likes Received:
    0
    Trophy Points:
    6
    Hello ! :)

    My server management company techs told me :
    - the account is 36 Gb in size.
    - the backup lasts long because it is halted due tot the server load,
    - mysql times out when performing the backup of my biggest database's table.

    Anyway, /scripts/pkgacct works... (but I have not tested the backup file to see if it's ok).

    What can I do now to backup this 36 Gb account ? Can you give me pieces of advice ?

    Thank you ! :)
     
  6. SecondSight

    SecondSight Well-Known Member

    Joined:
    Jun 30, 2004
    Messages:
    66
    Likes Received:
    0
    Trophy Points:
    6
    Hello ! :)

    Still trying to find a solution.

    1) Running backups as root works.
    2) The automatic WHM backup system doesn't work (slow and error messages).

    What are the differences between the two ? Can you tell me which user the automatic WHM backup task runs ?

    Thank you for your help. :)
     
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,678
    Likes Received:
    653
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
Loading...

Share This Page