Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Backups taking longer on CloudLinux and cPanel 66

Discussion in 'Data Protection' started by feanorknd, Sep 4, 2017.

Tags:
  1. feanorknd

    feanorknd Member

    Joined:
    Sep 28, 2005
    Messages:
    17
    Likes Received:
    0
    Trophy Points:
    151
    Hello:

    just have actually same experience.... backups are taking almost 18-20 hours..... are incremental, only keeping actual day, at NFS partition.

    Before last upgrade to cpanel v64, it was taking maybe 2-3 hours or less.

    Any information related???

    Thanks.
     
  2. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    There are a couple of cases referenced on the following post:

    Remote incremental backups - timeouts

    However, as far as the speed itself, could you open a support ticket using the link in my signature so we can take a closer look?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Hello

    Im having the same issue here.

    Let me tell my scenario: previous server running CL6 64bits, cpanel 11.66. Incremental backup took 2 or 3 hours to complete, every night. Low CPU load.

    So we moved all accounts to a new server, better CPU, double RAM, running CloudLinux 7 64 bits, cpanel 11.66. Now incremental backups need from 8 to 12 hours to complete. When the backup start at 01am, load rise from 0.XX to 2.00.

    Something is wrong, because it's same accounts (in fact less accounts because some users leave the new server), more CPU, more memory, new hard drives. We try replace the backup hard drive, but this dont fix the issue. How the same backup job took 3 hours on a old server and 8 to 12 hours on a more powerfull server?

    The main change beside the hardware is CL6 to CL7.

    (fully updated cpanel, kernel)

    Thank
     
  4. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Just a update:

    I use this small script to update the incremental backup:


    Code:
    KONTA=`ls -1A /var/cpanel/users/`
    for x in `echo -n "$KONTA"`;do
    if [ -d "/home/$x" ]; then
    /usr/local/cpanel/bin/pkgacct --incremental "$x" /backup/cpbackup/daily backup
    fi
    done

    Result? backup done in less than 3 hours - like on previous server.

    If I run incremental backup using the legacy backup OR the new backup system, the result is the same: 12 hours to complete backup.

    Something is throttling the backup script (legacy and new) of cpanel, I dont know if is CloudLinux 7 related or anything else.
     
  5. InterServed

    InterServed Well-Known Member

    Joined:
    Jul 10, 2007
    Messages:
    263
    Likes Received:
    4
    Trophy Points:
    68
    cPanel Access Level:
    DataCenter Provider
    Same problems for me. I had to split accounts into more cpanel servers just to make the backups complete in reasonable time.
    I believe it may have to do with the new meta files and/or because the changes cpanel made in the way incrementals are made/processed. Pruning of the old backup dir after the backup finishes as well , in my case just the pruning sometimes took 1-2 hours or more). So i personally never saw any backup improvements ever since v60 , it only degraded more in terms of performance/impact on the systems.

    One more thing that I'm not sure if it is related or not , but on systems with MariaDB 10.2 i notice very high memory increase , probably memory leaks.

    I will probably start testing the method mindnetcombr described , until cpanel will decide to remove that as well.
     
    #5 InterServed, Sep 9, 2017
    Last edited: Sep 9, 2017
  6. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    I dont use MariaDB, so I think is not related.

    I try use legacy backup system (incremental) and same issue. When I use a simple script to pkgacct all accounts (the script on my previous message above), server have no issues.

    The more strange for me is: this start only when I move from CL6 to CL7. I have servers running CL6 doing incremental backups daily in less than 3 hours (650 accounts average, 1Tb of data average).

    Sure meta files on new backup system is problem too - and like you said, pruning the old days cause high load on my servers.

    Based on my experience the new backup system is a total disaster.

    I hope they keep the script pkgacct - so I can run my own backup script, without any issues.
     
  7. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    Please feel free to open a support ticket if you are facing this issue so we can take a closer look. Post the ticket number here and we will update this thread with the outcome.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  8. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Update: now I have 2 servers on same condition, cPanel + CloudLinux 7, incremental backups take 12 hours.

    If I run the script below, backup complete in 3 hours:


    Code:
    KONTA=`ls -1A /var/cpanel/users/`
    for x in `echo -n "$KONTA"`;do
    if [ -d "/home/$x" ]; then
    /usr/local/cpanel/bin/pkgacct --incremental "$x" /backup/cpbackup/daily backup
    fi
    done
     
  9. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hi @mindnetcombr,

    Could you open a support ticket using the link in my signature so we can take a closer look at one of the affected systems? You can post the ticket number here and we will update this thread with the outcome.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Hi

    I have the ticket #8858129 handled by cPanel Shoshana.

    Your support team is looking on my two servers at same time, doing some research and I think they found a possibly culprit for this, but I'll wait for the investigation to finish before I say anything. You can read on ticket.

    Thank Michael!
     
  11. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    We are tracking reports of backups taking significantly longer to complete (compared to cPanel version 64) on servers using cPanel version 66 and CloudLinux as part of internal case CPANEL-15825. I'll monitor this case and update this thread with more information as it becomes available.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Hello,

    I have two servers running CloudLinux 7 and cPanel 11.66, in version 11.64 it took 3 hours, now in version 11.66 each backup takes 12 hours.

    This has turned into a daily hell, because backups take so long that they eventually reach business hours, and the server gets overloaded.

    I know that the developers are researching, it was me who opened the ticket that culminated in the opening of the investigation case they are doing, but the days are passing and there is no solution. I have this problem for 15 days.

    Is there any way to downgrade from 11.66 to 11.64? I believe the 11.64 version dont suffer this problem, If it were possible I might have an immediate solution.
     
  13. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    Our developers are currently in communication with CloudLinux to further investigate the cause of this issue. I'll continue to monitor the case and update this thread with more information as it becomes available.

    You can not downgrade to an earlier major version (e.g. 66 to 64) after updating.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  14. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    More info maybe usefull for someone else: CloudLinux 6 dont have this issue. I have servers running CloudLinux 6 and backup run as usual. But all my servers with CloudLinux 7 have the same isse.

    So if you have a server with CloudLinux 6 dont upgrade to 7 until this issue is fixed.
     
  15. zye

    zye Well-Known Member

    Joined:
    Dec 6, 2002
    Messages:
    114
    Likes Received:
    4
    Trophy Points:
    168
    my backup took also 11 hours today but i just noticed that it takes every 7 days that long... ( 1 TB of data )
     
  16. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
  17. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    45,442
    Likes Received:
    1,961
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hi @mindnetcombr,

    There's no update to report at this time. I'll continue to monitor the case and update this thread once new information is available.

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  18. InterServed

    InterServed Well-Known Member

    Joined:
    Jul 10, 2007
    Messages:
    263
    Likes Received:
    4
    Trophy Points:
    68
    cPanel Access Level:
    DataCenter Provider
    Hello,

    Backups run perfect using the latest v68 build and activating the "Disable phpMyAdmin information schema searches" which can be found in the Tweak settings. This is much better. Will monitor more servers in the next days.

    backups start at 00:01
    Code:
    v66
    -rw------- 1 root root 882K Oct 12 03:50 1507755722.log
    -rw------- 1 root root 880K Oct 13 03:26 1507842122.log
    -rw------- 1 root root 884K Oct 14 04:16 1507928521.log
    -rw------- 1 root root 882K Oct 15 03:36 1508014922.log
    -rw------- 1 root root 884K Oct 16 04:01 1508101322.log
    -rw------- 1 root root 885K Oct 17 04:03 1508187722.log
    -rw------- 1 root root 889K Oct 18 04:33 1508274122.log
    -rw------- 1 root root 877K Oct 19 03:38 1508360522.log
    -rw------- 1 root root 869K Oct 20 04:22 1508446921.log
    -rw------- 1 root root 869K Oct 21 02:49 1508533321.log
    v68
    -rw------- 1 root root 882K Oct 22 01:42 1508619722.log
    
     
    cPanelMichael likes this.
  19. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Hello InterServed,

    Thank you for the information. How are going the tests with cpanel 11.68? Are you using CloudLinux 6 or 7?

    Here I found this: if I reboot the server (CloudLinux 7) and run backup as soon server boot, the incremental backups run quick.

    But after 12-24 hours of a boot, the issue begins.
     
    Mostafizur likes this.
  20. mindnetcombr

    mindnetcombr Well-Known Member

    Joined:
    Sep 16, 2004
    Messages:
    65
    Likes Received:
    3
    Trophy Points:
    158
    Hello, InterServed

    Yesterday I update to 11.68.09 (release) CloudLinux 7 and this not fixed the issue.

    Unfortunately
     
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice