[Case CPANEL-1843] pkgacct on 11.50.2.0 under CentOS 5

tomfra

Well-Known Member
Sep 30, 2002
68
1
158
Hello everyone,

We had some serious problems earlier Today after a recent cpanel update. Every night cpanel updates itself and then runs a backup. We are using legacy backup still, it's been working OK for us, for several years.

Tonight at 1:39 AM local time we received the standard cpanel update log, so far so good. Backup normally begins at 1 AM. At 1:57 AM we received "server down" notification.

So apparently, it crashed during the backup.

Server was rebooted, raid resynced. Again, so far so good. I launched the backup process again as it did not finish before, via the standard cpanel backup script.

Some 15 minutes after it launched, server load got extremely high again (like 95+) and had to be rebooted again.

So apparently, the problem is in the backup process - but as I said, it's been working for years so the recent update is at fault apparently.

After some log research, it seems the backup was creating an archive and in /var/log/messages there was a "running out of memory" error. A bad process I suppose but have no further information.

Anyone experiences anything similar? I cannot use the backup system before it's fixed!

Thanks!

Tomas
 

qtgame

Registered
Feb 9, 2013
1
0
1
cPanel Access Level
Root Administrator
Hello Tomfra,

we have a same problem with the backup. After i started manual, 30 min later the LOAD very hight ( >140 ).
This server have 6GB memory and run out... server start using swap!
(I temporarily disabled the backup)
Any other solution?

Krisztian​
 

tomfra

Well-Known Member
Sep 30, 2002
68
1
158
I have also contacted customer support about this issue and they confirmed there seems to be a problem on CentOS 5 servers, and that it also affects the new/standard backup system. They are trying to find a solution.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,215
363
Hello :)

Internal case number CPANEL-1843 is open to address an issue where systems running CentOS 5 and 11.50.2.0 are locking up due to resource usage when pkgacct (Backups, Transfers) runs. This bug is considered a high priority, and it's likely a resolution will be released in the near future. In the meantime, the current workaround is to downgrade to cPanel version 11.50.1.3 (available on the "Stable" build tier). I will update this thread with more information as it becomes available.

Thank you.
 
  • Like
Reactions: daflame

Erik Knepfler

Member
Sep 2, 2014
19
1
53
Long Beach, California, United States
cPanel Access Level
Root Administrator
I just had two systems freeze this morning during modern backup process which had to be rebooted. Result was incomplete backups. cpbackup logs show it on "Creating archive ..." with many "....." and then it stopped logging when it froze. The drives are basically healthy (one has a handful of read errors but the other does not, on either the source or destination drive.) The odds of both failing in the same way on the same day is pretty low, though one also failed the day before the same way before the other did. One was using compression, one was not.

Both servers logged this in upcp logs the day it started (Oct 6 2015)
Code:
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync.asc (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync.bz2 (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync.asc (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
I wonder if there are issues with those.
 
Last edited by a moderator:

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,215
363
This issue is addressed in cPanel version 11.50.2.2:

Fixed case CPANEL-1853: pkgacct: Avoid overload on backup on CentOS 5.

You can update cPanel via "/scripts/upcp" if you are on the "Release" build tier to receive this update.

Thank you.
 
  • Like
Reactions: daflame

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,910
2,215
363
New I am having this problem with CloudLinux in 11.50.2.2 and had to roll back to 11.50.1.3
Could you provide more information about which version of Cloud Linux is installed on your system, the exact issue you experienced, and if you were able to verify that 11.50.2.2 was in-fact used "before" this happened?

Thank you.
 

@home

Well-Known Member
Nov 5, 2003
121
2
168
cPanel Access Level
Root Administrator
I had the same problems on 3 VPS machines running on Xen With WHM 11.50.2.0 under Centos 5,

Upgrade to 11.50.2.2 fixed it for me.
 
Last edited: