[Case CPANEL-1843] pkgacct on 11.50.2.0 under CentOS 5

[tomfra]

Hello everyone,

We had some serious problems earlier Today after a recent cpanel update. Every night cpanel updates itself and then runs a backup. We are using legacy backup still, it's been working OK for us, for several years.

Tonight at 1:39 AM local time we received the standard cpanel update log, so far so good. Backup normally begins at 1 AM. At 1:57 AM we received "server down" notification.

So apparently, it crashed during the backup.

Server was rebooted, raid resynced. Again, so far so good. I launched the backup process again as it did not finish before, via the standard cpanel backup script.

Some 15 minutes after it launched, server load got extremely high again (like 95+) and had to be rebooted again.

So apparently, the problem is in the backup process - but as I said, it's been working for years so the recent update is at fault apparently.

After some log research, it seems the backup was creating an archive and in /var/log/messages there was a "running out of memory" error. A bad process I suppose but have no further information.

Anyone experiences anything similar? I cannot use the backup system before it's fixed!

Thanks!

Tomas

 

[qtgame]

Hello Tomfra,

we have a same problem with the backup. After i started manual, 30 min later the LOAD very hight ( >140 ).
This server have 6GB memory and run out... server start using swap!
(I temporarily disabled the backup)
Any other solution?

Krisztian​

 

[tomfra]

I have also contacted customer support about this issue and they confirmed there seems to be a problem on CentOS 5 servers, and that it also affects the new/standard backup system. They are trying to find a solution.

 

[cPanelMichael]

Hello

Internal case number CPANEL-1843 is open to address an issue where systems running CentOS 5 and 11.50.2.0 are locking up due to resource usage when pkgacct (Backups, Transfers) runs. This bug is considered a high priority, and it's likely a resolution will be released in the near future. In the meantime, the current workaround is to downgrade to cPanel version 11.50.1.3 (available on the "Stable" build tier). I will update this thread with more information as it becomes available.

Thank you.

 

[Erik Knepfler]

I just had two systems freeze this morning during modern backup process which had to be rebooted. Result was incomplete backups. cpbackup logs show it on "Creating archive ..." with many "....." and then it stopped logging when it froze. The drives are basically healthy (one has a handful of read errors but the other does not, on either the source or destination drive.) The odds of both failing in the same way on the same day is pretty low, though one also failed the day before the same way before the other did. One was using compression, one was not.

Both servers logged this in upcp logs the day it started (Oct 6 2015)

update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync.asc (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync.bz2 (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync.asc (connected:1).......(request attempt 1/12)[email protected]%......request success......Done
update.1444117681.log:Fetching http://httpupdate.cpanel.net/cpanelsync/transfers_PUBLIC/pkgacct/.cpanelsync (connected:1).......(request attempt 1/12)[email protected].100%......request success......Done

I wonder if there are issues with those.

 

[Erik Knepfler]

Pretty sure my issue above is covered here:

https://forums.cpanel.net/threads/case-cpanel-1843-pkgacct-on-11-50-2-0-under-centos-5.501671/

 

[cPanelMichael]

This issue is addressed in cPanel version 11.50.2.2:

Fixed case CPANEL-1853: pkgacct: Avoid overload on backup on CentOS 5.

You can update cPanel via "/scripts/upcp" if you are on the "Release" build tier to receive this update.

Thank you.

 

[hostmoon]

I am having this problem with CloudLinux in 11.50.2.2 and had to roll back to 11.50.1.3

 

[cPanelMichael]

New I am having this problem with CloudLinux in 11.50.2.2 and had to roll back to 11.50.1.3

Could you provide more information about which version of Cloud Linux is installed on your system, the exact issue you experienced, and if you were able to verify that 11.50.2.2 was in-fact used "before" this happened?

Thank you.

 

[@home]

I had the same problems on 3 VPS machines running on Xen With WHM 11.50.2.0 under Centos 5,

Upgrade to 11.50.2.2 fixed it for me.