After update to 110.0.5, LFD crashes and Cpanel unresponsive

[samedaycomputer]

This morning we woke up to cpanel being completely unresponsive. I rebooted the server and got cpanel / whm back up and running but LFD is crashing every couple minutes.

I looked at the logs and this seems to be the precursor to the issues this morning:

May 2 00:15:13 vps lfd[48446]: cPanel upgrade detected, restarting ConfigServer services...
May 2 00:15:13 vps lfd[48446]: cPanel upgrade detected, restarting lfd


After that, every 5-30 minutes it crashes and has to restart. This is the lines in the lfd.log right before each crash:


May 2 05:22:47 vps lfd[90736]: *User Processing* PID:90680 Kill:0 User:redacted VM:507(MB) EXE:/opt/cpanel/ea-php74/root/usr/sbin/php-fpm CMD:php-fpm: pool redacted
May 2 05:23:48 vps lfd[80838]: *Error* cannot fork: Cannot allocate memory, at line 7615
May 2 05:23:48 vps lfd[80838]: daemon stopped
May 2 05:23:51 vps lfd[91249]: daemon started on redacted.com - csf v14.18 (cPanel)

Any thoughts? This started this morning and it appears it's immediately following a cpanel update.

The cPanel monitoring emails show me this about memory:

Used​	4.78 GB
Available​	9.13 GB
Installed​	13.9 GB

 

[samedaycomputer]

Shortly after writing this, IMAP started giving errors "FORK FAILED"

Appears to be a process limit issue, or memory. This all started with the latest CPANEL upgrade.

And now I have lost access to the server and must reboot.

This server hasn't had any major changes in over a year, except for the update today.

 

[cPRex]

Hey there! If the server was completely unresponsive to the point where you needed to reboot the system, that indicates an issue at a lower level than the cPanel software. Since multiple processes seem to be experiencing memory issues, it might be best to reach out to the hosting provider or datacenter to have them check the hardware on the machine, as the system itself seems to be reporting plenty of free memory.

You could also check /var/log/messages to see if there are any server-wide errors posted there about the health of the machine.

While the cPanel update itself wouldn't have caused it, the CPU and RAM usage could have been the tipping point to reveal a larger issue on the machine.

 

[samedaycomputer]

Hey there! If the server was completely unresponsive to the point where you needed to reboot the system, that indicates an issue at a lower level than the cPanel software. Since multiple processes seem to be experiencing memory issues, it might be best to reach out to the hosting provider or datacenter to have them check the hardware on the machine, as the system itself seems to be reporting plenty of free memory.

You could also check /var/log/messages to see if there are any server-wide errors posted there about the health of the machine.

While the cPanel update itself wouldn't have caused it, the CPU and RAM usage could have been the tipping point to reveal a larger issue on the machine.

I was connected to SSH when I lost access last time, I was trying to run PS and it wasn't able to run PS.

Two services have now declared "Fork Failed" so it seems like either memory or process cap.

But the problem is these errors started literally minutes from the cpanel update. My logs are clear until this morning starting with cpanel update.

It's on a VM, I can up the memory but I don't believe this is the issue as it has >8 gb "available" from the Free command minutes before this happened.

 

[cPRex]

I don't disagree that there's a memory issue happening, I just don't think a cPanel update could cause the issues you're seeing.

For the ultimate confirmation, it might be best to submit a ticket to our team so we can take a look directly.

 

[samedaycomputer]

Would love to submit a ticket but my email is hosting is on this box and I can't log in to submit a ticket!

 

[cPRex]

You can always start a ticket by emailing our Customer Service team at [email protected].

 

[cPRex]

Do you happen to be using the CloudLinux operating system? If so, your issue could be related to this problem they discovered here: ERROR anon_vma num_children for anon_vma