Out of Memory The process “php” was terminated

[LordLiverpool]

Hello

My Web Server keeps running out of memory which causes websites to fall over.

This has happened 12 times in the past 3 months, roughly once a week. But its happened 3 times in the last 7 days, which is why I'm posting here.

I get an email from cPanel Monitoring to say:

Out of memory: The process “php” was terminated because the system is low on memory.

But the last instance where the server fell over I also got these new messages:

• Out of memory: The process “mysqld” was terminated because the system is low on memory.
• The service “clamd” appears to be down.
• The service “cphulkd” appears to be down.
• The chkservd process has become non-responsive.

I suspect those services simple fell over at the same time due to lack of memory

My websites were down from between 30 seconds up to 10 minutes, before the server automatically recovered itself.

My server has 6GB of ram and runs just 11 small websites, none with any significant traffic. Which I thought was more than sufficient. So maybe I need to clear something down? Change a setting etc..

Any ideas what I can do to stop this from happening again or reduce the likelihood?

Thanks in advance.

 

[SysSachin]

You are right. You are facing this issues because of server resource shortages. You need to monitor your server process to find out the exact process which is using all server memory.

The following thread is a good for you.

Troubleshooting high server loads on Linux servers

 

[cPanelMichael]

Hello,

Yes, as mentioned, the following thread is a good place to start:

Troubleshooting high server loads on Linux servers

Thank you.

 

[LordLiverpool]

Thanks guys I will read the article and report back my findings.

Cheers

 

[LordLiverpool]

OK just to give an update.

I provisioned an additional 4GB of RAM for my VPS back in November, taking the server from 4GB to 8GB
and it initially made a difference.

I also backed up, archived and deleted all unwanted domains, to try to reduce the load on the server.
(Its not a particularly busy server, just a dozen or so small WordPress websites)

But since then my server has continued to keep on falling over, roughly once a week.

See Thumbnails for typical error messages:



About 3 weeks ago I installed CSF firewall and it keeps sending me emails to say PHP-CGI is

• a suspicious process.
• is using excessive resources.

Does this help prove PHP-CGI is the culprit?

I asked the nice folks at CSF why I was getting so many emails:



but they never replied, I appreciated it's probably because mine is a repeat question.

Based on the information in those last two emails from CSF I contact WordFence via the WordPress forum, in case ironically their WordPress firewall plugin had been infected, and the plugin author said in his opinion it looks like PHP-CGI is the culprit.

I read the thread you provided me regarding troubleshooting my server

I downloaded the script created by @JeffP.

But when I tried to Add a Chron Job via WHM I couldn't find the option anywhere in WHM. I am on version 64.



Does your documentation need updating? or am I on the wrong version of WHM???
Or must I use the CLI to create a Chron Job? So 1970's!

Would love to get to the bottom of this, Any help is appreciated.

If PHP-CGI is the culprit, what's the remedy?

Cheers

 

[cPanelMichael]

Hello,

The information you provided suggests the WordPress application referenced in the notifications you received is the likely culprit for the resource usage. The following URL may help:

WordPress Optimization « WordPress Codex

You may also want to seek out system administration services at the following URL:

System Administration Services | cPanel Forums

Note: You can setup root cron jobs via SSH with the "crontab -e" command.

Thank you.

 

[LordLiverpool]

@cPanelMichael

Firstly I appreciate your help, I fully understand how important your time is, so genuinely thank you for helping me.

Based on your feedback I decide to deactivate the WordFence Firewall Plugin.
After further reading it looks like the same functionality is offered by CSF

However after a sudden halt in emails from CSF I got another one to say the deactivated code was still a suspicious activity!
I was surprised as I thought that deactivated code was incapable of being run. So as I precaution I deleted the code.

But still the emails keep coming from CSF, albeit at a much reduced rate now i.e. from every 5 minutes down to once a day.

This make me wonder was WordFence a red herring? i.e. a symptom rather than the cause?
and that php-cgi is somehow misconfigured???
"out the box" I've not changed it.

'
Time: Wed Jun 7 16:25:56 2017 +0100
Account: adminm
Process Count: 20 (Not killed)

Process Information:

User:adminm PID:23418 PPID:20957 Run Time:0(secs) Memory:464572(kb) RSS:37140(kb) exe:/opt/cpanel/ea-php56/root/usr/bin/php-cgi cmd:/opt/cpanel/ea-php56/root/usr/bin/php-cgi
'

 

[cPanelMichael]

Hello,

It's normal to see that process itself running on the system, as it's your server's PHP handler for PHP version 5.6. The part you should pay attention to if it's listed in the notification is the "Network Connection" line to see if the PHP script is making outgoing connections. There's a thread on that topic at:

Outbound wp-login.php brute force attack from my cpanel server

Thank you.

 

[LordLiverpool]

@cPanelMichael

I really appreciate your help. I will read the link thoroughly and report back.

Genuinely thank you!

 

[NixTree]

Last time I got a server with wordpress outbound attack, there was a process running like
/usr/bin/host. Is there any such process ? Also can you search for any .so extension files under your account like bruteforce.so,.frsdfg , libworker.so etc ? tcpdump command is the best companion for you if that is an outbound attack