High load on server caused by PHP-FPM

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
I have VPS with 4 cores , but the load average reach 300 !

In Process Manager i see a lot of php-fpm: pool mywebsite_com

The problem is: I don't know which file cause problems or am I under attack.

The question is: How can I know what is the problem?
 

Jcats

Well-Known Member
PartnerNOC
May 25, 2011
806
156
168
New Jersey
cPanel Access Level
DataCenter Provider
I have VPS with 4 cores , but the load average reach 300 !

In Process Manager i see a lot of php-fpm: pool mywebsite_com

The problem is: I don't know which file cause problems or am I under attack.

The question is: How can I know what is the problem?

Enable slow PHP logging, it can in most cases narrow down the problematic scripts.

Modify: /opt/cpanel/ea-php*/root/etc/php-fpm.d/domain.conf

You can leave the asterisk next to php if you don't know the PHP version and just replace domain.com example if you use nano:

Code:
nano /opt/cpanel/ea-php*/root/etc/php-fpm.d/bob.com.conf
then at the bottom of the config, add:

Code:
request_slowlog_timeout = 10
slowlog = /home/bobcom/logs/bob.com.php.slow.log
replace bobcom with the cpanel user and bob.com with your actual domain.

then restart php-fpm

Code:
/scripts/restartsrv_apache_php_fpm
Now once a script has been serving a single request for more than 10 seconds, it will trace the process and output a bunch of hopefully useful info for you into that log file.

Secondly, you can tail the access log file to see your traffic:

Code:
tail -f /home/bobcom/access-logs/bob.com*
See anything suspicious? Example if running a WordPress site you might see lots of hits to xmlrpc.php and/or wp-login.php

Otherwise, it might just be your site's dynamic usage, again if its WordPress this can in most cases be resolved by simply installing a caching plugin.

Let us know what you find
 
  • Like
Reactions: cPanelLauren

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
What is the RAM of your server? It should be atleast 1 GB.

And by the way, browscap also causes high memory load. And about 5 seconds increase on page load.

Do not restart your server if you have browscap.ini and have less than 1GB or 1GB of RAM.
8 GB of ram, it was working very well and fast. But now sometimes the load reach 300
I think there is a problem in some php files in my website, but i can't see it process manager because PHP-FPM
When I disable PHP-FPM to see the file I get error 500 in my website.
 

Jcats

Well-Known Member
PartnerNOC
May 25, 2011
806
156
168
New Jersey
cPanel Access Level
DataCenter Provider
When I disable PHP-FPM to see the file I get error 500 in my website.
Did you check your error logs? If running PHP-FPM PHP errors get logged to:

Code:
/home/user/logs/domain.com.php.error.log
when PHP-FPM disabled, it will be in an error_log file where the script is throwing the error, you can search for all error_log files:

Code:
find /home/user/public_html/ -type f -name error_log
 
  • Like
Reactions: cPanelLauren

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
Enable slow PHP logging, it can in most cases narrow down the problematic scripts.

Modify: /opt/cpanel/ea-php*/root/etc/php-fpm.d/domain.conf

You can leave the asterisk next to php if you don't know the PHP version and just replace domain.com example if you use nano:

Code:
nano /opt/cpanel/ea-php*/root/etc/php-fpm.d/bob.com.conf
then at the bottom of the config, add:

Code:
request_slowlog_timeout = 10
slowlog = /home/bobcom/logs/bob.com.php.slow.log
replace bobcom with the cpanel user and bob.com with your actual domain.

then restart php-fpm

Code:
/scripts/restartsrv_apache_php_fpm
Now once a script has been serving a single request for more than 10 seconds, it will trace the process and output a bunch of hopefully useful info for you into that log file.

Secondly, you can tail the access log file to see your traffic:

Code:
tail -f /home/bobcom/access-logs/bob.com*
See anything suspicious? Example if running a WordPress site you might see lots of hits to xmlrpc.php and/or wp-login.php

Otherwise, it might just be your site's dynamic usage, again if its WordPress this can in most cases be resolved by simply installing a caching plugin.

Let us know what you find
Thank you very much, I added the code in the file.
In the same file i see this:

pm = ondemand
pm.max_children = 200
pm.max_requests = 1000
pm.max_spare_servers = 5
pm.min_spare_servers = 1
pm.process_idle_timeout = 1
pm.start_servers = 0

Is it good settings?
 
Last edited:

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
When I entered this code
/scripts/restartsrv_apache_php_fpm

I got
Service Error
(XID 5c2fjf) The “apache_php_fpm” service failed to restart because the restart script exited with an error:

apache_php_fpm has failed. Contact your system administrator if the service does not automagically recover.
 

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
[[email protected] ~]# /scripts/restartsrv_apache_php_fpm

apache_php_fpm restarted successfully.

Now it works

I will see what will happen
This is the first time i see the load reach 300. maybe it is ddos?

Now Load Averages: 2.55 1.76 1.22 which is normal
 
Last edited by a moderator:

Jcats

Well-Known Member
PartnerNOC
May 25, 2011
806
156
168
New Jersey
cPanel Access Level
DataCenter Provider
Something was entered in incorrectly then.

My guess is the path you entered in for slowlog is wrong.

Make sure its the path that exists already for your cPanel accounts logs folder.
 

Jcats

Well-Known Member
PartnerNOC
May 25, 2011
806
156
168
New Jersey
cPanel Access Level
DataCenter Provider
This is the first time i see the load reach 300. maybe it is ddos?
Impossible for me to say, we have no information to go off of other than your load is hitting 300.

Did you check the access logs yet?
 

Jcats

Well-Known Member
PartnerNOC
May 25, 2011
806
156
168
New Jersey
cPanel Access Level
DataCenter Provider
Use 'cat' to print out the entire access log, then scroll up to the time the issue was occuring. You can pipe it through less so you can use page up / page down to scroll through.

Code:
cat access.log | less
 

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
Use 'cat' to print out the entire access log, then scroll up to the time the issue was occuring. You can pipe it through less so you can use page up / page down to scroll through.

Code:
cat access.log | less
The first log at [07/Oct/2019:12:12:30 +0000]
But the problem was at 10:00am

Very nice codes thank you Jcats, I will use it when it happens again

I think i will immediately know what is the problem if i disable php-fpm so I can view file name in Process manager.

But when I disable it i got error 500 in my website
 

Jcats

Well-Known Member
PartnerNOC
May 25, 2011
806
156
168
New Jersey
cPanel Access Level
DataCenter Provider
That means the log file was rotated, you can view the rotated log file, example:

/home/user/logs/domain.com-ssl_log-Oct-2019.gz

But when I disable it i got error 500 in my website
You have to look in the error_log file, without it, we'll never know. Please check my earlier response:


If you are not seeing anything in the error_log, check

Code:
tail -f /usr/local/apache/logs/error_log
While tailing it, hit refresh a few times on the site and see if it logs anything.

You can do something similar for error_log, example run this in public_html(assuming that is where your site is).

Code:
tail -f $(find -name error_log)
then hit refresh on the site a few times and see if any error's get thrown.
 
  • Like
Reactions: cPanelLauren

ziadmm0

Active Member
Aug 11, 2019
26
3
3
usa
cPanel Access Level
Root Administrator
I disabled PHP-FPM now and I got
[Mon Oct 07 15:37:44.059871 2019] [:error] [pid 19248:tid 47142920025856] [client 151.255.14.183:20502] SoftException in Application.cpp:267: File "/home/xx/public_html/xx.php" is writeable by group
[Mon Oct 07 15:37:44.059930 2019] [core:error] [pid 19248:tid 47142920025856] [client 151.255.14.183:20502] End of script outputbefore headers: xx.php
[Mon Oct 07 15:37:44.115858 2019] [:error] [pid 19278:tid 47142938937088]