SOLVED Need help debugging Apache crashes

mvandemar

Well-Known Member
Jun 17, 2006
129
29
178
In the past 9 hours on one of my client's servers Apache has crashed 3 times, from 10-15 minutes each. The server does not appear to be under undue load. /var/log/dmesg doesn't show anything (it hasn't been modified since May 4th), the access_log entries just stop, nothing in /var/log/messages, and nothing in /var/log/apache2/error_log until just before it comes back up, each time with:

Code:
[Thu Sep 23 17:26:46.653241 2021] [mpm_event:notice] [pid 19225:tid 47581126034496] AH00491: caught SIGTERM, shutting down
When I say nothing, I mean nothing anywhere near the time frame of the actual crashes. After that it comes back up and just resumes normally as far as I can tell. In the ChkServd email from cpanel it tells me that the loads averages at the time it detected the service being down were 0.16 0.07 0.46, it was using 1.95 GB of 15.51 GB ram, with 13.57 GB available. The startup logs in the email, from the previous crash:

Code:
Sep 23 09:20:52 vps.theserver.net systemd[1]: Starting Apache web server managed by cPanel EasyApache...
Sep 23 09:20:52 vps.theserver.net systemd[1]: Can't open PID file /run/apache2/httpd.pid (yet?) after start: No such file or directory
Sep 23 09:20:52 vps.theserver.net systemd[1]: Started Apache web server managed by cPanel EasyApache.
Log messages:

Code:
[Thu Sep 23 09:20:52.827674 2021] [mpm_event:notice] [pid 5492:tid 47706683618368] AH00489: Apache/2.4.49 (cPanel) OpenSSL/1.1.1l mod_bwlimited/1.4 configured -- resuming normal operations
[Thu Sep 23 09:20:52.761787 2021] [:notice] [pid 5489:tid 47706683618368] ModSecurity for Apache/2.9.3 (http://www.modsecurity.org/) configured.
[Thu Sep 23 04:09:31.290396 2021] [mpm_event:notice] [pid 13272:tid 47566933357632] AH00489: Apache/2.4.49 (cPanel) OpenSSL/1.1.1l mod_bwlimited/1.4 configured -- resuming normal operations
[Thu Sep 23 04:09:31.227027 2021] [:notice] [pid 13269:tid 47566933357632] ModSecurity for Apache/2.9.3 (http://www.modsecurity.org/) configured.
[Thu Sep 23 04:09:26.195460 2021] [mpm_event:notice] [pid 12864:tid 47405335766080] AH00489: Apache/2.4.49 (cPanel) OpenSSL/1.1.1l mod_bwlimited/1.4 configured -- resuming normal operations
[Thu Sep 23 04:09:26.140720 2021] [:notice] [pid 12861:tid 47405335766080] ModSecurity for Apache/2.9.3 (http://www.modsecurity.org/) configured.
Those appear to be from the restart >5 hours prior. Top process:

Code:
PID     Owner     CPU %     Memory %     Command
19023     mysql     38.08     2.21     /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
19566     user01     31.90     0.53     php-fpm: pool domain_com
19574     user01     31.40     0.53     php-fpm: pool domain_com
19577     user01     30.58     0.53     php-fpm: pool domain_com
19576     user01     30.34     0.53     php-fpm: pool domain_com
And the disk was nowhere near full. I am not sure where to look next.

-Michael
 
  • Like
Reactions: inveress

inveress

Member
Apr 8, 2014
6
0
51
cPanel Access Level
Root Administrator

mvandemar

Well-Known Member
Jun 17, 2006
129
29
178
I have a server with Apache crashing every couple of hours like clockwork since the last round of server updates this morning. It's running PHP 5.6 amongst other versions. I'll check this out, thanks!
ssh in and run:

Code:
/scripts/upcp
They just patched it a few hours ago.

-Michael
 
  • Like
Reactions: inveress and cPRex