About once per day, Apache on this server fails

[ttremain]

About once a day, Apache on this server fails, and it cannot (or doesn't) restart for about 10-15 minutes.

I see nothing in the Apache error logs. The server fails just after midnight, and came back up at 12:12. It happens at different times on different days.

Are there other logs I should look at, or is it possible to increase the logging level?


[Fri Jan 14 00:00:21.059088 2022] [:error] [pid 1158486:tid 47642606323456] [client 103.144.157.150:32555] [client 103.144.157.150] ModSecurity: Warning. String match "faultCode" at RESPONSE_BODY. [file "/etc/apache2/conf.d/modsec_vendor_configs/imunify360-full-apache/004_i360_2_bruteforce.conf"] [line "293"] [id "33376"] [msg "IM360 WAF: XMLRPC fault response||MV:<?xmlversion=\\"1.0\\"encoding=\\"UTF-8\\"?><methodResponse><fault><value><struct><member><name>faultCode</name><value><int>403</int></value></member><member><name>faultString</name><value><string>Incorrectusernameorpassword.</string></value></member></struct></value></fault></methodResponse>||T:APACHE||"] [severity "NOTICE"] [tag "service_i360custom"] [tag "noshow"] [hostname "www.XXXXXXX.com"] [uri "/xmlrpc.php"] [unique_id "YeEtk7cyxrqzpSmKL9Eh-QAAAFg"]
body.xml:1: parser error : Document labelled UTF-16 but has UTF-8 content
<?xml version="1.0" encoding="utf-16" standalone="yes"?>
^
body.xml:2: parser error : XML declaration allowed only at the start of the document
<?xml version="1.0" encoding="iso-8859-1"?>
^
[Fri Jan 14 00:12:15.989269 2022] [mpm_event:notice] [pid 361409:tid 47642228945984] AH00491: caught SIGTERM, shutting down
[Fri Jan 14 00:12:19.818387 2022] [hostinglimits:notice] [pid 3980869:tid 47838909551680] mod_hostinglimits: use Min UID 1000
[Fri Jan 14 00:12:19.818470 2022] [hostinglimits:notice] [pid 3980869:tid 47838909551680] mod_hostinglimits: version 1.0-37. LVE mechanism enabled
[Fri Jan 14 00:12:19.818472 2022] [hostinglimits:notice] [pid 3980869:tid 47838909551680] mod_hostinglimits: found apr extention version 3
[Fri Jan 14 00:12:19.818478 2022] [hostinglimits:notice] [pid 3980869:tid 47838909551680] mod_hostinglimits: apr_lve_environment_init_group_minuid check ok
[Fri Jan 14 00:12:19.859605 2022] [ssl:warn] [pid 3980869:tid 47838909551680] AH01909: XXXX123.com:443:0 server certificate does NOT include an ID which matches the server name
[Fri Jan 14 00:12:19.902095 2022] [ssl:warn] [pid 3980869:tid 47838909551680] AH01909: YYY123.com:443:0 server certificate does NOT include an ID which matches the server name
[Fri Jan 14 00:12:19.913898 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity for Apache/2.9.3 (http://www.modsecurity.org/) configured.
[Fri Jan 14 00:12:19.913904 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity: APR compiled version="1.7.0"; loaded version="1.7.0"
[Fri Jan 14 00:12:19.913907 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity: PCRE compiled version="8.32 "; loaded version="8.32 2012-11-30"
[Fri Jan 14 00:12:19.913909 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity: LUA compiled version="Lua 5.1"
[Fri Jan 14 00:12:19.913911 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity: YAJL compiled version="2.0.4"
[Fri Jan 14 00:12:19.913912 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity: LIBXML compiled version="2.9.7"
[Fri Jan 14 00:12:19.913914 2022] [:notice] [pid 3980869:tid 47838909551680] ModSecurity: Status engine is currently disabled, enable it by set SecStatusEngine to On.
[Fri Jan 14 00:12:19.914304 2022] [suexec:notice] [pid 3980869:tid 47838909551680] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Jan 14 00:12:20.155332 2022] [hostinglimits:notice] [pid 3980878:tid 47838909551680] mod_hostinglimits: use Min UID 1000
[Fri Jan 14 00:12:20.155360 2022] [hostinglimits:notice] [pid 3980878:tid 47838909551680] mod_hostinglimits: version 1.0-37. LVE mechanism enabled
[Fri Jan 14 00:12:20.155362 2022] [hostinglimits:notice] [pid 3980878:tid 47838909551680] mod_hostinglimits: found apr extention version 3
[Fri Jan 14 00:12:20.155367 2022] [hostinglimits:notice] [pid 3980878:tid 47838909551680] mod_hostinglimits: apr_lve_environment_init_group_minuid check ok


Please advise!

 

[bellwood]

See the following thread:

SOLVED - UPS-421 - Apache 2.4.52 issues?

Anybody seeing a lot of Apache dying with the latest EA4 Apache? Upgraded to ea-apache24-2.4.52-1.1.3.cpanel.x86_64 earlier today across all of our servers, but I'm getting a lot of servers with the web server not responding. Not sure if it's related or not - but seems to be a bit more than...

forums.cpanel.net

Make sure your version of Apache is no LATER than this:

Server version: Apache/2.4.52 (cPanel)
Server built:   Dec 30 2021 14:32:42

The current version is:

Server version: Apache/2.4.52 (cPanel)
Server built:   Jan 12 2022 01:58:29

Note the build times.

 

[ttremain]

I've got this:
Server version: Apache/2.4.52 (cPanel)
Server built: Dec 23 2021 04:07:02

Also, could be useful:
ea-apache24-2.4.52-1.el7.cloudlinux.x86_64

 

[bellwood]

You are on CloudLinux, see this specific comment about the patch release availability on CloudLinux:

SOLVED - UPS-421 - Apache 2.4.52 issues?

Also experience this. Latest patch in Apache bugzilla appears to fix the issue. https://bz.apache.org/bugzilla/show_bug.cgi?id=65769 This affects servers in mpm event.

forums.cpanel.net

I have been asking the folks at CloudLinux via ticket. On 1 Jan 2022 they said:
> We already have a task in order to fix this issue, the task ID - EA4D-318.

And on 5 Jan 2022 they said:
> When the package is released, the news will appear on our blog: СloudLinux Blog | EasyApache 4

CloudLinux forum post:

https://cloudlinux.zendesk.com/hc/en-us/community/posts/4414417739922-mpm-event-will-crash-the-server

 

[ttremain]

CloudLinux forum post:

https://cloudlinux.zendesk.com/hc/en-us/community/posts/4414417739922-mpm-event-will-crash-the-server

I've applied this patch earlier today, and it seems stable so far. Watching it carefully.
Thank you.

 

[cPanelAnthony]

I'm happy to hear things are working so far. Let us know if you run into any issues.