Restart Apache after WHM Update

[swbrains]

WHM often prompts me to reboot the server to apply an update. Whenever I do this, the server reboots, but httpd service is not started again. It times out after ~90 seconds. (Even though the attached screenshot text is truncated, it IS a timeout error loading the httpd serviice.) If I SSH into the server and restart httpd manually at this point, it starts fine.

My admins updated the httpd.service file to include the line:
TimeoutStartSec=300
and tested a restart and httpd loaded properly (but this test did not include an update at this point as it had just been updated recently).

After the next WHM update/reboot request, I rebooted the server (gracefully) and the same timeout occurred after 90 seconds and httpd was not loaded. Again, loading it manually via SSH was successful.

It appears that the httpd.service file may have been overwritten during the update (or some other prior process), as the line was no longer there. I do not recall making any changes to the server other than perhaps a tweak in the Exim config page in WHM to update the queue-load parameter.

So I'm wondering if it's known that some of the WHM updates that require reboots would overwrite the httpd.service file, and if so, is there some way I can ensure that after such an update/reboot that httpd will start without timing out as it appears to do whenever these updates are installed and I am forced to reboot to apply them.

Thanks!

 

[cPanelLauren]

Hi @swbrains

This is not standard behavior, no. May I ask why you had them set the timeout to 300 seconds? I'd suggest reverting any customizations of that nature to see if the issue still occurs. Where were they adding the timeout, if not added to the correct location any customizations made to the apache configuration file directly will be overwritten.

 

[swbrains]

Actually, the admins performed the modification after some research on the issue, not at my behest. Initially, the timeout parameter was not set in httpd.service and httpd would not start automatically after a reboot to apply a cPanel update. It would report a timeout starting httpd. I could manually restart it if I logged in via console and issued the service restart command to start httpd. This has been occurring for more than 6 months now across many updates. After a required reboot to apply an update, httpd fails to load automatically. If I restart the server without an update, it starts httpd properly.

After a recent cPanel update, the server again failed to start httpd automatically. The admins checked and determined that the 300 second timeout was not in the httpd.service file even though they had previously added it, so they added it again and rebooted, and httpd started automatically. But this is exactly what happened in prior updates. It would fail during an update/reboot cycle, I would let them know, they would restart the server and it would start fine (likely because no cpanel update preceded the restart when they tried it).

Each time it fails, the log file shows the same timeout error starting httpd after about 90 seconds, so they tried the 300 second timeout parameter, but we don't think it was actually there when the next reboot occurred because after that update/reboot, the server failed to start httpd again. When httpd failed to start, the log showed a 90 second timeout (the default), and they said the timeout line was no longer in the httpd.service file. Hence, it seems like the cPanel update may have overwritten it before the reboot (unless there is some other process that would do this).

 

[cPanelLauren]

Hi @swbrains

My concern is that it shouldn't need to be there in the first place. The standard httpd.service file should look as follows:

 cat /usr/lib/systemd/system/httpd.service
[Unit]
Description=Apache web server managed by cPanel EasyApache
ConditionPathExists=!/etc/httpddisable
ConditionPathExists=!/etc/apachedisable
ConditionPathExists=!/etc/httpdisable

[Service]
Type=forking
ExecStart=/usr/local/cpanel/scripts/restartsrv_httpd --no-verbose
ExecStop=/usr/local/cpanel/scripts/restartsrv_httpd stop --no-verbose
ExecReload=/usr/local/cpanel/scripts/restartsrv_httpd --no-verbose
PIDFile=/run/apache2/httpd.pid

[Install]
WantedBy=multi-user.target

This will more than likely be overwritten when the apache package is updated.


Can you show the full error when the timeout occurs?

 

[swbrains]

This is the log of the error from a previous instance (I've masked the server identifier):

Jan 28 07:22:17 xxxxxxx systemd: httpd.service start operation timed out. Terminating.
Jan 28 07:22:18 xxxxxxx systemd: Failed to start Apache web server managed by cPanel EasyApache.
Jan 28 07:22:18 xxxxxxx systemd: Unit httpd.service entered failed state.
Jan 28 07:22:18 xxxxxxx systemd: httpd.service failed.

Here is the httpd.service file (with the timeout parameter added):

[Unit]
Description=Apache web server managed by cPanel EasyApache
ConditionPathExists=!/etc/httpddisable
ConditionPathExists=!/etc/apachedisable
ConditionPathExists=!/etc/httpdisable
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
ExecStart=/usr/local/cpanel/scripts/restartsrv_httpd --no-verbose
ExecStop=/usr/local/cpanel/scripts/restartsrv_httpd stop --no-verbose
ExecReload=/usr/local/cpanel/scripts/restartsrv_httpd --no-verbose
PIDFile=/run/apache2/httpd.pid
TimeoutStartSec=300

[Install]
WantedBy=multi-user.target

 

[cPanelLauren]

Hi @swbrains

That's not as telling as I'd hoped. Ultimately I'd like to see the need for adding that timeout resolved since it's really not the standard behavior.
Can you please open a ticket using the link in my signature? Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved.


Thanks!

 

[swbrains]

Hi,

I tried to create a support request via cPanel, but the ticket wizard gets to the step where it requires me to log into my cPanel account to grant authorization. Unfortunately, I don't know the password. I use the password reset link but the email never comes. I checked the cpanel email delivery tracking page and there is no attempted delivery of any message from cpanel, so it's like it never got sent. It has been more than 5 minutes since I requested the password reset and I don't see it. I tried creating a new cpanel account on the authorization login page, but it says the user id (email address) exists already. So I'm stuck. I have full access go log into my server as root, but can't seem to authorize cpanel to access it. Frustrating...

 

[swbrains]

Ok, luckily I had written down my password some place, so I was able to finally log into the cpanel customer portal and submit the ticket:
11526775 - httpd does not consistently restart when server is restarted
I noticed however on the final ticket submission page that it said the system could not access my server via SSH, so I guess I need to update the details for SSH, but I haven't found out where to do that yet.

 

[cPanelLauren]

Hi @swbrains

I am glad you were able to get the ticket open, I took a look at it today and it looks like we're waiting for the issue to show itself again at the next time you need to reboot. We'll update here more when there's more information but do let us know when the next time for reboot is.


Thanks!