cPanel 11.25 Log Processing

[blargman]

It is my assumption that safeapacherestart is waht is used to restart Apache when it is rotating logs correct?

What does this script do exactly?

I may need to wrap it so that I can restart my front-end nginx server. Everytime stats are run and the logs are rotated it quits logging as the filehandle has changed.

[cPanelDon]

It is my assumption that safeapacherestart is waht is used to restart Apache when it is rotating logs correct?

What does this script do exactly?

I may need to wrap it so that I can restart my front-end nginx server. Everytime stats are run and the logs are rotated it quits logging as the filehandle has changed.

It is correct that the script mentioned is one component involved in handling restarts of Apache/httpd and it is correct that it can be executed or triggered by cpanellogd; however, I do not recommend attempting to wrap-around it as that would not be an ideal solution to catch all stops, starts, restarts, and graceful restarts of Apache/httpd, such as manual restarts performed by a server administrator.

I would consider the following two viable methods if needing to hook into Apache restarts:

1. Provide a custom apachectl script (at "/usr/local/apache/bin/apachectl"); this is used to perform the graceful restarts, usually triggered by cpanellogd. Any such script should handle at least the 'graceful' command-line parameter; an ideal customization would still allow for the other stock-default CLI arguments to be accepted and handled appropriately.
2. Provide a custom httpd init script (at "/etc/init.d/httpd"); this is used when a full stop and start cycle is needed.

For in-depth development-related discussions I recommend the following forums area: cPanel and WHM Developer Discussions

[usercpanel]

This is with ref from post - http://forums.cpanel.net/617717-post159.html

I am stil having the problems regarding Graceful apache restarts at different intervals. Could you please shed some light on this to avoid Graceful apache restarts and sending notfications to the server admin.

Please tell me the short and sweet way to avoid this. I would really appreciate your help.



Below are my findings on my server

apache error logs
[Wed Mar 31 02:11:33 2010] [notice] Graceful restart requested, doing restart
[Wed Mar 31 04:08:29 2010] [notice] Graceful restart requested, doing restart
[Wed Mar 31 06:05:20 2010] [notice] Graceful restart requested, doing restart
[Wed Mar 31 08:02:09 2010] [notice] Graceful restart requested, doing restart
[Wed Mar 31 09:01:01 2010] [notice] Graceful restart requested, doing restart
[Wed Mar 31 10:16:03 2010] [notice] Graceful restart requested, doing restart

----------------------------------------------------------
cpanel version using
11.25.0-RELEASE_43473
----------------------------------------------------------

# ll log_rotation/cp__apache__error_log.cpanellogd
lrwxrwxrwx 1 root root 32 Jan 8 2009 log_rotation/cp__apache__error_log.cpanellogd -> /usr/local/apache/logs/error_log
root@[/var/cpanel]#
----------------------------------------------------------
domlogs of domain.com
388M domain.com.bkup
----------------------------------------------------------
# cat /var/cpanel/cpanel.config | grep 600
rotatelogs_size_threshhold_in_megabytes=600
----------------------------------------------------------
#cat /usr/local/cpanel/logs/error_log | grep cpanellogd | grep 2010-03-31
[2010-03-31 10:24:49 -0300] warn [cpanellogd] The backup file '/usr/local/apache/domlogs/domain.com.bkup' still exists, process it instead of '/usr/local/apache/domlogs/domain.com'.
----------------------------------------------------------
Apache Server Status for localhost

Server Version: Apache
Server Built: Mar 31 2010 06:11:19
_________________________________________________________________

Current Time: Wednesday, 31-Mar-2010 10:29:33 BRT
Restart Time: Wednesday, 31-Mar-2010 06:21:59 BRT
Parent Server Generation: 3
Server uptime: 4 hours 7 minutes 34 seconds
Total accesses: 340973 - Total Traffic: 12.1 GB
CPU Usage: u99.55 s9.05 cu0 cs0 - .731% CPU load
23 requests/sec - 0.8 MB/second - 37.3 kB/request
256 requests currently being processed, 0 idle workers

KRWWKKWKKKKRWKKKKKKKKKKCKKWKKKKKKKKWKKKWKKKWKRKCKKWKKKKKKKKKKWKK
KKWKGKKKKKKKKKKKKKWWWKCKKKKKKKKKKKKKKKKKKWKRKKKKKCKWCKRKKKKKKKKK
KKKKKKKWKKKWKKKKKKKWKKKWWKKKKKWKWWKKKWRKKKKKKKKKKKWKKKKKRCKWWWKC
KKKKKWKKRKKKKKKKKKKKKKWWKKRRKKRKRKKKKKKWKKKKWKKKKKCKWWKKKKKKKKCC

----------------------------------------------------------

PHP 5.2.9 (cli) (built: Mar 31 2010 06:17:17)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2009 Zend Technologies
with Zend Optimizer v3.3.9, Copyright (c) 1998-2009, by Zend Technologies

----------------------------------------------------------
Server version: Apache/2.0.63
Server built: Mar 31 2010 06:11:19
Cpanel::Easy::Apache v3.2.0 rev5053
----------------------------------------------------------
#cat /etc/httpd/conf/httpd.conf | grep Max
MaxKeepAliveRequests 100
# cat /etc/httpd/conf/httpd.conf | grep Spare
# cat /etc/httpd/conf/httpd.conf | grep Timeout
KeepAliveTimeout 15
Timeout 300
SSLSessionCacheTimeout 300
# cat /etc/httpd/conf/httpd.conf | grep RLimit
#cat /etc/httpd/conf/httpd.conf | grep Limit
----------------------------------------------------------
# php -i | grep max
log_errors_max_len => 1024 => 1024
max_execution_time => 0 => 0
max_input_nesting_level => 64 => 64
max_input_time => -1 => -1
post_max_size => 8M => 8M
upload_max_filesize => 20M => 20M
mysql.max_links => Unlimited => Unlimited
mysql.max_persistent => Unlimited => Unlimited
mysqli.max_links => Unlimited => Unlimited
session.gc_maxlifetime => 1440 => 1440
----------------------------------------------------------

[Infopro]

Have you read the first post of this thread, in particular item 3.7 in that list?

[cPanelDon]

Have you read the first post of this thread, in particular item 3.7 in that list?

As suggested by Infopro, an option available is that of the piped logging feature for Apache domain access logs; this may be enabled via WHM. For full details and reference information please see the following post: cPanel Forums - View Single Post - cPanel 11.25 Log Processing

[usercpanel]

Hello Don,

Thanks for your time. I will give it a check. I will update you once I fix this issue on the server.

[ANewDay]

Any updates on this issue? I'm afraid of upgrading to 11.25.

[cpanelkenneth]

Any updates on this issue? I'm afraid of upgrading to 11.25.

Please be more specific. Which issue concerns you?

[VeZoZ]

3.3 So this means no more Apache restarts every 15 minutes, right?

As mentioned above the Apache error_log is not handled via the pipe method, therefore if that log needs rotated an Apache restart will be required. To lengthen the time between Apache restarts for that log file one can do the following:

• Fix the problems that cause the Apache error_log to grow quickly. Often a broken PHP application will cause the Apache error log to swiftly grow in size. Determing the problem and resolving it will stem the growth of that file and thereby lengthen the time needed for the error_log to exceed the threshold.
• Increase the log rotation threshold. This is done in Tweak Settings by increasing the value of Threshold above which cpanellogd will rotate files.

Well this is disappointing because this is not true it still issues restarts. We've turned on piped logging and to test we run the /scripts/runweblogs username and it still says:

info [safeapacherestart] Restart elapsed seconds: 2


At this point we're exploring options of getting rid of cPanel's ability to even restart our web servers. Ever since the stats processing changed it's caused us nothing problems with no solution in sight. I thought piped logging would be the solution but even it's not the solution.


Also one other thing I'd like to point out that I found when exploring how to stop cPanel from issuing restarts when doing stats processing.

/usr/local/cpanel/Cpanel/HttpUtils/ApRestart.pm line:359


This function determines http pid's really needs to be re-worked. It picks up non Apache processes. It will pick up anything ran as root or nobody with httpd in the process. So I can do "pico httpd", "vim httpd" and it will think that is a web server process. Other portions of cPanel use this logic to determine things for example it will use this pid list to issue kills when it thinks the web server did not respond to a graceful restart.

[VeZoZ]

Still no explanation as to why piped logging does not stop the frequent restarts as claimed.

So I've been changing the code:

/usr/local/cpanel/Cpanel/Logd.pm


I'm guessing there is possibly some bugs with loading piped logging configuration. Just from looking at the logic it's there in parts for checking if it's enabled. Though for whatever reason I'm guessing it's not working. So commented out the restarts since we do not need them to be happening for apache.

[prettypete]

I downgraded to get rid of this problem, and now that I have upgraded again 6 months later I can't believe this graceful restart issue still exists. Unbelievable.

[cPanelDon]

Well this is disappointing because this is not true it still issues restarts. We've turned on piped logging and to test we run the /scripts/runweblogs username and it still says:

info [safeapacherestart] Restart elapsed seconds: 2

At this point we're exploring options of getting rid of cPanel's ability to even restart our web servers. Ever since the stats processing changed it's caused us nothing problems with no solution in sight. I thought piped logging would be the solution but even it's not the solution.

Also one other thing I'd like to point out that I found when exploring how to stop cPanel from issuing restarts when doing stats processing.

/usr/local/cpanel/Cpanel/HttpUtils/ApRestart.pm line:359

This function determines http pid's really needs to be re-worked. It picks up non Apache processes. It will pick up anything ran as root or nobody with httpd in the process. So I can do "pico httpd", "vim httpd" and it will think that is a web server process. Other portions of cPanel use this logic to determine things for example it will use this pid list to issue kills when it thinks the web server did not respond to a graceful restart.

Still no explanation as to why piped logging does not stop the frequent restarts as claimed.

So I've been changing the code:

/usr/local/cpanel/Cpanel/Logd.pm

I'm guessing there is possibly some bugs with loading piped logging configuration. Just from looking at the logic it's there in parts for checking if it's enabled. Though for whatever reason I'm guessing it's not working. So commented out the restarts since we do not need them to be happening for apache.

I downgraded to get rid of this problem, and now that I have upgraded again 6 months later I can't believe this graceful restart issue still exists. Unbelievable.

Please be aware that suspected bugs should be reported via our ticket system to ensure proper attention, investigation, and progress tracking; please see the link in the upper-right-corner of forums, labeled Bugs to submit a bug report.

If you are experiencing any persistent difficulty with log processing please submit a support request; this will be your best course of action to have the issue properly diagnosed and investigated thoroughly.

As indicated, you may need to consider availing of the aforementioned support resources, including using the provided links to either submit an official bug report with all necessary details or to submit a support request if your situation requires.