The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Why is apache restarting every two hours?

Discussion in 'EasyApache' started by aww, Jan 23, 2012.

  1. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    I remember the problems people had awhile back under 11.25 with log rotation causing restarts
    but this is 11.30

    Almost every two hours (but not exactly) apache will restart.
    It's not a segfault error, at least not any listed in the logs.

    WHM 11.30.5 (build 6)
    CENTOS 6.2 x86_64 xenpv
    (mod_php+apc+suhosin)

    I'd appreciate any advice in tracking this down - it's annoying because it's clearing the opcode cache too often

    Code:
    [Fri Jan 20 18:09:21 2012] [notice] Graceful restart requested, doing restart
    [Fri Jan 20 20:06:09 2012] [notice] Graceful restart requested, doing restart
    [Fri Jan 20 22:02:58 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 00:18:01 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 02:07:16 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 04:14:56 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 06:11:50 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 08:08:37 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 10:05:27 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 12:02:16 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 14:15:45 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 16:12:34 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 18:09:22 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 20:06:10 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 21 22:02:58 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 00:23:59 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 02:05:22 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 04:14:51 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 06:11:41 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 08:08:28 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 10:05:15 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 12:02:03 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 14:15:31 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 16:12:19 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 18:09:08 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 20:05:55 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 22 22:02:44 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 00:16:14 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 02:06:55 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 04:01:13 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 06:14:44 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 08:11:31 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 10:08:20 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 23 12:05:10 2012] [notice] Graceful restart requested, doing restart
    
     
    #1 aww, Jan 23, 2012
    Last edited: Jan 23, 2012
  2. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Looking back further in the logs it looks like this started happening January 17th for what it's worth.

    Before then restarts were "only" three time a day at 0,2 and 22 hours

    So the question is did I change a setting or did cpanel change something.

    Code:
    [Tue Jan 10 00:10:37 2012] [notice] Graceful restart requested, doing restart
    [Tue Jan 10 02:07:25 2012] [notice] Graceful restart requested, doing restart
    [Tue Jan 10 22:12:35 2012] [notice] Graceful restart requested, doing restart
    [Wed Jan 11 00:04:53 2012] [notice] Graceful restart requested, doing restart
    [Wed Jan 11 02:01:40 2012] [notice] Graceful restart requested, doing restart
    [Wed Jan 11 22:01:55 2012] [notice] Graceful restart requested, doing restart
    [Thu Jan 12 00:16:05 2012] [notice] Graceful restart requested, doing restart
    [Thu Jan 12 02:12:52 2012] [notice] Graceful restart requested, doing restart
    [Thu Jan 12 22:13:03 2012] [notice] Graceful restart requested, doing restart
    [Fri Jan 13 00:26:18 2012] [notice] Graceful restart requested, doing restart
    [Fri Jan 13 02:06:25 2012] [notice] Graceful restart requested, doing restart
    [Fri Jan 13 22:06:37 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 14 00:11:41 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 14 02:03:51 2012] [notice] Graceful restart requested, doing restart
    [Sat Jan 14 22:04:02 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 15 00:13:10 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 15 02:02:53 2012] [notice] Graceful restart requested, doing restart
    [Sun Jan 15 22:03:07 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 16 00:30:20 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 16 02:02:53 2012] [notice] Graceful restart requested, doing restart
    [Mon Jan 16 22:03:04 2012] [notice] Graceful restart requested, doing restart
    
     
  3. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    I've been searching for possible causes

    The log rotation is at the default of 300MB

    While the error_log is fine, I noticed there are at least three domlogs which are way, way over 300MB

    One domlog is 918801950 bytes for example

    Is it possible these are causing periodic apache restart?
    Why are logs failing to rotate?

    researching more...


    oh! Piped Apache Logs is disabled by default?
    any reason why I *wouldn't* want to turn that on ?
     
    #3 aww, Jan 23, 2012
    Last edited: Jan 23, 2012
  4. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    I cannot think of a reason why you wouldn't want to enable piped logging. I'd actually suggest enabling that logging option in WHM > Apache Configuration > Piped Log Configuration area to see if the graceful restarts every 2 hours cease to occur.
     
  5. alphawolf50

    alphawolf50 Well-Known Member

    Joined:
    Apr 28, 2011
    Messages:
    186
    Likes Received:
    2
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    What executable are the logs being piped to? rotatelogs? Is the processing being spawned without a shell? The closest thing to an answer I found was:
    "A common target process" seems like a rather generic answer :)
     
  6. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    If you enable piped logging, here are the results on a machine:

    Basically, enabling piped logging will add entries for splitlogs into /usr/local/apache/conf/httpd.conf file and, if you check the processes, it shows what binary is being used.
     
  7. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Ah twelve hours uptime without restart - finally fixed.

    I thought after 11.25 it was the default, I guess not.

    Everyone with an opcode cache (which should be everyone) should have piped logging enabled.

    The crazy oddity is why was it only 2-3 times a day before the 17th and every 2 hours afterwards.

    Anyway, the error log is realtime and that's all that matters to me, so I don't care there's a helper splitting it later.
     
  8. Brian

    Brian Well-Known Member

    Joined:
    Dec 1, 2010
    Messages:
    117
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Texas
    cPanel Access Level:
    Root Administrator
    By default, Apache will restart every 2 hours as well as every 24 hours. This is normal and intended.

    Why at those intervals?

    Those are the default log/statistics processing intervals for a cPanel & WHM server.

    WHM -> Statistics Software Configuration
    Log Processing Frequency [ 24 ] hours
    Bandwidth Processing Frequency [ 2 ] hours

    If your Apache is not restarting at those two configured intervals (whatever they may be) then this means either...

    [1] You have Piped Logging enabled
    - or -
    [2] Something is broken on your server that needs investigation.


    The reason the restarts occur is that Apache must be restarted to grab the new logs' file handles (since the logs being processed are rotated out during processing). If Apache were not restarted, bandwidth/stats would fail to log and that data be lost.

    Piped Logging, however, aims to do two things.
    [1] Reduce the number of open file handles on a server
    -and-
    [2] Eliminate the need for Apache to be restarted during stats/bandwidth logging processing.

    In prior releases, Piped Logging was considered experimental. It no longer carries that label and is certainly beneficial. It's certainly plausible that this will become the default in the future, although I can't speak to when/if that will occur.

    Note that Apache will restart in one other situation as well, when a cPanel user performs a change that involves modifying httpd.conf (adding/removing domains, for example). When that occurs, Apache must be restarted.

    Any other restarts would be manually requested restarts or something like chkservd restarting the service if it believes it to be offline/down.
     
  9. aww

    aww Well-Known Member

    Joined:
    Feb 10, 2005
    Messages:
    152
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Ah and that explains why it changed on the 17th, I vaguely remember tinkering with the stats processing intervals.

    Pleased with piped so far, zero problems.
     
  10. alphawolf50

    alphawolf50 Well-Known Member

    Joined:
    Apr 28, 2011
    Messages:
    186
    Likes Received:
    2
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    People using suPHP can't use an opcode cache :)
     
Loading...

Share This Page