The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Apache's mod_status causes high IOWait.

Discussion in 'General Discussion' started by virtualreality, Jan 9, 2015.

  1. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Recently I've read Apache's mod_status causes high IOWait. I've also found Apache's mod_status, among other server software, can raise IOWait in our server.

    mod_status.c is listed live in our server, when I check the loaded Apache modules in our server with /usr/local/apache/bin/httpd -l,

    and also, status_module (static) is listed live in our server, when I check with /usr/local/apache/bin/httpd -M.

    I want to disable and totally remove Apache's mod_status in the Apache loaded modules' lists.

    Is there a cPanel server command for us to disable and remove Apache's mod_status?

    I've searched the cPanel forum for some cPanel server command to disable Apache's mod_status but haven't found one.

    Anyone knows how to totally disable Apache's mod_status on the cPanel server?

    Any help is highly expected and appreciated!!!
     
    #1 virtualreality, Jan 9, 2015
    Last edited: Jan 9, 2015
  2. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Or, what is exactly the Apache loaded modules folder in the cPanel server that I can disable and remove Apache mod_status module?

    This morning, I received 4 emails titled as "High 1 minute load average alert":

    PHP:
    Time:                    Mon Jan 12 12:04:26 2015 -0600
    1 Min Load Avg
    :          2.67
    5 Min Load Avg
    :          1.51
    15 Min Load Avg
    :         0.71
    Running
    /Total Processes:  4/96
    PHP:
    Time:                    Mon Jan 12 11:03:48 2015 -0600
    1 Min Load Avg
    :          4.47
    5 Min Load Avg
    :          1.98
    15 Min Load Avg
    :         0.81
    Running
    /Total Processes:  8/98
    PHP:
    Time:                    Mon Jan 12 10:03:48 2015 -0600
    1 Min Load Avg
    :          2.39
    5 Min Load Avg
    :          1.10
    15 Min Load Avg
    :         0.50
    Running
    /Total Processes:  3/96
    PHP:
    Time:                    Mon Jan 12 09:03:39 2015 -0600
    1 Min Load Avg
    :          2.75
    5 Min Load Avg
    :          1.17
    15 Min Load Avg
    :         0.49
    Running
    /Total Processes:  4/97
    And the vmstat.txt says:

    PHP:
    Output from vmstat:
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
      
    r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id  wa  st
      2  0     0   948280   0    54476   0    0    14    1    0    0  0  0  15  84  0
    See wa is 84. It is VERY high!

    In one way, the hourly high IOwait is related to Apache's mod_status??

    In another way, the hourly high IOwait is also raised by Chkservd, as my previous post Chkservd hangs our website almost every hour spoke about. In this thread, the Chkservd HANG problem is still not really fixed even after I shut down the Chkservd functions in the cPanel server as much as possible. --- I did send a support ticket for this problem and the cPanel technician who received my ticket did try to do some fixing work on the problem, but because I already shut down most of the Chkservd functions to reduce the current Chkservd HANG occurrences, so the ticket had to be canceled with my request. Therefore, simply I didn't know how to answer cPanelMichael's question in the thread: Would you mind sharing the resolution on this thread in-case other users experience a similar issue? Sorry Michael, the Chkservd problem still remains. And now there is another one (Apache's mod_status) that needs to be disabled.
     
  3. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    675
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    Mod_Status is a base module in Apache 2.4. To clarify, are you experiencing high disk I/O on your system as a result of mod_status or is this a preventative measure? Please review this thread for information on troubleshooting load issues:

    Troubleshooting High Loads

    Thank you.
     
  4. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Thank you Michael for helpful response!

    I use Apache 2.2. For the full compatibility of our long-lived website system and the Php version based on the website system we cannot upgrade the Apache version to 2.4 now.

    I've checked disk I/O and in most hours the disk I/O looks normal and not high except the First 4-5 Minutes of almost every hour yesterday and this morning today. In one way the high IOwait is raised by Chkservd and/or another cPanel server monitoring system. After I shut down most of the Chkservd functions, I've found the Chkservd Hang occurrences have reduced, and the hourly high IOwait occurrences also have reduced, from almost every hour to about every 4~5 hours. However, yesterday and today, the high IOwait occurrences have resumed.
     
    #4 virtualreality, Jan 12, 2015
    Last edited: Jan 12, 2015
  5. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    675
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Do you notice actual reduced performance or are you simply worried about the disk I/O usage numbers? The following thread offers some advice on how to obtain additional information about the usage:

    Troubleshooting High Loads

    Thank you.
     
  6. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Thank you Michael for help!

    High IOwait and high CPU load can be caused by several things. I'm still looking for some effective ways trying to solve the frequent high server IOwait and CPU load, and still looking for a way to disable Apache mod_status hoping to help reduce the IOwait and CPU load.
     
  7. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    675
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    You could disable the following entry in /usr/local/apache/conf/httpd.conf:

    Code:
    <Location /whm-server-status>
        SetHandler server-status
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1 ::1
    </Location>
    Information on modifying the Apache configuration file

    However, I'm not sure that's the real cause of your disk I/O.

    Thank you.
     
  8. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Thank you Michael for help!

    I've alreaday done with your method in the main Apache Config file httpd as the following:

    #ExtendedStatus on
    #<Location /whm-server-status>
    #SetHandler server-status
    #Order deny,allow
    #Deny from all
    #Allow from 127.0.0.1 ::1
    #</Location>

    However, the IOwait is still high frequently. The Apache status_module still functions at some extent? I've also checked CSF/lfd. All those IOwait-related CSF/lfd settings (such as System Integrity Checking, processeses sanity check, etc.) were disabled earlier.

    I checked /etc/my.cnf and increased tmp_table_size to 256M and max_heap_table_size to 256M, hoping to reduce the disk's in and out and the disk's load. I also largely increased key_buffer_size hoping to help lower some IOwait.

    In the Apache server, I reduced ServerLimit and MaxClients, lowered StartServers, MinSpareServers, MaxSpareServers, KeepAliveTimeout and MaxKeepAliveRequests. The MaxRequestsPerChild is 2000 and it's a reasonable value.

    Although with all the above done already, the high server IOwait and high CPU load occur at the beginning of almost every hour or every two hours.

    Something is confusing.

    The high IOwait and CPU load should be caused by several factors, and one of the factors is queueprocd?queueprocd can also raise the high IOwait and CPU load?

    Is queueprocd frequently increasing Writes to disk?? I'll keep watching it for some time.
     
  9. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    queueprocd is frequently increasing Writes to the disk. Now this is a major cause of frequent high IOwait on the cPanel server.

    Just now, I've received the "High 1 minute load average alert" again:

    PHP:
    Time:                    Thu Jan 15 08:02:26 2015 -0600
    1 Min Load Avg
    :          3.83
    5 Min Load Avg
    :          1.42
    15 Min Load Avg
    :         0.62
    Running
    /Total Processes1/72
    And the vmstat:

    PHP:
    Output from vmstat:
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
      
    r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
      1  0      0 930668      0  59352    0    0    13     1    0    0  1  0 15 84  0

    cPanel Team, how to fix this?
     
  10. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    675
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    In "WHM Home » Server Configuration » Tweak Settings", under the "Stats and Logs" tab, I presume the following option is already enabled:

    "Use delayed sql inserts in the eximstats process to reduce the number of disk writes"

    Also, in "Tweak Settings", under the "System" tab, see if the following option is enabled:

    "Conserve memory - Conserve memory at the expense of more CPU usage and disk I/O."

    Beyond that, feel free to open a support ticket using the link in my signature so we can take a closer look. It's possible the issue is with your hardware. You can post the ticket number here so we can update this thread with the outcome.

    Thank you.
     
  11. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Thank you Michael for help!

    I've sent a support ticket using the link in your signature. The support ticket number is 5973315.

    In WHM's Tweak Settings, I select "Use delayed sql inserts in the eximstats process to reduce the number of disk writes." And I never select "Conserve memory". I do consider CPU usage and disk I/O are more important than memory. I feel in a VPS environment, what often makes trouble is high disk I/O and CPU usage.

    Our web hosting company uses SSD in our VPS hosting. The hardware is fine, stable and fast, because previously we did not encounter this high IOwait problem in our cPanel server for several years. We've continuously used the same web hosting company hosting for about 5 years until now. Their main node and network are fast, stable and excellent. I must say this after using their hosting service for about 5 years continually. The high IOwait problem does not come from the web hosting company.

    I feel since we upgraded our WHM/cPanel to WHM 11.44.1 (build 18) (stable version), there appears the high IOwait problem in our cPanel server.

    Now continuously, some new "High 1 minute load average alert" until now today:

    PHP:
    Time:                    Thu Jan 15 12:04:31 2015 -0600
    1 Min Load Avg
    :          2.48
    5 Min Load Avg
    :          1.28
    15 Min Load Avg
    :         0.60
    Running
    /Total Processes1/73
    PHP:
    Time:                    Thu Jan 15 11:04:28 2015 -0600
    1 Min Load Avg
    :          2.42
    5 Min Load Avg
    :          1.50
    15 Min Load Avg
    :         0.75
    Running
    /Total Processes1/70
    PHP:
    Time:                    Thu Jan 15 10:04:27 2015 -0600
    1 Min Load Avg
    :          3.38
    5 Min Load Avg
    :          1.74
    15 Min Load Avg
    :         0.83
    Running
    /Total Processes1/72
    PHP:
    Time:                    Thu Jan 15 09:03:54 2015 -0600
    1 Min Load Avg
    :          2.05
    5 Min Load Avg
    :          0.91
    15 Min Load Avg
    :         0.44
    Running
    /Total Processes2/69
    And the :
    PHP:
    Output from vmstat:
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
      
    r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
      0  0      0 840828      0 138636    0    0    13     1    0    0  1  0 15 84  0
    Sincerely expect some better customizations and update need to be done for queueprocd and/or some other cPanel system components.
     
  12. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    And during the high IOwait, our WHM, cPanel and website are all stuck. It is really frustrated.
     
  13. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Some new "High 1 minute load average alert" until now today:

    PHP:
    Time:                    Thu Jan 15 15:05:37 2015 -0600
    1 Min Load Avg
    :          4.84
    5 Min Load Avg
    :          2.68
    15 Min Load Avg
    :         1.24
    Running
    /Total Processes1/75
    PHP:
    Time:                    Thu Jan 15 14:05:14 2015 -0600
    1 Min Load Avg
    :          4.75
    5 Min Load Avg
    :          2.67
    15 Min Load Avg
    :         1.23
    Running
    /Total Processes5/80
    PHP:
    Time:                    Thu Jan 15 13:04:46 2015 -0600
    1 Min Load Avg
    :          2.61
    5 Min Load Avg
    :          1.39
    15 Min Load Avg
    :         0.69
    Running
    /Total Processes1/72
    The vmstat:
    PHP:
    Output from vmstat:
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
      
    r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
      2  1      0 863248      0 107904    0    0    13     1    0    0  1  0 15 84  0
     
  14. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    Hi cPanel Friends,

    Now today, things are clear:

    The frequent high IOwait problem DOES NOT come from the cPanel systems.

    The frequent high IOwait problem has NOTHING to do with the cPanel systems.


    I am really sorry for suspecting something in the cPanel caused the frequent high IOwait, at first. I apologize for my suspicion of some cPanel component causing the frequent high IOwait. The cPanel systems are excellent.

    I really don't know what caused the frequent high IOwait problem, but since the VPS was migrated to a new host node last night, I've checked until now there are no more IOwait problems. Now, the VPS performance is smooth, robust, and stable.
     
  15. virtualreality

    virtualreality Well-Known Member

    Joined:
    Jan 18, 2008
    Messages:
    76
    Likes Received:
    0
    Trophy Points:
    6
    And, the cPanel support team members have given me a lot of professional support.

    Thank you cPanel from my heart for your excellent support!
     
  16. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    675
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    I am happy to see the issue has been addressed. Thank you for updating us with the outcome.
     

Share This Page