The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

High CPU load and lots of '/usr/sbin/httpd -k start processes'

Discussion in 'General Discussion' started by FabianNL, Dec 5, 2016.

  1. FabianNL

    FabianNL Registered

    Joined:
    Nov 29, 2016
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Arnhem, The Netherlands
    cPanel Access Level:
    Root Administrator
    Almost everyday my server 'hangs' for about 2 hours, because there are 20+ processes '/usr/sbin/httpd -k start', owned by nobody. If I kill these processes, everything returns to normal. I checked all the logs and also the Apache connections to see if a specific site is causing higher than normal traffic, but that's not the case.
    I've searched Google and these forums, but I can't find anyone with the exact same problems.

    This is my configuration:
    Code:
    /etc/redhat-release:CentOS release 6.8 (Final)
    /usr/local/cpanel/version:11.60.0.26
    /var/cpanel/envtype:kvm
    CPANEL=release
    Server version: Apache/2.4.23 (cPanel)
    Server built:   Nov  8 2016 16:57:01
    ea-php-cli Copyright 2016 cPanel, Inc.
    PHP 5.6.28 (cli) (built: Nov 14 2016 15:20:37)
    Copyright (c) 1997-2016 The PHP Group
    Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
        with the ionCube PHP Loader v4.7.5, Copyright (c) 2002-2014, by ionCube Ltd., and
        with Zend Guard Loader v3.3, Copyright (c) 1998-2014, by Zend Technologies
        with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2015, by Zend Technologies
    mysql  Ver 15.1 Distrib 10.0.28-MariaDB, for Linux (x86_64) using readline 5.1
    
    Does this problem sound familiair to someone?
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,287
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    It's normal to see "/usr/sbin/httpd -k start" processes owned by the "nobody" user. That's the standard username Apache runs as. The following thread is a good place to start if you want to determine what's causing the high load average:

    Troubleshooting high server loads on Linux servers

    Thank you.
     
  3. FabianNL

    FabianNL Registered

    Joined:
    Nov 29, 2016
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Arnhem, The Netherlands
    cPanel Access Level:
    Root Administrator
    Hi Michael,

    Thanks for your answer. I already found that thread before posting this question. I've tried the solutions mentioned there, but with no luck. Can I somehow see what these processes are doing? It now occurs multiple times each day and everytime I have to manually stop multiple processes (30+) in order to make my server responsive again. Killing those processes doesn't seem to have any negative impact on the running sites.
    It looks as if these httpd processes are running something outside of the normal websites. Is there someway I can check this?
     
  4. popeye

    popeye Well-Known Member

    Joined:
    May 23, 2013
    Messages:
    330
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Texas
    cPanel Access Level:
    Root Administrator
    What sites do you host is it Wordpress
     
  5. FabianNL

    FabianNL Registered

    Joined:
    Nov 29, 2016
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Arnhem, The Netherlands
    cPanel Access Level:
    Root Administrator
    Different platforms, WordPress (about 75%), Magento and some CakePHP sites.
     
  6. popeye

    popeye Well-Known Member

    Joined:
    May 23, 2013
    Messages:
    330
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Texas
    cPanel Access Level:
    Root Administrator
    Are you sure it's not xmlrpc attacks ?
     
  7. FabianNL

    FabianNL Registered

    Joined:
    Nov 29, 2016
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Arnhem, The Netherlands
    cPanel Access Level:
    Root Administrator
    I think I have found the cause of the problems. I've disabled the ModSecurity OWASP package about 36 hours ago and since then I've seen no high loads anymore. If this is still the case tomorrow, I will enable them again to see if the problem returns. If so, what can be wrong with this?
     
  8. popeye

    popeye Well-Known Member

    Joined:
    May 23, 2013
    Messages:
    330
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Texas
    cPanel Access Level:
    Root Administrator
    I would not disable ModSecurity
     
  9. inveress

    inveress Registered

    Joined:
    Apr 8, 2014
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Hey all,

    Just wondering if there was any cause established with this as I'm having the same issue.
    Very few visitors on the server, multiple "/usr/sbin/httpd -k start" tasks showing under Process Manager, high load on server (10-15 on a 6 CPU server).
    If I restart Apache, it's fine.
    Seems to happen most/every morning, not exactly at the same time.

    Curiously (per FabianNL's comment), I also recently added the OWASP vendor ruleset to ModSecurity (during a recent cPanel/WHM update - 64.0.17, I believe).

    I've tried disabling the automatic updates for the ruleset and this hasn't helped, so next step might be disabling the ruleset altogether, but of course I'd rather not...

    Any further info?

    Thanks!
    Peter.
     
  10. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,287
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    You may want to try some of the investigation tips referenced on the following thread to help determine why the server load is high:

    Troubleshooting high server loads on Linux servers

    There's a script you can setup that will run on a cron job and allow you to see results from when the server load is high.

    Thank you.
     
  11. zuronam

    zuronam Member

    Joined:
    Aug 2, 2016
    Messages:
    9
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    Zimbabwe
    cPanel Access Level:
    Root Administrator
    Hi CpanelMichael

    I've just experienced this - after an update to 64.0.24, with OWASP Ruleset 3, my server because completely unstable. I was seeing load averages over 15mintes as high and 6.8 and over 1 minute as high as 17 at times!... and on a 6 core VPS - with over 400 customers - meant lots of disgruntled users.

    Upon reading this forum (and after trying virtually everything, cPanel support, CloudLinux Support) I've disabled OWASP core ruleset, and v3.0 on my server - lo and behold my server load is back around 0.48 0.67 1.95

    As much as I dislike disabling Mod Security - high load and unhappy customers are worse to deal with, esp when you have a server loading with 6-7 nobody processes using 100% cpu time - all the while websites are unreachable, no Webmail access for customers - basically a server admin's worst nightmare - stretched over a number of days

    Any ideas on what the next step is now from here - as I've identified the ruleset to be the cause of load on the server
     
  12. popeye

    popeye Well-Known Member

    Joined:
    May 23, 2013
    Messages:
    330
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Texas
    cPanel Access Level:
    Root Administrator
    We stopped using cpanel rules a long time ago, your much better off with Comodo rules.
     
  13. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    37,064
    Likes Received:
    1,287
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Could you verify the size of the /var/cpanel/secdatadir/ip.pag file on the system? EX:

    Code:
    du -sh /var/cpanel/secdatadir/ip.pag
    Thanks.
     
  14. zuronam

    zuronam Member

    Joined:
    Aug 2, 2016
    Messages:
    9
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    Zimbabwe
    cPanel Access Level:
    Root Administrator
    Hi cPpanelMichael,

    I had checked the size of the file as I saw when I would check the apache status

    httpd status

    I'd get a lot of L's for logging


    WLLLL..LLLLLL..LLLLLL - something like that

    When I checked the file size, it was 26mb if memory serves. I even cleared it out and tailed the file as it grew to around 400kb, then quit as the load situation was not improving.

    I took the suggestion offered by popeye, and after disabling a few rules that did not apply - my server has been performing as expected


    upload_2017-6-22_15-46-18.png

    mind the low uptime is due to the recent kernel updates

    Regards
     
    cPanelMichael likes this.
Loading...

Share This Page