The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Load High, 100% CPU httpd Threads, clock_gettime?

Discussion in 'Workarounds and Optimization' started by jvanv8, Mar 4, 2011.

  1. jvanv8

    jvanv8 Member

    Joined:
    Nov 10, 2006
    Messages:
    22
    Likes Received:
    0
    Trophy Points:
    1
    I have about 3-4 httpd threads that when I use strace it appears that they are polling and nothing more. A sample looks like this:
    HTML:
    poll([{fd=91, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {3140, 207436246}) = 0
    
    thousands and thousands of times...

    top (after a 51 minutes up):
    HTML:
    10297 nobody    25   0  185m  14m 4248 R 100.0  0.2  35:33.18 httpd                                                                                                                                                                                                
    11347 nobody    25   0  184m  13m 4780 R 100.0  0.2  32:30.62 httpd                                                                                                                                                                                                
     9924 nobody    25   0  185m  15m 4884 R 99.6  0.2  35:49.04 httpd 
    Any ideas?
    Thanks!
     
    #1 jvanv8, Mar 4, 2011
    Last edited: Mar 4, 2011
  2. syslint

    syslint Well-Known Member

    Joined:
    Oct 9, 2006
    Messages:
    249
    Likes Received:
    6
    Trophy Points:
    18
    Location:
    India
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hope you are in preforker MPM . Can you switch to worker MPM and check it
     
  3. jvanv8

    jvanv8 Member

    Joined:
    Nov 10, 2006
    Messages:
    22
    Likes Received:
    0
    Trophy Points:
    1
    The module "prefork" is loaded but I'm a bit of a newbie at this and am not really familiar with how to change to "worker" or what the implications of the change are.
    Here's another sample from top:
    HTML:
    top - 16:42:13 up  3:15,  1 user,  load average: 40.41, 41.19, 39.28
    Tasks: 366 total,  38 running, 321 sleeping,   6 stopped,   1 zombie
    Cpu(s): 18.7%us, 81.2%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.1%si,  0.0%st
    Mem:   8174168k total,  3693952k used,  4480216k free,   202440k buffers
    Swap:  2104504k total,        0k used,  2104504k free,  2506036k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                              
    20194 nobody    15   0  280m 110m 6064 R 51.1  1.4   5:04.97 httpd                                                                                                                                                                                                 
    23152 nobody    25   0  185m  15m 4264 R 26.9  0.2  60:30.39 httpd                                                                                                                                                                                                 
    15156 nobody    25   0  186m  16m 4932 R 26.6  0.2   6:10.44 httpd                                                                                                                                                                                                 
     7845 nobody    25   0  184m  14m 4824 R 26.2  0.2  12:05.19 httpd                                                                                                                                                                                                 
    27697 nobody    25   0  184m  13m 4592 R 25.6  0.2  38:52.67 httpd                                                                                                                                                                                                 
    17269 nobody    25   0  187m  17m 4776 R 24.9  0.2   4:31.01 httpd                                                                                                                                                                                                 
    27560 nobody    25   0  184m  13m 4780 R 24.6  0.2  38:00.86 httpd                                                                                                                                                                                                 
     3312 nobody    25   0  182m  10m 4100 R 24.2  0.1  18:02.83 httpd                                                                                                                                                                                                 
    14440 nobody    25   0  182m  10m 4072 R 23.9  0.1   7:09.11 httpd                                                                                                                                                                                                 
    16708 nobody    25   0  183m  13m 4764 R 23.6  0.2   5:29.71 httpd                                                                                                                                                                                                 
    30853 nobody    25   0  187m  16m 4760 R 23.6  0.2  30:21.56 httpd                                                                                                                                                                                                 
    11045 nobody    25   0  182m  11m 4836 R 23.2  0.1   8:15.32 httpd                                                                                                                                                                                                 
    15288 nobody    25   0  185m  15m 4824 R 23.2  0.2   6:15.49 httpd                                                                                                                                                                                                 
    18056 nobody    25   0  187m  16m 4904 R 23.2  0.2   3:31.00 httpd                                                                                                                                                                                                 
    22790 nobody    25   0  182m  10m 4012 R 23.2  0.1  62:16.74 httpd                                                                                                                                                                                                 
     1634 nobody    25   0  182m  10m 4044 R 22.9  0.1  22:22.69 httpd                                                                                                                                                                                                 
    14460 nobody    25   0  182m  10m 3924 S 22.9  0.1   7:19.62 httpd                                                                                                                                                                                                 
    21521 nobody    25   0  184m  13m 4896 R 21.6  0.2  61:31.62 httpd                                                                                                                                                                                                 
    28403 nobody    25   0  184m  13m 4768 R 20.6  0.2  35:56.49 httpd                                                                                                                                                                                                 
     8092 nobody    25   0  182m  10m 4092 R 20.3  0.1  12:20.71 httpd                                                                                                                                                                                                 
    17511 nobody    25   0  186m  16m 4836 R 20.3  0.2   4:11.25 httpd                                                                                                                                                                                                 
    21051 nobody    25   0  186m  16m 4784 R 20.3  0.2  66:57.50 httpd                                                                                                                                                                                                 
    28520 nobody    25   0  184m  13m 4760 R 20.3  0.2  35:35.02 httpd                                                                                                                                                                                                 
     1997 nobody    25   0  184m  13m 4528 R 19.9  0.2  20:59.58 httpd                                                                                                                                                                                                 
     8266 nobody    25   0  182m  11m 4408 R 19.9  0.1  12:07.31 httpd                                                                                                                                                                                                 
    11429 nobody    25   0  185m  15m 4612 R 19.9  0.2   8:35.06 httpd                                                                                                                                                                                                 
    12654 nobody    25   0  184m  13m 4720 R 19.9  0.2   8:03.82 httpd                                                                                                                                                                                                 
    27974 nobody    25   0  182m  11m 4428 S 19.9  0.1  37:52.26 httpd                                                                                                                                                                                                 
    31684 nobody    25   0  182m  11m 4248 R 19.9  0.1  27:36.75 httpd                                                                                                                                                                                                 
    31749 nobody    25   0  182m  10m 4000 R 19.9  0.1  28:01.86 httpd                                                                                                                                                                                                 
     3378 nobody    25   0  185m  15m 4832 R 19.6  0.2  17:41.87 httpd                                                                                                                                                                                                 
    22065 mysql     17   0  290m  51m 4116 S 18.9  0.6   0:00.80 mysqld                                                                                                                                                                                                
    19951 nobody    15   0  307m 135m 6244 R 15.6  1.7   3:02.78 httpd                                                                                                                                                                                                 
    21908 nobody    15   0  182m  12m 4780 S  9.6  0.2   0:02.25 httpd                                                                                                                                                                                                 
     8455 nobody    25   0  182m  10m 4064 R  5.0  0.1  11:41.38 httpd                                                                                                                                                                                                 
    22010 nobody    15   0  185m  15m 4556 S  4.3  0.2   0:00.21 httpd                                                                                                                                                                                                 
    27443 nobody    25   0  184m  13m 4884 R  3.7  0.2  36:55.42 httpd                                                                                                                                                                                                 
    22068 mysql     16   0  290m  51m 4116 S  3.3  0.6   0:00.10 mysqld                                                                                                                                                                                                
    27958 nobody    25   0  182m  11m 4768 R  3.3  0.1  37:44.92 httpd                                                                                                                                                                                                 
    21777 nobody    15   0  184m  14m 4832 S  3.0  0.2   0:00.67 httpd                                                                                                                                                                                                 
    21927 nobody    15   0  185m  14m 3844 S  3.0  0.2   0:00.26 httpd                                                                                                                                                                                                 
    22018 mysql     16   0  290m  51m 4116 S  2.3  0.6   0:00.19 mysqld                                                                                                                                                                                                
    21941 nobody    15   0  185m  15m 4824 S  1.7  0.2   0:00.84 httpd                                                                                                                                                                                                 
    21789 nobody    15   0  202m  31m 4884 S  1.3  0.4   0:03.49 httpd                                                                                                                                                                                                 
    21817 nobody    15   0  184m  14m 4832 S  1.3  0.2   0:00.77 httpd                                                                                                                                                                                                 
    21916 nobody    15   0     0    0    0 Z  0.3  0.0   0:00.41 httpd <defunct> 
    .... [truncated] ...
     
     
    #3 jvanv8, Mar 4, 2011
    Last edited: Mar 4, 2011
  4. jvanv8

    jvanv8 Member

    Joined:
    Nov 10, 2006
    Messages:
    22
    Likes Received:
    0
    Trophy Points:
    1
    any ideas? Server load has been 80-100 for 3 days.
     
  5. hackboys

    hackboys Active Member

    Joined:
    Feb 12, 2008
    Messages:
    31
    Likes Received:
    1
    Trophy Points:
    8
    Some problem!
    100% CPU httpd Threads, clock_gettime
    Please help how to fix?


    clock_gettime(CLOCK_MONOTONIC, {4557, 12307472}) = 0
    poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {4557, 12496472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 12553472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 12609472}) = 0
    select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
    clock_gettime(CLOCK_MONOTONIC, {4557, 12787472}) = 0
    poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {4557, 12899472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 12954472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13015472}) = 0
    select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
    clock_gettime(CLOCK_MONOTONIC, {4557, 13095472}) = 0
    poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {4557, 13159472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13190472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13218472}) = 0
    select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
    clock_gettime(CLOCK_MONOTONIC, {4557, 13290472}) = 0
    poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {4557, 13353472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13384472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13416472}) = 0
    select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
    clock_gettime(CLOCK_MONOTONIC, {4557, 13490472}) = 0
    poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {4557, 13554472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13584472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13613472}) = 0
    select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
    clock_gettime(CLOCK_MONOTONIC, {4557, 13685472}) = 0
    poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    clock_gettime(CLOCK_MONOTONIC, {4557, 13748472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13779472}) = 0
    clock_gettime(CLOCK_MONOTONIC, {4557, 13808472}) = 0
    select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})




    HTML:
    Tasks: 784 total,   6 running, 778 sleeping,   0 stopped,   0 zombie
    Cpu(s): 19.9%us, 31.6%sy,  0.0%ni, 47.5%id,  0.4%wa,  0.0%hi,  0.6%si,  0.0%st
    Mem:  37038568k total, 20630780k used, 16407788k free,   343540k buffers
    Swap: 39792996k total,        0k used, 39792996k free, 14803328k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     6877 mysql      5 -10  665m 308m 3616 S 180.2  0.9  69:24.14 mysqld
    14673 nobody    25   0  321m  21m 6300 R 99.0  0.1   3:09.48 httpd
    18913 nobody    25   0  319m  21m 5856 R 97.7  0.1   2:45.95 httpd
    19182 nobody    25   0  319m  21m 6040 R 97.4  0.1   2:51.96 httpd
    18590 nobody    25   0  324m  23m 6024 S 96.1  0.1   0:50.08 httpd
     8202 nobody    25   0  325m  28m 6544 R 92.4  0.1   3:28.31 httpd
    13678 nobody    15   0  337m  38m 6684 S 14.6  0.1   0:08.55 httpd
    13072 nobody    15   0  322m  24m 6180 S 13.2  0.1   0:06.67 httpd
    19212 nobody    16   0  318m  19m 5616 S  7.9  0.1   0:00.95 httpd
    16387 nobody    15   0  327m  26m 6172 S  5.6  0.1   0:05.41 httpd
    19606 nobody    15   0  321m  23m 5912 S  5.3  0.1   0:01.05 httpd
    18329 nobody    15   0  319m  21m 6076 S  4.6  0.1   0:02.12 httpd
    14076 nobody    15   0  324m  24m 6308 S  4.3  0.1   0:07.87 httpd
    19899 nobody    15   0  319m  19m 5352 S  4.3  0.1   0:00.29 httpd
    18939 nobody    15   0  322m  24m 5924 S  3.6  0.1   0:02.00 httpd
    19635 nobody    15   0  321m  22m 5452 S  3.6  0.1   0:00.88 httpd
    19131 nobody    15   0  325m  26m 6080 S  3.3  0.1   0:02.14 httpd
    19588 nobody    15   0  318m  19m 5728 S  3.3  0.1   0:00.69 httpd
     
    #5 hackboys, Apr 19, 2011
    Last edited: Apr 19, 2011
  6. ramindia

    ramindia Well-Known Member

    Joined:
    Apr 3, 2011
    Messages:
    71
    Likes Received:
    0
    Trophy Points:
    6
    Hi Jvanv8

    try Timeout 30 and restart apache see any change, if not install Cpanel Firewall

    i had same problem i did the same i see some change, but iam still tuning myself for better performance.
     
  7. nobodyk

    nobodyk Well-Known Member

    Joined:
    Aug 1, 2010
    Messages:
    90
    Likes Received:
    0
    Trophy Points:
    6
    Have you checked it's not a ddos attack?

    can you post your apache conf? (just the cpanel variables)
     
  8. cPanelTristan

    cPanelTristan Quality Assurance Analyst
    Staff Member

    Joined:
    Oct 2, 2010
    Messages:
    7,623
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    somewhere over the rainbow
    cPanel Access Level:
    Root Administrator
    I have a post that I did recently to troubleshoot a possible DoS attack:

    http://forums.cpanel.net/f5/how-restart-apache-trough-cpanel-command-line-187252.html#post766101

    I'd suggest trying the commands noted to see if this might be one. Since you have a very high number of Apache processes, it might well be an attack.

    Alternatively, you might be hitting MaxClients on the machine. If you are, you can use the information at the end of that post to increase the MaxClients and ServerLimit values.
     
Loading...

Share This Page