Load High, 100% CPU httpd Threads, clock_gettime?

jvanv8

Member
Nov 10, 2006
22
0
151
I have about 3-4 httpd threads that when I use strace it appears that they are polling and nothing more. A sample looks like this:
HTML:
poll([{fd=91, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {3140, 207436246}) = 0
thousands and thousands of times...

top (after a 51 minutes up):
HTML:
10297 nobody    25   0  185m  14m 4248 R 100.0  0.2  35:33.18 httpd                                                                                                                                                                                                
11347 nobody    25   0  184m  13m 4780 R 100.0  0.2  32:30.62 httpd                                                                                                                                                                                                
 9924 nobody    25   0  185m  15m 4884 R 99.6  0.2  35:49.04 httpd
Any ideas?
Thanks!
 
Last edited:

jvanv8

Member
Nov 10, 2006
22
0
151
The module "prefork" is loaded but I'm a bit of a newbie at this and am not really familiar with how to change to "worker" or what the implications of the change are.
Here's another sample from top:
HTML:
top - 16:42:13 up  3:15,  1 user,  load average: 40.41, 41.19, 39.28
Tasks: 366 total,  38 running, 321 sleeping,   6 stopped,   1 zombie
Cpu(s): 18.7%us, 81.2%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:   8174168k total,  3693952k used,  4480216k free,   202440k buffers
Swap:  2104504k total,        0k used,  2104504k free,  2506036k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                              
20194 nobody    15   0  280m 110m 6064 R 51.1  1.4   5:04.97 httpd                                                                                                                                                                                                 
23152 nobody    25   0  185m  15m 4264 R 26.9  0.2  60:30.39 httpd                                                                                                                                                                                                 
15156 nobody    25   0  186m  16m 4932 R 26.6  0.2   6:10.44 httpd                                                                                                                                                                                                 
 7845 nobody    25   0  184m  14m 4824 R 26.2  0.2  12:05.19 httpd                                                                                                                                                                                                 
27697 nobody    25   0  184m  13m 4592 R 25.6  0.2  38:52.67 httpd                                                                                                                                                                                                 
17269 nobody    25   0  187m  17m 4776 R 24.9  0.2   4:31.01 httpd                                                                                                                                                                                                 
27560 nobody    25   0  184m  13m 4780 R 24.6  0.2  38:00.86 httpd                                                                                                                                                                                                 
 3312 nobody    25   0  182m  10m 4100 R 24.2  0.1  18:02.83 httpd                                                                                                                                                                                                 
14440 nobody    25   0  182m  10m 4072 R 23.9  0.1   7:09.11 httpd                                                                                                                                                                                                 
16708 nobody    25   0  183m  13m 4764 R 23.6  0.2   5:29.71 httpd                                                                                                                                                                                                 
30853 nobody    25   0  187m  16m 4760 R 23.6  0.2  30:21.56 httpd                                                                                                                                                                                                 
11045 nobody    25   0  182m  11m 4836 R 23.2  0.1   8:15.32 httpd                                                                                                                                                                                                 
15288 nobody    25   0  185m  15m 4824 R 23.2  0.2   6:15.49 httpd                                                                                                                                                                                                 
18056 nobody    25   0  187m  16m 4904 R 23.2  0.2   3:31.00 httpd                                                                                                                                                                                                 
22790 nobody    25   0  182m  10m 4012 R 23.2  0.1  62:16.74 httpd                                                                                                                                                                                                 
 1634 nobody    25   0  182m  10m 4044 R 22.9  0.1  22:22.69 httpd                                                                                                                                                                                                 
14460 nobody    25   0  182m  10m 3924 S 22.9  0.1   7:19.62 httpd                                                                                                                                                                                                 
21521 nobody    25   0  184m  13m 4896 R 21.6  0.2  61:31.62 httpd                                                                                                                                                                                                 
28403 nobody    25   0  184m  13m 4768 R 20.6  0.2  35:56.49 httpd                                                                                                                                                                                                 
 8092 nobody    25   0  182m  10m 4092 R 20.3  0.1  12:20.71 httpd                                                                                                                                                                                                 
17511 nobody    25   0  186m  16m 4836 R 20.3  0.2   4:11.25 httpd                                                                                                                                                                                                 
21051 nobody    25   0  186m  16m 4784 R 20.3  0.2  66:57.50 httpd                                                                                                                                                                                                 
28520 nobody    25   0  184m  13m 4760 R 20.3  0.2  35:35.02 httpd                                                                                                                                                                                                 
 1997 nobody    25   0  184m  13m 4528 R 19.9  0.2  20:59.58 httpd                                                                                                                                                                                                 
 8266 nobody    25   0  182m  11m 4408 R 19.9  0.1  12:07.31 httpd                                                                                                                                                                                                 
11429 nobody    25   0  185m  15m 4612 R 19.9  0.2   8:35.06 httpd                                                                                                                                                                                                 
12654 nobody    25   0  184m  13m 4720 R 19.9  0.2   8:03.82 httpd                                                                                                                                                                                                 
27974 nobody    25   0  182m  11m 4428 S 19.9  0.1  37:52.26 httpd                                                                                                                                                                                                 
31684 nobody    25   0  182m  11m 4248 R 19.9  0.1  27:36.75 httpd                                                                                                                                                                                                 
31749 nobody    25   0  182m  10m 4000 R 19.9  0.1  28:01.86 httpd                                                                                                                                                                                                 
 3378 nobody    25   0  185m  15m 4832 R 19.6  0.2  17:41.87 httpd                                                                                                                                                                                                 
22065 mysql     17   0  290m  51m 4116 S 18.9  0.6   0:00.80 mysqld                                                                                                                                                                                                
19951 nobody    15   0  307m 135m 6244 R 15.6  1.7   3:02.78 httpd                                                                                                                                                                                                 
21908 nobody    15   0  182m  12m 4780 S  9.6  0.2   0:02.25 httpd                                                                                                                                                                                                 
 8455 nobody    25   0  182m  10m 4064 R  5.0  0.1  11:41.38 httpd                                                                                                                                                                                                 
22010 nobody    15   0  185m  15m 4556 S  4.3  0.2   0:00.21 httpd                                                                                                                                                                                                 
27443 nobody    25   0  184m  13m 4884 R  3.7  0.2  36:55.42 httpd                                                                                                                                                                                                 
22068 mysql     16   0  290m  51m 4116 S  3.3  0.6   0:00.10 mysqld                                                                                                                                                                                                
27958 nobody    25   0  182m  11m 4768 R  3.3  0.1  37:44.92 httpd                                                                                                                                                                                                 
21777 nobody    15   0  184m  14m 4832 S  3.0  0.2   0:00.67 httpd                                                                                                                                                                                                 
21927 nobody    15   0  185m  14m 3844 S  3.0  0.2   0:00.26 httpd                                                                                                                                                                                                 
22018 mysql     16   0  290m  51m 4116 S  2.3  0.6   0:00.19 mysqld                                                                                                                                                                                                
21941 nobody    15   0  185m  15m 4824 S  1.7  0.2   0:00.84 httpd                                                                                                                                                                                                 
21789 nobody    15   0  202m  31m 4884 S  1.3  0.4   0:03.49 httpd                                                                                                                                                                                                 
21817 nobody    15   0  184m  14m 4832 S  1.3  0.2   0:00.77 httpd                                                                                                                                                                                                 
21916 nobody    15   0     0    0    0 Z  0.3  0.0   0:00.41 httpd <defunct> 
.... [truncated] ...
 
Last edited:

hackboys

Active Member
Feb 12, 2008
34
2
58
Some problem!
100% CPU httpd Threads, clock_gettime
Please help how to fix?


clock_gettime(CLOCK_MONOTONIC, {4557, 12307472}) = 0
poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {4557, 12496472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 12553472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 12609472}) = 0
select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
clock_gettime(CLOCK_MONOTONIC, {4557, 12787472}) = 0
poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {4557, 12899472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 12954472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13015472}) = 0
select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
clock_gettime(CLOCK_MONOTONIC, {4557, 13095472}) = 0
poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {4557, 13159472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13190472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13218472}) = 0
select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
clock_gettime(CLOCK_MONOTONIC, {4557, 13290472}) = 0
poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {4557, 13353472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13384472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13416472}) = 0
select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
clock_gettime(CLOCK_MONOTONIC, {4557, 13490472}) = 0
poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {4557, 13554472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13584472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13613472}) = 0
select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})
clock_gettime(CLOCK_MONOTONIC, {4557, 13685472}) = 0
poll([{fd=173, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
clock_gettime(CLOCK_MONOTONIC, {4557, 13748472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13779472}) = 0
clock_gettime(CLOCK_MONOTONIC, {4557, 13808472}) = 0
select(174, [173], [173], [], {15, 0}) = 1 (out [173], left {15, 0})




HTML:
Tasks: 784 total,   6 running, 778 sleeping,   0 stopped,   0 zombie
Cpu(s): 19.9%us, 31.6%sy,  0.0%ni, 47.5%id,  0.4%wa,  0.0%hi,  0.6%si,  0.0%st
Mem:  37038568k total, 20630780k used, 16407788k free,   343540k buffers
Swap: 39792996k total,        0k used, 39792996k free, 14803328k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 6877 mysql      5 -10  665m 308m 3616 S 180.2  0.9  69:24.14 mysqld
14673 nobody    25   0  321m  21m 6300 R 99.0  0.1   3:09.48 httpd
18913 nobody    25   0  319m  21m 5856 R 97.7  0.1   2:45.95 httpd
19182 nobody    25   0  319m  21m 6040 R 97.4  0.1   2:51.96 httpd
18590 nobody    25   0  324m  23m 6024 S 96.1  0.1   0:50.08 httpd
 8202 nobody    25   0  325m  28m 6544 R 92.4  0.1   3:28.31 httpd
13678 nobody    15   0  337m  38m 6684 S 14.6  0.1   0:08.55 httpd
13072 nobody    15   0  322m  24m 6180 S 13.2  0.1   0:06.67 httpd
19212 nobody    16   0  318m  19m 5616 S  7.9  0.1   0:00.95 httpd
16387 nobody    15   0  327m  26m 6172 S  5.6  0.1   0:05.41 httpd
19606 nobody    15   0  321m  23m 5912 S  5.3  0.1   0:01.05 httpd
18329 nobody    15   0  319m  21m 6076 S  4.6  0.1   0:02.12 httpd
14076 nobody    15   0  324m  24m 6308 S  4.3  0.1   0:07.87 httpd
19899 nobody    15   0  319m  19m 5352 S  4.3  0.1   0:00.29 httpd
18939 nobody    15   0  322m  24m 5924 S  3.6  0.1   0:02.00 httpd
19635 nobody    15   0  321m  22m 5452 S  3.6  0.1   0:00.88 httpd
19131 nobody    15   0  325m  26m 6080 S  3.3  0.1   0:02.14 httpd
19588 nobody    15   0  318m  19m 5728 S  3.3  0.1   0:00.69 httpd
 
Last edited:

ramindia

Well-Known Member
Apr 3, 2011
71
0
56
Hi Jvanv8

try Timeout 30 and restart apache see any change, if not install Cpanel Firewall

i had same problem i did the same i see some change, but iam still tuning myself for better performance.
 

nobodyk

Well-Known Member
Aug 1, 2010
90
0
56
Have you checked it's not a ddos attack?

can you post your apache conf? (just the cpanel variables)
 

cPanelTristan

Quality Assurance Analyst
Staff member
Oct 2, 2010
7,607
38
248
somewhere over the rainbow
cPanel Access Level
Root Administrator
I have a post that I did recently to troubleshoot a possible DoS attack:

http://forums.cpanel.net/f5/how-restart-apache-trough-cpanel-command-line-187252.html#post766101

I'd suggest trying the commands noted to see if this might be one. Since you have a very high number of Apache processes, it might well be an attack.

Alternatively, you might be hitting MaxClients on the machine. If you are, you can use the information at the end of that post to increase the MaxClients and ServerLimit values.