|
|||
|
Need help troubleshooting high load
Hi there -
Recently, my server has began showing a high CPU load. The server is lightly loaded for the most part and I need some help nailing down the cause of the high load. Here is a snapshot of my top CPU processes - Pid Owner Priority Cpu % Mem % Command 3854 nobody 0 7.9 0.8 /usr/local/apache/bin/httpd -DSSL 28015 nobody 0 6.9 0.9 /usr/local/apache/bin/httpd -DSSL 9853 nobody 0 3.7 0.6 /usr/local/apache/bin/httpd -DSSL 9918 nobody 0 2.9 0.6 /usr/local/apache/bin/httpd -DSSL 20631 nobody 0 2.1 0.8 /usr/local/apache/bin/httpd -DSSL 9854 nobody 0 1.9 0.4 /usr/local/apache/bin/httpd -DSSL 9857 nobody 0 1.7 0.5 /usr/local/apache/bin/httpd -DSSL 3572 nobody 0 0.3 1.5 /usr/local/apache/bin/httpd -DSSL 9955 root 0 0.3 0.0 0 top -n 2 -b - 3569 nobody 0 0.1 0.8 /usr/local/apache/bin/httpd -DSSL 16437 nobody 0 0.1 0.8 /usr/local/apache/bin/httpd -DSSL 17050 nobody 0 0.1 0.8 /usr/local/apache/bin/httpd -DSSL 22806 nobody 0 0.1 0.7 /usr/local/apache/bin/httpd -DSSL 7147 nobody 0 0.1 0.7 /usr/local/apache/bin/httpd -DSSL 9065 nobody 0 0.1 0.8 /usr/local/apache/bin/httpd -DSSL 9072 nobody 0 0.1 0.5 /usr/local/apache/bin/httpd -DSSL 9872 nobody 0 0.1 0.3 /usr/local/apache/bin/httpd -DSSL 9921 nobody 0 0.1 0.2 /usr/local/apache/bin/httpd -DSSL I've done a lot of searching, but need some help in where to go from here. Thanks. Mike
__________________
Get your hosting business listed on www.HostPremiums.com. Visit www.decomodo.com for cool stuff you can't live without. |
|
|||
|
It looks like you have a user with a very heavy web page but its hard to tell because the top of top is missing, lol.
info about the server would be helpful too. Oh, you were looking via whm. Use ssh and the top -c command and post back. Although its not very accurate, in whm you can look at the cpu/memory/mysql usage in whm and see where the calls are coming from. Again its not very accurate but an easy starting point for you to look at. Last edited by jayh38; 02-28-2007 at 05:43 PM. |
|
|||
|
Hmmm - Top of top missing? I copied/pasted all of it.
Does this help? (top -c from shell instead) - 32381 nobody 15 0 17580 17M 1552 S 2.7 0.8 1:26 0 /usr/local/apache/bin/httpd -DSSL 16437 nobody 15 0 17608 17M 1572 S 2.3 0.8 4:01 0 /usr/local/apache/bin/httpd -DSSL 3678 nobody 15 0 17484 17M 1456 S 0.3 0.8 3:52 0 /usr/local/apache/bin/httpd -DSSL 3572 nobody 15 0 30876 30M 1328 S 0.1 1.5 7:29 0 /usr/local/apache/bin/httpd -DSSL 27173 nobody 15 0 13048 12M 1624 S 0.1 0.6 0:00 0 /usr/local/apache/bin/httpd -DSSL 29278 root 15 0 1104 1104 776 R 0.1 0.0 0:00 0 top -c 1 root 15 0 112 112 56 S 0.0 0.0 0:04 0 init 2 root RT 0 0 0 0 SW 0.0 0.0 0:00 0 migration/0 3 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 keventd 4 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 kapmd 5 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0 8 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 bdflush 6 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 kswapd 7 root 15 0 0 0 0 SW 0.0 0.0 0:35 0 kscand 9 root 15 0 0 0 0 SW 0.0 0.0 0:01 0 kupdated 10 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd 18 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 scsi_eh_0 19 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 scsi_eh_1 22 root 15 0 0 0 0 SW 0.0 0.0 0:01 0 kjournald 77 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd 2679 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald 2680 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 kjournald 2681 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald 2682 root 15 0 0 0 0 SW 0.0 0.0 0:02 0 kjournald 2683 root 15 0 0 0 0 SW 0.0 0.0 0:09 0 kjournald 2684 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald 3150 root 15 0 256 256 172 S 0.0 0.0 0:03 0 syslogd -m 0 3154 root 15 0 60 60 0 S 0.0 0.0 0:00 0 klogd -x 3190 root 25 0 60 60 0 S 0.0 0.0 0:00 0 /usr/sbin/courierlogger -pid=/var/spool/authda 3191 root 15 0 100 100 4 S 0.0 0.0 0:00 0 /usr/libexec/courier-authlib/authdaemond 3201 root 15 0 176 176 116 S 0.0 0.0 0:00 0 mdadm --monitor --scan -f Server details - Processors 1 Model Intel(R) Pentium(R) 4 CPU 3.20GHz CPU Speed 3.19 GHz Cache Size 512 KB System Bogomips 6370.09 RAM = 2 GB Server Version: Apache/1.3.37 (Unix) mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.4.4 FrontPage/5.0.2.2635.SR1.2 mod_ssl/2.8.28 OpenSSL/0.9.7a Server Built: Aug 16 2006 11:58:54 Kernel Version 2.4.21-40.ELsmp (SMP) Distro Name Red Hat Enterprise Linux ES release 3 (Taroon Update 8) WHM 10.8.0 cPanel 10.9.0-C138 Is there any additional info that would be helpful? Thanks so much. Mike
__________________
Get your hosting business listed on www.HostPremiums.com. Visit www.decomodo.com for cool stuff you can't live without. |
|
|||
|
Quote:
|
|
|||
|
Thank you for that clarification -
Here's a current snapshot of everything this time ![]() top -c 19:13:06 up 16:51, 2 users, load average: 0.35, 0.31, 0.33 196 processes: 193 sleeping, 2 running, 1 zombie, 0 stopped CPU states: cpu user nice system irq softirq iowait idle total 7.9% 0.0% 1.9% 0.1% 0.0% 6.5% 83.2% Mem: 2054948k av, 1975584k used, 79364k free, 0k shrd, 134172k buff 1425108k actv, 285128k in_d, 26348k in_c Swap: 2040212k av, 19880k used, 2020332k free 849096k cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND 22802 nobody 15 0 17600 17M 1772 S 2.5 0.8 2:21 0 /usr/local/apache/bin/httpd -DSSL 27759 mailnull 15 0 3120 3120 2044 S 0.7 0.1 0:00 0 /usr/sbin/exim -bd -q60m 20631 nobody 15 0 18508 18M 2132 S 0.3 0.9 3:03 0 /usr/local/apache/bin/httpd -DSSL 29278 root 15 0 1188 1188 776 R 0.3 0.0 0:29 0 top -c 2680 root 15 0 0 0 0 SW 0.1 0.0 0:05 0 kjournald 3570 nobody 15 0 31340 30M 1744 S 0.1 1.5 9:13 0 /usr/local/apache/bin/httpd -DSSL 3572 nobody 15 0 31284 30M 1736 S 0.1 1.5 7:39 0 /usr/local/apache/bin/httpd -DSSL 20847 nobody 15 0 31472 30M 1904 S 0.1 1.5 4:36 0 /usr/local/apache/bin/httpd -DSSL 16437 nobody 15 0 17776 17M 1740 S 0.1 0.8 4:14 0 /usr/local/apache/bin/httpd -DSSL 5110 nobody 15 0 17960 17M 2456 S 0.1 0.8 0:25 0 /usr/local/apache/bin/httpd -DSSL 12194 nobody 15 0 14896 14M 2676 S 0.1 0.7 0:03 0 /usr/local/apache/bin/httpd -DSSL 27679 nobody 15 0 10972 10M 2508 S 0.1 0.5 0:00 0 /usr/local/apache/bin/httpd -DSSL 1 root 15 0 112 112 56 S 0.0 0.0 0:04 0 init 2 root RT 0 0 0 0 SW 0.0 0.0 0:00 0 migration/0 3 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 keventd 4 root 15 0 0 0 0 SW 0.0 0.0 0:05 0 kapmd 5 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0 8 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 bdflush 6 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 kswapd 7 root 15 0 0 0 0 SW 0.0 0.0 0:42 0 kscand 9 root 15 0 0 0 0 SW 0.0 0.0 0:02 0 kupdated 10 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd 18 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 scsi_eh_0 19 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 scsi_eh_1 22 root 15 0 0 0 0 SW 0.0 0.0 0:01 0 kjournald 77 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd 2679 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald 2681 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald 2682 root 15 0 0 0 0 SW 0.0 0.0 0:02 0 kjournald 2683 root 15 0 0 0 0 SW 0.0 0.0 0:11 0 kjournald 2684 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald 3150 root 15 0 256 256 172 S 0.0 0.0 0:04 0 syslogd -m 0 3154 root 15 0 60 60 0 S 0.0 0.0 0:00 0 klogd -x 3190 root 25 0 60 60 0 S 0.0 0.0 0:00 0 /usr/sbin/courierlogger -pid=/var/spool/authda 3191 root 15 0 116 116 20 S 0.0 0.0 0:00 0 /usr/libexec/courier-authlib/authdaemond 3201 root 15 0 176 176 116 S 0.0 0.0 0:00 0 mdadm --monitor --scan -f 3203 root 15 0 200 200 100 S 0.0 0.0 0:00 0 /usr/libexec/courier-authlib/authdaemond
__________________
Get your hosting business listed on www.HostPremiums.com. Visit www.decomodo.com for cool stuff you can't live without. |
|
|||
|
As of that snapshot, yes. But we're seeing wide fluctuations, spiking up to > 25 at some point in time yesterday - although, generally spiking up between 5 and 10.
I'll grab a snapshot when the load peaks - until then, any other suggestions? Thanks. Mike
__________________
Get your hosting business listed on www.HostPremiums.com. Visit www.decomodo.com for cool stuff you can't live without. |
|
|||
|
Keep in mind that load can be disk usage as well .. and it doesnt show up well in TOP
You will see the load go up but not always whats using the disk to cause it .. doing a cpanel backup is a great example the zipping of the files is both cpu and disk intensive but its the disk usage that drives up the load on my box .. |
|
|||
|
Yes, we would definitely need to see your top when you have the loading issue.
Although, its just a snapshot of that moment, you do not seem to have many users to warrant 196 tasks. So I would guess this is shortly after experiencing your trouble? Here is a command to tally actual connections at the time of your spike. See if its a connection issue to help narrow things down. Run this constantly and get a feel for the tasks and loading during normal times. netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | wc -l You do have a bit of a swap file so at one point or several, you ran low on ram which would also cause your loading. Try restarting apache to wipe some of the unused cache away. also run this utility to monitor statistics. The first column is important and should not exceed your cpu count by much or at all. vmstat -3 type man vmstat for more details of the report |
|
|||
|
This is extremely helpful info. Thank you so much for your help.
Mike
__________________
Get your hosting business listed on www.HostPremiums.com. Visit www.decomodo.com for cool stuff you can't live without. |
![]() |
| Thread Tools | |
| Display Modes | |
|
|