The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

High CPU load help!

Discussion in 'General Discussion' started by pphillips, Dec 2, 2005.

  1. pphillips

    pphillips Well-Known Member

    Joined:
    Nov 14, 2003
    Messages:
    71
    Likes Received:
    0
    Trophy Points:
    6
    I have a cpanel server running WHM 10.8.0/cPanel 10.8.1 and I can't figure out why the load shoots up to 12+. It is a P4, 2.4Ghz 1GB RAM and this started about a week ago, bringing the server to a crawl. It is about 2 years old and we haven't put anything else on it for about a year, just the automatic cPanel updates. I've tried the following looking for the problem:

    1. chkrootkit of course, nothing found
    2. lsof | grep LISTEN, no ports open that shouldn't be.
    3. ps auxf, nothing out of the ordinary.
    4. top, cpu load will be high but list of services does not indicate this.
    5. I recompiled apache, didn't help. reboot server, didn't help

    Here is output from top just now:

    PHP:
    top 15:01:47 up 2 days23:33,  1 user,  load average8.184.154.41
    Tasks
    173 total,   2 running167 sleeping,   1 stopped,   3 zombie
    Cpu
    (s): 31.9us,  3.9sy,  0.0ni,  0.0id62.2wa,  1.0hi,  1.0si
    Mem
    :   1034856k total,  1014012k used,    20844k free,    31296k buffers
    Swap
    :  2096440k total,     3436k used,  2093004k free,   505936k cached

      PID USER      PR  NI  VIRT  RES  SHR S 
    %CPU %MEM    TIME+  COMMAND
    26390 nobody    16   0 22516  11m  14m S  6.8  1.1   0
    :01.21 httpd
    27994 nobody    15   0 20600  11m  12m S  6.2  1.2   0
    :00.19 httpd
    23288 nobody    17   0 24228  15m  12m S  3.9  1.6   0
    :04.03 httpd
    27317 nobody    15   0 19404  10m  12m S  3.6  1.1   0
    :00.51 httpd
    21789 thehappy  15   0 37608  31m 6388 D  3.3  3.1   0
    :27.56 spamd
    23279 nobody    15   0 25860  15m  14m S  2.3  1.5   0
    :04.32 httpd
    23287 nobody    15   0 22984  12m  14m S  1.6  1.2   0
    :03.48 httpd
    23291 nobody    15   0 26072  15m  14m S  1.0  1.5   0
    :03.16 httpd
    27762 diamondd  16   0  2600 1080 2184 D  1.0  0.1   0
    :00.76 imapd
    21790 ritzocat  15   0 37388  31m 6388 D  0.7  3.1   0
    :10.07 spamd
    24526 nobody    16   0 24592  13m  14m S  0.7  1.4   0
    :02.20 httpd
    26398 nobody    15   0 19552  10m  12m S  0.7  1.1   0
    :00.77 httpd
     7722 mysql     16   0 42564  27m 3760 S  0.3  2.7   0
    :02.15 mysqld
     8682 root      19   0 18860 9712  12m S  0.3  0.9   0
    :04.18 httpd
    21711 root      16   0  3996 1700 2792 S  0.3  0.2   0
    :04.44 antirelayd
    27514 root      16   0  2552  984 1624 R  0.3  0.1   0
    :00.52 top
    27988 moocom    16   0 32556  26m 6388 D  0.3  2.6   0
    :00.18 spamd
    28001 nobody    15   0 18860 9804  12m S  0.3  0.9   0
    :00.01 httpd
    28002 nobody    16   0 18992 9844  12m S  0.3  1.0   0
    :00.01 httpd
        1 root      16   0  1816  460 1320 S  0.0  0.0   0
    :06.86 init
        2 root      34  19     0    0    0 S  0.0  0.0   0
    :00.87 ksoftirqd/0
        3 root       5 
    -10     0    0    0 S  0.0  0.0   0:00.01 events/0
        4 root      10 
    -10     0    0    0 S  0.0  0.0   0:00.00 khelper
        5 root      15 
    -10     0    0    0 S  0.0  0.0   0:00.00 kacpid
       19 root       5 
    -10     0    0    0 S  0.0  0.0   0:00.00 kblockd/0
       29 root      15   0     0    0    0 S  0.0  0.0   0
    :19.57 pdflush
       30 root      15   0     0    0    0 S  0.0  0.0   0
    :22.07 pdflush
       32 root      10 
    -10     0    0    0 S  0.0  0.0   0:00.00 aio/0
       20 root      15   0     0    0    0 S  0.0  0.0   0
    :00.00 khubd
       31 root      16   0     0    0    0 S  0.0  0.0   1
    :39.13 kswapd0
      105 root      25   0     0    0    0 S  0.0  0.0   0
    :00.00 kseriod
      184 root      15   0     0    0    0 S  0.0  0.0   0
    :07.18 md0_raid1
      185 root      15   0     0    0    0 S  0.0  0.0   2
    :46.90 kjournald
     1081 root      15   0     0    0    0 S  0.0  0.0   0
    :00.00 kjournald
     1635 root      15   0  3416  568 1300 S  0.0  0.1   1
    :04.94 syslogd
     1639 root      15   0  2648  444 1248 S  0.0  0.0   0
    :00.35 klogd
     1672 root      16   0  2208  556 1300 S  0.0  0.1   0
    :00.10 rpc.idmapd
     1741 root      15   0  2068  744 1372 S  0.0  0.1   0
    :00.02 smartd
     1750 root      19   0  2844  460 1240 S  0.0  0.0   0
    :00.00 acpid
     3187 named     18   0 38368 4784 4780 S  0.0  0.5   0
    :00.00 named
     3222 root      15   0  4248 1456 3444 S  0.0  0.1   0
    :09.74 sshd
     3235 root      16   0  2888  832 1688 S  0.0  0.1   0
    :01.73 xinetd
     3253 thehappy  15   0  2600 1000 2184 S  0.0  0.1   0
    :01.51 imapd
     3414 thehappy  15   0  2596 1004 2184 S  0.0  0.1   0
    :00.55 imapd
     3559 root      16   0  6396 1800 5644 S  0.0  0.2   0
    :00.71 pure-ftpd
     3562 root      15   0  6692  636 5264 S  0.0  0.1   0
    :00.15 pure-authd
     3585 root      16   0  3260  652 1360 S  0.0  0.1   0
    :00.62 crond
     4503 postgres  16   0 20268 2052  17m S  0.0  0.2   0
    :00.28 postmaster
     4508 postgres  16   0 11068 1852 7980 S  0.0  0.2   0
    :00.01 postmaster
     4509 postgres  15   0 10076 1848 7980 S  0.0  0.2   0
    :00.01 postmaster
     4544 dbus      18   0  3228  808 1704 S  0.0  0.1   0
    :00.00 dbus-daemon-1                 

    I did notice something while at the datacenter today, this server's hdd LED was blinking quite a bit faster than the rest of our servers. Could it be a HDD I/O issue? Saw something while searching this form but not sure what a HDD I/O issue could mean.

    Thanks for your help!!
     
  2. nyjimbo

    nyjimbo Well-Known Member

    Joined:
    Jan 25, 2003
    Messages:
    1,125
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    New York
    Since that is just a moments snapshot its really hard to tell but if you sit for a minute or two and really eyeball it see if spamd or exim is sitting at the top of top for a while.

    There is a huge amount of virus loaded spam making the rounds and if you are processing them you will see it take a big hit. We were getting massive loads until we started using a more complete RBL/DNSBL blocking especially with the ORDB lists or anything that would find more recent zombie systems.

    If you do find it to be spamd/exim/whatever you have to do something about finding out what IP's are flooding you with these virus files (typically 75-80k in size right now) or increase your blocking lists or adjust your exim to handle zip/virus attachments differently to reduce load.
     
  3. panayot

    panayot Well-Known Member

    Joined:
    Nov 18, 2004
    Messages:
    125
    Likes Received:
    0
    Trophy Points:
    16
    I have a similar problem but still could not find anything. server usually has load 0.60 and then for half hour it gets to 4 and stays there.

    I have tried stopping different things, so you could also try for example while you have the load:

    service exim stop

    and check if load goes down. Also:

    service mysql stop
    service cpanel stop

    I also went to /home directory and did to serveral sites:

    mv user/ user1/

    but there are 500 sites and it is difficult to disable them temporarily like that one by one. And I think that a php script that is already running will not stop if you move its directory.

    also in WHM check Apache Status. See the column Req if is too big for some script.

    check if during high load your server is using more bandwidth with mrtg or bwm-ng (http://www.gropp.org/)

    Hope some of this leads you to a clue (did not help me yet)
    actually stopping apache brought load down to 0.35 so I guess in my case it has to do something with web sites.

    I wonder if it is possible that there is some process that does not show in ps ? I am running daily chkrootkit and rkhunter and nothing there.
     
    #3 panayot, Dec 3, 2005
    Last edited: Dec 3, 2005
  4. pphillips

    pphillips Well-Known Member

    Joined:
    Nov 14, 2003
    Messages:
    71
    Likes Received:
    0
    Trophy Points:
    6
    I tried a lot of that, I use ibmonitor for live bandwidth monitoring on the server, but usage is still under 2Mbps at all times.

    I think a lot of it may be IMAP. The server is the busiest during normal business hours when people are POPing and IMAPing the server. I do notice a lot of IMAP processes in top when the load is high. Anyway to optimize IMAP somehow?
     
  5. Zaf

    Zaf Well-Known Member

    Joined:
    Aug 22, 2005
    Messages:
    119
    Likes Received:
    0
    Trophy Points:
    16
    Switch to maildir, if you havent so far. The conversion will replace your POP/IMAP service to Courier and that is known to help reduce load substantially.
     
Loading...

Share This Page