The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Very high load (30!)

Discussion in 'General Discussion' started by asmar, Mar 24, 2005.

  1. asmar

    asmar Well-Known Member

    Joined:
    Jul 16, 2004
    Messages:
    135
    Likes Received:
    0
    Trophy Points:
    16
    Every 1-2 hours and sometimes more, the load goes up to 30 and the only way to reduce that is to restart the httpd deamon.
    Doing a netstat -an|grep :80|sort|more doesn't show something unsual.
    Bellow is a ps output the time that his happening.

    root@ [/scripts]# ps ax
    PID TTY STAT TIME COMMAND
    1 ? S 0:05 init
    2 ? SW 0:00 [migration/0]
    3 ? SW 0:00 [migration/1]
    4 ? SW 0:00 [keventd]
    5 ? SWN 0:02 [ksoftirqd_CPU0]
    6 ? SWN 0:00 [ksoftirqd_CPU1]
    11 ? SW 0:00 [bdflush]
    7 ? SW 0:23 [kswapd]
    8 ? SW 0:00 [kscand/DMA]
    9 ? SW 0:40 [kscand/Normal]
    10 ? SW 0:18 [kscand/HighMem]
    12 ? SW 0:02 [kupdated]
    13 ? SW 0:00 [mdrecoveryd]
    17 ? SW 0:01 [kjournald]
    75 ? SW 0:00 [khubd]
    281 ? SW 0:00 [kjournald]
    282 ? SW 0:02 [kjournald]
    283 ? SW 0:00 [kjournald]
    284 ? SW 0:01 [kjournald]
    285 ? SW 0:05 [kjournald]
    652 ? S 0:02 syslogd -m 0
    656 ? S 0:00 klogd -x
    719 ? S 0:00 proftpd: (accepting connections)
    744 ? S 0:00 /usr/sbin/sshd
    758 ? S 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
    776 ? S 0:00 chkservd
    904 ? S 0:00 crond
    1223 ? S 0:00 entropychat
    1229 ? S 0:00 /usr/local/cpanel/bin/startmelange
    1250 ? S 0:01 /usr/bin/perl /usr/local/cpanel/bin/eximstats
    1335 ? S 0:00 /usr/sbin/portsentry -tcp
    1530 ? S 0:00 /usr/bin/perl /usr/libexec/webmin/miniserv.pl /etc/we
    1533 tty1 S 0:00 /sbin/mingetty tty1
    1534 tty2 S 0:00 /sbin/mingetty tty2
    1535 tty3 S 0:00 /sbin/mingetty tty3
    1536 tty4 S 0:00 /sbin/mingetty tty4
    1537 tty5 S 0:00 /sbin/mingetty tty5
    1538 tty6 S 0:00 /sbin/mingetty tty6
    2983 ? S 0:00 /usr/bin/stunnel-4.04local /usr/local/cpanel/etc/stun
    11901 ? S 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql
    11921 ? S 0:05 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql
    2086 ? S 0:00 postsuexecinstall - searching for suexec problems (14
    2099 ? S 0:02 /usr/local/apache/bin/httpd -DSSL
    6939 ? S 0:27 /usr/sbin/named -u named
    8763 ? SL 0:01 proftpd: eventco - 62.103.29.108: STOR dragshow1.zip
    11448 ? S 0:00 /usr/sbin/sshd
    11464 ? S 0:00 /usr/sbin/sshd
    11465 pts/1 S 0:00 -bash
    11500 pts/1 S 0:09 top
    11612 ? S 0:00 /usr/sbin/sshd
    11616 pts/2 S 0:00 -bash
    11936 ? SL 0:01 proftpd: eventco - 62.103.29.108: STOR freestyle1.zip
    11937 ? S 0:00 /usr/sbin/sshd
    11953 pts/3 S 0:00 -bash
    12067 pts/3 S 0:05 top
    13010 ? S 0:00 /usr/sbin/sshd
    13030 ? S 0:00 /usr/sbin/sshd
    13031 pts/4 S 0:00 -bash
    13062 pts/4 S 0:00 su
    13063 pts/4 S 0:00 bash
    16915 ? SL 0:00 proftpd: eventco - 62.103.29.108: IDLE
    17217 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17220 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17221 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17222 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17223 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17224 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17225 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17226 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17227 ? S 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/b
    17241 ? SN 0:00 cpanellogd - setting up logs for nouras
    17240 ? SN 0:00 cpanellogd - setting up logs for nouras
    17254 ? S 0:00 cppop - accepting on port 110
    17288 ? S 0:00 cpsrvd - waiting for connections
    17399 ? SL 0:00 proftpd: adminw - 62.103.29.108: IDLE
    17531 ? SL 0:00 proftpd: irven - 213.5.24.179: IDLE
    17903 ? S 0:00 /usr/sbin/exim -bd -q60m
    17915 ? S 0:00 /usr/sbin/exim -tls-on-connect -bd -oX 465
    17920 pts/2 S 0:00 antirelayd
    18230 ? S 0:00 whostmgrd - serving 127.0.0.1
    18231 ? S 0:00 /usr/local/cpanel/whostmgr/bin/whostmgr ./checktro
    18235 ? S 0:00 /usr/bin/perl /scripts/findtrojans
    18236 ? D 0:17 /usr/lib/rpm/rpmv -V --all
    18335 ? SL 0:00 proftpd: lwreth - 213.5.55.179: IDLE
    18404 pts/4 S 0:00 tail -f /var/log/messages
    18508 ? SL 0:00 proftpd: connected: 127.0.0.1 (127.0.0.1:36829)
    18670 ? SN 0:00 cpanellogd - http logs for nouras
    18671 ? SN 0:00 cpanellogd - http logs for nouras
    18672 ? SN 0:00 cpanellogd - http logs for nouras
    18673 ? SN 0:00 cpanellogd - http logs for nouras
    18674 ? SN 0:00 /usr/local/cpanel/bin/logrunner 5.0 /usr/local/cpanel
    18675 ? SN 0:00 /usr/local/cpanel/bin/logrunner 5.0 /usr/local/cpanel
    18677 ? TN 0:00 /usr/local/cpanel/3rdparty/bin/english/webalizer -N 1
    18676 ? TN 0:00 /usr/local/cpanel/3rdparty/bin/english/webalizer -N 1
    18768 ? S 0:00 /usr/bin/perl /usr/local/cpanel/bin/leechprotect
    18769 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18770 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18771 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18772 ? R 0:00 /usr/local/apache/bin/httpd -DSSL
    18774 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18778 ? S 0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql
    18783 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18787 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18796 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18797 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18804 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18806 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18808 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18810 ? D 0:00 /usr/local/apache/bin/httpd -DSSL
    18811 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18812 pts/2 R 0:00 ps ax
    18814 ? R 0:00 /usr/local/apache/bin/httpd -DSSL
    18815 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18816 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18817 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18819 ? R 0:00 /usr/local/apache/bin/httpd -DSSL
    18820 ? D 0:00 /usr/local/apache/bin/httpd -DSSL
    18821 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18822 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18823 ? S 0:00 cppop - serving 219.89.81.114 - TRANSACTION - shaun@b
    18825 ? R 0:00 cppop - serving 62.1.225.112 - AUTHORIZATION
    18826 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18827 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18828 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18829 ? S 0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql
    18830 ? S 0:00 /usr/local/apache/bin/httpd -DSSL
    18831 ? S 0:00 /usr/local/apache/bin/httpd -DSSL


    Any suggestions?
    The box is running cPanel stable and ReHat 9

    Thanks in advance
     
  2. kahoz

    kahoz Well-Known Member

    Joined:
    Nov 7, 2003
    Messages:
    78
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Portugal
    instead of ps ax use top ;)
     
  3. ramprage

    ramprage Well-Known Member

    Joined:
    Jul 21, 2002
    Messages:
    667
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Canada
    Try using top as suggested and also in WHM View the Apache and MySQL status.
    Generally when the load spikes like that you have a big process like a backup happening, although it could be something else.
     
  4. asmar

    asmar Well-Known Member

    Joined:
    Jul 16, 2004
    Messages:
    135
    Likes Received:
    0
    Trophy Points:
    16
    It is defenetly not the backup.
    I disable backup, change ssh port, optimize apache,mysql but problem remains. It happens once a day and load goes down only if I restart apache.

    Unde WHM apache status and mysql is not showing something unsual.

    Any ideas?

    Here is a pstree output as well:

    user@ [~]# pstree -apG
    init,1
    ├─antirelayd,27358
    ├─(bdflush,11)
    ├─chkservd,776
    ├─cpanellogd,27459
    ├─cppop,27480
    ├─cpsrvd,27563
    │ ├─cpsrvd,24809
    │ ├─cpsrvd,24745
    │ ├─cpsrvd,24746
    │ └─cpsrvd,24785
    ├─crond,29872
    ├─entropychat,1223
    ├─exim,27238 -bd -q60m
    │ └─exim,24794 -bd -q60m
    ├─exim,27243 -tls-on-connect -bd -oX 465
    ├─exim,27288 -oX 26 -bd
    ├─eximstats,1250 /usr/local/cpanel/bin/eximstats
    ├─httpd,3663 -DSSL
    │ ├─httpd,24454 -DSSL
    │ ├─httpd,24456 -DSSL
    │ ├─httpd,24474 -DSSL
    │ ├─httpd,24493 -DSSL
    │ ├─httpd,24495 -DSSL
    │ ├─httpd,24496 -DSSL
    │ ├─httpd,24501 -DSSL
    │ ├─httpd,24503 -DSSL
    │ ├─httpd,24504 -DSSL
    │ ├─httpd,24544 -DSSL
    │ ├─httpd,24545 -DSSL
    │ ├─httpd,24572 -DSSL
    │ ├─httpd,24577 -DSSL
    │ ├─httpd,24579 -DSSL
    │ ├─httpd,24580 -DSSL
    │ ├─httpd,24581 -DSSL
    │ ├─httpd,24582 -DSSL
    │ ├─httpd,24583 -DSSL
    │ ├─httpd,24584 -DSSL
    │ ├─httpd,24602 -DSSL
    │ ├─httpd,24625 -DSSL
    │ ├─httpd,24647 -DSSL
    │ ├─httpd,24732 -DSSL
    │ ├─httpd,24756 -DSSL
    │ └─leechprotect,18903 /usr/local/cpanel/bin/leechprotect
    ├─(keventd,4)
    ├─(khubd,75)
    ├─(kjournald,17)
    ├─(kjournald,281)
    ├─(kjournald,282)
    ├─(kjournald,283)
    ├─(kjournald,284)
    ├─(kjournald,285)
    ├─klogd,30291 -x
    ├─(kscand/DMA,8)
    ├─(kscand/HighMem,10)
    ├─(kscand/Normal,9)
    ├─(ksoftirqd_CPU0,5)
    ├─(ksoftirqd_CPU1,6)
    ├─(kswapd,7)
    ├─(kupdated,12)
    ├─mailmanctl,27433 /usr/local/cpanel/3rdparty/mailman/bin/mailmanctl ...
    │ ├─python2,27437 ...
    │ ├─python2,27438 ...
    │ ├─python2,27439 ...
    │ ├─python2,27440 ...
    │ ├─python2,27441 ...
    │ ├─python2,27442 ...
    │ ├─python2,27443 ...
    │ └─python2,27444 ...
    ├─(mdrecoveryd,13)
    ├─melange,1229
    ├─mingetty,1533 tty1
    ├─mingetty,1534 tty2
    ├─mingetty,1535 tty3
    ├─mingetty,1536 tty4
    ├─mingetty,1537 tty5
    ├─mingetty,1538 tty6
    ├─mysqld_safe,31012 /usr/bin/mysqld_safe --datadir=/var/lib/mysql ...
    │ └─mysqld,31040 --basedir=/ --datadir=/var/lib/mysql ...
    │ └─mysqld,31041 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31042 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31043 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31044 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31045 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31055 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31341 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31342 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2715 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2717 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2718 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2719 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2720 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2721 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,2723 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,31651 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,344 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,443 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,462 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,901 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,902 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,905 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,921 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,924 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,929 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,934 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,950 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,962 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,967 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,988 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,990 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,1003 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,1004 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14257 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14265 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14274 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14276 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14277 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14292 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14296 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14297 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14307 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14313 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14315 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14321 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14322 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14323 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14329 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14333 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14358 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14379 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14380 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14381 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14383 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14385 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14386 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14390 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14405 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14408 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14434 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14435 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14436 --basedir=/ --datadir=/var/lib/mysql ...
    │ ├─mysqld,14438 --basedir=/ --datadir=/var/lib/mysql ...
    │ └─mysqld,14440 --basedir=/ --datadir=/var/lib/mysql ...
    ├─named,6939 -u named
    ├─perl,22448
    ├─portsentry,1335 -tcp
    ├─proftpd,719
    │ ├─proftpd,22639
    │ ├─proftpd,24770
    │ └─proftpd,22219
    ├─spamd,14630
    │ ├─spamd,14635
    │ ├─spamd,14636
    │ ├─spamd,14637
    │ ├─spamd,14638
    │ └─spamd,14639
    ├─sshd,744
    │ ├─sshd,19729
    │ │ └─sshd,19759
    │ │ └─jailshell,19760
    │ │ └─bash,19789
    │ │ └─top,19813
    │ └─sshd,24747
    │ └─sshd,24763
    │ └─bash,24768
    │ └─pstree,24808 -apG
    ├─stunnel-4.04loc,2983 ...
    ├─syslogd,30287 -m 0
     
  5. Martin Blank

    Martin Blank Registered

    Joined:
    Apr 25, 2004
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    Could be the phpBB worm

    My box still gets hit with it periodically, and for a few minutes, loads can spike very high, with dozens more Apache threads than normal. If you have a phpBB forum, and you have administrative access, check the number of users on during that time. If you see a few hundred guests, that's probably it.
     
  6. asmar

    asmar Well-Known Member

    Joined:
    Jul 16, 2004
    Messages:
    135
    Likes Received:
    0
    Trophy Points:
    16
    It is defenetly not the phpbb worm.

    However, I found the solution.
    I change the priorities in some processes and now everything is working great.

    Thanks
     
  7. Sabaote

    Sabaote Well-Known Member

    Joined:
    Dec 19, 2003
    Messages:
    63
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Ribeirao Preto, Brazil
    Please share with us :)
     
  8. asmar

    asmar Well-Known Member

    Joined:
    Jul 16, 2004
    Messages:
    135
    Likes Received:
    0
    Trophy Points:
    16
    Hi Sabaote,

    Visit http://www.rfxnetworks.com/proj.php and download/configure the following tools:

    APF
    SIM
    LSM
    BFD
    PRM
    SPRI

    After careful configuration I managed to sort out the problem.
    The above tools are free and simply perfect!

    Let me know if you need anything else.

    Thanks
     
  9. desihost

    desihost Active Member

    Joined:
    Feb 2, 2005
    Messages:
    25
    Likes Received:
    0
    Trophy Points:
    1
    IM getin emails frm root frm usin SIM (System Integrity Monitor) every 5 mins... does any1 know how to change the minutes for sendin updated info..

    ps these scripts made my server smooth again...
     
  10. asmar

    asmar Well-Known Member

    Joined:
    Jul 16, 2004
    Messages:
    135
    Likes Received:
    0
    Trophy Points:
    16
    There is a cron running every 5 minutes under /etc/cron.d
    Edit the file and change that.

    Thanks
     
  11. desihost

    desihost Active Member

    Joined:
    Feb 2, 2005
    Messages:
    25
    Likes Received:
    0
    Trophy Points:
    1
    cheers mate..... mch apriciated.... :rolleyes:
     
  12. desihost

    desihost Active Member

    Joined:
    Feb 2, 2005
    Messages:
    25
    Likes Received:
    0
    Trophy Points:
    1
    could find that thier.... any other methods....
     
  13. asmar

    asmar Well-Known Member

    Joined:
    Jul 16, 2004
    Messages:
    135
    Likes Received:
    0
    Trophy Points:
    16
    Did you install it correctly?
    Type as root:

    updatedb

    and then:

    locate sim

    See if you find the cron there. There is also a command which install the cron as well. Read the readme file as I don't remember it.

    Hope that helps
     
Loading...

Share This Page