The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

High Loads daily at the same times

Discussion in 'Workarounds and Optimization' started by ggouweloos, Dec 17, 2010.

  1. ggouweloos

    ggouweloos Member

    Joined:
    Nov 26, 2008
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    I am monitoring our WHM/Cpanel VPS and notice there are a lot of High Loads during the day.
    Most of them show up every day at the same time.

    The VPS is a 16 CPU system on a hardware node that has several VPS's running. Not sure how much.

    Specific those High Loads are occuring every three hours. But some of them seem to be on other times too, but still daily. Seen peaks of 9.00 Loads for almost 10 minutes of time. Sites are very slow and all WHM/Cpanel functions (webmail etc) are extremely slow during the peaks.

    We changed some settings in LFD so every Peak of above two will be reported.

    Strange thing is that nothing can be seen during the High Loads, no process is showing that is taking the CPU resources. I checked both, during the Peak via SSH shell (using ps, netstat, vmstat, mpstat) and the logs that are sent by LFD.

    Also tried to disable crond for some hours, statistics and backups are only scheduled during the night. Even shut-sown Apache during a one of the High Loads. No effect at all.

    What can cause those High Loads?
    Why isn't there any process showing up that is consuming all those resources?
    How to investigate this further?

    Any help is appreciated.
     
  2. dmmcintyre3

    dmmcintyre3 Member

    Joined:
    Aug 26, 2010
    Messages:
    9
    Likes Received:
    0
    Trophy Points:
    1
    Ask your VPS provider, I'm having the same issue on one of my VPSs.
     
  3. ggouweloos

    ggouweloos Member

    Joined:
    Nov 26, 2008
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    The VPS provider cannot find the cause. :(

    The High Loads are there every 90 minutes.
    We tried:
    - Stopped service cron
    - Stopped service LFD/CSF

    Why isn't anything showing up with a 'ps auxf' during the High Loads ??
    What can be the cause of this?
     
  4. Tam

    Tam Well-Known Member

    Joined:
    Jul 31, 2004
    Messages:
    103
    Likes Received:
    0
    Trophy Points:
    16
    It is possible that it is someone else on the node and not you at all, go ask your VPS provider to check it out first.

    You should re-enable CSF and set it to send you notifications (process tracking) when the server load passes (say) 9, it will email you with a snapshot of the process list showing cpu and memory usage of everything running ;) If it is you that is using up the resources then it will show you where.
     
  5. ggouweloos

    ggouweloos Member

    Joined:
    Nov 26, 2008
    Messages:
    14
    Likes Received:
    0
    Trophy Points:
    1
    High Loads are there:
    11:33:43 PM load average alert - 4.80
    10:03:16 PM load average alert - 3.03
    08:33:06 PM load average alert - 3.69
    07:03:39 PM load average alert - 4.47
    05:32:21 PM load average alert - 3.50
    04:02:56 PM load average alert - 5.00
    02:33:05 PM load average alert - 3.41
    01:02:29 PM load average alert - 4.18
    11:32:49 AM load average alert - 4.19
    10:01:48 AM load average alert - 3.42
    08:32:28 AM load average alert - 3.05
    07:05:57 AM load average alert - 3.35
    05:31:44 AM load average alert - 3.12
    04:04:36 AM load average alert - 3.28

    Those Loads take around 8 to 12 minutes before getting below 1.00 again. All other times (in between) the Loads are between 0.08 and sometimes peak to 0.8.


    Herr the results from the 1PM load:

    Output from ps:

    USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
    root 1 0.0 0.1 1716 604 ? Ss 2010 0:02 init [3]
    root 7604 0.0 0.1 1620 528 ? Ss 2010 0:28 syslogd -m 0
    root 7608 0.0 0.0 1568 372 ? Ss 2010 0:00 klogd -x
    root 7673 0.0 0.1 4132 1040 ? Ss 2010 0:00 /usr/sbin/sshd
    root 7687 0.0 0.1 2188 792 ? Ss 2010 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
    root 11559 0.0 0.0 1580 408 ? Ss 2010 0:00 /usr/sbin/portsentry -tcp
    root 15861 0.0 0.2 7104 1340 ? Ss 2010 0:02 pure-ftpd (SERVER)
    root 15891 0.0 0.1 6820 1004 ? S 2010 0:00 /usr/sbin/pure-authd -s /var/run/ftpd.sock -r /usr/sbin/pureauth
    named 19856 0.0 1.0 195640 5404 ? Ssl 2010 2:55 /usr/sbin/named -u named
    mailnull 5817 0.0 0.3 7544 2068 ? Ss 2010 0:01 /usr/sbin/exim -bd -q60m
    root 11439 0.0 6.3 35764 33412 ? Ss 2010 0:05 /usr/bin/spamd -d --allowed-ips=127.0.0.1 --pidfile=/var/run/spamd.pid --max-children=3 --max-spare=1
    root 3441 0.4 13.0 70572 68512 ? S 10:04 0:51 \_ spamd child
    root 7868 0.0 0.8 6172 4644 ? S 2010 0:00 queueprocd - wait to process a task
    root 8121 0.0 1.0 7900 5464 ? S 2010 0:05 tailwatchd
    root 19762 0.1 1.1 8808 5796 ? S 13:01 0:00 \_ tailwatchd
    root 27900 1.2 0.7 5464 3852 ? D 13:02 0:00 \_ /usr/bin/perl /scripts/restartsrv_mysql --check
    root 8191 0.0 0.4 4096 2372 ? S 2010 0:00 cPhulkd - processor
    root 10122 0.0 1.7 13952 9068 ? S 2010 0:00 cpdavd - accepting connections on 2077 and 2078
    root 11338 0.0 0.3 3896 1632 ? SN 2010 0:00 cpanellogd - sleeping for logs
    root 13334 0.0 1.4 13440 7748 ? S 2010 0:11 cpsrvd - waiting for connections
    root 17795 0.0 0.1 1944 676 ? Ss 2010 0:07 /usr/sbin/dovecot
    root 17796 0.0 0.1 2372 1016 ? S 2010 0:08 \_ dovecot-auth
    dovecot 18095 0.0 0.3 4100 2056 ? S 2010 0:13 \_ pop3-login
    dovecot 18097 0.0 0.3 4100 2012 ? S 2010 0:01 \_ pop3-login
    dovecot 18098 0.0 0.3 4072 2056 ? S 2010 0:05 \_ imap-login
    dovecot 18099 0.0 0.3 3980 1888 ? S 2010 0:00 \_ imap-login
    root 30434 0.0 0.1 5764 1048 ? Ss Jan02 0:00 crond
    root 16256 0.0 1.3 12792 6816 ? Ss Jan02 0:08 /usr/local/apache/bin/httpd -k start -DSSL
    root 15944 0.0 1.0 8120 5664 ? S 12:07 0:00 \_ /usr/bin/perl /usr/local/cpanel/bin/leechprotect
    nobody 3573 0.0 1.2 12928 6576 ? S 12:28 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 26348 0.0 1.2 12928 6484 ? S 12:38 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 22133 0.0 1.2 12928 6456 ? S 12:40 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 25712 0.0 1.2 12928 6388 ? S 12:45 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 11627 0.0 1.2 12928 6572 ? S 12:46 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    italiew 17776 0.7 3.3 32144 17372 ? S 13:01 0:00 | \_ /usr/bin/php
    nobody 26195 0.0 1.2 12928 6372 ? S 12:50 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 26403 0.0 1.2 12928 6456 ? S 12:50 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 9323 0.0 1.2 12928 6428 ? S 12:52 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 3529 0.0 1.2 12928 6552 ? S 12:53 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 32618 0.0 1.2 12928 6344 ? S 13:00 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    nobody 27941 0.0 1.0 12792 5448 ? S 13:02 0:00 \_ /usr/local/apache/bin/httpd -k start -DSSL
    root 18312 0.0 0.2 5392 1244 ? S Jan02 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/servername.pid
    mysql 18383 0.1 4.7 134500 24832 ? Sl Jan02 5:33 \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/servername.pid --skip-external-locking --port=3306 --socket=/var/lib/mysql/mysql.sock
    root 28089 0.0 1.9 15396 10448 ? Ss 00:00 0:11 lfd - sleeping
    root 7297 0.0 1.7 15396 9424 ? S 13:00 0:00 \_ lfd - checking system integrity
    root 7298 0.2 0.1 4708 536 ? D 13:00 0:00 | \_ /usr/bin/md5sum --check /etc/csf/csf.tempint
    root 17757 0.1 1.8 15528 9880 ? D 13:01 0:00 \_ lfd - checking directories
    root 28254 0.0 1.8 15396 9588 ? S 13:02 0:00 \_ lfd - (child) checking load...
    root 28453 0.0 0.1 2396 796 ? R 13:02 0:00 \_ /bin/ps axuf



    Output from vmstat:

    procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
    r b swpd free buff cache si so bi bo in cs us sy id wa
    0 4 0 318736 0 0 0 0 56 15 0 1 0 0 97 3




    I cannot see the process that is causing the High Load, does this mean another VPS on the node is causing this every 90 minutes? So are the Load values shown in WHM, or when using shell, are showing the loads of all VPS'ses on the node? Why are the loads shown 5.00 or even up to 12.00 when there is no process on the VPS peaking?
     
    #5 ggouweloos, Jan 4, 2011
    Last edited: Jan 4, 2011
Loading...

Share This Page