Major Cpanel Problems URGENT HELP NEEDED....

[blackice]

upgraded to cpanel last night however the server has become very slow it is only running a single forum with a 30mb db (invisionboard 2.0.2), cpanel and all other services are slow... there is only 1 user online on the board.

Server is a 2,2 Ghz Intel processor with 512 MB DDR RAM running Fedora Core 2.

Please help....

Many Thanks in advance

Richard

BlackICE Hosting

 

[ntwaddel]

are there any processes running that are using up all your cpu power? how much cpu usage are you at now?

 

[GOT]

Take a look at the top output and see what is dragging yuor server down.

You can PM me if you want more help.

 

[morpheous]

Take a look at the top output and see what is dragging yuor server down.

You can PM me if you want more help.

[email protected] [~]# ps aux
USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  3328  460 ?        S    Dec22   0:01 init [3]
root         2  0.0  0.0     0    0 ?        SWN  Dec22   0:00 [ksoftirqd/0]
root         3  0.0  0.0     0    0 ?        SW<  Dec22   0:00 [events/0]
root         4  0.0  0.0     0    0 ?        SW<  Dec22   0:00 [khelper]
root         5  0.0  0.0     0    0 ?        SW<  Dec22   0:00 [kacpid]
root        19  0.0  0.0     0    0 ?        SW<  Dec22   0:00 [kblockd/0]
root        29  0.0  0.0     0    0 ?        SW   Dec22   0:00 [pdflush]
root        30  0.0  0.0     0    0 ?        SW   Dec22   0:00 [pdflush]
root        32  0.0  0.0     0    0 ?        SW<  Dec22   0:00 [aio/0]
root        20  0.0  0.0     0    0 ?        SW   Dec22   0:00 [khubd]
root        31  0.0  0.0     0    0 ?        SW   Dec22   0:01 [kswapd0]
root       105  0.0  0.0     0    0 ?        SW   Dec22   0:00 [kseriod]
root       175  0.0  0.0     0    0 ?        SW   Dec22   0:01 [kjournald]
root      1099  0.0  0.0     0    0 ?        SW   Dec22   0:00 [kjournald]
root      1100  0.0  0.0     0    0 ?        SW   Dec22   0:00 [kjournald]
root      1501  0.0  0.0  2048  448 ?        S    Dec22   0:00 klogd -x
root      1534  0.0  0.1  1904  560 ?        S    Dec22   0:00 rpc.idmapd
root      1614  0.0  0.1  2888  740 ?        S    Dec22   0:00 /usr/sbin/smartd
root      1623  0.0  0.0  1552  464 ?        S    Dec22   0:00 /usr/sbin/acpid
root      1668  0.0  0.3  5108 1460 ?        S    Dec22   0:00 /usr/sbin/sshd
root      1681  0.0  0.1  3748  808 ?        S    Dec22   0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
root      1777  0.0  0.1  2704  644 ?        S    Dec22   0:00 crond
cpanel    1973  0.0  0.4 34948 1964 ?        S    Dec22   0:00 /usr/bin/stunnel-4.04local /usr/local/cpanel/etc/stunnel/defau
root      1997  0.0  0.0  1420  476 ?        S    Dec22   0:00 /usr/sbin/portsentry -tcp
root      2015  0.0  0.0  2444  320 ?        S    Dec22   0:00 mdadm --monitor --scan
root      2030  0.0  0.0  3128  344 tty1     S    Dec22   0:00 /sbin/mingetty tty1
root      2031  0.0  0.0  2872  344 tty2     S    Dec22   0:00 /sbin/mingetty tty2
root      2032  0.0  0.0  2556  340 tty3     S    Dec22   0:00 /sbin/mingetty tty3
root      2033  0.0  0.0  2260  340 tty4     S    Dec22   0:00 /sbin/mingetty tty4
root      2034  0.0  0.0  2192  344 tty5     S    Dec22   0:00 /sbin/mingetty tty5
root      2035  0.0  0.0  2004  340 tty6     S    Dec22   0:00 /sbin/mingetty tty6
root      2402  0.0  0.2  2056 1028 ?        S    Dec22   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-fi
mysql     2422  0.0  3.3 33220 15972 ?       S    Dec22   0:01 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2423  0.0  3.3 33220 15972 ?       S    Dec22   0:01 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2424  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2425  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2426  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2427  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2428  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2429  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2430  0.0  3.3 33220 15972 ?       S    Dec22   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
mysql     2431  0.0  3.3 33220 15972 ?       S    Dec22   0:01 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=m
root      2623  0.0  3.0 16116 14792 ?       S    Dec22   0:00 /usr/local/cpanel/whostmgr/bin/whostmgr2 ./dobuildapache
root      2624  0.0  0.2  3272 1188 ?        S    Dec22   0:00 /usr/bin/perl /scripts/easyapache
root      2643  0.0  0.2  2060 1056 ?        S    Dec22   0:00 sh buildapache.sea
root      2695  0.0  0.1  2056  948 ?        S    Dec22   0:00 /bin/sh ./bs
root      2697  0.0  0.3  3536 1456 ?        S    Dec22   0:00 /usr/bin/perl ./buildapache
root      2698  0.0  0.2  2060  976 ?        S    Dec22   0:00 /bin/sh ./checkrpms
root      2727  0.0  0.3  3536 1708 ?        S    Dec22   0:00 /usr/bin/perl /scripts/safeyum -y install wget
root      2731  0.0  1.5 11804 7504 ?        S    Dec22   0:00 /usr/bin/python /usr/bin/yum -y install wget
root     17436  0.0  0.9  9304 4376 ?        S    14:36   0:00 /usr/local/apache/bin/httpd -DSSL
nobody   19352  0.2  2.7 19864 13396 ?       S    16:10   0:51 /usr/local/apache/bin/httpd -DSSL
nobody   19357  0.2  2.7 19960 13504 ?       S    16:10   0:47 /usr/local/apache/bin/httpd -DSSL
root     22885  0.0  0.4  8768 2164 ?        S    20:13   0:00 sshd: [email protected]/0
root     22888  0.0  0.2  5616 1340 pts/0    S    20:14   0:00 -bash
root     23167  0.0  0.6  7580 3260 ?        S    20:22   0:00 chkservd
nobody   23870  0.3  2.1 14712 10340 ?       S    20:52   0:01 /usr/local/apache/bin/httpd -DSSL
nobody   23871  0.3  2.1 14892 10524 ?       S    20:52   0:01 /usr/local/apache/bin/httpd -DSSL
nobody   23872  0.2  2.6 17076 12696 ?       S    20:52   0:01 /usr/local/apache/bin/httpd -DSSL
nobody   23873  0.1  1.4 11580 7172 ?        S    20:52   0:00 /usr/local/apache/bin/httpd -DSSL
nobody   23874  0.0  1.8 13360 8884 ?        S    20:52   0:00 /usr/local/apache/bin/httpd -DSSL
nobody   23875  0.1  1.8 13340 8868 ?        S    20:52   0:00 /usr/local/apache/bin/httpd -DSSL
nobody   23878  0.5  2.1 14628 10264 ?       S    20:52   0:02 /usr/local/apache/bin/httpd -DSSL
nobody   23880  0.4  2.3 15828 11468 ?       S    20:52   0:01 /usr/local/apache/bin/httpd -DSSL
nobody   23881  0.2  2.2 15508 11096 ?       S    20:52   0:01 /usr/local/apache/bin/httpd -DSSL
nobody   23882  0.2  2.1 14964 10596 ?       S    20:52   0:01 /usr/local/apache/bin/httpd -DSSL
nobody   23937  0.3  2.1 14840 10480 ?       S    20:54   0:01 /usr/local/apache/bin/httpd -DSSL
named    24037  0.0  0.5 36148 2724 ?        S    20:55   0:00 /usr/sbin/named -u named
root     24190  0.0  0.3  3404 1812 ?        S    20:55   0:00 antirelayd
root     24262  0.0  1.2  7920 6076 ?        SN   20:58   0:00 cpanellogd - sleeping for logs
root     24268  0.0  1.3  7812 6508 ?        S    20:58   0:00 cppop - accepting on port 110
root     24285  0.0  1.4  8160 6832 ?        S    20:58   0:00 cpsrvd - waiting for connections
root     24301  0.1  0.4  8768 2168 ?        S    20:58   0:00 sshd: [email protected]/1
root     24303  0.0  0.2  6540 1364 pts/1    S    20:59   0:00 -bash
root     24370  0.0  0.1  3092  740 pts/1    R    21:00   0:00 ps aux

That's a ps aux output on the server.

Help? can't see anything that should be killing it.

 

[morpheous]

Seems it might be our colocaters.... is there a program available to check up/down speeds from the server?

 

[haze]

Looks to me as if you need a bit of optimization there. From the output you provided, looks to me as if you might be swapping to disk pretty hard. Here's a few suggestions,

Log into WHM, and under the "Security" heading click on "Modify Apache Memory Usage"

Next, take a look at your /etc/my.cnf consult the mysql docs for information on variables you can tweak, what they do and hour they work: http://dev.mysql.com/doc/mysql/en/Server_system_variables.html

There are also example mysql configuration files here:
/usr/share/mysql
They are:
my-huge.cnf
my-large.cnf
my-medium.cnf
my-small.cnf

I wouldn't suggest using any of them straight out, but they are well commented and will help you in building your own.

I'd then try and figure out why your named process is so large. Are you using your local name servers as reseolvers? If so, this is a serious security risk, ask your server provider for their DNS resolvers that you can use on your server.

I'd also recommend you do an audit on your system and remove any services you don't use or need, such as "mdadm" and "rpc.idmapd". Having these installed creates 1. A possible security risk and 2. uses resources you otherwise need.

The above is just a starting point. I'd recommend if you have little to no experience with working on a linux box, to look into hireing someone that does and can help you fix up the box and keep it secure.

Remember to make sure all your OS updates are installed, including the latest kernel.

 

[morpheous]

I killed named and everything else that's not apache/mysql.

It's still slow- if slower, so i'm definate on connection now pretty much... our host's site is slow too. I can't even load WHM/cPanel now... or shell.

Hit the memory usage button.... limited apache to 153MB

 

[chirpy]

If you're still having problems and Beau's comments don't mean anything to you, then I would suggest that you hire a server administrator (or contact your NOC if they provide OS support) to investigate the problem for you. Tracking something like this down requires an understanding of your OS and the apps running on it.

 

[morpheous]

Fixed it- turns out our hosts had decided to get hit with a DoS attack without telling us :D


Works fine now. Thanks everyone! Learned a few tricks even if they were for the wrong problem.

Cheers,
Morph