The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

MySQL Overload

Discussion in 'General Discussion' started by Salman75, Mar 23, 2005.

  1. Salman75

    Salman75 Well-Known Member

    Joined:
    Jan 20, 2004
    Messages:
    102
    Likes Received:
    0
    Trophy Points:
    16
    My server shows this:

    load average: 5.15, 4.98, 3.92

    Code:
    12359 mysql     21   0 21688  21M  1664 S    13.8  2.1   0:01   1 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    12358 mysql     21   0 21688  21M  1664 S    12.7  2.1   0:01   0 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    12361 mysql     20   0 21688  21M  1664 S    12.7  2.1   0:01   1 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    12360 mysql     21   0 21688  21M  1664 S    12.3  2.1   0:01   1 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    12340 mysql     16   0 21688  21M  1664 R    10.6  2.1   0:02   1 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    12379 mysql     23   0 21688  21M  1664 S     6.7  2.1   0:00   0 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    10755 nobody    16   0 16048  10M  3640 S     0.4  1.0   0:04   1 /usr/local/apache/bin/httpd -DSSL
    30646 nobody    15   0 14440 9480  3492 S     0.2  0.9   0:09   1 /usr/local/apache/bin/httpd -DSSL
    12354 nobody    15   0 12680 6900  3460 S     0.2  0.6   0:00   0 /usr/local/apache/bin/httpd -DSSL
    12364 nobody    15   0 12620 6676  3468 S     0.2  0.6   0:00   1 /usr/local/apache/bin/httpd -DSSL
    12372 nobody    15   0 12836 6936  3476 S     0.2  0.6   0:00   0 /usr/local/apache/bin/httpd -DSSL
    11709 nobody    16   0 54088  47M  3592 S     0.1  4.7   0:01   0 /usr/local/apache/bin/httpd -DSSL
    12363 nobody    18   0 12628 6816  3408 S     0.1  0.6   0:00   0 /usr/local/apache/bin/httpd -DSSL
    12365 nobody    15   0 12412 6432  3392 S     0.1  0.6   0:00   0 /usr/local/apache/bin/httpd -DSSL
    

    The server load is constantly v high. Also have this from WHM:

    Code:
    Top Process %CPU 99.9 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.com.pid --skip-locking 
    Top Process %CPU 95.2 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.com.pid --skip-locking 
    Top Process %CPU 87.2 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.com.pid --skip-locking 
    
    I cant seem to understand the problem. Why is the server load so high and why so many mysql processes?

    Salman
     
    #1 Salman75, Mar 23, 2005
    Last edited: Mar 23, 2005
  2. AndyReed

    AndyReed Well-Known Member
    PartnerNOC

    Joined:
    May 29, 2004
    Messages:
    2,222
    Likes Received:
    3
    Trophy Points:
    38
    Location:
    Minneapolis, MN
    To find out what casuing the high load on your server, SSH to your server and run this command at the prompt:
    ps aufx

    High load can be a script related, DOS or UDP attack, or a Web site with very high traffic. Overall, it is really hard to say without looking at the processes running on your server.
     
  3. Salman75

    Salman75 Well-Known Member

    Joined:
    Jan 20, 2004
    Messages:
    102
    Likes Received:
    0
    Trophy Points:
    16
    Thanks. I did that. here is the result:

    Code:
    root         3  0.0  0.0     0    0 ?        SW   Mar22   0:00 [migration/1]
    root         2  0.0  0.0     0    0 ?        SW   Mar22   0:00 [migration/0]
    root         1  0.0  0.0  1520  440 ?        S    Mar22   0:05 init
    root         4  0.0  0.0     0    0 ?        SW   Mar22   0:00 [keventd]
    root         5  0.0  0.0     0    0 ?        SWN  Mar22   0:00 [ksoftirqd/0]
    root         6  0.0  0.0     0    0 ?        SWN  Mar22   0:00 [ksoftirqd/1]
    root         9  0.0  0.0     0    0 ?        SW   Mar22   0:00 [bdflush]
    root         7  0.0  0.0     0    0 ?        SW   Mar22   0:31 [kswapd]
    root         8  0.0  0.0     0    0 ?        SW   Mar22   1:13 [kscand]
    root        10  0.0  0.0     0    0 ?        SW   Mar22   0:04 [kupdated]
    root        11  0.0  0.0     0    0 ?        SW   Mar22   0:00 [mdrecoveryd]
    root        18  0.0  0.0     0    0 ?        SW   Mar22   0:00 [scsi_eh_0]
    root        19  0.0  0.0     0    0 ?        SW   Mar22   0:00 [scsi_eh_1]
    root        22  0.0  0.0     0    0 ?        SW   Mar22   0:28 [kjournald]
    root        77  0.0  0.0     0    0 ?        SW   Mar22   0:00 [khubd]
    root      2104  0.0  0.0     0    0 ?        SW   Mar22   0:00 [kjournald]
    root      2252  0.0  0.0     0    0 ?        SW   Mar22   0:00 [kjournald]
    root      4438  0.0  0.0  1584  536 ?        S    Mar22   0:06 syslogd -m 0
    root      4443  0.0  0.0  1532  440 ?        S    Mar22   0:00 klogd -x
    named     5589  0.0  0.3 49596 3892 ?        S    Mar22   0:28 /usr/sbin/named -u named
    root      5603  0.0  0.0  3648 1008 ?        S    Mar22   0:00 /usr/sbin/sshd
    root     13891  0.0  0.1  6864 1960 ?        S    14:26   0:00  \_ sshd: root@pts/0
    root     13926  0.0  0.1  4256 1364 pts/0    S    14:26   0:00      \_ -bash
    root     14146  0.0  0.0  2868  872 pts/0    R    14:28   0:00          \_ ps aufx
    root      5617  0.0  0.0  2132  708 ?        S    Mar22   0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
    lourdas  14038  0.0  0.1  2640 1228 ?        S    14:27   0:00  \_ imapd
    lourdas  14039  0.0  0.1  2632 1196 ?        S    14:27   0:00  \_ imapd
    root      5681  0.0  0.1  7020 1668 ?        S    Mar22   0:01 chkservd
    mailnull  5737  0.0  0.1  6600 1100 ?        S    Mar22   0:02 /usr/sbin/exim -bd -q60m
    mailnull  5741  0.0  0.0  6564  932 ?        S    Mar22   0:00 /usr/sbin/exim -tls-on-connect -bd -oX 465
    root      5748  0.0  0.1  3448 1108 ?        S    Mar22   0:39 antirelayd
    root      5792  0.0  0.2 23608 2272 ?        S    Mar22   0:22 /usr/local/apache/bin/httpd -DSSL
    root      7150  0.0  0.1  7844 1264 ?        S    13:51   0:00  \_ /usr/bin/perl /usr/local/cpanel/bin/leechprotect
    nobody    7151  0.4  1.3 28092 13732 ?       S    13:51   0:09  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7152  0.2  4.9 65804 50924 ?       S    13:51   0:05  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7154  0.1  4.8 65736 50188 ?       S    13:51   0:02  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7155  0.3  1.4 29432 14948 ?       S    13:51   0:07  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7157  0.3  4.9 65896 51164 ?       S    13:51   0:08  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7166  0.1  1.2 27292 12412 ?       S    13:51   0:03  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7170  0.2  4.9 66080 51196 ?       S    13:51   0:04  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7172  0.2  1.0 25448 10492 ?       S    13:51   0:05  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7174  0.4  4.9 66016 51020 ?       S    13:51   0:10  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7176  0.2  3.4 65800 35760 ?       S    13:51   0:05  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7178  0.1  1.1 27508 12104 ?       S    13:51   0:02  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7180  0.2  1.2 28004 13228 ?       S    13:51   0:06  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7181  0.4  1.1 26740 11800 ?       S    13:51   0:09  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7183  0.2  1.3 28824 13616 ?       S    13:51   0:05  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7186  0.2  5.0 66072 51300 ?       S    13:51   0:04  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7189  0.2  1.4 29692 14892 ?       S    13:51   0:06  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7190  0.1  0.9 25416 10032 ?       S    13:51   0:02  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7191  0.3  1.3 28732 13492 ?       S    13:51   0:08  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7193  0.2  1.1 27268 12056 ?       S    13:51   0:05  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7354  0.3  1.2 27816 12708 ?       S    13:52   0:07  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7482  0.3  1.3 28348 13396 ?       S    13:53   0:06  \_ /usr/local/apache/bin/httpd -DSSL
    nobody    7825  0.3  4.9 65832 51216 ?       S    13:58   0:06  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   12128  0.1  4.8 65796 49424 ?       S    14:11   0:01  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   12211  0.0  0.8 24248 8912 ?        S    14:12   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   12719  0.3  1.0 25400 10796 ?       S    14:17   0:02  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13036  0.7  1.4 28800 14752 ?       S    14:19   0:03  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13818  0.0  0.8 24508 8512 ?        S    14:25   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13819  0.1  0.9 25240 9236 ?        S    14:25   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13820  0.2  0.9 26376 9816 ?        S    14:25   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13823  0.2  1.0 26876 10612 ?       S    14:25   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13853  0.0  0.8 24252 9208 ?        S    14:26   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    nobody   13856  0.0  1.0 25336 10316 ?       S    14:26   0:00  \_ /usr/local/apache/bin/httpd -DSSL
    root      5805  0.0  0.0  1564  572 ?        S    Mar22   0:00 crond
    root      6532  0.0  0.0  6740  532 ?        S    Mar22   0:00 /usr/bin/perl /usr/local/bin/ipalert_statd
    root      6567  0.0  0.0  1532  392 ?        S    Mar22   0:00 /usr/sbin/portsentry -tcp
    root      6585  0.0  0.1  6056 1080 ?        S    Mar22   0:00 pure-ftpd (SERVER)
    harisp   28930  0.0  0.1  7216 1412 ?        S    12:59   0:00  \_ pure-ftpd (IDLE)
    busbys   11219  0.0  0.1  7180 1376 ?        S    14:06   0:00  \_ pure-ftpd (IDLE)
    grnpgs   12863  0.0  0.1  7180 1416 ?        S    14:18   0:00  \_ pure-ftpd (IDLE)
    manolo   13009  0.0  0.1  6156 1420 ?        S    14:19   0:00  \_ pure-ftpd (IDLE)
    root      6597  0.0  0.0  5596  820 ?        S    Mar22   0:00 /usr/sbin/pure-authd -s /var/run/ftpd.sock -r /usr/sbin/pureau
    root      6601  0.0  0.0     0    0 ?        SW   Mar22   0:01 [loop0]
    root      6602  0.0  0.0     0    0 ?        SW   Mar22   0:01 [kjournald]
    root      6607  0.0  0.0  1500  348 tty1     S    Mar22   0:00 /sbin/mingetty tty1
    root      6608  0.0  0.0  1500  348 tty2     S    Mar22   0:00 /sbin/mingetty tty2
    root      6609  0.0  0.0  1500  348 tty3     S    Mar22   0:00 /sbin/mingetty tty3
    root      6610  0.0  0.0  1500  348 tty4     S    Mar22   0:00 /sbin/mingetty tty4
    root      6611  0.0  0.0  1500  348 tty5     S    Mar22   0:00 /sbin/mingetty tty5
    root      6612  0.0  0.0  1500  348 tty6     S    Mar22   0:00 /sbin/mingetty tty6
    postgres  7241  0.0  0.1  9600 1196 ?        S    Mar22   0:00 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
    postgres  7243  0.0  0.1  9160 1092 ?        S    Mar22   0:00  \_ postgres: stats buffer process
    postgres  7244  0.0  0.1  8168 1144 ?        S    Mar22   0:00      \_ postgres: stats collector process
    root     14682  0.2  1.0 19932 10456 ?       SN   Mar22   2:38 cpanellogd - updating bandwidth for carolin
    root     14145  0.0  0.0  1560  516 ?        RN   14:28   0:00  \_ chown -R venue:venue /home/venue/tmp
    root     14708  0.0  0.2  8164 2088 ?        S    Mar22   0:12 cppop - accepting on port 110
    mailman  14722  0.0  0.0  7208  920 ?        S    Mar22   0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/mailma
    mailman  14723  0.0  0.1  7184 1868 ?        S    Mar22   0:08  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14724  0.0  0.1  7136 1856 ?        S    Mar22   0:09  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14726  0.0  0.1  7124 1824 ?        S    Mar22   0:08  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14727  0.0  0.1  7292 1940 ?        S    Mar22   0:08  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14728  0.0  0.1  7224 1832 ?        S    Mar22   0:08  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14729  0.0  0.2  7456 2064 ?        S    Mar22   0:09  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14730  0.0  0.1  7304 1856 ?        S    Mar22   0:08  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    mailman  14731  0.0  0.1  7188 1388 ?        S    Mar22   0:00  \_ /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qr
    nobody   14732  0.0  0.0  1736  364 ?        S    Mar22   0:00 /usr/local/cpanel/bin/startmelange
    
     
  4. Salman75

    Salman75 Well-Known Member

    Joined:
    Jan 20, 2004
    Messages:
    102
    Likes Received:
    0
    Trophy Points:
    16
    The board wouldnt let me post the complete result in one post. SO here is the second part:

    Code:
    nobody   14738  0.0  0.0  4060  496 ?        S    Mar22   0:00 entropychat
    root      2613  0.0  0.0 24888  652 ?        S    01:22   0:00 /usr/bin/spamd -d --allowed-ips=127.0.0.1 --pidfile=/var/run/s
    root      2628  0.0  2.1 34064 22340 ?       S    01:22   0:45  \_ spamd child
    root      2629  0.0  2.1 33496 21772 ?       S    01:22   0:41  \_ spamd child
    root      2630  0.0  2.1 32648 21864 ?       S    01:22   0:40  \_ spamd child
    root      2631  0.0  2.2 31984 22676 ?       S    01:22   0:44  \_ spamd child
    root      2632  0.0  2.1 32904 22404 ?       S    01:22   0:41  \_ spamd child
    cpanel    3022  0.0  0.0 35744 1020 ?        S    01:23   0:00 /usr/bin/stunnel-4.04local /usr/local/cpanel/etc/stunnel/defau
    root      3025  0.0  0.1  9520 1264 ?        S    01:23   0:03 cpsrvd - waiting for connections
    root     14120  0.2  0.1  9528 1788 ?        S    14:28   0:00  \_ whostmgrd - serving
    root     14127  0.0  0.1  9528 1776 ?        S    14:28   0:00  \_ cpaneld - serving
    root      6152  0.0  0.0  4196  952 ?        S    11:03   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-fi
    mysql     6173  0.0  0.8 54088 9076 ?        S    11:03   0:01  \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --us
    mysql     6175  0.0  0.8 54088 9076 ?        S    11:03   0:01      \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql
    mysql     6176  0.0  0.8 54088 9076 ?        S    11:03   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6177  0.0  0.8 54088 9076 ?        S    11:03   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6178  0.0  0.8 54088 9076 ?        S    11:03   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6179  0.0  0.8 54088 9076 ?        S    11:03   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6182  0.0  0.8 54088 9076 ?        S    11:03   0:02          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6183  0.0  0.8 54088 9076 ?        S    11:03   0:01          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6184  0.0  0.8 54088 9076 ?        S    11:03   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     6185  0.0  0.8 54088 9076 ?        S    11:03   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     7550  0.0  0.8 54088 9076 ?        S    11:10   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql     7164  0.0  0.8 54088 9076 ?        S    13:51   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    12522  0.4  0.8 54088 9076 ?        S    14:15   0:03          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    12594  0.4  0.8 54088 9076 ?        S    14:15   0:03          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    12604  0.4  0.8 54088 9076 ?        S    14:16   0:03          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    12659  0.3  0.8 54088 9076 ?        S    14:16   0:02          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    13031  0.6  0.8 54088 9076 ?        S    14:19   0:03          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    14072 24.9  0.8 54088 9076 ?        S    14:27   0:06          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    14085 31.3  0.8 54088 9076 ?        S    14:27   0:06          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    14099 14.8  0.8 54088 9076 ?        S    14:28   0:02          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    14100 15.6  0.8 54088 9076 ?        S    14:28   0:02          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    14102 21.5  0.8 54088 9076 ?        R    14:28   0:02          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mysql    14110 16.8  0.8 54088 9076 ?        S    14:28   0:01          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/my
    mailnull  7549  0.0  0.1  8808 1940 ?        S    11:10   0:01 /usr/bin/perl /usr/local/cpanel/bin/eximstats
    root     14133  6.0  0.3  7548 3896 ?        S    14:28   0:00 /usr/sbin/exim -Mc 1DECSe-0003fw-57
    mailnull 14136  0.0  0.3  7548 3948 ?        S    14:28   0:00  \_ /usr/sbin/exim -Mc 1DECSe-0003fw-57
    
     
    #4 Salman75, Mar 23, 2005
    Last edited: Mar 23, 2005
  5. Secret Agent

    Secret Agent Guest

    I'm having the same trouble

    Code:
    User Domain %CPU %MEM Mysql Processes 
    mysql  31.84 433.41 0.0 
    Top Process %CPU 42.7 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.mydomain.com.pid --skip-locking --port3306 
    Top Process %CPU 42.5 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.mydomain.com.pid --skip-locking --port3306 
    Top Process %CPU 41.3 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/server.mydomain.com.pid --skip-locking --port3306 
    
    Server load went down after i had no choice but to reboot completely, so I cannot show ps auxf results right now. In the meantime, any suggestions?
     
  6. DigitalN

    DigitalN Well-Known Member

    Joined:
    Sep 23, 2004
    Messages:
    420
    Likes Received:
    1
    Trophy Points:
    18
    How much memory do you have in the server? How much is being used?

    # free -m

    You may need to add some more, do you have swap space , is it being used?

    Check out MyTop to see what is using MySQL resources, http://jeremy.zawodny.com/mysql/mytop/

    http://dev.mysql.com has lots of information on how to optimise MySQL servers.
     
  7. Secret Agent

    Secret Agent Guest

    Code:
    root@server [~]# free -m
                 total       used       free     shared    buffers     cached
    Mem:          2027       1959         67          0        259       1113
    -/+ buffers/cache:        586       1440
    Swap:         1764          1       1763
    
    I have 2GB ECC Memory
     
  8. ladierainy

    ladierainy Well-Known Member

    Joined:
    Dec 1, 2003
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    6
    This is mine - just had my server unplugged for an outbound attack .... I know some of the attack scripts are showing up here ... how can I delete them and then protect against them in the future?

    USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
    root 2 0.0 0.0 0 0 ? SW 12:40 0:00 [migration/0]
    root 1 0.0 0.0 1524 432 ? S 12:40 0:04 init [3]
    root 3 0.0 0.0 0 0 ? SW 12:40 0:00 [keventd]
    root 4 0.0 0.0 0 0 ? SWN 12:40 0:00 [ksoftirqd/0]
    root 7 0.0 0.0 0 0 ? SW 12:40 0:00 [bdflush]
    root 5 0.0 0.0 0 0 ? SW 12:40 0:02 [kswapd]
    root 6 0.0 0.0 0 0 ? SW 12:40 0:02 [kscand]
    root 8 0.0 0.0 0 0 ? SW 12:40 0:01 [kupdated]
    root 9 0.0 0.0 0 0 ? SW 12:40 0:00 [mdrecoveryd]
    root 13 0.0 0.0 0 0 ? SW 12:40 0:17 [kjournald]
    root 69 0.0 0.0 0 0 ? SW 12:40 0:00 [khubd]
    root 765 0.0 0.0 0 0 ? SW 12:40 0:00 [kjournald]
    root 1521 0.0 0.0 1576 524 ? S 12:40 0:01 syslogd -m 0
    root 1525 0.0 0.0 1528 356 ? S 12:40 0:00 klogd -x
    root 1612 0.0 0.0 3652 600 ? S 12:40 0:00 /usr/sbin/sshd
    root 30016 0.0 0.1 6864 1712 ? S 18:36 0:00 \_ sshd: root@pt
    root 30020 0.0 0.1 4260 1400 pts/0 S 18:36 0:00 \_ -bash
    root 1019 0.0 0.0 2868 808 pts/0 R 18:59 0:00 \_ ps au
    root 1626 0.0 0.0 2140 648 ? S 12:40 0:00 xinetd -stayalive
    root 1651 0.0 0.2 7444 2092 ? S 12:40 0:01 chkservd
    mailnull 1707 0.0 0.0 6628 648 ? S 12:40 0:00 /usr/sbin/exim -b
    root 30238 0.0 0.1 6592 1688 ? S 18:40 0:00 \_ /usr/sbin/exi
    root 719 0.0 0.3 7292 3300 ? S 18:54 0:00 \_ /usr/sbin
    mailnull 720 0.0 0.3 7292 3344 ? S 18:54 0:00 \_ /usr/
    mailnull 1711 0.0 0.0 6588 628 ? S 12:40 0:00 /usr/sbin/exim -b
    mailnull 1715 0.0 0.0 6588 484 ? S 12:40 0:00 /usr/sbin/exim -t
    root 1721 0.0 0.1 3452 1048 ? S 12:40 0:07 antirelayd
    root 1776 0.0 0.1 13816 1508 ? S 12:40 0:02 /usr/local/apache
    nobody 1831 0.2 0.9 20120 9968 ? S 12:40 1:02 \_ /usr/local/ap
    nobody 1832 0.2 0.9 20248 10036 ? S 12:40 0:50 \_ /usr/local/ap
    nobody 1833 0.2 0.9 20188 9572 ? S 12:40 1:03 \_ /usr/local/ap
    nobody 1834 0.2 0.9 20088 10028 ? S 12:40 0:49 \_ /usr/local/ap
    nobody 1835 0.2 0.9 20036 9812 ? S 12:40 0:52 \_ /usr/local/ap
    nobody 1860 0.2 0.9 19708 9692 ? S 12:41 0:58 \_ /usr/local/ap
    nobody 1875 0.2 0.9 19684 9948 ? S 12:41 1:00 \_ /usr/local/ap
    nobody 1876 0.2 1.0 20344 10492 ? S 12:41 0:58 \_ /usr/local/ap
    nobody 1898 0.3 0.8 19336 9004 ? S 12:41 1:12 \_ /usr/local/ap
    nobody 1899 0.3 1.0 20176 10328 ? S 12:41 1:09 \_ /usr/local/ap
    nobody 31598 0.2 0.9 18644 9336 ? S 18:50 0:01 \_ /usr/local/ap
    nobody 31660 0.1 0.9 18752 9612 ? S 18:51 0:00 \_ /usr/local/ap
    nobody 31879 0.0 0.2 13952 2840 ? S 18:51 0:00 \_ /usr/local/ap
    nobody 31880 0.0 0.5 15576 5912 ? S 18:51 0:00 \_ /usr/local/ap
    nobody 31881 0.0 0.5 15604 5748 ? S 18:51 0:00 \_ /usr/local/ap
    nobody 31883 0.1 0.9 18488 9304 ? S 18:51 0:00 \_ /usr/local/ap
    nobody 32224 0.0 0.2 13908 2396 ? S 18:53 0:00 \_ /usr/local/ap
    nobody 962 0.0 0.2 13816 2556 ? S 18:57 0:00 \_ /usr/local/ap
    nobody 963 0.4 0.7 16972 7212 ? S 18:57 0:00 \_ /usr/local/ap
    nobody 1001 0.0 0.2 13816 2360 ? S 18:58 0:00 \_ /usr/local/ap
    nobody 1002 0.0 0.2 13816 2716 ? S 18:59 0:00 \_ /usr/local/ap
    nobody 1005 0.0 0.2 13816 2684 ? S 18:59 0:00 \_ /usr/local/ap
    nobody 1006 1.0 0.6 16664 6916 ? S 18:59 0:00 \_ /usr/local/ap
    root 1812 0.0 1.1 24928 11636 ? S 12:40 0:01 /usr/bin/spamd -d
    root 2040 0.1 2.1 27680 22136 ? S 12:41 0:44 \_ spamd child
    root 2062 0.3 2.1 27988 22464 ? S 12:41 1:28 \_ spamd child
    root 2077 0.2 2.1 27844 22364 ? S 12:41 0:45 \_ spamd child
    root 2090 0.1 2.2 28496 23092 ? S 12:41 0:42 \_ spamd child
    root 2092 0.1 2.1 27820 22444 ? S 12:41 0:45 \_ spamd child
    root 1836 0.0 0.0 1560 564 ? S 12:40 0:00 crond
    named 1877 0.0 0.1 36408 2032 ? S 12:41 0:00 /usr/sbin/named -
    root 2193 0.0 0.0 6024 792 ? S 12:41 0:00 pure-ftpd (SERVER
    closet 29177 0.0 0.1 6188 1056 ? S 18:27 0:00 \_ pure-ftpd (ID
    nobody 2198 0.0 0.1 4064 1320 ? S 12:41 0:00 entropychat
    root 2200 0.0 0.0 5588 580 ? S 12:41 0:00 /usr/sbin/pure-au
    nobody 2205 0.0 0.0 1728 596 ? S 12:41 0:00 /usr/local/cpanel
    cpanel 2231 0.0 0.1 34556 1472 ? S 12:41 0:00 /usr/bin/stunnel-
    root 2244 0.0 0.0 3552 508 ? S 12:41 0:00 rhnsd --interval
    root 2275 0.0 0.0 1532 428 ? S 12:41 0:00 /usr/sbin/portsen
    root 2294 0.0 0.0 1504 388 tty1 S 12:41 0:00 /sbin/mingetty tt
    root 2295 0.0 0.0 1500 392 tty2 S 12:41 0:00 /sbin/mingetty tt
    root 2296 0.0 0.0 1508 392 tty3 S 12:41 0:00 /sbin/mingetty tt
    root 2297 0.0 0.0 1508 392 tty4 S 12:41 0:00 /sbin/mingetty tt
    root 2298 0.0 0.0 1496 388 tty5 S 12:41 0:00 /sbin/mingetty tt
    root 2299 0.0 0.0 1516 392 tty6 S 12:41 0:00 /sbin/mingetty tt
    root 2300 0.0 0.0 1520 396 ttyS0 S 12:41 0:00 /sbin/agetty -L 9
    root 2341 0.0 0.1 4256 1152 ? S 12:41 0:00 /bin/sh /usr/bin/
    mysql 2369 0.0 1.9 33444 19932 ? S 12:41 0:02 \_ /usr/sbin/mys
    mysql 2371 0.0 1.9 33444 19932 ? S 12:41 0:03 \_ /usr/sbin
    mysql 2372 0.0 1.9 33444 19932 ? S 12:41 0:00 \_ /usr/
    mysql 2373 0.0 1.9 33444 19932 ? S 12:41 0:00 \_ /usr/
    mysql 2374 0.0 1.9 33444 19932 ? S 12:41 0:00 \_ /usr/
    mysql 2375 0.0 1.9 33444 19932 ? S 12:41 0:00 \_ /usr/
    mysql 2376 0.0 1.9 33444 19932 ? S 12:41 0:07 \_ /usr/
    mysql 2377 0.0 1.9 33444 19932 ? S 12:41 0:04 \_ /usr/
    mysql 2378 0.0 1.9 33444 19932 ? S 12:41 0:00 \_ /usr/
    mysql 2380 0.0 1.9 33444 19932 ? S 12:41 0:00 \_ /usr/
    mysql 9165 0.0 1.9 33444 19932 ? S 12:58 0:00 \_ /usr/
    mysql 9381 0.0 1.9 33444 19932 ? S 13:00 0:00 \_ /usr/
    mysql 9385 0.0 1.9 33444 19932 ? S 13:00 0:00 \_ /usr/
    root 9164 0.0 0.4 9040 4340 ? S 12:58 0:05 /usr/bin/perl /us
    mailman 28351 0.0 0.3 7232 3276 ? S 18:21 0:00 /usr/bin/python2
    mailman 28354 0.0 0.3 7188 3444 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28355 0.0 0.3 7156 3468 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28356 0.0 0.3 7132 3444 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28358 0.0 0.3 7196 3448 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28359 0.0 0.3 7232 3480 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28361 0.0 0.3 7272 3536 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28362 0.0 0.3 7196 3444 ? S 18:21 0:00 \_ /usr/bin/pyth
    mailman 28363 0.0 0.3 7188 3248 ? S 18:21 0:00 \_ /usr/bin/pyth
    root 28380 0.1 0.7 10772 7952 ? SN 18:21 0:04 cpanellogd - slee
    root 28395 0.0 0.3 7028 3348 ? S 18:21 0:00 cppop - accepting
    root 28399 0.0 0.3 8244 3932 ? S 18:21 0:00 cpsrvd - waiting
     
    #8 ladierainy, Apr 25, 2005
    Last edited: Apr 25, 2005
  9. ladierainy

    ladierainy Well-Known Member

    Joined:
    Dec 1, 2003
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    6
    This is what ev1servers informaed me of:

    nobody 2597 0.0 0.0 1488 20 ? S Apr24 0:00 ./r0nin.htm
    nobody 6183 0.0 0.0 1488 96 ? S Apr24 0:00 \_ ./r0nin.htm
    nobody 6184 0.0 0.0 2208 588 ttyp1 S Apr24 0:00 \_ sh -i
    nobody 31556 72.0 0.2 3760 2068 ttyp1 R 11:26 51:42 \_ perl
    udp.pl 66.80.139.72 0 0
    nobody 3260 0.0 0.0 1152 860 ? S Apr24 0:56 ./ntpd

    You will need to remove the attack scripts located via /tmp, /var/tmp and /home/*******/public_html/********/modules/coppermine

    -rw-r--r-- 1 nobody nobody 6 Apr 20 13:02 \377
    -rw-r--r-- 1 nobody nobody 1946 Apr 20 13:02 bot.log
    -rw-r--r-- 1 nobody nobody 252 Apr 20 13:02 bot.state
    -rw-r--r-- 1 nobody nobody 252 Apr 20 13:02 bot.state~
    -rw-r--r-- 1 nobody nobody 1267 Apr 20 13:01 franca.txt
    -rwxrwxrwx 1 nobody nobody 19138 Oct 22 2004 hide*
    -rw-r--r-- 1 nobody nobody 7858 Apr 20 13:01 hide.zip
    drwxr-xr-x 2 nobody nobody 4096 Apr 24 03:38 .iro/
    -rwxrwxrwx 1 nobody nobody 235038 Apr 20 13:00 ps*
    drwxr-xr-x 2 nobody nobody 4096 Apr 21 05:13 .up/

    Please refer to the directory /root/**.**.**.**-2/ for more information on the results of this investigation.

    I can find the stuff just not sure how to successfully remove itand then block it.

    The gallery they I assume entered through has been deleted - and all the files within that module.

    I know there is more I need to kill off - can anyone help me or at least doect me to a place where I can read what steps I need to be taking to fix this?

    Thanks!
     
  10. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    I would suggest you search the forum on advice for securing your server, there are some good threads if you use the search.

    As for cleaing it up, you need to stop any processes accessing those files. You could use:

    lsof | grep /tmp

    To list all the files open in the directory and then trawl through them. then delete all of those files from /tmp. then read up on securing your server in the forums. you should also check that yo have not suffered a root compromise and that there aren't compromise files in other places on the server, e.g.:

    /var/tmp
    /dev/shm
    /usr/local/apache/proxy/

    If you're not comfortable doing this, you can always hire someone to do it for you.
     
  11. ladierainy

    ladierainy Well-Known Member

    Joined:
    Dec 1, 2003
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    6
    I've been doing searches thorugh the forums, always do before I ask - and I post when I find myself wading through a ton of stuff that has little or no bearing - just to see if I can get an answer quicker than me just searching.

    I'll still keep looking for my own answer - just hoping for help along the way.

    I could pay someone to do it - but then I wouldn't be learning how to do this - and not really sure how you find a trustworthy person to secure your secure even if I wanted to go that route.

    But thanks for the tips you did give me!
     
  12. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
  13. webignition

    webignition Well-Known Member

    Joined:
    Jan 22, 2005
    Messages:
    1,880
    Likes Received:
    0
    Trophy Points:
    36
    As for finding someone trustworthy, I've heard a lot of good things about Chirpy and used his services twice for the installation of MailScanner and would trust him with my root password (which says a lot).

    From what I've read, AndyReed both knows his stuff very well and is a very good person to work with.
     
  14. ladierainy

    ladierainy Well-Known Member

    Joined:
    Dec 1, 2003
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    6
    Bless your hearts! Thanks for the info!

    I'll check it out!
     
  15. DannyM

    DannyM Member

    Joined:
    Jul 31, 2003
    Messages:
    13
    Likes Received:
    0
    Trophy Points:
    1
    Is there any news about this issue? We are having the same problem and I am certain that we are not under attack..

    Please help!
     
Loading...

Share This Page