(hacked) cPanel & whm slow & time out

[thanatopsizer]

Yesterday I noticed my server was running very slowly, so I checked the running processes and say the following:

./gma xxx.xxx.xxx.xxx 0 0 0


and in place of the x's was an ip. My pure-ftpd had been hacked and my server was performing a denial of service attack. Because I could not deal with the problem at the time, I shut the server off. I have restarted it and switched from pure-ftpd to pro-ftpd, and everything apears to be running normal. I checked all of my logs; and the only suspicious things I found were a feq connection attempts within one second of eatchother which all disconnected the same second. Anyway, now that I have eveything backonline, cpanel is extremely slow, and using the ssl port times out; as does whm, so I am at a loss for what to do; I know the hacked process isn't running, and I have restarted cpanel as well as the whole server, but it has not helped the situation. Any suggestions would be greatly appreciated.

 

[casey]

You need a security audit. There are quite a few reputable places in the ads forum.

 

[thanatopsizer]

That is a good thought, but I am more interested in getting everything on my server backonline first.

I am of course having an odd problem now, if I view the services running on my system, cpanel says that httpd, exim, ftpd, imap, and cppop have all failed; and yet all of them are running, and all of them work; web pages still load, I can still send and recieve emails, and my messages log file shows everything starting fine...

Mar 29 14:32:32 challenger exim: exim shutdown failed
Mar 29 14:32:32 challenger exim: antirelayd shutdown failed
Mar 29 14:32:32 challenger exim: spamd shutdown failed
Mar 29 14:32:32 challenger exim: exim startup succeeded
Mar 29 14:32:32 challenger exim: exim startup succeeded
Mar 29 14:32:33 challenger exim: antirelayd startup succeeded
Mar 29 14:32:43 challenger proftpd[5964]: challenger.pixelop.com - ProFTPD killed (signal 15)
Mar 29 14:32:43 challenger proftpd[5964]: challenger.pixelop.com - ProFTPD 1.2.9 standalone mode SHUTDOWN
Mar 29 14:32:43 challenger proftpd: proftpd shutdown succeeded
Mar 29 14:32:43 challenger proftpd[6516]: challenger.pixelop.com - ProFTPD 1.2.9 (stable) (built Fri Dec 19 18:21:13 EST 2003) standalone mode STARTUP
Mar 29 14:32:43 challenger proftpd: proftpd startup succeeded
Mar 29 14:33:19 challenger xinetd: xinetd shutdown failed
Mar 29 14:33:19 challenger xinetd[6561]: Server in.ntalkd is not executable [file=/etc/xinetd.d/ntalk] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Error parsing attribute server - DISABLING SERVICE [file=/etc/xinetd.d/ntalk] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Server in.qpopper is not executable [file=/etc/xinetd.d/pop-3] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Error parsing attribute server - DISABLING SERVICE [file=/etc/xinetd.d/pop-3] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Server in.talkd is not executable [file=/etc/xinetd.d/talk] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Error parsing attribute server - DISABLING SERVICE [file=/etc/xinetd.d/talk] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Server in.telnetd is not executable [file=/etc/xinetd.d/telnet] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Error parsing attribute server - DISABLING SERVICE [file=/etc/xinetd.d/telnet] [line=8]
Mar 29 14:33:19 challenger xinetd[6561]: Must specify a server in ntalk
Mar 29 14:33:19 challenger xinetd[6561]: Must specify a server in pop-3
Mar 29 14:33:19 challenger xinetd[6561]: Must specify a server in talk
Mar 29 14:33:19 challenger xinetd[6561]: Must specify a server in telnet
Mar 29 14:33:19 challenger xinetd[6561]: xinetd Version 2.3.12 started with libwrap loadavg options compiled in.
Mar 29 14:33:19 challenger xinetd[6561]: Started working: 1 available service
Mar 29 14:33:19 challenger xinetd: xinetd startup succeeded


Any ideas why cpanel's status is showint them as failed? One other thing, Cpanel still will not load in ssl mode, but will load in non ssl. I do not expect complete instructions on how to resolve everything (although it would be nice) but any suggestions would be helpful.

 

[chirpy]

For the status, make sure chksrvd is running:

/etc/init.d/chkservd stop
/etc/init.d/chkservd start

As for SSL. Shutdown httpd and check that all the processes have definitely shutdown, then start:

/etc/init.d/httpd stop
ps axf | grep -v grep | grep httpd
(repeat until all gone)
/etc/init.d/httpd start

However, if your server has been hacked you can no longer trust it. Unless you have a forensic security audit of the whole server done by a professional company that specialises in Linux security (which could cost $1000's) it is impossible to say that it is 100% clean.

You should backup all your user data and have an OS restore done of the server.

 

[Tom Pyles]

Stupid question for you. Are you positive that your server was hacked? I have seen FTP sessions that were not terminated properly. Server load will climb causing things to slow down depending on the server specs. Killing the process will return things to normal. I can't say for sure that this is what happened in your case.

Have you run chkrootkit to see if anything was found? Regardless, any suspicion that you were hacked, you should have an expert look into it.

 

[thanatopsizer]

I backed up everything on my server to its second hard drive, but just so there are no doubts, this is what my output liiked like...

That along with a process showed as ./gma xxx.xxx.xxx.xxx 0 0 0 makes me very suspicious; and there were 2 instances of my ftp server running (I'm again not sure how) but it's all taken care of now...