Server Hacked, please help

[encryption]

I got a strange email this morning from CPanel and running a search on here yields no results. The email states

[hackcheck] cp4nel has a uid 0 account
IMPORTANT: Do not ignore this email.
This message is to inform you that the account cp4nel has user id 0 (root privs).
This could mean that your system was compromised (OwN3D). To be safe you should
verify that your system has not been compromised.

Moreover, running a "top" yields the following result

top - 07:53:40 up 4 days, 22:03, 2 users, load average: 2.23, 2.24, 2.19
Tasks: 149 total, 3 running, 146 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.3% us, 0.3% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.2% hi, 0.2% si
Mem: 2073820k total, 1963448k used, 110372k free, 118008k buffers
Swap: 2096472k total, 648k used, 2095824k free, 1279696k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28155 root 25 0 7116 5888 516 R 99.9 0.3 2082:33 john
26566 root 25 0 7116 5892 516 R 97.9 0.3 2087:30 john
1 root 16 0 1744 600 516 S 0.0 0.0 0:01.41 init
2 root RT 0 0 0 0 S 0.0 0.0 0:00.09 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/0
4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT 0 0 0 0 S 0.0 0.0 0:04.39 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:00.09 ksoftirqd/1
7 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/0
9 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1
10 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
11 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.59 kblockd/0
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.07 kblockd/1
16 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
108 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khubd

What is "john"

[darren.nolan]

http://www.cyberciti.biz/faq/unix-li...hn-the-ripper/

This is all I could find. Kill off the process imho.

[encryption]

I got to the bottom of it, this guy installed a rootkit "shv5_rootkit" and is sending spam (the Bank of America Hack I imagine) I was able to get a list of commands executed and saw exactly where he got in from and what he has done.

One of my clients seems to have installed an email list program and he gained access through the "temp" file on that program.

[encryption]

http://www.cyberciti.biz/faq/unix-li...hn-the-ripper/

This is all I could find. Kill off the process imho.

wow thanks for that, I have killed the process, changed root ports and updated passwords. Going to have to reinstall this system but shit I'm so curious to nail this sucker right now. He even managed to delete all my firewall logs (I run CSF and surprised he got to that as well). Digging deeper.

One of the commands he ran was

root@tk [~]# cd /lib/ld-lsb.so.3/john-1.7.0.2/run

but I cant even file that so.3 file at that location.

[darren.nolan]

Start going crazy on security.

Ensure you use something like suPHP (to run all PHP scripts as the user).

Start using suExec - which makes cgi-bin scripts run as the user (like when a user accesses a perl script to send an email).

Ensure that you use the WHM's compiler tweak - so that people who do actually get on your system, can't compile the tools to further hack the system.

[encryption]

Thanks for that, what would be the best way to migrate all the accounts from the server is SSH has been disabled and WHM from the new server is unable to establish a connection to the compromised server.

How exactly would I setup backup to remote FTP ?

[dwykofka]

You should have all of your clients change their passwords, John the ripper is typically used to decrypt user passwords. depending on length / complexity / Salted or unsalted he could have cracked the passwords live or downloaded a copy of the shadow file to offline processing. Once the passwords are uncovered he can use them to re-gain access to your server unless you change all of your users passwords.

[encryption]

Thanks for that, that is on my agenda to complete. Just cant believe the guy got in the way he did.......

[darren.nolan]

Thanks for that, that is on my agenda to complete. Just cant believe the guy got in the way he did.......

Which was?

Uploaded a file to your tmp directory - ran it?

[bsdjunk]

The most important thing to do is have cpanel update itself automatically, enable the hardened password thing cpanel provides. I have been admining servers for a long time and I have never paid much attention to uptime nor do I go with servers which have 400 day uptimes as I don't trust them. I rather have a low uptime server with all security updates than one which has good uptime. make sure all users use strong passwords. setting the password hardener to 75 or higher is recomended pisses users off but it's for there own good as much as your servers. as it isn't good for your bussiness if you've been hacked.

[jayh38]

Also consider using a private key instead of password for ssh.

[encryption]

Which was?

Uploaded a file to your tmp directory - ran it?

Not entirely sure, the techs at the datacenter state he may have exploited a recently discovered hole in FC4 kernel but when I check the bash_history, he suddenly appears in the system and starts executing commands referencing a clients mailer program folder. I think that folder is how he gained access to the system but its unclear to me how

[encryption]

so I recompiled Apache 2.2 with the latest version of PHP and enabled Mod Security but now none of the sites are working.... I get the following error. Any clues ?

Not Found

The requested URL /~aaronmos/ was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8b mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635

[darren.nolan]

Sounds like you don't have usrdir enabled on your system.

Try the following.

Log into WHM -> Security Center -> mod_userdir ->

Ensure that it's enabled on the default host - if you want to allow your customers to access their sites via their username (lots of discussion about that).

[encryption]

cheers darren, you've been mighty helpful all along, mod_security didnt have any rules configured so I chose the default configuration, is there anyplace you recommend I obtain a relatively well configured ruleset for use in WHM?

also the default config breaks the use of .htaccess, how would I change that ?

(I've searched on the forums a bit but not finding anything constructive)