Server Hacked, please help

encryption

Well-Known Member
Jun 24, 2005
74
1
158
I got a strange email this morning from CPanel and running a search on here yields no results. The email states

[hackcheck] cp4nel has a uid 0 account
IMPORTANT: Do not ignore this email.
This message is to inform you that the account cp4nel has user id 0 (root privs).
This could mean that your system was compromised (OwN3D). To be safe you should
verify that your system has not been compromised.
Moreover, running a "top" yields the following result

top - 07:53:40 up 4 days, 22:03, 2 users, load average: 2.23, 2.24, 2.19
Tasks: 149 total, 3 running, 146 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.3% us, 0.3% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.2% hi, 0.2% si
Mem: 2073820k total, 1963448k used, 110372k free, 118008k buffers
Swap: 2096472k total, 648k used, 2095824k free, 1279696k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28155 root 25 0 7116 5888 516 R 99.9 0.3 2082:33 john
26566 root 25 0 7116 5892 516 R 97.9 0.3 2087:30 john

1 root 16 0 1744 600 516 S 0.0 0.0 0:01.41 init
2 root RT 0 0 0 0 S 0.0 0.0 0:00.09 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/0
4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT 0 0 0 0 S 0.0 0.0 0:04.39 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:00.09 ksoftirqd/1
7 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/0
9 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1
10 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 khelper
11 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 kthread
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.59 kblockd/0
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.07 kblockd/1
16 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid
108 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khubd
What is "john"
 

encryption

Well-Known Member
Jun 24, 2005
74
1
158
I got to the bottom of it, this guy installed a rootkit "shv5_rootkit" and is sending spam (the Bank of America Hack I imagine) I was able to get a list of commands executed and saw exactly where he got in from and what he has done.

One of my clients seems to have installed an email list program and he gained access through the "temp" file on that program.
 

encryption

Well-Known Member
Jun 24, 2005
74
1
158
wow thanks for that, I have killed the process, changed root ports and updated passwords. Going to have to reinstall this system but **** I'm so curious to nail this sucker right now. He even managed to delete all my firewall logs (I run CSF and surprised he got to that as well). Digging deeper.

One of the commands he ran was

[email protected] [~]# cd /lib/ld-lsb.so.3/john-1.7.0.2/run

but I cant even file that so.3 file at that location.
 

darren.nolan

Well-Known Member
Oct 4, 2007
257
0
66
Start going crazy on security.

Ensure you use something like suPHP (to run all PHP scripts as the user).

Start using suExec - which makes cgi-bin scripts run as the user (like when a user accesses a perl script to send an email).

Ensure that you use the WHM's compiler tweak - so that people who do actually get on your system, can't compile the tools to further hack the system.
 

encryption

Well-Known Member
Jun 24, 2005
74
1
158
Thanks for that, what would be the best way to migrate all the accounts from the server is SSH has been disabled and WHM from the new server is unable to establish a connection to the compromised server.

How exactly would I setup backup to remote FTP ?
 

dwykofka

Well-Known Member
Aug 6, 2003
394
3
168
You should have all of your clients change their passwords, John the ripper is typically used to decrypt user passwords. depending on length / complexity / Salted or unsalted he could have cracked the passwords live or downloaded a copy of the shadow file to offline processing. Once the passwords are uncovered he can use them to re-gain access to your server unless you change all of your users passwords.
 

encryption

Well-Known Member
Jun 24, 2005
74
1
158
Thanks for that, that is on my agenda to complete. Just cant believe the guy got in the way he did.......
 

bsdjunk

Member
Jan 15, 2008
16
0
51
The most important thing to do is have cpanel update itself automatically, enable the hardened password thing cpanel provides. I have been admining servers for a long time and I have never paid much attention to uptime nor do I go with servers which have 400 day uptimes as I don't trust them. I rather have a low uptime server with all security updates than one which has good uptime. make sure all users use strong passwords. setting the password hardener to 75 or higher is recomended pisses users off but it's for there own good as much as your servers. as it isn't good for your bussiness if you've been hacked.
 

encryption

Well-Known Member
Jun 24, 2005
74
1
158
Which was?

Uploaded a file to your tmp directory - ran it?
Not entirely sure, the techs at the datacenter state he may have exploited a recently discovered hole in FC4 kernel but when I check the bash_history, he suddenly appears in the system and starts executing commands referencing a clients mailer program folder. I think that folder is how he gained access to the system but its unclear to me how
 
Last edited:

encryption

Well-Known Member
Jun 24, 2005
74
1
158
so I recompiled Apache 2.2 with the latest version of PHP and enabled Mod Security but now none of the sites are working.... I get the following error. Any clues ?

Not Found

The requested URL /~aaronmos/ was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Apache/2.2.8 (Unix) mod_ssl/2.2.8 OpenSSL/0.9.8b mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
 
Last edited:

darren.nolan

Well-Known Member
Oct 4, 2007
257
0
66
Sounds like you don't have usrdir enabled on your system.

Try the following.

Log into WHM -> Security Center -> mod_userdir ->

Ensure that it's enabled on the default host - if you want to allow your customers to access their sites via their username (lots of discussion about that).
 

encryption

Well-Known Member
Jun 24, 2005
74
1
158
cheers darren, you've been mighty helpful all along, mod_security didnt have any rules configured so I chose the default configuration, is there anyplace you recommend I obtain a relatively well configured ruleset for use in WHM?

also the default config breaks the use of .htaccess, how would I change that ?

(I've searched on the forums a bit but not finding anything constructive)
 

twhiting9275

Well-Known Member
Sep 26, 2002
560
28
178
cPanel Access Level
Root Administrator
Twitter
FC4 - Years out of date. There's no question how they got in, the system was out of date, or let go. That's like saying "Oh, I'm running redhat 7.3, but I'm just going to let it go".

Every couple of years you need to have your system replaced and updated. When you do so, you should update the OS as well (ie: fc4 to fc8 now, etc). Hardware (even server hardware) degrades over time, and needs to be replaced. It happens, like I said, every couple years or so, you should be upgrading for better performance.

John (aka JTR) is a password cracking system. So, not only have you been hacked, but your passwords have been compromised. Not just ONE password, but ALL of them are gone. You'd best mail your users or it will happen again!

While you're at it, require ssh keys to login to the server, NOT just as root, but as any user. This will help solve issues.

Mod_security is iffy, you're going to have problems there. Best result is use something like suhosin that doesn't go out of it's way to break functionality, yet provides at least a bit of security.