Reactive Security Measures [Case 56087]

[cPanelDavidG]

I recently had a discussion with a Partner NOC and they were discussing how they sometimes need to go into their customers' unmanaged servers to clean up the remains of a security breach caused by users running outdated website software.

The issue

While proactive measures exist such as using something like cPAddons/Site Software or a third party utility that permits forced upgrading of outdated website software, since these are customers' unmanaged servers, such proactive measures are often out of their hands.

What happens is a vulnerable (often outdated) website script will be compromised and a shell is uploaded, and by looking at the directories in /home or using other methods, they can determine the usernames of users on that server. They would then use a URL like the following to propagate the attack:

http://(domain of compromised website)/~(username)/(attack URL)

This can be prevented by enabling mod_userdir protection, but the Partner NOC often is compelled to clean up the mess regardless of how server admins configure their cPanel&WHM servers. Keep in mind, SuPHP and SuExec, while it prevents one account from accessing another account, does not prevent a script running on another computer from calling any URL it wants.

Often, there are indicators in the SSH, mail and other logs that clearly indicate an attack is underway. While tools such as the free third party utility ConfigServer Security & Firewall exist to proactively combat these issues including by means of monitoring logs for suspicious activity and preemptively banning such IPs, again the servers are outside the control of the Partner NOC so such proactive measures are deemed impractical since these are unmanaged servers.

One suggestion

This Partner NOC suggested adding an option to WHM's Password Modification screen to indicate that an account has been compromised. This would somehow produce information based on how that account was compromised including:

• Check the last logs to see when unrecognized IPs initially got into SSH
• Use find -mtime to discover what they changed/created/etc.
• Check for SSH keys
• Check if they added FrontPage Authorization
• Look at the Apache error logs for other sites they have attacked
• Analyze the error codes in the Apache error log to reflect what they may have successfully hit
• Analyze the secure.log and cPanel logs to see if they ever logged into the accounts' cPanel or WHM interfaces

Ultimately, scanning this list of items could help reduce re-infections and missed infections.

[fkatzenb]

Interesting concept. Maybe you could do something event based. If abc happens, do/log xyz.

[Kent Brockman]

Yep, you could load the server with checks for attack parameters, and mark those accounts that seems to be vulnerable, but it is going towards and entire new security feature, already managed by scripts like CSF. I think unmanaged servers should be given to end clients with thos optimizations to avoid security breaches. I say, even with the CSF script installed and configurated to its basics. If cPanel will work in a great security solution it should work even better than CSF, and that's a great catch.

[fkatzenb]

Dave,
This topic has been sitting in my mind and has given me some ideas...

1. cPanel password changes require e-mail confirmation by the user before they can ssh and such.
2. Reseller password changes require e-mail confirmation before they can modify accounts within WHM.

If they don't confirm a password, their account is locked out and it notifies root.

Of course DomainKeys and DKIM need to be implemented to reach Yahoo folks.

Thanks,
Frank

[cPanelDavidG]

Dave,
This topic has been sitting in my mind and has given me some ideas...

1. cPanel password changes require e-mail confirmation by the user before they can ssh and such.
2. Reseller password changes require e-mail confirmation before they can modify accounts within WHM.

If they don't confirm a password, their account is locked out and it notifies root.

Of course DomainKeys and DKIM need to be implemented to reach Yahoo folks.

Thanks,
Frank

Interesting. I presume email changes shouldn't be effective until after an email confirmation to prevent someone from changing the email *then* changing the password?

Also, the SSH being discussed in my original post, this is being done by means of a PHP script that effectively provides shell access like C99 which provides a webpage-based interface for SSH, rather than SSH in the more traditional sense of using a command line.

[fkatzenb]

Interesting. I presume email changes shouldn't be effective until after an email confirmation to prevent someone from changing the email *then* changing the password?

Correct!

Also, the SSH being discussed in my original post, this is being done by means of a PHP script that effectively provides shell access like C99 which provides a webpage-based interface for SSH, rather than SSH in the more traditional sense of using a command line.

Understand that! Scary situation


As you mentioned above, CSF is great, but if they do get in, there is nothing it or anyone can do. I would love to see SSH service get terminated for X amount of time and restarted on a different predetermined port IF someone has a successful login without keys.


Frank

[cPanelDavidG]

Correct!

This particular issue seems moot with the introduction of our source IP checking feature. I would advise enabling that functionality.

[SoftDux]

I think it's a great idea that better security measures be added to cPanel.

One should also be able to lock-down an account, but still give the client access to the files to fix-up the compromised website if needed. Somehow ..... ?

[cPanelDavidG]

In talking with our Product Manager, it seems some of these issues could be addressed proactively rather than by means of the "Limit logins to verified IP Addresses"/Security questions functionality we introduced after our initial conversation. This can be configured in the Configure Security Policies screen in WHM. This, along with CSRF protection and source IP checking on cookies, can help curb many issues where cPanel&WHM is the point of entry.

Our main concern with checking for things like the enabling of FrontPage extensions, error logs etc. is generating false positives, potentially so many that the tool is rendered useless. If you have any ideas on how to reduce false positives, that would be useful.

Regarding Apache error codes in particular (in response to people probing your server for exploits), have you considered mod_security with some rules specifically designed for cPanel&WHM environments (many seem to recommend the rules from GotRoot.com) to proactively stop this issue?

I know this request is primarily about a second line of defense, however we have a strong culture of promoting proactive solutions rather than reactive solutions. In fact, we're moving away from solutions that generate excessive false positives such that they are essentially useless like the old Scan for Trojans tool. Hence the emphasis on reducing false positives.

I look forward to hearing your input on this.