The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Dealing with customers with compromised sites?

Discussion in 'Security' started by benito, Jun 22, 2015.

  1. benito

    benito Well-Known Member

    Joined:
    Jan 8, 2004
    Messages:
    296
    Likes Received:
    1
    Trophy Points:
    18
    Location:
    Mar del Plata - Argentina
    To who sell virtual hosting services, how do you deal with customers who had wordpress, joomla, etc CMS sites compromised?

    Usually those sites get filed modified, send spam and sometime have some php tools to browse files on server.

    We have all accounts restrictes so we do not have mayor risks, out biggest problem is the spam those scripts send.

    When we detect this i usually clean the site and the customer never knows.

    Now we want to let the customer clean his mess and we don't know how to deal with it, i mean if we suspend the account the customer can't access to clean.

    How do you do this this problem?
     
  2. quizknows

    quizknows Well-Known Member

    Joined:
    Oct 20, 2009
    Messages:
    940
    Likes Received:
    55
    Trophy Points:
    28
    cPanel Access Level:
    DataCenter Provider
    First time around we usually restore from clean backups if available, or clean the site if it is possible, and then advise them to take proper steps to secure the site (CMS/plugin updates, code review, change admin passwords, allow us to install ModSecurity if they are not using it, etc.).

    If they get compromised again after that, and it is clear they did not follow our advice, then we suspend the account. We may clean or restore one more time as a courtesy as long as they expressly agree to the above steps that are their responsibility.

    If it keeps reoccurring we suspend or terminate the account, or force them to rebuild a new site from scratch if they wish to not be terminated for ToS violations. Our ToS/AUP states they are responsible for spam/malware even if they did not intend for it to happen.

    The hardest part of all of this is teaching customers that it is not your server which was hacked, but rather their website. I've found people are a lot easier to work with if you can communicate that effectively.
     
    mythosis likes this.
  3. mythosis

    mythosis Member

    Joined:
    Oct 29, 2013
    Messages:
    8
    Likes Received:
    2
    Trophy Points:
    3
    cPanel Access Level:
    Reseller Owner
    I agree with quizknows 100%. As a Web Dev/Server Admin the best way to approach this is customer knowledge, and clear communication. There are lots of idle hosts out there hoping to cash in and not monitor servers, only to find out they are blacklisted due to malicious scripts. We have taken over websites and cleaned/updated them for free.

    You cant blame the CMS's really as there will always be ways to bend code. If anything using a CMS makes it easier to trouble shoot and secure sites via upgrades and proper coding conventions.

    I have found for joomla sites common attack vectors are usually out of date components that handle upload functions (JCE, OSFileman, eXtplorer). Keeping these up to date is a must.

    CSX scans daily, monitoring your mail queue and just generally being proactive about managing your server resources will ensure better customer service and less overall headaches.
     
  4. sparek-3

    sparek-3 Well-Known Member

    Joined:
    Aug 10, 2002
    Messages:
    1,381
    Likes Received:
    23
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
    This is an issue that I really don't think gets talked about enough and therefore a lot of people are uneducated in regards to this topic.

    If you are a shared web hosting provider (multiple web hosting accounts on a web server), then you need to decide if you are a managed web hosting provider or an unmanaged web hosting provider. If you are a managed web hosting provider, what specifically does that entail?

    To me, a managed web hosting provider (not to be confused with managed server management) is a web hosting provider that actively installs, updates, and keeps all of the web hosting sites on a web server up to date and clean. For example, a managed web hosting provider would update all of the web hosting accounts that use WordPress to the latest version, whenever a new version of WordPress comes out.

    Obviously with that definition of managed web hosting provider you have to define what you provide. You probably can't support every single CMS application and every single CMS plugin that is out there. And you also have to define what happens when a client goes rogue and installs a CMS application you don't support or a CMS plugin you don't support.

    Most web hosting companies are probably unmanaged web hosting providers. An unmanaged web hosting provider just provides space on a web server for the end user to do what he pleases, as long as he stays within the TOS of that web hosting provider.

    If you are an unmanaged web hosting provider, who is responsible for the security of an end user's web hosting account? Who is responsible for keeping an end user's web hosting account up to date. Who is responsible when abuse or malicious activity takes place on an end user's web hosting account? These are all questions that need to be defined and addressed. And since it's unmanaged, I assume that the bulk of that responsibility falls on the end user.

    People have this huge thinking complex, where they believe they can install something and then they're done. "Wow, I can install WordPress and now my website is complete." Managing a website and a web hosting account is never-ending, and most end users (in my experience) fail to grasp this. Perhaps this is the fault of the web hosting industry as a whole, not educating end users enough. When you install a script, WordPress, Joomla!, Drupal, anything, you constantly have to be in check with the script's developers for when a new version is released and if that release is a security release, you have to to update as soon as possible. You also can't go around installing every plugin, component, theme, etc just because it "does something neat" or "looks cool". Well written components and themes from reputable developers are key. You have to pay attention to password security, keeping your passwords secure, knowing who has admin level accounts and their passwords, etc.

    In my opinion, once an account is hacked or compromised, it has to be completely reset to avoid any future hackings or compromises. Can you clean a compromised account? Perhaps. But do you go through each file on the account, line by line, comparing the code that is there, to the code that is suppose to be there? If someone has gotten access to an account, they can inject code anywhere they want in any files on the account. So you might remove the spam script or the defaced message on an account and updated the script/plugins, but if that backdoor code is still there, then they can still get into the account.

    An ounce of prevention is worth a pound of cure. End users, or whoever is responsible for managing a web hosting account, needs to be aware of the importance of preventing compromises instead of dealing with them afterwards. Perhaps that's a failure on the part of the web hosting industry or perhaps it's a failure in the script developing industry, I do not know. But the lack of care in terms of web hosting account security is mind boggling and I suspect it to only get worse.
     
    quizknows likes this.
  5. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    14,447
    Likes Received:
    195
    Trophy Points:
    63
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    Great post, @sparek-3.

    Managed Hosting is also a nice upsell for smaller Hosting Providers that have the talent, time and care about every little thing that happens on their servers.
     
  6. quizknows

    quizknows Well-Known Member

    Joined:
    Oct 20, 2009
    Messages:
    940
    Likes Received:
    55
    Trophy Points:
    28
    cPanel Access Level:
    DataCenter Provider
    There are different levels of managed hosting though, and sparek-3 did touch on that. A large "managed" hosting provider cannot typically update every single users WP site, etc.

    In my position what we offer is support; we manage the server itself (OS and LAMP Stack, and cPanel) but we are pretty hands off with the web content itself. We are not web developers. However, if the customer requires support or help updating, then we provide it. We sell managed servers, not managed websites, which sparek-3 mentioned the differences of.

    In regards to cleaning; it takes experience but it can be done. You don't necessarily have to review every single file, because there are timestamps and such. You can compare to backups, etc. Unless the server is root compromised it is pretty easy to determine every file changed since the hack and address those. Even if the mtime is spoofed, the ctime cannot be (again, unless the server is rooted). The best course of action is usually to find the oldest hacked file and work from there; consult the logs and determine exactly how they "got in" (be it password, CMS exploit, etc.) and then restore to backups before the hack, patching said vector before bringing the site online. If there are no backups then you have to be very good at your job to properly clean it, but it can be done. Database review is quite ugly though, and nobody likes to do that. Anyway, only in some cases with really bad long-running compromised sites, or hacks that inject every single file, do we insist that the site be entirely rebuilt. One that comes to mind is cryptoPHP, since it was a deep infection caused by installing bootleg plugins, we would refuse to clean it and insist on site rebuild.

    One thing I do in my day to day job is manage modsecurity rules for "managed" customers, and write new rules if needed. If numerous customers get hacked due to the same out-dated plugin, we will push updates to our modsecurity rule set to "virtual patch" the plugin, and then inform customers via e-mail or other means to update for good measure. This lowers our ticket volume significantly.

    @sparek-3 you are right about a lack of care, and in many cases a lack of understanding. I recognize that while I may be able to clean a site, the vast majority of end users cannot, and are better off just rebuilding. The problem I run into is nobody is willing to rebuild (or at least the vast majority are not), perhaps because they just don't understand. This is one of the reasons I've spent so long learning how to properly clean infected sites, because my other options are to lose a customer or leave an infected site, because I know they just won't be willing to rebuild. Obviously I can't just leave it because of spam, IP reputation, support requests, etc., so it boils down to "be able to clean it, or the customer will leave or be terminated for ToS"

    In regards to responsibility, that needs to be outlined in a providers ToS. Generally though it comes down to the end user being responsible no matter what (at least from what I've seen).
     
    #6 quizknows, Jun 24, 2015
    Last edited: Jun 24, 2015

Share This Page