Ebury infection on Server

[isp]

So here I am .... completely gutted. My server has an Ebury infection.

This frustrates me, because I literally took every single precaution I could:

I made sure that my root password was at least 64 characters long. It was completely random and consisted of numbers, letters and special characters. Only two people have the root password. Both of us use linux desktops to avoid exploitation.

I kept the system updated. I install CSF firewall. I run daily malware scans. Since we have had to reinstall systems in the past we took just about every single precaution humanly possible to prevent our system being exploited. The only thing I didnt do was limit from which machine you can ssh from (for practicle reasons).

I am now sitting with an Ebury infection. Apparently it came in an SSH package that cPanel gave me :/

https://www.cert-bund.de/ebury-faq

I can confirm my machine is infected:

[email protected] [~]# ipcs -m

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x000009e4 0 root 666 3179912 0

[email protected] [~]# find /lib* -type f -name libkeyutils.so* -exec ls -la {} \;
-rwxr-xr-x 1 root root 32904 Jun 22 2012 /lib64/libkeyutils.so.1.3

So I am 100% definitely infected.

So now what do I do? The only solution that is offered is to reinstall the WHOLE server. I have almost 600 accounts on the one server. How on earth do I do this so there is no down time?

How do I even do this?

I feel that I did take ever possible precaution and that it was not due to negligence on my side but rather my software vendor.

Are cPanel offering assistance to clean clients servers? Is the responsibility solely ours? Is there any assistance or advice someone can give to avoid down time?

This could literally hurt our business in a massive way. I feel I have been let down by cPanel :/

 

[cPanelMichael]

Hello

There is a useful post here that should help to answer your questions:

SSHD Rootkit

Thank you.

 

[quizknows]

Have you checked the machine that you SSH to this machine from? That one may be infected.

Unless you had a cPanel ticket in a very small window of time where they had an infected workstation, this was more likely (sadly) from something on your end and not cPanel.

The only cases I've seen aside from stolen root passwords are due to SSHing to the newly infected box from an already infected box, or a vulnerable exploited web application existed on a system with an outdated kernel that allowed privelege escalation. If your kernel is more than a year old this easily could have happened, especially with hundreds of accounts.

You do need to move your clients to a machine with a new OS. The last person that tried to have me just replace the compromised RPMs ended up with a non-functioning system 1-2 days later; it would not even boot. This is a very very nasty rootkit.

Now might be a good time to consider investing in 6 smaller servers with 100 accounts each rather than having 600 accounts on one server. Just make sure you PULL data from the infected server, do NOT log in to your new server(s) from the infected one. Typically logging into the infected one via a clean machine won't risk the clean machine, but it's best to boot to a guest operating system to get your data (i.e. live boot CD / Jump drive).