Cpanel compronised?

[rhm.geerts]

So we got a root alert mail today from csf/lfd. But we god a hard and long password for root.

We are looking at the logs and found this in the access log.

[03/28/2021:07:27:55 -0000] "GET / HTTP/1.1" 200 0 "-" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:27:55 -0000] "GET / HTTP/1.1" 200 0 "-" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:28:02 -0000] "POST /login/ HTTP/1.1" 307 0 "https://www.serverdomain.com:2087/" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:28:05 -0000] "POST /login/ HTTP/1.1" 307 0 "https://serverdomain.com:2087/" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:28:27 -0000] "GET /cpsess6056748524/json-api/listaccts?api.version=1 HTTP/1.1" 200 0 "-" "Mozilla/5.0" "s" "-" 2087
[03/28/2021:07:28:30 -0000] "GET /cpsess8188774263/json-api/listaccts?api.version=1 HTTP/1.1" 200 0 "-" "Mozilla/5.0" "s" "-" 2087

so the http 200 code worries me.

Checking the session log and finding this:

[2021-03-28 09:28:03 +0200] info [whostmgrd] 31.xx.xx.xx NEW root:3_Nbyd_W6BGvujLW address=31.xx.xx.xx,app=whostmgrd,creator=root,method=handle_form_login,path=form,possessed=0
[2021-03-28 09:28:05 +0200] info [whostmgrd] 31.xx.xx.xx NEW root:KLXRp89PA5ITPQne address=31.xx.xx.xx,app=whostmgrd,creator=root,method=handle_form_login,path=form,possessed=0

So checking those session keys in /var/cpanel/sessions/raw and they match.

Looks to me this one got in and used the list accounts option. I did not see any other action untill now and we changed the root pass ofcourse.

But how did this one get in?
Nothing to be found in /var/log/messages and not in /var/log/secure for this ip.
Nothing in the /usr/local/cpanel/logs/login.log either so that is odd.

 

[cPRex]

Hey there! I'm sorry to hear about the root compromise on the machine. It's hard to say for sure how the system was accessed, but one of the most common ways is keylogger software on a user's computer that has access to root.

If you'd like to submit a ticket we could have our team do some additional tests to see if we can determine how this happened, or you could use tools like chrootkit to see if there are any known exploits on the system.

It's important to note that the only secure way to keep things safe would be to migrate the accounts to a clean server.

 

[rhm.geerts]

Luckily we have to move to a clean server soon anyway. But it would be good if you could have a look in the meantime, because it can take several weeks before we go over.

I will send in a ticket so your team can have a look.

 

[cPRex]

Thanks! If you could post the ticket number here I can follow along and keep this thread updated with our findings.

 

[rhm.geerts]

Oke that would be nice. Ticket number is 94311130.

 

[cPRex]

Thanks for that - I'm following along with the ticket now.

 

[rhm.geerts]

As for the logs, probably a script is used, because otherwise we would have seen at least things like favicon.ico loaded. Nothing like that is present.
After investigation of a cPanel technician it might have been the hacker entered via/because of a WHMCS token (installed by the server owner) which had way to many privileges. Not sure, but it's the most obvious in this case, they certainly did not get in via SSH. Only via WHM or WHM api (via whmcs token).
Token removed and renewed, minimum rights, root pass changed, this is the solution for the short term.

Since the server was compromised, the only thing to do is create a new installation and transferring all accounts which will be done probably next week.

I've seen via the cPanel tools it's easy to migrate accounts.

Is there also an option that all system configuration is migrated to a new server? If possible including adjusted templates?
But especially the installed php configurations and selected modules?

 

[cPRex]

Transfer Tool can move various configuration files listed here:

Transfer Tool | cPanel & WHM Documentation

This interface copies multiple accounts from a remote server to your cPanel & WHM server.

docs.cpanel.net

Moving the EA4 profile will get the PHP versions and modules.

There isn't a way to move custom templates as that would need to happen manually with a tool like rsync or the work would need to be duplicated on the new machine.

 

[rhm.geerts]

Thank you.
I've indeed seen the Transfer tool documentation, but I didn't see the EA4 profile. But if that is in there, it's fine with me.

As for the custom templates, I can use rsync that's no problem.
The only thing changed is that for mail I changed domain.com to mail.domain.com and created an A record for mail rather then a cname record.
So I guess this is some DNS template, but I don't know where these changed templates are residing in cPanel.

If you could tell me where to find them, then I can push them over with rsync next week when we create the new server.

 

[cPRex]

A custom DNS template would be located in /var/cpanel/zonetemplates, named in the format of the one you edited based off this information in the WHM >> Edit Zone Template interface:

simple - Domains with an A entry only
standard - Domains on a dedicated IP address, parked domains, and addon domains
standardvirtualftp - Domains on a shared IP address

 

[rhm.geerts]

If I'm correct I changed the standard template.

That will do.
Thank you very much.

If I run into issues I will contact you on the forums again.

 

[cPRex]

That sounds great!

 

[rhm.geerts]

Sorry to bother you again but I'm just wondering.

I've installed the new server now. Still very bare at the moment.

Is it possible to do a test run with the transfer tool? Like try now and see if everythings works, and then do it again at the moment the transfer will be definate?
So suppose I would do a transfer now via the tool. And I would do it again on Saturday, will this just overwrite the current accounts and stuff?

Because if I do a transfer now and leave it this way and change the nameservers this weekend, a lot of mail for example, and maybe content on forums and webshops will be lost. Which wouldn't be the case on a new transfer. But I'm a bit afraid that this will create duplicate content or other issues.

 

[cPDavidL]

Greetings!

The key point in the Transfer Tool you're going to want to pay attention to, is the Live Transfer.

Transfer Tool | cPanel & WHM Documentation

This interface copies multiple accounts from a remote server to your cPanel & WHM server.

docs.cpanel.net

To make sure the source server is left operating as normal, make sure live transfer is off.
You can then use transfer tool again later to sync all the mailbox contents and/or changes since the last one, by using the Overwrite option. There should be no "duplicate" content in this step, because it will overwrite existing files with the new transfer.

Please let us know if we may be of further assistance.

 

[rhm.geerts]

That is great, thank you very much.

 

[rhm.geerts]

Well.... it almost worked great, except that the easyapache config was not copied, so wrong php versions were used which messed up some sites it seems.

I don't know why, I found this:

Starting “TRANSFER” for “Apache” “Easy Apache”.
Creating config package on remote server …
cpanel::easy::apache failed: Could not determine mod security version at /usr/local/cpanel/Whostmgr/ModSecurity/Settings.pm line 60.
Failed: Could not determine remote path from cpconftool run.

But I did not select mod_security, we didn't use it on the old server. but we did use php 5.6, 7.0, 7.1 and 7.2 and now on the new server there is 7.3, 7.4 and 8.0 and that is not what i expected.
I thought also all apache and php settings would be synchronized by the transfer tool?

 

[rhm.geerts]

Can be closed/set to solved. It should not happen this error.

But in the meantime, yesterday I already exported the custom eapache profile and imported it on the new server and provisioned and working now.