rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
So we got a root alert mail today from csf/lfd. But we god a hard and long password for root.

We are looking at the logs and found this in the access log.
Code:
[03/28/2021:07:27:55 -0000] "GET / HTTP/1.1" 200 0 "-" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:27:55 -0000] "GET / HTTP/1.1" 200 0 "-" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:28:02 -0000] "POST /login/ HTTP/1.1" 307 0 "https://www.serverdomain.com:2087/" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:28:05 -0000] "POST /login/ HTTP/1.1" 307 0 "https://serverdomain.com:2087/" "Mozilla/5.0" "-" "-" 2087
[03/28/2021:07:28:27 -0000] "GET /cpsess6056748524/json-api/listaccts?api.version=1 HTTP/1.1" 200 0 "-" "Mozilla/5.0" "s" "-" 2087
[03/28/2021:07:28:30 -0000] "GET /cpsess8188774263/json-api/listaccts?api.version=1 HTTP/1.1" 200 0 "-" "Mozilla/5.0" "s" "-" 2087
so the http 200 code worries me.

Checking the session log and finding this:
Code:
[2021-03-28 09:28:03 +0200] info [whostmgrd] 31.xx.xx.xx NEW root:3_Nbyd_W6BGvujLW address=31.xx.xx.xx,app=whostmgrd,creator=root,method=handle_form_login,path=form,possessed=0
[2021-03-28 09:28:05 +0200] info [whostmgrd] 31.xx.xx.xx NEW root:KLXRp89PA5ITPQne address=31.xx.xx.xx,app=whostmgrd,creator=root,method=handle_form_login,path=form,possessed=0
So checking those session keys in /var/cpanel/sessions/raw and they match.

Looks to me this one got in and used the list accounts option. I did not see any other action untill now and we changed the root pass ofcourse.

But how did this one get in?
Nothing to be found in /var/log/messages and not in /var/log/secure for this ip.
Nothing in the /usr/local/cpanel/logs/login.log either so that is odd.
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
4,483
565
273
cPanel Access Level
Root Administrator
Hey there! I'm sorry to hear about the root compromise on the machine. It's hard to say for sure how the system was accessed, but one of the most common ways is keylogger software on a user's computer that has access to root.

If you'd like to submit a ticket we could have our team do some additional tests to see if we can determine how this happened, or you could use tools like chrootkit to see if there are any known exploits on the system.

It's important to note that the only secure way to keep things safe would be to migrate the accounts to a clean server.
 

rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
Luckily we have to move to a clean server soon anyway. But it would be good if you could have a look in the meantime, because it can take several weeks before we go over.

I will send in a ticket so your team can have a look.
 

rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
As for the logs, probably a script is used, because otherwise we would have seen at least things like favicon.ico loaded. Nothing like that is present.
After investigation of a cPanel technician it might have been the hacker entered via/because of a WHMCS token (installed by the server owner) which had way to many privileges. Not sure, but it's the most obvious in this case, they certainly did not get in via SSH. Only via WHM or WHM api (via whmcs token).
Token removed and renewed, minimum rights, root pass changed, this is the solution for the short term.

Since the server was compromised, the only thing to do is create a new installation and transferring all accounts which will be done probably next week.

I've seen via the cPanel tools it's easy to migrate accounts.

Is there also an option that all system configuration is migrated to a new server? If possible including adjusted templates?
But especially the installed php configurations and selected modules?
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
4,483
565
273
cPanel Access Level
Root Administrator
Transfer Tool can move various configuration files listed here:


Moving the EA4 profile will get the PHP versions and modules.

There isn't a way to move custom templates as that would need to happen manually with a tool like rsync or the work would need to be duplicated on the new machine.
 
  • Like
Reactions: rhm.geerts

rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
Thank you.
I've indeed seen the Transfer tool documentation, but I didn't see the EA4 profile. But if that is in there, it's fine with me.

As for the custom templates, I can use rsync that's no problem.
The only thing changed is that for mail I changed domain.com to mail.domain.com and created an A record for mail rather then a cname record.
So I guess this is some DNS template, but I don't know where these changed templates are residing in cPanel.

If you could tell me where to find them, then I can push them over with rsync next week when we create the new server.
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
4,483
565
273
cPanel Access Level
Root Administrator
A custom DNS template would be located in /var/cpanel/zonetemplates, named in the format of the one you edited based off this information in the WHM >> Edit Zone Template interface:

Code:
simple - Domains with an A entry only
standard - Domains on a dedicated IP address, parked domains, and addon domains
standardvirtualftp - Domains on a shared IP address
 
  • Like
Reactions: rhm.geerts

rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
Sorry to bother you again but I'm just wondering.

I've installed the new server now. Still very bare at the moment.

Is it possible to do a test run with the transfer tool? Like try now and see if everythings works, and then do it again at the moment the transfer will be definate?
So suppose I would do a transfer now via the tool. And I would do it again on Saturday, will this just overwrite the current accounts and stuff?

Because if I do a transfer now and leave it this way and change the nameservers this weekend, a lot of mail for example, and maybe content on forums and webshops will be lost. Which wouldn't be the case on a new transfer. But I'm a bit afraid that this will create duplicate content or other issues.
 

cPDavidL

Linux Analyst II
Staff member
Oct 15, 2012
65
12
133
cPanel Access Level
Root Administrator
Greetings!

The key point in the Transfer Tool you're going to want to pay attention to, is the Live Transfer.


To make sure the source server is left operating as normal, make sure live transfer is off.
You can then use transfer tool again later to sync all the mailbox contents and/or changes since the last one, by using the Overwrite option. There should be no "duplicate" content in this step, because it will overwrite existing files with the new transfer.

Please let us know if we may be of further assistance.
 
  • Like
Reactions: rhm.geerts

rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
Well.... it almost worked great, except that the easyapache config was not copied, so wrong php versions were used which messed up some sites it seems.

I don't know why, I found this:
Starting “TRANSFER” for “Apache” “Easy Apache”.
Creating config package on remote server …
cpanel::easy::apache failed: Could not determine mod security version at /usr/local/cpanel/Whostmgr/ModSecurity/Settings.pm line 60.
Failed: Could not determine remote path from cpconftool run.
But I did not select mod_security, we didn't use it on the old server. but we did use php 5.6, 7.0, 7.1 and 7.2 and now on the new server there is 7.3, 7.4 and 8.0 and that is not what i expected.
I thought also all apache and php settings would be synchronized by the transfer tool?
 

rhm.geerts

Well-Known Member
Jul 29, 2008
134
12
68
Maastricht
cPanel Access Level
Root Administrator
Can be closed/set to solved. It should not happen this error.

But in the meantime, yesterday I already exported the custom eapache profile and imported it on the new server and provisioned and working now.
 
  • Like
Reactions: cPRex