SOLVED [CPANEL-27445] cPHulk Countries blacklist not working?

apaulo

Member
May 14, 2019
7
0
1
Portugal
cPanel Access Level
Reseller Owner
Hi,

I have cPHulk Brute Force Protection On and an extensive list of blacklisted countries.

When I wake up this morning I have over 6000 "⚠ Excessive Number of Failed Login Attempts from 103.231.xxx.xxx (Iran, Islamic Republic of:IR)" warnings - not only from Iran, but also from several other countries included in the blacklist - most (if not all) from countries included on my blacklist for months.

Also over 27000 one-day-blocks have been created by CPanel for the suspicious IPs.

Is there a known issue with cPHulk?

Thanks.

Alexandre
 
Last edited by a moderator:

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
7,360
584
263
Houston
cPanel Access Level
DataCenter Provider
Hi @apaulo

There is not a known issue with country management. If you go to WHM>>Security Center>>cPHulk Brute Force Detection -> Countries Management -> Filter: Blacklisted are the ranges you previously selected still blacklisted including Iran?

Also, is anything listed in the cPHulkd error logs? You can find them here:
Code:
/usr/local/cpanel/logs/cphulkd_errors.log
/usr/local/cpanel/logs/cphulkd.log
 

apaulo

Member
May 14, 2019
7
0
1
Portugal
cPanel Access Level
Reseller Owner
Hi @apaulo

There is not a known issue with country management. If you go to WHM>>Security Center>>cPHulk Brute Force Detection -> Countries Management -> Filter: Blacklisted are the ranges you previously selected still blacklisted including Iran?
Yes, all countries are blacklisted - as I explained, the warnings were not only from Iran, but also from several other countries included in the blacklist - most (if not all) from countries included on my blacklist for months.

Iran as just an example - it was the most active (about 26000 hits in 8 hours).


Hi @apaulo
Also, is anything listed in the cPHulkd error logs? You can find them here:
Code:
/usr/local/cpanel/logs/cphulkd_errors.log
/usr/local/cpanel/logs/cphulkd.log
On cphulkd_errors.log, this one is quite recurring:

Code:
[2019-05-14 11:20:44 +0100] die [cPhulkd] Timeout while waiting for response at /usr/local/cpanel/Cpanel/Hulkd.pm line 487.
        Cpanel::Hulkd::die(Cpanel::Hulkd=HASH(0x1382908), "Timeout while waiting for response") called at /usr/local/cpanel/Cpanel/Hulkd.pm line 417
        Cpanel::Hulkd::__ANON__(__CPANEL_HIDDEN__) called at /usr/local/cpanel/Cpanel/Hulkd/Processor.pm line 289
        eval {...} called at /usr/local/cpanel/Cpanel/Hulkd/Processor.pm line 289
        Cpanel::Hulkd::Processor::run(Cpanel::Hulkd::Processor=HASH(0x1ca34c8), undef) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 423
        Cpanel::Hulkd::__ANON__() called at /usr/local/cpanel/3rdparty/perl/528/lib/perl5/cpanel_lib/Try/Tiny.pm line 97
        eval {...} called at /usr/local/cpanel/3rdparty/perl/528/lib/perl5/cpanel_lib/Try/Tiny.pm line 88
        Try::Tiny::try(CODE(0x1761ce8), Try::Tiny::Catch=REF(0x176dfa8)) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 427
        Cpanel::Hulkd::handle_one_connection(Cpanel::Hulkd=HASH(0x1382908), Cpanel::Socket::INET=GLOB(0x176dd50), undef) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 594
        Cpanel::Hulkd::_handle_accepted_socket_and_reset_idleloops(Cpanel::Hulkd=HASH(0x1382908), Cpanel::Socket::INET=GLOB(0x176dd50)) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 341
        Cpanel::Hulkd::main_loop(Cpanel::Hulkd=HASH(0x1382908), Cpanel::Socket::UNIX=GLOB(0x13c0450), Cpanel::Socket::INET=GLOB(0x13c06a8)) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 244
        Cpanel::Hulkd::processor_run(Cpanel::Hulkd=HASH(0x1382908)) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 159
        Cpanel::Hulkd::__ANON__(__CPANEL_HIDDEN__) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 170
        Cpanel::Hulkd::launcher(Cpanel::Hulkd=HASH(0x1382908), 0) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 139
        Cpanel::Hulkd::start_daemon(Cpanel::Hulkd=HASH(0x1382908), 0) called at /usr/local/cpanel/Cpanel/Hulkd.pm line 93
        Cpanel::Hulkd::run_daemon(Cpanel::Hulkd=HASH(0x1382908)) called at libexec/cphulkd.pl line 32
[2019-05-14 11:20:44 +0100] info [cPhulkd] The system encountered an error while processing a request: exit level [die] [pid=15597] (Timeout while waiting for response)

On cphulkd.log I just get loads and loads of entries like:
Code:
[2019-05-14 14:54:32 +0100] info [cPhulkd] Login Blocked: The IP address is blacklisted. [Service]=[dovecot] [Local IP Address]=[94.126.171.194] [Remote IP Address]=[103.231.139.176] [Authentication Database]=[mail] [Username]=[[email protected]]
That at some point, it seems it gets back to normal, because the above messages stop and start getting
Code:
[2019-05-16 16:21:44 +0100] info [cPhulkd] Login Blocked: The country is blacklisted. [Service]=[dovecot] [Local IP Address]=[94.126.171.213] [Local Port]=[143] [Remote IP Address]=[220.233.42.61] [Remote Port]=[49532] [Authentication Database]=[mail] [Username]=[[email protected]]
So, I am guessing that this was some automatic update that messed up things, and got back to normal the following day, thanks to a new automatic update...

According to logs, it seems it is back to normal...
 
Last edited by a moderator:

apaulo

Member
May 14, 2019
7
0
1
Portugal
cPanel Access Level
Reseller Owner
Meh... it is not solved... just enabled the "Send a notification when the system detects a brute force user", again and started being bombarded with emails "⚠ Excessive Number of Failed Login Attempts from 78.9.51.10 (Poland:PL)"

Poland is blacklisted - double checked.

For this specific message, on "cphulkd.log":
Code:
[2019-05-16 17:48:42 +0100] info [cPhulkd] Login Blocked: IP reached maximum auth failures for a one day block [Service]=[dovecot] [Local IP Address]=[94.126.171.213] [Local Port]=[143] [Remote IP Address]=[78.9.51.10] [Remote Port]=[53582] [Authentication Database]=[mail] [Username]=[justin_fields] (1/6 failures) (blocked until [Fri May 17 16:48:42 2019 UTC/Fri May 17 17:48:42 2019 LOCAL])
[2019-05-16 17:48:42 +0100] info [cPhulkd] Login Blocked: The country is blacklisted. [Service]=[dovecot] [Local IP Address]=[94.126.171.213] [Local Port]=[143] [Remote IP Address]=[78.9.51.10] [Remote Port]=[53582] [Authentication Database]=[mail] [Username]=[justin_fields]
That same IP is now also on the "One day block"... twice!

On my view, the login it should be only "Login Blocked: The country is blacklisted"... period.

So cPHulk is not ignoring the country blacklist... but it is not checking it BEFORE allowing the user to try to login.
 

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
7,360
584
263
Houston
cPanel Access Level
DataCenter Provider
Iran as just an example - it was the most active (about 26000 hits in 8 hours).
I understood that, I was using it as an example of what to look for as well :)

According to logs, it seems it is back to normal...
That's good news, if it does start to occur again and you see those same errors in the cphulk error log I'd suggest opening a ticket which you can do using the link in my signature.
 

apaulo

Member
May 14, 2019
7
0
1
Portugal
cPanel Access Level
Reseller Owner
It's behavior I don't believe is to check prior but the login will always fail due to the country being blocked.
Ok, but
...so why send me a message?
...so why send the IP to one-day blocks?

The country is blocked - period. No need for one-day blocks. No need to populate the firewall with tens of thousands IPs... that is the point of country management.

I changed nothing on CPanel configurations... it just started out of the blue.

Now I cannot have notifications - and I need them (I did not block all countries). I must be able to be notified if someone is trying on a not-blackelisted country and take action.
 

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
7,360
584
263
Houston
cPanel Access Level
DataCenter Provider
Yea, I see the issue with the behavior you're experiencing. Can you please open a ticket using the link in my signature? I'd like to see if there's more to the issue and possibly if an update caused it to behave differently. Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved.


Thanks!
 

keat63

Well-Known Member
Nov 20, 2014
1,364
103
43
cPanel Access Level
Root Administrator
I'm not entirely sure Country Code blocking is working correctly either.
I have China Blacklisted, yet I can see an IP entry, clearly identified as CN, with an expiry date of the 23rd of May.
Yet the same IP tried again two days later on the 10th.

1. How could CN even be given the opportunity to login to Dovecot when CN is blacklisted.
2. Why was it allowed a second opportinuty two days later when it should still be under a CPHULK block ?

image attached.
 

Attachments

Last edited by a moderator:

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
7,360
584
263
Houston
cPanel Access Level
DataCenter Provider
I've been talking to some folks about this and doing some research, it looks like the GeoIP database isn't 100% correct. For example I took the IP listed in @apaulo's initial response and ran it through the geo-ip database here: IP to Country Database (IPV4 and IPV6)

The output was as follows:


Code:
IP    220.233.42.61
Numeric    3,706,268,221
Start    220.233.0.0
End    220.233.127.255
Hosts    32,768
CIDR    220.233.0.0/17
Reg    apnic
Alloc    Feb 08, 2004
CC    AU
Host    Click
Country    Australia
Details    Not Available
This is being reported by GeoIP as an Australian IP address

Then another one of the examples:

Code:
IP    78.9.51.10
Numeric    1,309,225,738
Start    78.8.0.0
End    78.11.255.255
Hosts    262,144
CIDR    78.8.0.0/14
Reg    ripencc
Alloc    Feb 25, 2007
CC    PL
Host    No Hostname
Country    Poland
Details    Not Available
Flag    Poland Start 78.8.0.0 - 78.11.255.255 [Hosts 262,144]
GeoIP shows this as a polish IP

@keat63 can you run your IP example through the GeoIP database and let me know if it's coming up as a blocked country?

You may also want to have a look at the thread here as well: Login Denied Country is Blacklisted?
 

grant-la

Registered
May 18, 2019
1
0
1
Minnesota
cPanel Access Level
Root Administrator
@apaulo I'm seeing the exact same issue you are. I have a very extensive IP blacklist on countries, but suddenly last week sometime I started getting bombarded.

Just found this post because I'm having the exact issues, suddenly. I actually attempted to block all countries except 3 (US/GB/Canada) and still attempts are flowing in.
 

keat63

Well-Known Member
Nov 20, 2014
1,364
103
43
cPanel Access Level
Root Administrator
I can confirm that the country code database isn't 100% accurate, as I have a feature request open for a more accurate database.

As for my example, its coming back as China

Code:
IP 58.216.13.23
Numeric 987,237,655
Start 58.208.0.0
End 58.223.255.255
Hosts 1,048,576
CIDR 58.208.0.0/12
Reg apnic
Alloc Jun 23, 2005
CC CN
Host No Hostname
Country China
 

keat63

Well-Known Member
Nov 20, 2014
1,364
103
43
cPanel Access Level
Root Administrator
Apologies on my part, i wasn't paying attention.

I recall that I had country code blocking disabled for a few weeks during March/April to try and figure out a user email login problem.
These Chinese logins could well have been during this period.

I got a reply back from Cpanel support if it helps.

"If an IP address is blocked by cPhulk, that IP can still attempt to login to the server. In this case, even if they type a correct password, it will fail. This is why you are seeing the IP addresses showing in the logs, even though they are blocked by cPhulk."

I have only 3 country codes whitelisted in CPHULK.
I don't see any attempted connections outside of these 3 countries since April 10th, so maybe it is working for me.
 
  • Like
Reactions: cPanelLauren

daflame

Member
Oct 7, 2015
6
0
1
Beloeil Canada
cPanel Access Level
Root Administrator
On my side, all countries in cPHulk Brute Force Protection are blacklisted except Canada and USA but since few weeks, I started to receive ton of email notifications about Excessive Number of Failed Login Attempts from many countries (China, Brazil, France, Spain, etc). I received 14000 email notifications since may 21. What's wrong?
 

cPanelLauren

Forums Analyst II
Staff member
Nov 14, 2017
7,360
584
263
Houston
cPanel Access Level
DataCenter Provider
We have a case open for this specific behavior @daflame - CPANEL-27445 - In 80 cPHulkd sends notifications for blocked logins from blacklisted IPs/Countries. This doesn't mean they are no longer blacklisted just that the notifications are being sent now. This is an improvement case and I'll update here when/if any changes are made.
 
  • Like
Reactions: CyberPepe