Is anyone good with CSF custom regex ?

[keat63]

I reguarly see the follwing error in my logs.

Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622

I believe caused by unauthorised cpanel login attempts.

Whilst I have HAC configured to only allow a small number of IP's cpanel access, i'd still like CSF to add the offending IP to the blocklist, but i've absolutely no idea how to write a custom regex.
I've asked a few times on the CSF forum, but never had a reply.

Are there any regex guru's on here who could help ?

 

[cPanelLauren]

Hi @keat63

That message is a result of using Host Access control to limit access to the server. I know you mentioned you'd posted to the CSF forums already but I did find a thread called "Custom REGEX rules for CSF." which has quite a few examples of which you could most likely modify to suit your needs

 

[keat63]

I too have seen that thread, however, it might as well be written in a foriegn langauge as it means absolutely nothing to me.
It's just a series of numbers and commas.

 

[fuzzylogic]

Two requests.

1. Could you post the full line of that log message? (anonymizing IP, username or host name)
   Knowing the date, time formats, number of spaces etc. before the word "Dropping" allows for much better performing regex.
2. Which log file does that line occur in? (I am assuming /var/log/messages)

 

[keat63]

I recreated this using my mobile phone, which is not included in HAC.

The message appears in usr/local/cpanel/logs/error_log

No server names or time stamps just the following message.

Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622.

 

[fuzzylogic]

Actual version for Keat because his errors have no date/time stamp or hostname.
Sample log line...

Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers

In /etc/csf/csf.conf at about line 2600 (40 lines from end of file) set...

CUSTOM1_LOG = "/usr/local/cpanel/logs/error_log"

Then save - Hint: this edit can not be done from within the CSF ui, it must be done by ssh console

In /usr/local/csf/bin/regex.custom.pm
Add the following code...

if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /^Dropping connection from (\S+) because of tcp_wrappers/)) {
    return ("5 cPanel login attempts from IP not in Host Access Control list",$1,"keats_hammer_4","5","2077,2078,2082,2083,2086,2087,2095,2096","1");
}

Restart lfd
Generate 5 test log lines either using your mobile phone as you posted or...
by using the following command from a ssh console (in which you can use any valid ip address)...

echo "Dropping connection xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622" >> /usr/local/cpanel/logs/error_log

This should achieve blocking of the ip
Depending on your settings in /etc/csf/csf.conf the block can take 2 forms...
If

LF_TRIGGER = 0
LF_TRIGGER_PERM = 1

You will get permanent block with ip added to deny list

Read the following from /usr/local/csf/bin/regex.custom.pm to acheive temperary blocking or port specific blocking.

# Example:
#    if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /^\S+\s+\d+\s+\S+ \S+ pure-ftpd: \(\?\@(\d+\.\d+\.\d+\.\d+)\) \[WARNING\] Authentication failed for user/)) {
#        return ("Failed myftpmatch login from",$1,"myftpmatch","5","20,21","1");
#    }
#
# The return values from this example are as follows:
#
# "Failed myftpmatch login from" = text for custom failure message
# $1 = the offending IP address
# "myftpmatch" = a unique identifier for this custom rule, must be alphanumeric and have no spaces
# "5" = the trigger level for blocking
# "20,21" = the ports to block the IP from in a comma separated list, only used if LF_SELECT enabled
# "1" = n/temporary (n = number of seconds to temporarily block) or 1/permanant IP block, only used if LF_TRIGGER is disabled

# In my test the ip was added to the deny list and the following email was generated...
-----
Time: Tue Apr 10 17:13:04 2018 +1000
IP: xxx.xxx.xxx.xxx (CN/China/-)
Failures: 5 (keats_hammer_4)
Interval: 86400 seconds
Blocked: Permanent Block [LF_CUSTOMTRIGGER]

Log entries:

Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622
Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622
Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622
Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622
Dropping connection from xxx.xxx.xxx.xxx because of tcp_wrappers at cpsrvd.pl line 3622
-----

 

[keat63]

I followed the instructions, but it CSF doesn't seem to block the IP.
Initially, I used the echo method, and after a few attempts discovered a slight syntax error. (missing from)
After it still didn't add the IP, i tried from my phone.
I can see my IP is generating the string, but still doesn't seem to add my IP to CSF block list.

LF_TRIGGER = 0
LF_TRIGGER_PERM = 1

CUSTOM1_LOG = "usr/local/cpanel/logs/error_log"
CUSTOM2_LOG = "/var/log/customlog"
CUSTOM3_LOG = "/var/log/customlog"
CUSTOM4_LOG = "/var/log/customlog"
CUSTOM5_LOG = "/var/log/customlog"
CUSTOM6_LOG = "/var/log/customlog"
CUSTOM7_LOG = "/var/log/customlog"
CUSTOM8_LOG = "/var/log/customlog"
CUSTOM9_LOG = "/var/log/customlog"

#!/usr/local/cpanel/3rdparty/bin/perl
###############################################################################
# Copyright 2006-2015, Way to the Web Limited
# URL: http://www.configserver.com
# Email: [email protected]
###############################################################################
sub custom_line {
   my $line = shift;
   my $lgfile = shift;

# Do not edit before this point
###############################################################################
#
# Custom regex matching can be added to this file without it being overwritten
# by csf upgrades. The format is slightly different to regex.pm to cater for
# additional parameters. You need to specify the log file that needs to be
# scanned for log line matches in csf.conf under CUSTOMx_LOG. You can scan up
# to 9 custom logs (CUSTOM1_LOG .. CUSTOM9_LOG)
#
# The regex matches in this file will supercede the matches in regex.pm
#
# Example:
#   if (($globlogs{CUSTOM1_LOG}{$lgfile}) and ($line =~ /^\S+\s+\d+\s+\S+ \S+ pure-ftpd: \(\?\@(\d+\.\d+\.\d+\.\d+)\) \[WARNING\] Authentication failed for user/)) {
#       return ("Failed myftpmatch login from",$1,"myftpmatch","5","20,21","1");
#   }
#
# The return values from this example are as follows:
#
# "Failed myftpmatch login from" = text for custom failure message
# $1 = the offending IP address
# "myftpmatch" = a unique identifier for this custom rule, must be alphanumeric and have no spaces
# "5" = the trigger level for blocking
# "20,21" = the ports to block the IP from in a comma separated list, only used if LF_SELECT enabled. To specify the protocol use 53;udp,53;tcp
# "1" = n/temporary (n = number of seconds to temporarily block) or 1/permanant IP block, only used if LF_TRIGGER is disabled
# If the matches in this file are not syntactically correct for perl then lfd
# will fail with an error. You are responsible for the security of any regex
# expressions you use. Remember that log file spoofing can exploit poorly
# constructed regex's


if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /^Dropping connection from (\S+) because of tcp_wrappers/)) {
    return ("5 cPanel login attempts from IP not in Host Access Control list",$1,"keats_hammer_4","5","2077,2078,2082,2083,2086,2087,2095,2096","1");
}

###############################################################################
# Do not edit beyond this point

   return 0;
}

1;

I can see the string in the email though.
Maybe I put the regex in the wrong place ??

 

[fuzzylogic]

You have...
CUSTOM1_LOG = "usr/local/cpanel/logs/error_log"
in /etc/csf/csf.conf
I posted...
CUSTOM1_LOG = "/usr/local/cpanel/logs/error_log"

I forgot to add that after editing /etc/csf/csf.conf cfs should be restarted.

 

[fuzzylogic]

Your custom regex appears to be in the right place and identical to my working version.

 

[keat63]

You have...
CUSTOM1_LOG = "usr/local/cpanel/logs/error_log"
in /etc/csf/csf.conf
I posted...
CUSTOM1_LOG = "/usr/local/cpanel/logs/error_log"

I looked at that for about 2 minutes, thinking, I don't see where the problem is.
Then I spotted the preceeding /

Guess what, it now works.

123.123.123.123 # lfd: (keats_hammer_4) 5 cPanel login attempts from IP not in Host Access Control list 123.123.123.123 (CN/China/Beijing/Beijing/-): 5 in the last 3600 secs - Tue Apr 10 13:10:33 2018

This issue has taken me at least 2 years to finally get a resolution.
I owe you a beer.

Would you mind me sharing this on the CSF forum. ?

 

[fuzzylogic]

I.m OK wit you adding it to Serge's list of CSF custom regex.
Seems appropriate for that thread.

 

[jeffschips]

Hello. I've been following this thread and have a follow up question:

Joomla has log files stored in domain/administrator/logs/error.php and they look like this:

2019-10-12T22:02:41+00:00 INFO xx.xx.xxx.xxx joomlafailure Username
and password do not match or you do not have an account yet.
2019-10-12T22:02:46+00:00 INFO xx.xx.xxx.xx joomlafailure Username
and password do not match or you do not have an account yet.
2019-10-12T22:02:54+00:00 INFO xx.xxx.xxx.xx joomlafailure Username
and password do not match or you do not have an account yet.

NOTE: I believe those are tabs and not spaces between the words. On this forum they look like spaces, but when viewed in a text editor they are tabs.

I've been attempting to make csf recognize the following regex but getting nowhere. Perhaps a regex or csf expert can pipe in?

if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /\bjoomlafailure/)) {return ("5 failed Joomla logins for blablabla",$3,"blablablareport","2","443","360","0");}

CUSTOM1_LOG in csf.conf is:

CUSTOM1_LOG = "/home/domain/public_html/domain.com/administrator/logs/error.php"

csf login failure blocks and alerts has LF_TRIGGER = "0" and I'm assuming that if the /usr/local/csf/bin/regex.custom.pm has a regex in it for CUSTOM1_LOG then I'm assuming regex.custom.pm would override the LF_TRIGGER settings, or maybe that's not necessary.

Thanks.

 

[fuzzylogic]

I'm sure you have checked the source code of...
/home/domain/public_html/domain.com/administrator/logs/error.php
and it is in fact a log file, not a dynamic php file.

If it's source is in fact multiple lines with the text...
2019-10-12T22:02:54+00:00 INFO xx.xxx.xxx.xx joomlafailure Username and password do not match or you do not have an account yet.

And it's path is exactly...
/home/domain/public_html/domain.com/administrator/logs/error.php

Then the following custom-rexex rule should work...

if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /\\S+(?:\t|\s)INFO(?:\t|\s)(\S+)(?:\t|\s)joomlafailure/)) {
    return ("5 Joomla login failures",$1,"joomlafailurechecker","5","80,443","1");
}

In my tests the regex worked with the log lines you posted with either tab characters or space characters separating the values.
I did not test it on an actual Joomla site and do not have the time to do so.
So any further troubleshooting is in your hands.

 

[jeffschips]

Thank you! I will verify and report back. I'm a bit stumped on how the regex calculates the $1 as the ip address. In the php file which is holding the log file entries, the ip address is the third entry in each line output, so why is the $1 used versus $3?

As well I'm assuming the block is *only* for that ip against those ports. If I wanted to block the ip system-wide regardless of port, do we leave those entries blank?

Thank you for the excellent and timely response.

 

[jeffschips]

Well, I'm seeing no joy.
running the command:
file error.php
error.php: ASCII text
so that's a text file.
CUSTOM1_LOG = "/home/domain/public_html/domain.com/administrator/logs/error.php"
LF_TRIGGER = "0"
LF_TRIGGER_PERM = "1"
LF_SELECT = "0"
restarting csf with csf -x and then csf -e shows no errors and systemctl status lfd and csf shows csf operational.

I've double-checked all paths. The entries are indeed being added to the error.php file. I thought maybe it was a permissions issue so temporarly moved the log files to the website root, but that didn't' help.

I'm pretty much stumped at the moment.

 

[jeffschips]

SOLVED:
The suggested regex appeared to have an error:

if (($lgfile eq $config{CUSTOM1_LOG}) and ($line =~ /\\S+(?:\t|\s)INFO(?:\t|\s)(\S+)(?:\t|\s)joomlafailure/)) {
return ("5 Joomla login failures",$1,"joomlafailurechecker","5","80,443","1");
}

I had to remove one backslash here, changed:

($line =~ /\\S
to:
($line =~ /\S

now it works!

 

[fuzzylogic]

You are correct that extra backslash was a copy paste error on my part. Good Spotting. They can be hard to see.
As I said I tested the regex but not in place on a cPanel server.

Look to the regex to understand the $1 and $3 rather than the log line. Regex is ignorant of what it is parsing (to it every character, including white space characters either matches or does not )
The thing that is output by $1 or $3 is named capture group1 or capture group3.
It is a very basic type of string variable that regex generates.
A capture occurs when a regex is in brackets.
Brackets also have a 2nd purpose of containing a regex fragment such as a pipe OR syntax, which without the brackets would mean OR the rest of the regular expression.
(joomlafailure) would capture the string joomlafailure when it occured and could be output by $1
We are capturing the ip address with (\S+)
Why is it then $1 when it is the third group of brackets in the expression?
I used (?:\t|\s) to match tab OR space but explicitly do not capture.
If I used (\t|\s) it would have still matched but would have captured the space or tab to $1 and $2 (which is a sensless waste of memory, even if a very small waste)
The ip address then would have been in $3

Glad you got it to work.

 

[jeffschips]

You are correct that extra backslash was a copy paste error on my part. Good Spotting. They can be hard to see.
As I said I tested the regex but not in place on a cPanel server.

Look to the regex to understand the $1 and $3 rather than the log line. Regex is ignorant of what it is parsing (to it every character, including white space characters either matches or does not )
The thing that is output by $1 or $3 is named capture group1 or capture group3.
It is a very basic type of string variable that regex generates.
A capture occurs when a regex is in brackets.
Brackets also have a 2nd purpose of containing a regex fragment such as a pipe OR syntax, which without the brackets would mean OR the rest of the regular expression.
(joomlafailure) would capture the string joomlafailure when it occured and could be output by $1
We are capturing the ip address with (\S+)
Why is it then $1 when it is the third group of brackets in the expression?
I used (?:\t|\s) to match tab OR space but explicitly do not capture.
If I used (\t|\s) it would have still matched but would have captured the space or tab to $1 and $2 (which is a sensless waste of memory, even if a very small waste)
The ip address then would have been in $3

Glad you got it to work.

 

[jeffschips]

So to be clear then the $some-number captures the item represented by the numerical position of the captured item contained in parenthesis - you said brackets but I think you mean parenthesis.

 

[fuzzylogic]

Almost. (and yes parentheses not brackets)

(regex1)regex2(?:regex3)regex4(regex5)

The value of capture group 1 = regex1
It can be output in the scripting language using the regex using the $1 syntax.

The $x syntax is a type of getter - get value where key = x

The value of capture group 2 = regex5
It can be output in the scripting language by using $2

regex2 and regex4 are not captured at all because they are not inside of any capture syntax.

regex3 is not captured because it is inside brackets which have the DO NOT CAPTURE syntax.
(?:regex)