Advice on the use of a blocklist within CSF

bloatedstoat

Well-Known Member
Jun 14, 2012
184
24
68
Victoria, Australia
cPanel Access Level
Root Administrator
Looking for a little advice on the use of a blocklist within CSF.

Of late one of my servers has been getting hammered with brute force attacks on mail accounts - mostly distributed IMAP attacks; having captured and recorded a substantial amount of the offending IP addresses I ran a comparison between what I've collected and the IPs listed in http://lists.blocklist.de/lists/mail.txt and pretty much all of them are on the list, as such, to me, it's a high value blocklist ideal for deployment.

Thing is, the list itself contains roughly 24,000 IP addresses.

I've currently set my own config' to pull a meagre 2000 [as per below]. I would ideally like to go much, much higher to realise the full benefits of the list - but I have to admit I'm unsure how high I can actually go without causing serious server performance issues and as such hesitant to tinker too much on a production server.

Code:
# Name: BLOCKLIST_DE_MAIL
# Category: attacks
# Maintainer: Blocklist.de
# Maintainer URL: https://www.blocklist.de/
# Information: All IP addresses which have been reported within the last 48 hours as having run attacks on the service Mail, Postfix.
BLOCKLIST_DE_MAIL|86400|2000|http://lists.blocklist.de/lists/mail.txt
Has anyone deployed this list or any other list that has such abundant entries within CSF with Faststart and IPSet enabled?

This particular machine is reasonably specified: 4 core 2.3GHz, 16GB RAM.

Thanks
 

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
I run over 35,000 IPSet entries in CSF between the Blocklists and Country Codes etc, without any discernable performance impact.

8 core 3.40GHz, 8GB RAM

Everything I have read suggests that up to 100,000 entries are easily handled, but I have never tried to load it up that far !
 

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
I did look at the www.blocklist.de site and a couple of things struck me:

It is a shame that their blocklists are published with the IPs sorted into numerical order rather than in the order of 'most recent detected IP on top'. That would enable one to use the CSF max IP addresses setting in the csf.blocklists most effectively. Perhaps they would benefit from some feedback suggesting that, so they could boast of compatibility with CSF as well

An alternative to considerer might be to use the mail.bl.blocklist.de in a custom RBL setting in your exim configuration. I have not tested this, but it might be less invasive for the server.
 

bloatedstoat

Well-Known Member
Jun 14, 2012
184
24
68
Victoria, Australia
cPanel Access Level
Root Administrator
I agree with you totally, there are some blocklists listed at https://store.delmarvagroup.com/index.php?rp=/knowledgebase/4192/CSF-BlockLists.html that offer chronological and hit based ordered lists, I have not tried them. I only tried the one I mentioned originally purely on the basis that it focusses on my problem area - mail. After I ran my comparisons against it from data collected it produced so many hits I never even bothered to try the others.

I actually added the mail.bl.blocklist.de as an RBL in my Exim config but never enabled it, after I had added it I thought to myself that brute force attacks target Dovecot and not Exim. I'm trying to repel brute force at the moment. Please correct me if I am wrong. Every day is a school day :)
 

Infopro

Well-Known Member
May 20, 2003
17,076
524
613
Pennsylvania
cPanel Access Level
Root Administrator
Twitter
so they could boast of compatibility with CSF as well
They're already in here:

/etc/csf/csf.blocklists

Code:
# Blocklist.de
# Set URLGET in csf.conf to use LWP as this list uses an SSL connection
# Details: https://www.blocklist.de
# This first list only retrieves the IP addresses added in the last hour
BDE|3600|0|https://api.blocklist.de/getlast.php?time=3600
# This second list retrieves all the IP addresses added in the last 48 hours
# and is usually a very large list (over 10000 entries), so be sure that you
# have the resources available to use it
#BDEALL|86400|0|http://lists.blocklist.de/lists/all.txt
 

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
I was basing my suggestion on their listing on the www.blocklist.de/en/download.html#services page.
The following service-names can be parsed:

mail:
mail, postfix-blacklist, postfix, exim, postfix2, exim4, postfix-550, postfix550, postfix-554, postfix-blacklst, smtp, postfix-gl, sendmail, postfix-bl, exim-relay, postfix-strict, postfix-connection, postfixblacklist, postfix-tcpwrapper, postfix-rejected, postfix-spamers550, plesk-postfix, mail-ban, postfix-554-3, postfix-550-2, exim-greylist, postfix-554-2, postfix-450
Possibly a stretch, but since they parse exim data for their reports, I shouldn't be surprised if the blacklists contain exim generated IPs. (I might be way off track in how I am interpreting their pages)
 

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
Wow I am getting behind - I don't have the BDE entry in my CSF either - and it updated last night :(
 

Infopro

Well-Known Member
May 20, 2003
17,076
524
613
Pennsylvania
cPanel Access Level
Root Administrator
Twitter

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
OK the BDE was added on the CSF v7.50 but for existing installs, it can be found in /etc/csf/csf.blocklists.new

Darn it - he beat me to it ..... again !! Thanks @Infopro
 
  • Like
Reactions: Infopro

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
# Blocklist.de
# Set URLGET in csf.conf to use LWP as this list uses an SSL connection
#Details: www.blocklist.de
# This first list only retrieves the IP addresses added in the last hour
BDE|3600|0|https://api.blocklist.de/getlast.php?time=3600
For full disclosure, this list is currently loading 3686 entries - which I think is a great compromise as it is updated every hour, and doesn't look as if it would overwhelm the server resources even if IPSet wasn't used.
 

bloatedstoat

Well-Known Member
Jun 14, 2012
184
24
68
Victoria, Australia
cPanel Access Level
Root Administrator
Thank you @rpvw and @Infopro for your support.

I ended up using the http://lists.blocklist.de/lists/mail.txt file in its entirety, just shy of 24,000 entries on its last load.

I initially scaled the amount of entries retrieved upwards throughout the day whilst monitoring performance. I am now pulling down the entire list with IPSet registering just under 30,000 entries all up with no noticeable difference in performance.

The effect on the brute force nonsense has been astonishing.

Do either of you know how much the implementation will affect boot time upon reboot with so many entries?

Thanks.
 
  • Like
Reactions: cPanelLauren

rpvw

Well-Known Member
Jul 18, 2013
1,101
470
113
UK
cPanel Access Level
Root Administrator
My understanding is that it won't affect boot time at all. The use of IPSet reads the IPs from an indexed data structure rather than reading them in from the iptables linear file, which would have to read all the IPs into memory during the boot process. This is why boot can fail with huge iptables lists consuming large amounts of memory, and why one needs to off-load as many of the IPs as possible into IPSet.

This is from CSF
Using ipset moves the onus of ip matching against large lists away from iptables rules and to a purpose built and optimised database matching utility. It also simplifies the switching in of updated lists