Help with Globally blocking Bad Bots

[360webfirm]

Hello Everyone,

I am running several c panel servers using CentOS 7 and I am experiencing a lot of bandwidth usage on a lot of websites by bots.

Mainly these ones:

• Unknown robot identified by \*bot
• MJ12bot

Is there a good way to block these two Globally in WHM? I have been searching and trying with plugins on the wordpress sites, but they are not working.

Thank you so much for your help.

Steve

 

[dalem]

there is a Mod security rule in this thread you can use
Just edit the rule to add the bots you want, the MJ12bot is in the thread

SOLVED - Blocking bad bots

Hi! I have seen lots of bots accessing my websites on my VPS. For now i just block IPs temporarily using CSF, but i would like to have a better and global solution. So, i'm thinking in 2 options... first: Apache Configuration -> Include Editor -> “Pre Main Include” SetEnvIfNoCase...

forums.cpanel.net

 

[360webfirm]

That above thread is all over the place and I dont know exactly what to use, so can you PLEASE post the solution here so I can see exactly what I need to use and where to use it. I have 8 servers using C Panel and would like your help with this please..

Below is a code I added, please let me know if this would work. If not, please tell me the code I need to add with the example BOTS I have below.

 

[360webfirm]

I have this in Apache Configuration, Include Editor:

<Directory “/home”>

SetEnvIfNoCase User-Agent “MJ12bot” bad_bots

SetEnvIfNoCase User-Agent “AhrefsBot” bad_bots

SetEnvIfNoCase User-Agent “SemrushBot” bad_bots

SetEnvIfNoCase User-Agent “Baiduspider” bad_bots

SetEnvIfNoCase User-Agent “YandexBot” bad_bots

SetEnvIfNoCase User-Agent “SeznamBot” bad_bots

SetEnvIfNoCase User-Agent “DotBot” bad_bots

SetEnvIfNoCase User-Agent “MauiBot” bad_bots

<RequireAll>

Require all granted

Require not env bad_bots

</RequireAll>

</Directory>

Does this not work? I really need to stop these BOTS as some are using so much of my bandwidth.

 

[dalem]

Use the Mod security rule as I suggested
& use it in conjunction with csf to ban the IP's

 

[dalem]

The include editor even if it worked would do would still allow the bad bots connect to your server

Post 11 if you want to try each one by themselves you should be able to combine them then you can tweak the syntax for each rule

 

[360webfirm]

What I have done is added this rule in the mod security, add rules section. Can you tell me if this would work please.

SecRule REQUEST_HEADERS:User-Agent "@rx ^(?:MJ12bot|AhrefsBot|SemrushBo|Baiduspider|YandexBot|SeznamBot|DotBot|MauiBot)$" "msg:'Spiderbot blocked',phase:1,log,id:777777,t:none,block,status:403"

Thanks so much for all your help,

Steve

 

[dalem]

How Would I know if it worked ??

check your apache error logs & the hit list in the mod security section in WHM

 

[360webfirm]

Silly question, but where exactly can I find the Mod Security logs? I was looking at ther raw files in all c panel accounts.

 

[cPanelLauren]

If you've set ModSecurity to log at WHM>>Security Center>>ModSecurity Configuration - > Audit Log Level it will log to /var/log/modsec_audit.log

 

[dalem]

In addition to that or if it not enabled it wil log to
/var/log/apache2/error_log