mod_sec - best way to ID user agents to block?

WorkinOnIt

Well-Known Member
Aug 3, 2016
294
52
78
UK
cPanel Access Level
Root Administrator
Hi Forum

I am new here so greetings to all !

I'm working on the best way to identify and stop bad user agents. I've read multiple threads and googled a lot - so far it seems there is no "clear" or obvious best practice so to speak.

At the moment I've implemented a mod_sec rule to include a txt file with a list of known bad bots etc.... but looking at logs of various user accounts on the server there appear to be a lot of hits from unknown spiders and bots - so how do you mitigate these in the txt file if you don't have a user agent name? What would be the best way to target "unknown bots"

Also - where would one find a list of bad user agents to block?

Thank you
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,883
2,256
463
Hello,

You can use the term "user agent list" on a search engine and find several results with lists of user agents that you may want to block. One of the third-party URLs in the results that looks helpful is:

Identify and block bad robots from website

Thank you.