What is recommended way to detect IP addresses that make junk httpd requests?

[Jelf]

By junk request I mean a request where the first part is legit but the second part consists of folders/files/scripts that are not on my server. Sometimes the same IP will send many such requests.

I see these junk requests in the apache 2.4 error.log

Is there an automated way to detect these junk requests or do I need to write my own php script?

 

[ejsolutions]

CSF allows you to set a limit of 404 errors, which can help to some degree. I usually go with a limit of 60 (the minimum) and a temporary ban of a few hours. Useful but not effective long-term. [You might lock out some poxy Mac that's polling for multiple non-existent favicons, plus perhaps missing files (background images in CSS, for example) by mistake.]

I use a variation (customised by me over a decade ago) of IP Trap Secure your site with an IP trap
It has been pretty successful over the years, though I never took the time to refine it and deploy to my few client sites.
I have it running in a fairly minimalist way i.e. fairly simple to implement. I can to try reverse engineer what I've done, if interested, and no other options are offered here.

Basics of what it does:
I don't have my own e-commerce site, so if some one/bot tries to access mydomain.tld/catalog/index.php they get banned (excluding dumb Google,Bing etc.).

 

[Jelf]

Thanks for the IP Trap link. I will take a look at that.

 

[ejsolutions]

Try version 5.3a - only minor changes required eg. change the variable DIR_WS_INCLUDES to a hard-coded includes.
See the readme file, installation section, for details.

 

[cPanelWilliam]

Hello! You can also look into implementing ModSecurity and/or mod_evasive if you haven't already. Both of those can be useful in blocking bad requests/brute force attacks:

How to install mod_evasive

How to tune mod_evasive

How do I enable Apache Module: ModSecurity®?

 

[Jelf]

Thanks.
I have not yet looked into ModSecurity mod_evasive but will do so.

 

[ejsolutions]

I always add ModSecurity, even if just for default OWASP rules (with some whitelisting/disabled, using ConfigServer ModSecurity Control). IME, it doesn't tackle what you asked i.e. detecting non-existent files, in a particular place.
I haven't taken the time to explore mod_evasive but if @cPanelWilliam could explain the advantages, over CSF, that'd be handy. ;-)

 

[cPRex]

We have a lot more details on mod_evasive here: Apache Module: Evasive | cPanel & WHM Documentation

While it won't handle any large-scale attacks, it can help deal with smaller-scale DoS attacks.