SOLVED ModSecurity: IP whitelisting doesn't work

[serpent_driver]

Hello,

I have added exclude rule to ModSecurity in /etc/apache2/conf.d/modsec/modsec2.user.conf to whitelisting Googlebot from being blocked, but it doesn't work. Googlebot will still blocked if accessing to robots.txt.

SecRule REMOTE_ADDR "^66\.249\.xxx\.xxx$" "id:1,phase:1,nolog,allow,ctl:ruleEngine=Off"

What is wrong with my rule?

Thank you
Michael

 

[cPRex]

Hey there! Can you let me know specifically how you are seeing Google being blocked from the system? Are there entries in the ModSecurity log showing that IP address being blocked?

 

[serpent_driver]

From modsec_audit.log


--46acc7c7-A--
[08/Dec/2020:23:26:06 +0100] -jheB94ktFAAXppPXCmZsvqH 66.249.66.214 63716 xxx.xxx.xxx.xxx:80 80
--46acc7c7-B--
GET / HTTP/1.1
Host: xxx.xxx.xxx.xxx // My Server IP removed
AMP-Cache-Transform: google;v="1..5"
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/signed-exchange;v=b3,application/xml;q=0.9,*/*;q=0.8
From: googlebot(at)googlebot.com
User-Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Accept-Encoding: gzip,deflate,br

Message: Access denied with code 403 (phase 2). Test 'REQUEST_HEADERS:Host' against '@rx ^[\d.:]+$' is true. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP3/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "735"] [id "920350"] [msg "Host header is a numeric IP address"] [logdata "xxx.xxx.xxx.xxx"] [severity "WARNING"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [tag "PCI/6.5.10"] [MatchedString "xxx.xxx.xxx.xxx"]


 

[cPRex]

Thanks for the additional details. You may want to allow the user-agent itself rather than the IP address in case that changes in the future, as shown here:

Mod Security Rules Incorrectly Blocking Googlebot

Hello everyone I'm using CSF Firewall in my blog and the problem is the Mod_Security rules incorrectly blocks Googlebot regularly. It is happening almost every week. In past I have removed a Mod Security rule which was blocking Googlebot and now a new Mod Security rule 950005 is blocking...

forums.cpanel.net

Could you try that instead and see if you get better results?

 

[serpent_driver]

I replaced rule with REQUEST_HEADERS rule, but this rule is missing an ID. I corrected it and will watch it. If it works this rule is no good idear. Everybody can fake its User Agent, so why doesn't IP exeption work?

 

[cPRex]

Your IP address rule looks correct from everything I am seeing. Could you try adding this rule directly from the WHM interface through ModSecurity Tools >> Rules List >> Add Rule to see if that gets you better results?

 

[serpent_driver]

Done, but just found a new issue with ModSecurity related to the current issue. I have just installed Matomo (Dev Release), a analytics software like Google Analytics. For development a plugin it is necessary to generate fake requests. These fake requests are done by Matomo function, but executed by me. In ModSecurity Tools I have now hundreds of entries with my own IP and log shows " Access denied with code 403 ", but I wasn't blocked. Is it because I am whitelisted in cPHulk or has ModSecurity a (huge) malfunction?

 

[cPRex]

cPHulk shouldn't have any interaction with ModSecurity, so it seems like there may be something else happening with that system. It might be best to get a ticket submitted to our team so we could take a look at that directly on your system.

If you put in that ticket please post the number here so I can follow along and post an update for the community.

 

[serpent_driver]

Okay, but I will wait if the current changes work to have a final status.
Thank you for help.

btw. If a new rule has been added in WHM -> ModSecurity Tools and not by adding it in modsec2.user.conf such rule has to be enabled in ModSecurity Tools first to get it work.

 

[serpent_driver]

Update and new status

My rule set for whitelisting Google IP works, but don't know why it works now, but not before. I have changed nothing, but only restart ModSecurity and LSWS. (Already done before more than once). So the new status is: Everything works as it should and there is no (more) issue with ModSecurity and IP whitelisting. Issue can be set to solved.

Thank you for help!

 

[cPRex]

I'm glad it's working well for you now!