The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

modsec errors

Discussion in 'Security' started by Spork Schivago, Aug 9, 2016.

  1. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    Hello,

    I have these errors in my modsec log file. Are they normal or should I worry?

    Code:
        [Tue Aug 09 12:23:46.751981 2016] [:error] [pid 25520:tid 140358038025984] [client 180.76.15.12] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/REQUEST-20-PROTOCOL-ENFORCEMENT.conf"] [line "299"] [id "960015"] [rev "3"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "Host: example.com"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "example.com"] [uri "/"] [unique_id "V6oDkijphM-boQz4tY@byQAAAI0"]
        [Tue Aug 09 12:23:21.440134 2016] [:error] [pid 25483:tid 140358038025984] [client 180.76.15.141] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/REQUEST-20-PROTOCOL-ENFORCEMENT.conf"] [line "299"] [id "960015"] [rev "3"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "Host: example.com"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "example.com"] [uri "/"] [unique_id "V6oDeU-dXvZ3MKBwtByNrAAAAA0"]
        [Tue Aug 09 11:29:14.271924 2016] [:error] [pid 25520:tid 140358079985408] [client 180.76.15.17] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/REQUEST-20-PROTOCOL-ENFORCEMENT.conf"] [line "299"] [id "960015"] [rev "3"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "Host: example.com"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "example.com"] [uri "/"] [unique_id "V6n2yijphM-boQz4tY@brAAAAIk"]
        [Tue Aug 09 11:28:36.761229 2016] [:error] [pid 25516:tid 140358079985408] [client 180.76.15.34] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/REQUEST-20-PROTOCOL-ENFORCEMENT.conf"] [line "299"] [id "960015"] [rev "3"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "Host: example.com"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "example.com"] [uri "/"] [unique_id "V6n2pAhxWZGr@fprNLDS6AAAAEk"]
    
    I think what worries me is how they all say "[:error]"

    I don't think that's correct. It makes me think that perhaps modsecurity is configured incorrectly or something.

    If I look at /var/log/apache2/modsec_audit.log and go to the very end, I see stuff like:
    Code:
    --6083872e-H--
    Message: Rule processing failed.
    Apache-Handler: default-handler
    ...
    
    If I go up to --6083872e-A-- and look through, I see a spider.html tried getting /robots.txt,
    then received a 404 (I don't have a robots.txt yet).

    So is the Rule processing failed normal as well? There's a whole bunch of them.
     
    #1 Spork Schivago, Aug 9, 2016
    Last edited by a moderator: Aug 10, 2016
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,764
    Likes Received:
    662
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello,

    You can verify if the request was actually blocked by reviewing the entry in the following log file:

    Code:
    /usr/local/apache/logs/modsec_audit.log
    Thank you.
     
    Spork Schivago likes this.
  3. rpvw

    rpvw Well-Known Member

    Joined:
    Jul 18, 2013
    Messages:
    121
    Likes Received:
    34
    Trophy Points:
    28
    Location:
    Spain
    cPanel Access Level:
    Root Administrator
    Hi Spork Schivago,

    I was getting the same entries in the logs, and non of the rules seemed to be being processed. Now I have to admit I did not spend the amount of time that I perhaps should have done in diagnosing why I was getting these errors - what I did find was switching OFF the rules/REQUEST-10-IP-REPUTATION.conf in Home » Security Center » ModSecurity™ Vendors » Select Vendor Rule Sets seemed to fix everything, and there were no more errors and everything worked as I expected.
     
  4. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    Okay. /usr/local/apache/logs is a symbolic link to /etc/apache2/logs which is a symbolic link to /var/log/apache2, so the modsec_audit.log in the /var/log/apache2 directory is the correct modsec_audit.log that I've been looking at.

    This is what I see for those specific entries I listed in post #1. There's a ton of them and that one IP address has connected multiple times over multiple days. This worries me a bit also. I don't see why csf / lfd isn't blocking the IP address. I would think modsec would at least permanently block the address so it couldn't continue to try and attack my server in the future. Obviously, I'm not understanding something here. Blocking the hacking attempts is good, but the attacking IP address should be banned permanently so they cannot continue to attack. I thought this was what modsecurity was for.

    Code:
    --abde3a2d-A--
    [09/Aug/2016:12:23:46 --0400] V6oDkijphM-boQz4tY@byQAAAI0 180.76.15.12 49267 104.238.117.105 80
    --abde3a2d-B--
    GET / HTTP/1.1
    Host: example.com
    Accept-Language: zh-cn
    Connection: close
    User-Agent: Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
    
    --abde3a2d-F--
    HTTP/1.1 301 Moved Permanently
    Location: https://example.com/
    Content-Length: 227
    Connection: close
    Content-Type: text/html; charset=iso-8859-1
    
    --abde3a2d-E--
    
    --abde3a2d-H--
    Message: Rule processing failed.
    Message: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/REQUEST-20-PROTOCOL-ENFORCEMENT.conf"] [line "299"] [id"960015"] [rev "3"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/3.0.0"] [maturity "9"] [accuracy "8"] [tag "Host: example.com"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"]
    Stopwatch: 1470759826750832 1928 (- - -)
    Stopwatch2: 1470759826750832 1928; combined=1317, p1=207, p2=746, p3=0, p4=89, p5=203, sr=69, sw=72, l=0, gc=0
    Response-Body-Transformed: Dechunked
    Producer: ModSecurity for Apache/2.9.0 (http://www.modsecurity.org/); OWASP_CRS/3.0.0.
    Server: Apache
    Engine-Mode: "ENABLED"
    
    --abde3a2d-Z--
    

    That's one of the ones from the first post. It's a pain copying and pasting the text because of the very long lines, so I only did one of the entries but they're all similar. How can I tell from that one entry there if the block actually succeeded or not? To me, the whole Rule processing failed says it did not succeed.

    If you'd like, I can copy and paste the very long OWASP rule ID #: 960015. Just doing a quick search through the modsec_audit.log file, I searched for Rule processing failed and see a whole bunch. But from the very first rule processing failed search in the file, I'm thinking the rule that fails is not the rules that it's reporting on. For instance, I see the message of Rules processing failed, then I see a bunch of rules after that which seems to of succeeded.

    Just to make sure I'm saying this properly, I think with the one I put in the code tags, id # "960015" succeeded but something before that failed. There's nothing the modsec_debug.log file. I wonder if this could be with the local loopback ip address rule I added a while back with the help of some cPanel users. This is the rule I added:

    Code:
    # ...
    # REQUEST-01-COMMON-EXCEPTIONS.conf
    # ...
    #
    # Exception for cPanel scripts (whm-server-status, etc)
    #
    SecRule REMOTE_ADDR "^127.0.0.1" phase:1,nolog,allow,id:'981022',ctl:ruleEngine=off
    
    I don't think this is the problem though. id # 960015 gets processed before rule id # 981022, right? If there was something wrong with my rule (it worked before), then I should see the message about the failure after the rule id # 960015 is processed, not before, right?

    Thanks.
     
    #4 Spork Schivago, Aug 9, 2016
    Last edited by a moderator: Aug 10, 2016
  5. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    Thank you. I guess I'll have to go through one by one, turning off rules to try and figure out which file contains the offending rule. Does anyone have any suggestions on how I could try to get modsec to trigger? For example, let's say I disable the REQUEST-01-COMMON-EXCEPTIONS.conf files...how could I get modsecurity2 to trip and start processing rules, to see if the message pops up?
     
  6. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    I have to go to a class here so I won't be back until around 8PM or so. But when I was searching (I didn't go through all of the entries because the file is very big), I noticed every single one that I looked at that had the rule processing failed show
    REQUEST-20-PROTOCOL-ENFORCEMENT.conf rules processed. Perhaps the bad rule is in REQUEST-20-PROTOCOL-ENFORCEMENT.conf

    The files in /etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/ show they were last accessed, modified and changed on 2016-08-09 @ 01:06:56

    This is around the time upcp gets ran on my system. Was there an update to the OWASP rules? Did something recently change? Wish I had an old copy of the directory so I could compare the difference between the files.

    Thanks.
     
  7. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    I enabled debug level 4 for modsec temporarily and I see some issues. For the rule processing failed one, right before the message, I see in the modsec_debug.log file:
    Code:
    Reciper: Invoking rule 7f9da819d868; [file "/etc/apache2/conf.d/modsec_vendor_configs/OWASP/rules/REQUEST-10-IP-REPUTATION.conf"] [line "60"] [id "981138"].
    Transformation completed in 0 usec.
    Executing operator "rbl" with param "dnsbl.httpbl.org" against TX:real_ip.
    RBL httpBl called but no key defined: set SecHttpBlKey
    Operator completed in 25 usec.
    Operator error: RBL httpBl called but no key defined: set SecHttpBlKey
    Rule returned -1.
    Rule processing failed.
    
    It shows the person (or webcrawler) was visiting cpanel.example.com. I'd like to think the cpanel, whm, webmail, etc subdomains have a valid robots.txt so search engines don't return results for that stuff. If not, I'd like to setup a robots.txt for that.

    Any idea how I go about setting SecHttpBlKey and what I'd set it to? I believe this has something to do with Project HoneyPot, correct? Thanks!
     
  8. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    Okay, I got my Project HoneyPot HTTP:BL key and I put it in WHM's ModSec Project HoneyPot Http:BL API Key's section, to set the SecHttpBlKey.

    Now, I just gotta figure out how to download and setup the Google Safe Browsing Database and the Geolocation Database SecGeoLookupDb. Then I think I'll be all set.

    Thanks.
     
  9. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    For the Geolocation Database, does cPanel support the GeoLite2 Databases or should I be using the legacy databases?
     
  10. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    Hello,

    I think perhaps the error message I was getting was because I did not have a Project HoneyPot API key configured under the Modsecurity Configuration. I would have thought that without the key, modsecurity2 wouldn't be trying to use the API. Once I obtained a free key and plopped it in, I was good to go. Perhaps you should consider getting one as well? They're free and the HTTP:Bl stuff is much like the normal Project HoneyPot stuff, but instead of being used for spam, it's used for bad people.

    For example, let's say my IP address, 192.168.2.2, is attacking lots of sites. If I understand everything correctly, my IP address will get added to the HTTP:Bl database. When I try connecting to someone's site who has modsecurity2 configured and running, along with an HTTP:Bl API key, I'll automatically be blocked, instead of being allowed to the their site.
     
    rpvw likes this.
  11. rpvw

    rpvw Well-Known Member

    Joined:
    Jul 18, 2013
    Messages:
    121
    Likes Received:
    34
    Trophy Points:
    28
    Location:
    Spain
    cPanel Access Level:
    Root Administrator
    Got the key and configured it and switched that rule-set back in - will update tomorrow how we are getting on. Thank you very much.
     
  12. rpvw

    rpvw Well-Known Member

    Joined:
    Jul 18, 2013
    Messages:
    121
    Likes Received:
    34
    Trophy Points:
    28
    Location:
    Spain
    cPanel Access Level:
    Root Administrator
    Overnight logs and modsec rewrites and blocks indicate that it is now working perfectly with all the OSWAP vendor rules enabled.

    Thank you very much for the tip about the Project Honey Pot Http:BL API Key - perhaps the docs could be updated to mention that the rules/REQUEST-10-IP-REPUTATION.conf needs this key configured to run?
     
    Spork Schivago likes this.
  13. Spork Schivago

    Spork Schivago Well-Known Member

    Joined:
    Jan 21, 2016
    Messages:
    276
    Likes Received:
    21
    Trophy Points:
    18
    Location:
    corning, ny
    cPanel Access Level:
    Website Owner
    You're very welcome. I suspect perhaps it's a bug? I would think without the API key configured properly, modsec wouldn't try using rules based on it.

    There's a new version of modsecurity2 coming out, modsecurity 3.0. When that comes, hopefully cPanel implements it. Currently, I get lots of messages in my log files about some mutex failing. This should be fixed in modsecurity 3.0. Also, the way they're writing it, there shouldn't be anymore conflicts between modsec and modRUID2.

    I have problems with my system now. Chkrootkit found something new, something it never found, 2 hidden processes. I believe it's a false warning, it discovered the LKM rootkit....but there's some real weird stuff going on now so I'm going to open a new thread about it..
     
    #13 Spork Schivago, Aug 11, 2016
    Last edited: Aug 11, 2016
Loading...

Share This Page