The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

ModSecurity is logging but not blocking (integration with ConfigServer)

Discussion in 'Security' started by shacker23, Oct 28, 2014.

  1. shacker23

    shacker23 Well-Known Member

    Joined:
    Feb 20, 2005
    Messages:
    263
    Likes Received:
    1
    Trophy Points:
    16
    I feel like I'm missing something with the new ModSecurity tools. I have a single rule (Malicious Bots) set up, which is logging 10,000+ hits per day. I have both "Connections Engine" and "Rules Engine" set to "Process Rules". And yet I'm not seeing the decrease in bandwidth usage that I would expect when blocking all these malicious bots.

    I have heard that ConfigServer Firewall can prevent ModSecurity from processing rules, but haven't been able to find instructions on how to prevent it from doing that.

    Cpanel's modsecurity appears to log to /usr/local/apache/logs/modsec_audit.log (is this correct, or is there another logging location?). In CSF configuration, I set MODSEC_LOG to that path and restarted CSF.

    So then I tail -f /var/log/lfd.log . I see CSF's normal block entries showing up in lfd.log, but no sign of the tons of hits that are showing up in the ModSecurity log watcher in the gui.

    Have I got things wired up wrong, or am I looking in the wrong place?

    Is it intended that cPanel ModSecurity will only log but not block? Is there a recipe for getting the two to play nice together?

    Thanks.
     
  2. quizknows

    quizknows Well-Known Member

    Joined:
    Oct 20, 2009
    Messages:
    940
    Likes Received:
    55
    Trophy Points:
    28
    cPanel Access Level:
    DataCenter Provider
    cPanel changes in 11.46 are a bit confusing even for me, and I'm very experienced deploying ModSecurity.

    Check the apache error log; cPanel (at least used to) clears the audit log hourly to dump the hits into a database. This results in losing a lot of valuable data (like POST payloads, but that's a discussion for another day).

    Anyway, in /usr/local/apache/conf/modsec2.conf (which is what the main interface modifies) you should see this line:

    SecDefaultAction "phase:2,deny,log,status:406"

    This should deny any requests matching rules, unless the rule has a different action (allow, pass) than the "deny" here.

    The apache error log should let you know if it's actually blocking stuff, or simply matching but allowing it through.

    edit; I've never had problems using both CSF and ModSecurity, at least in a WHM 11.44 and before world.
     
    #2 quizknows, Oct 28, 2014
    Last edited: Oct 28, 2014
  3. shacker23

    shacker23 Well-Known Member

    Joined:
    Feb 20, 2005
    Messages:
    263
    Likes Received:
    1
    Trophy Points:
    16
    Thanks quizknows. My modsec2.conf does say

    SecDefaultAction "phase:2,deny,log,status:406"

    but I do not see ModSecurity hits in the apache error_log. If I

    grep -i security /usr/local/apache/error_log

    the only hits I see are of the form:

    [Tue Oct 28 07:46:07 2014] [error] [client 143.179.210.200] ModSecurity: Audit log: Failed to unlock global mutex: Identifier removed [hostname "domain.com"] [uri "/index.php"] [unique_id "VE@q6tNcAAF7@ffkAAAAJ"]

    (details tweaked for privacy).

    I'm not sure what to make of that - what it means or whether it's related to seeing no actual removals happening in error_log. Theories?

    Thanks.
     
  4. quizknows

    quizknows Well-Known Member

    Joined:
    Oct 20, 2009
    Messages:
    940
    Likes Received:
    55
    Trophy Points:
    28
    cPanel Access Level:
    DataCenter Provider
    Make sure that /usr/local/apache/conf/modsec2.cpanel.conf (if it exists, otherwise use modsec2.user.conf) has this line:

    SecRuleEngine On
     
  5. shacker23

    shacker23 Well-Known Member

    Joined:
    Feb 20, 2005
    Messages:
    263
    Likes Received:
    1
    Trophy Points:
    16
    It does. Though, oddly, "On" is not in quotes like it is in the other directives there. Should I quote "On"?

    # WHM-managed ModSecurity configuration directives

    SecAuditEngine "On"
    SecRuleEngine On

    SecConnEngine "On"
     
  6. quizknows

    quizknows Well-Known Member

    Joined:
    Oct 20, 2009
    Messages:
    940
    Likes Received:
    55
    Trophy Points:
    28
    cPanel Access Level:
    DataCenter Provider
    The quotes should not be necessary, I've never used them.

    Here's what I would try next, add a custom rule such as this one:
    Code:
    #generic path recurision sig
    SecRule REQUEST_URI "/\.\./" "id:1000825"
    
    Restart apache so the rule is active.

    Then go to:
    Code:
    yoursite.com/some.php?something=../../etc
    
    (some.php doesn't even need to exist).

    If modsec is working, you should be greeted with a 500 or 406 error for the above, and see it logged.
     
    #6 quizknows, Oct 28, 2014
    Last edited: Oct 28, 2014
  7. deka

    deka Member

    Joined:
    Jul 24, 2014
    Messages:
    17
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Root Administrator
    Since the upgrade to 11.46 I'm not sure if mod_security is doing anything now. I see two tables, one named hits that is growing/recording but the other one named modsec isn't recording anything now. Mod_security has been working for years for me, now I just feel my system is vulnerable.

    I pretty sure modsec2.cpanel.conf wasn't in my system pre update - can it be deleted as I haven't enabled anything in the WHM *new feature*?
     
  8. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,743
    Likes Received:
    662
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Feel free to open a support ticket using the link in my signature if you want us to take a closer look. You can post the ticket number here so we can update this thread with the outcome.

    Thank you.
     
  9. Brian

    Brian Well-Known Member

    Joined:
    Dec 1, 2010
    Messages:
    117
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Texas
    cPanel Access Level:
    Root Administrator
    While I can't speak for the ConfigServer integration issues, I can say that we have not changed how ModSecurity produces its log file, where it produces its log file, or anything with the ModSecurity Apache module itself with the recent ModSecurity revamp. You should have experienced no functionality change when upgrading from 11.44 to 11.46 with any existing rules you had setup. If you did experience functionality change to existing rules simply by updating cPanel & WHM, please open a ticket so we can investigate.

    The only major changes made were:

    • Complete revamp of the WHM UI for ModSecurity
    • Creation of "ModSecurity Domain Manager" in cPanel UI
    • Deprecation of modsec.modsec table (no longer used)
    • Creation of modsec.hits table (new structure/table for storing parsed log data)
    • Creation of /usr/local/apache/conf/modsec2.cpanel.conf to hold persistent ModSecurity config changes made via WHM UI, rule disable entries, and future ModSecurity functionality coming in 11.48

    The error you're seeing leading to ModSecurity not enforcing blocks seems to be outside of the changes we've made (an "Audit log: Failed to unlock global mutex: Identifier removed" error would hint to system level errors), but I couldn't say with 100% certainty without our support team further investigating. Have you opened a support ticket? What is the ticket number?
     
  10. shacker23

    shacker23 Well-Known Member

    Joined:
    Feb 20, 2005
    Messages:
    263
    Likes Received:
    1
    Trophy Points:
    16
    Many thanks for the information Brian - very useful.

    Yes, ticket 5636595 is open, and a tech has done some work on this. I'll follow up with him on that ticket and post the results here when we get everything worked out.

    Thanks.
     
  11. shacker23

    shacker23 Well-Known Member

    Joined:
    Feb 20, 2005
    Messages:
    263
    Likes Received:
    1
    Trophy Points:
    16
    OK, several things to report:

    - cPanel support *rocks* - thanks Brian and Peter for the assist in getting this all working.

    - After waiting 24 hours so I could see what the impact on overall traffic was via awstats on various domains, we have success! I'm seeing 25-50% reduction across the hardest-hit sites. And now when I

    tail -f /var/log/lfd.log

    I am now seeing lots of firewall blockage entries put there by mod_security, which means we're getting server-wide blockage for the worst actors. That means CSF is getting its queues correctly from apache's error_log.

    - Important lesson: cPanel's ModSecurity does some validation when you're entering rules, but that's only syntax validation - you can easily enter a non-working rule without getting any error messages. Getting that bad rule removed and some good ones installed was the key here.

    - cPanel techs helped to install a good set of basic rules that actually are working. My apache error_log now shows lots of entries from mod_security.

    - But the ultimate goal was to block bad actors server-wide in the CSF firewall. To get that working make sure you have these two lines in /etc/csf/csf.conf:

    HTACCESS_LOG = "/usr/local/apache/logs/error_log"
    MODSEC_LOG = "/usr/local/apache/logs/error_log"

    and restart CSF. With those in place, I am now seeing lots of entries in /var/log/lfd.log related to mod_security.

    So:

    1) ModSecurity rules block individual matching hits and log their actions to the apache error_log

    2) CSF watches error_log and, when it finds more than x matches on a given IP (where x = the value in CSF's LF_MODSEC setting), that IP gets firewalled server-wide.

    Yay!
     
  12. quizknows

    quizknows Well-Known Member

    Joined:
    Oct 20, 2009
    Messages:
    940
    Likes Received:
    55
    Trophy Points:
    28
    cPanel Access Level:
    DataCenter Provider
    Glad they got you straightened out. I don't know how anyone runs an Apache webserver without ModSecurity.
     
  13. Infopro

    Infopro cPanel Sr. Product Evangelist
    Staff Member

    Joined:
    May 20, 2003
    Messages:
    14,460
    Likes Received:
    195
    Trophy Points:
    63
    Location:
    Pennsylvania
    cPanel Access Level:
    Root Administrator
    Twitter:
    This comment makes it sound like you did not have these set properly. Is that so?
     
Loading...

Share This Page