Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

mod_security Rule execution error - PCRE limits exceeded (-8): (null).

Discussion in 'Security' started by ikillbill, May 10, 2010.

  1. cPanelDon

    cPanelDon cPanel Quality Assurance Analyst Staff Member

    Joined:
    Nov 5, 2008
    Messages:
    2,544
    Likes Received:
    10
    Trophy Points:
    268
    Location:
    Houston, Texas, U.S.A.
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    To help clarify, please be aware it is not required to restart MySQL nor is it required to use a new, separate configuration file to insert the customized ModSecurity directives.

    I recommend using the standard file at the following path that is intended for user customizations:
    Code:
    /usr/local/apache/conf/modsec2.user.conf
    Please keep in mind that the IfModule section is already accounted for in the parent configuration file that includes the "modsec2.user.conf" file.

    The "modsec2.user.conf" configuration file can be safely modified using WebHost Manager via the following menu path: WHM: Main >> Plugins >> Mod Security >> Edit Config

    I would consider placing the directives at the top or beginning of the aforementioned "modsec2.user.conf" configuration file.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  2. Secmas

    Secmas Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    331
    Likes Received:
    2
    Trophy Points:
    168
    @cPanelDon,
    a few months ago I wrote the following post at http://forums.cpanel.net/f185/modsecurity-auto-updater-147745-p3.html#post658046, in there I described how to set the commands for PCRE. Also, I wrote what Brian Rectanus saids about using a big match limit, here is an excerpt of what I wrote in that time:
    I just want to mention this because using a 150,000 limit could be a load problem.

    Regards,

    Sergio
     
  3. cPanelDon

    cPanelDon cPanel Quality Assurance Analyst Staff Member

    Joined:
    Nov 5, 2008
    Messages:
    2,544
    Likes Received:
    10
    Trophy Points:
    268
    Location:
    Houston, Texas, U.S.A.
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Thank you Sergio for the excellent feedback. I agree that an excessive limit would likely be detrimental to system stability, and with that, I believe only the smallest possible values should be used that would still allow mod_security to function under normal circumstances.

    For comparison, the documentation for Mod Security states it uses a PCRE match limit of 1500, by default, and allows use of the appropriate configuration directive to override the value at run-time. As clarified by the detail Sergio (Secmas) provided, I would stress the importance of using caution when increasing the mod_security default match limit of 1500 so as to not introduce system instability. While worth it in my opinion, I believe it may take several tries to find a suitable value that is high enough to allow for normal rule processing while not inadvertently allowing for excessive use of system resources. Further reading in the official PCRE documentation (man pages) leads me to believe that the recursion limit may be useful only when set lower than the match limit; taking this into consideration, I would imagine that testing should include the use of a recursion limit set lower than the match limit, as shown by the following example from Sergio (Secmas) in the thread he mentioned, titled ModSecurity Auto Updater, discussing an automatic updater for mod_security rules:
    Similarly for comparison, the documentation for PHP states it uses a PCRE match limit of 100000 by default; however, I am not sure that I fully understand why the PHP default PCRE match limit would need to be adjusted higher (i.e., increased). My understanding is that the PCRE match limit error, as discussed in this thread, is triggered by the use of PCRE in mod_security rather than the use of PCRE in PHP. As I have not experienced the PCRE match limit errors first-hand, I am curious to know if anyone has seen PHP-specific log information that may be related.

    If anyone has any further insight I believe it will be appreciated.

    For additional reference and clarification, here are a few related resources that I referenced while researching the topic:
    • Mod Security documentation: SecPcreMatchLimit - Configuration Directives - ModSecurity® Reference Manual
      Code:
      [B]SecPcreMatchLimit[/B]
      
      [I]Description:[/I] Sets the the match limit in the PCRE library.
      	     See the pcre_extra field in the pcreapi man page.
      [I]Syntax:[/I] SecPcreMatchLimit value
      [I]Example Usage:[/I] SecPcreMatchLimit 1500
      [I]Processing Phase:[/I] N/A
      [I]Scope:[/I] Global
      [I]Version:[/I] 2.5.12
      [I]Dependencies/Notes:[/I] Default is set at compile (1500 by default)
      The --enable-pcre-match-limit=val configure option will set a custom default
      and the --disable-pcre-match-limit option will resort to the compiled PCRE
      library default.
      
      
      [B]SecPcreMatchLimitRecursion[/B]
      
      [I]Description:[/I] Sets the the match limit recursion in the PCRE library.
      	     See the pcre_extra field in the pcreapi man page.
      [I]Syntax:[/I] SecPcreMatchLimitRecursion value
      [I]Example Usage:[/I] SecPcreMatchLimitRecursion 1500
      [I]Processing Phase:[/I] N/A
      [I]Scope:[/I] Global
      [I]Version:[/I] 2.5.12
      [I]Dependencies/Notes:[/I] Default is set at compile (1500 by default)
      The --enable-pcre-match-limit-recursion=val configure option will set a custom
      default and the --disable-pcre-match-limit-recursion option will resort to the
      compiled PCRE library default.
    • PHP documentation: PHP: PCRE Runtime Configuration - Manual
      Code:
      [B]Runtime Configuration[/B]
      The behaviour of these functions is affected by settings in php.ini.
      
      [B]PCRE Configuration Options[/B]
      Name			Default		Changeable	Changelog
      [URL="http://www.php.net/manual/en/pcre.configuration.php#ini.pcre.backtrack-limit"]pcre.backtrack_limit[/URL] 	"100000" 	PHP_INI_ALL 	Available since PHP 5.2.0.
      [URL="http://www.php.net/manual/en/pcre.configuration.php#ini.pcre.recursion-limit"]pcre.recursion_limit[/URL] 	"100000" 	PHP_INI_ALL 	Available since PHP 5.2.0.
      For further details and definitions of the PHP_INI_* modes,
      see the [URL="http://www.php.net/manual/en/configuration.changes.modes.php"]Where a configuration setting may be set[/URL].
      
      Here's a short explanation of the configuration directives.
      
      [I]pcre.backtrack_limit[/I] [URL="http://www.php.net/manual/en/language.types.integer.php"]integer[/URL]
          PCRE's backtracking limit.
      
      [I]pcre.recursion_limit[/I] [URL="http://www.php.net/manual/en/language.types.integer.php"]integer[/URL]
          PCRE's recursion limit. Please note that if you set this value to a high
      number you may consume all the available process stack and eventually crash
      PHP (due to reaching the stack size limit imposed by the Operating System).
      
    • PCRE ReadMe document: http://www.pcre.org/readme.txt or http://vcs.pcre.org/viewvc/code/trunk/README?view=co
      Code:
      . PCRE has a counter that can be set to limit the amount of resources it uses.
        If the limit is exceeded during a match, the match fails. The default is ten
        million. You can change the default by setting, for example,
        --with-match-limit=500000
        on the "configure" command. This is just the default; individual calls to
        pcre_exec() can supply their own value. There is more discussion on the
        pcreapi man page.
      
      . There is a separate counter that limits the depth of recursive function calls
        during a matching process. This also has a default of ten million, which is
        essentially "unlimited". You can change the default by setting, for example,
        --with-match-limit-recursion=500000
        Recursive function calls use up the runtime stack; running out of stack can
        cause programs to crash in strange ways. There is a discussion about stack
        sizes in the pcrestack man page.
    • PCRE man pages: http://www.pcre.org/pcre.txt or http://vcs.pcre.org/viewvc/code/trunk/doc/pcre.txt?view=co
      PCRE man pages in HTML: PCRE specification
      Code:
      ------------------------------------------------------------------------------
      PCREBUILD(3)                                                      PCREBUILD(3)
      PCRE BUILD-TIME OPTIONS
      [...]
      AVOIDING EXCESSIVE STACK USAGE
             When matching with the pcre_exec() function, PCRE implements backtrack-
             ing by making recursive calls to an internal function  called  match().
             In  environments  where  the size of the stack is limited, this can se-
             verely limit PCRE's operation. (The Unix environment does  not  usually
             suffer from this problem, but it may sometimes be necessary to increase
             the maximum stack size.  There is a discussion in the  pcrestack  docu-
             mentation.)
      [...]
      LIMITING PCRE RESOURCE USAGE
             Internally,  PCRE has a function called match(), which it calls repeat-
             edly  (sometimes  recursively)  when  matching  a  pattern   with   the
             pcre_exec()  function.  By controlling the maximum number of times this
             function may be called during a single matching operation, a limit  can
             be  placed  on  the resources used by a single call to pcre_exec(). The
             limit can be changed at run time, as described in the pcreapi  documen-
             tation.  The default is 10 million, but this can be changed by adding a
             setting such as
               --with-match-limit=500000
             to  the  configure  command.  This  setting  has  no  effect   on   the
             pcre_dfa_exec() matching function.
      
             In  some  environments  it is desirable to limit the depth of recursive
             calls of match() more strictly than the total number of calls, in order
             to  restrict  the maximum amount of stack (or heap, if --disable-stack-
             for-recursion is specified) that is used. A second limit controls this;
             it  defaults  to  the  value  that is set for --with-match-limit, which
             imposes no additional constraints. However, you can set a  lower  limit
             by adding, for example,
               --with-match-limit-recursion=10000
             to  the  configure  command.  This  value can also be overridden at run
             time.
      [...]
      ------------------------------------------------------------------------------
      PCREAPI(3)                                                          PCREAPI(3)
      PCRE NATIVE API
      [...]
      MATCHING A PATTERN: THE TRADITIONAL FUNCTION
      [...]
         Extra data for pcre_exec()
      [...]
             The match_limit field provides a means of preventing PCRE from using up
             a vast amount of resources when running patterns that are not going  to
             match,  but  which  have  a very large number of possibilities in their
             search trees. The classic example is a pattern that uses nested  unlim-
             ited repeats.
      
             Internally,  PCRE uses a function called match() which it calls repeat-
             edly (sometimes recursively). The limit set by match_limit  is  imposed
             on  the  number  of times this function is called during a match, which
             has the effect of limiting the amount of  backtracking  that  can  take
             place. For patterns that are not anchored, the count restarts from zero
             for each position in the subject string.
      
             The default value for the limit can be set  when  PCRE  is  built;  the
             default  default  is 10 million, which handles all but the most extreme
             cases. You can override the default  by  suppling  pcre_exec()  with  a
             pcre_extra     block    in    which    match_limit    is    set,    and
             PCRE_EXTRA_MATCH_LIMIT is set in the  flags  field.  If  the  limit  is
             exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT.
      
             The  match_limit_recursion field is similar to match_limit, but instead
             of limiting the total number of times that match() is called, it limits
             the  depth  of  recursion. The recursion depth is a smaller number than
             the total number of calls, because not all calls to match() are  recur-
             sive.  This limit is of use only if it is set smaller than match_limit.
      
             Limiting  the  recursion  depth  limits the amount of stack that can be
             used, or, when PCRE has been compiled to use memory on the heap instead
             of the stack, the amount of heap memory that can be used.
      
             The  default  value  for  match_limit_recursion can be set when PCRE is
             built; the default default  is  the  same  value  as  the  default  for
             match_limit.  You can override the default by suppling pcre_exec() with
             a  pcre_extra  block  in  which  match_limit_recursion  is   set,   and
             PCRE_EXTRA_MATCH_LIMIT_RECURSION  is  set  in  the  flags field. If the
             limit is exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT.
      [...]
         Error return values from pcre_exec()
      [...]
               PCRE_ERROR_MATCHLIMIT     (-8)
             The backtracking limit, as specified by  the  match_limit  field  in  a
             pcre_extra  structure  (or  defaulted) was reached. See the description
             above.
      [...]
               PCRE_ERROR_RECURSIONLIMIT (-21)
             The internal recursion limit, as specified by the match_limit_recursion
             field  in  a  pcre_extra  structure (or defaulted) was reached. See the
             description above.
      [...]
      ------------------------------------------------------------------------------
      PCREPERFORM(3)                                                  PCREPERFORM(3)
      PCRE PERFORMANCE
             Two  aspects  of performance are discussed below: memory usage and pro-
             cessing time. The way you express your pattern as a regular  expression
             can affect both of them.
      [...]
      STACK USAGE AT RUN TIME
             When  pcre_exec()  is  used  for matching, certain kinds of pattern can
             cause it to use large amounts of the process stack.  In  some  environ-
             ments  the default process stack is quite small, and if it runs out the
             result is often SIGSEGV.  This issue is probably  the  most  frequently
             raised  problem  with  PCRE. Rewriting your pattern can often help. The
             pcrestack documentation discusses this issue in detail.
      
      PROCESSING TIME
             Certain items in regular expression patterns are processed  more  effi-
             ciently than others. It is more efficient to use a character class like
             [aeiou]  than  a  set  of   single-character   alternatives   such   as
             (a|e|i|o|u).  In  general,  the simplest construction that provides the
             required behaviour is usually the most efficient.
      [...]
      ------------------------------------------------------------------------------
      PCRECPP(3)                                                          PCRECPP(3)
      SYNOPSIS OF C++ WRAPPER
      [...]
      PASSING MODIFIERS TO THE REGULAR EXPRESSION ENGINE
      [...]
                        Setting match_limit to a non-zero value will limit the  exe-
             cution  of pcre to keep it from doing bad things like blowing the stack
             or taking an eternity to return a result.  A  value  of  5000  is  good
             enough  to stop stack blowup in a 2MB thread stack. Setting match_limit
             to  zero  disables  match  limiting.  Alternatively,   you   can   call
             match_limit_recursion()  which uses PCRE_EXTRA_MATCH_LIMIT_RECURSION to
             limit how much  PCRE  recurses.  match_limit()  limits  the  number  of
             matches PCRE does; match_limit_recursion() limits the depth of internal
             recursion, and therefore the amount of stack that is used.
      [...]
      ------------------------------------------------------------------------------
      PCRESTACK(3)                                                      PCRESTACK(3)
      PCRE DISCUSSION OF STACK USAGE
      [...]
         Limiting pcre_exec()'s stack usage
             You can set limits on the number of times that match() is called,  both
             in  total  and recursively. If a limit is exceeded, pcre_exec() returns
             an error code. Setting suitable limits should prevent it  from  running
             out  of  stack.  The  default  values of the limits are very large, and
             unlikely ever to operate. They can be changed when PCRE is  built,  and
             they  can  also be set when pcre_exec() is called. For details of these
             interfaces, see the pcrebuild documentation and the  section  on  extra
             data for pcre_exec() in the pcreapi documentation.
      
             As a very rough rule of thumb, you should reckon on about 500 bytes per
             recursion. Thus, if you want to limit your  stack  usage  to  8Mb,  you
             should  set  the  limit at 16000 recursions. A 64Mb stack, on the other
             hand, can support around 128000 recursions.
      
             In Unix-like environments, the pcretest test program has a command line
             option (-S) that can be used to increase the size of its stack. As long
             as the stack is large enough, another option (-M) can be used  to  find
             the  smallest  limits  that allow a particular pattern to match a given
             subject string. This is done by  calling  pcre_exec()  repeatedly  with
             different limits.
      [...]
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. JinGu

    JinGu Member

    Joined:
    Mar 26, 2006
    Messages:
    17
    Likes Received:
    0
    Trophy Points:
    151
    It shouldn't be too difficult to experience them first-hand, just try using the excellent collection of rules created and maintained by gotroot.com - the latest free version is available at http://updates.atomicorp.com/channels/rules/delayed/modsec-2.5-free-latest.tar.gz After downloading there are 3 things to do before using them : (1) delete the RBL, Scanner + Data Loss files, as they are apparently incompatible with cPanel. (2) running the shell command '/etc/init.d/httpd configtest' will probably show 1 error due to a non-existent file referenced in one of the rules files, you can fix that by creating an empty file at that location 'touch /etc/asl/whitelist'. (3) Check the cPanel-specific instructions on this page : Special_notes_for_CPANEL_users

    Now restart Apache and watch first-hand as your logs fill with PCRE match limit errors! Increasing the recursion setting to 150000 definitely helps but doesn't completely eliminate the problem, and could cause major system load issues, but it seems to be necessary if you wish to use these excellent gotroot rules. The gotroot people have their customized ASL mod_security build which apparently just disables the reporting of this error since it's not currently possible to find the exact cause, see the second post at ModSecurity: Rule execution error - PCRE limits exceeded - they're also working on a cPanel compatible version of their ASL custom mod_security build.
     
    #24 JinGu, Dec 29, 2010
    Last edited: Dec 29, 2010
  5. budway

    budway Well-Known Member

    Joined:
    Apr 16, 2003
    Messages:
    189
    Likes Received:
    0
    Trophy Points:
    166
    cPanel could update modsecurity, looks like they fixed to show witch rules tringers the PCRE limit reach.


    https://www.modsecurity.org/tracker/sr/jira.issueviews:searchrequest-fullcontent/temp/SearchRequest.html?pid=10000&component=10030&sorter/field=issuekey&sorter/order=DESC&tempMax=1000
    [MODSEC-135] PCRE Limits Exceeded message does not indicate which rule Created: 13/Feb/10 Updated: 01/Nov/10 Resolved: 01/Nov/10
    Status: Closed
    Project: ModSecurity
    Component/s: Logging
    Affects Version/s: 2.5.12
    Fix Version/s: 2.5.13
    Security Level: Normal

    Type: Improvement Priority: Normal
    Reporter: Brian Rectanus Assignee: Breno Silva Pinto
    Resolution: Fixed Votes: 3
    Labels: None
    Remaining Estimate: Not Specified
    Time Spent: Not Specified
    Original Estimate: Not Specified

    Severity: Minor

    Description
    Need to add some indication of which rule caused a PCRE limit to be exceeded.

    See also (MODSEC-119)
     
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice