Comodo WAF ModSecurity ruleset leading to large secdatadir cache files

[weblinks]

CLOUDLINUX 7.6 [] v78.0.23

Hi,

I am getting this alert

Time: Mon May 20 04:00:08 2019 +0500
ModSecurity persistent IP database (/var/cpanel/secdatadir/ip.pag) size is 51.73GB
This requires further investigation otherwise it will start to affect server performance.

but when I am checking after an hour of this email the file size showing there is

1857786880 = 1771 MB

2 days ago, I follow the steps mentioned there to
ModSecurity SDBM Utility - EasyApache 4 - cPanel Documentation
to purge expired entries / purge the cache file

pls suggest..

 

[cPanelMichael]

Hello @weblinks,

Can you open a support ticket so we can take a closer look at your system to see why the ModSecurity SDBM utility isn't working? You can post the ticket number here and we'll link this thread to it.

Thank you.

 

[internetbug256]

Hello @weblinks,

Can you open a support ticket so we can take a closer look at your system to see why the ModSecurity SDBM utility isn't working? You can post the ticket number here and we'll link this thread to it.

Thank you.

Hi there. I am getting exactly the same error notification, with that same size (51.73GB). When I go to that folder, this is what I see:

root@host [/var/cpanel/secdatadir]# ls -la
total 2336
drwxrwx--T. 2 root nobody 4096 May 24 18:01 ./
drwx--x--x. 119 root root 20480 May 24 18:17 ../
-rw-r-----. 1 nobody nobody 61440 May 24 07:54 default_SESSION.dir
-rw-r-----. 1 nobody nobody 442839040 May 24 18:16 default_SESSION.pag
-rw-r-----. 1 nobody nobody 0 Oct 24 2015 global.dir
-rw-r-----. 1 nobody nobody 0 Oct 24 2015 global.pag
-rwxr-xr-x. 1 nobody nobody 77824 May 24 18:17 ip.dir*
-rwxr-xr-x. 1 nobody nobody 784045056 May 24 18:17 ip.pag*

That is less than a GB, so I assume it's a bug.
Do I open a ticket too, or maybe you have a way to clear this false alarm?

Thanks in advance

 

[cPanelMichael]

Hello @internetbug256,

I couldn't find a support ticket opened by the original poster. Could you submit a ticket to report this issue and post the ticket number here once it's opened?

Thank you.

 

[dusanf]

Hi all,

I`ve noted that /var/cpanel/secdatadir/default_SESSION.pag file sometimes grows too big.

Does anyone know what is being logged in this file?

I noted that when modsecurity is off, this file doesn't grow but cant determine what gets logged there. At first I thought its related to joomla only but it seems I was wrong about it.

 

[cPanelMichael]

Hello Everyone,

A. We've seen recent reports of the third-party Comodo WAF ModSecurity ruleset leading to excessive entries in the /var/cpanel/secdatadir/default_SESSION.pag file. We are tracking this as part of internal case UPS-134. I'll follow this case and update this thread with more information as it becomes available.

In the meantime, the temporary workaround is to manually prune /var/cpanel/secdatadir/default_SESSION.pag using the following steps:

1. Execute the following command to install the ModSecurity™ SDBM utility if it's not already installed on the system:

yum install ea-modsec-sdbm-util

2. Execute the following command to prune default_SESSION.pag:

modsec-sdbm-util -s /var/cpanel/secdatadir/default_SESSION.pag

B. Additionally, case CPANEL-27451 is open to consider adding automatic rotation/pruning support for /var/cpanel/secdatadir/default_SESSION.pag. I'll monitor this case and update this thread with more information on it's status as it becomes available.

Thank you.

 

[dusanf]

@cPanelMichael

Do you have info on rules that are using default_SESSION.pag so we can disable them?

 

[cPanelMichael]

Do you have info on rules that are using default_SESSION.pag so we can disable them?

Can you confirm if you are using the third-party Comodo WAF ModSecurity ruleset, or do you mean the rule types in general?

Thank you.

 

[dusanf]

@cPanelMichael

I can confirm that use Comodo WAF using cPanel plugin provided by Comodo, yes.

I tried running
modsec-sdbm-util -s /var/cpanel/secdatadir/default_SESSION.pag

on 4 servers but it didnt reduce the size of pag file.

 

[cPanelMichael]

I can confirm that use Comodo WAF using cPanel plugin provided by Comodo, yes.

Hello @dusanf,

Comodo yes not yet published a workaround for the issue with their rules. As a workaround, you can manually purge the
/var/cpanel/secdatadir/default_SESSION.pag cache file per the commands listed under You can also run the following commands in a shell to purge the cache file on the link below:

ModSecurity SDBM Utility - EasyApache 4 - cPanel Documentation

Replace "ip.pag" with "default_SESSION.pag" in the example provided on the link above.

Thank you.

 

[dusanf]

@cPanelMichael

It works with manual way, I hope we get an update from comodo soon

 

[markhard]

in my server the one that grows to 52GB is nobody-ip.pag, running the SDBM utility resulted:

$ modsec-sdbm-util -s nobody-ip.pag
Opening file: nobody-ip.pag
Database ready to be used.
 [-] 550 records so far.
Total of 556 elements processed.
0 elements removed.
Expired elements: 22, inconsistent items: 0
Fragmentation rate: 3.96% of the database is/was dirty data.

however the size didn't reduced, and the website hosted in the server remains slow to open. only after deleting the file did it help speed up web server response. however the file size built up back to 52GB pretty soon which causes the slowness of web response.

in other thread: [EA-8506] ModSecurity 2.9.3 results in Apache service failures it's mentioned to downgrade ModSecurity which I followed.

my current ModSecurity version is ea-apache24-mod_security2-2.9.3-2.2.1.cpanel.x86_64 (which have this issue). following that thread instruction, i downgraded to version ea-apache24-mod_security2-2.9.2-11.11.7.cpanel.x86_64 and it doesn't seem to have this issue. however cpanel update, updates the modsecurity to 2.9.3 version and this issue comes back.

note that I'm using Comodo's rule

 

[cPanelMichael]

Hello Everyone,

We've reached out to Comodo to report the issue with their ruleset, however we have not yet received a response. I'll continue to monitor internal case UPS-134 and report any updates to this thread.

A more permanent workaround is to disable the Comodo WAF ModSecurity ruleset in lieu of an alternative such as OWASP:

Updating to OWASP v.3.1 question

Thank you.

 

[markhard]

Hi Michael,

i tried to use OWASP ModSecurity Core Rule Set V3.0 instead of Comodo's rule, however i see a lot of false positives in that rule set causing a lot of my clients got blocked.

the link you gave pointed to a request ticket which already 4 months old and has no update. disabling mod_security is not an option for me.

so is there a way to keep using apache24-mod_security2-2.9.2-11.11.7.cpanel.x86_64? cpanel update keeps updating it to version 2.9.3 which have this issue. i did try to add mod_security in yum.conf exclude file but it's ignored by cpanel update

 

[dusanf]

@cPanelMichael

Is there any update on this? Now I can see nobody-ip.pag is 5GB.

 

[cPanelMichael]

i tried to use OWASP ModSecurity Core Rule Set V3.0 instead of Comodo's rule, however i see a lot of false positives in that rule set causing a lot of my clients got blocked.

the link you gave pointed to a request ticket which already 4 months old and has no update. disabling mod_security is not an option for me.

so is there a way to keep using apache24-mod_security2-2.9.2-11.11.7.cpanel.x86_64? cpanel update keeps updating it to version 2.9.3 which have this issue. i did try to add mod_security in yum.conf exclude file but it's ignored by cpanel update

Hello @markhard,

Could you open a support ticket so we can take a closer look at your system and recommend the best possible workaround? Post the ticket number here once it's opened and we'll share the outcome.

I can confirm that use Comodo WAF using cPanel plugin provided by Comodo, yes.

Is there any update on this? Now I can see nobody-ip.pag is 5GB.

Hello @dusanf,

We've not yet received a solution from Comodo. I'll continue to monitor internal case UPS-134 and report any updates to this thread.
Disabling the COMODO WAF Mod_Security ruleset, manually removing the large nobody-ip.pag file, and restarting Apache (/scripts/restartsrv_httpd) are still the applicable workaround steps. At that point, you should consider an alternative ruleset such as OWASP:

OWASP ModSecurity CRS - cPanel Knowledge Base - cPanel Documentation

Thank you.

 

[cPanelMichael]

Hello Everyone,

We received an update from Comodo regarding this topic noting the issue should be fixed as of the most recent rule updates on 05-22-2019 and 05-31-2019 seen on the link below:

Rules Updates: Changelog - Free Modsecurity rules - Comodo Web Application Firewall | Page 13

If you've already cleared the Mod_Security .pag cache file and it still continues to grow to a large size, please confirm the Comodo rule version by executing the following command:

cat /etc/apache2/conf.d/modsec_vendor_configs/comodo_apache/rules.dat

If it doesn't state version 1.208 or higher, then you must update your ruleset to the latest version provided by Comodo.

Thank you.

 

[dusanf]

1.210 fixes a lot of issues and returns bruteforce protection, highly recommend to upgrade to it

 

[sumi21kav]

1.211 version, still experiencing the same issue. Nobody-ip.pag is getting too big too fast

Any permanent solution ?

 

[cPanelMichael]

1.211 version, still experiencing the same issue. Nobody-ip.pag is getting too big too fast

Could you open a support ticket so we can take a closer look at your system? Include a reference to internal case UPS-134 in the subject of your ticket and post the ticket number here once it's opened so we can link this thread to it.

Thank you.