ModSecurity Auto Updater

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
NOTE:
It seems that Atomicorp has forgotten to update the VERSION file on his payed rules, if your autoupdater relays on that info you will need to check the rules manually.

Regards,

Sergio
 

mikegotroot

Well-Known Member
Verifed Vendor
Apr 29, 2008
85
1
58
Although I can't reproduce your issue, the VERSION file for the real time rules is update to date (see below), can you tell me how you are getting this error?

ASL_VERSION=3.0.15
APPINV_VERSION=201107281511
CLAMAV_VERSION=201112011125
GEOMAP_VERSION=201111262006
GRSEC_VERSION=0
KERNEL_VERSION=2.6.32.43-6
MODSEC_VERSION=201112011700
OSSEC_VERSION=201111171741
WAF_DELAYED_VERSION=201109071352
 
Last edited:

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
Although I can't reproduce your issue, the VERSION file for the real time rules is update to date (see below), can you tell me how you are getting this error?

ASL_VERSION=3.0.15
APPINV_VERSION=201107281511
CLAMAV_VERSION=201112011125
GEOMAP_VERSION=201111262006
GRSEC_VERSION=0
KERNEL_VERSION=2.6.32.43-6
MODSEC_VERSION=201112011700
OSSEC_VERSION=201111171741
WAF_DELAYED_VERSION=201109071352
The file was not updated since Nov. 23, I see that it has been updated todays, thanks!
 

k-planethost

Well-Known Member
Sep 22, 2009
199
10
68
Athens Greece
one of the rulesets on a specific domain all the time return me this
Failed to write to DBM file "/var/asl/data/msa/global": Invalid argument
on ModSecurity Log Entries ruleset is unknown
are there any suggestions how to investigate further which ruleset returns this?
 

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
one of the rulesets on a specific domain all the time return me this
Failed to write to DBM file "/var/asl/data/msa/global": Invalid argument
on ModSecurity Log Entries ruleset is unknown
are there any suggestions how to investigate further which ruleset returns this?
Check if the following files are created inside "msa" directory, if not create them:

global.dir nobody(99)/nobody(99) 640
global.pag nobody(99)/nobody(99) 640
ip.dir nobody(99)/nobody(99) 640
ip.pag nobody(99)/nobody(99) 640

Regards,

Sergio
 

k-planethost

Well-Known Member
Sep 22, 2009
199
10
68
Athens Greece
thanks inside on /msa i already have this
SESSION.dir 0 02-Dec-2011 10:08:13 nobody(99)/nobody(99) 640
SESSION.pag 0 02-Dec-2011 10:08:13 nobody(99)/nobody(99) 640
USER.dir 0 02-Dec-2011 10:18:19 nobody(99)/nobody(99) 640
USER.pag 0 02-Dec-2011 10:18:19 nobody(99)/nobody(99) 640 move
default_SESSION.dir 4.0k 02-Dec-2011 11:22:54 nobody(99)/nobody(99) 640
default_SESSION.pag 333.0k 02-Dec-2011 11:23:13 nobody(99)/nobody(99) 640
global.dir 0 27-Sep-2011 22:37:26 nobody(99)/nobody(99) 640
global.pag 1.0k 03-Dec-2011 17:01:49 nobody(99)/nobody(99) 640
ip.dir 4.0k 02-Dec-2011 11:22:07 nobody(99)/nobody(99) 640
ip.pag 540.0k 02-Dec-2011 12:24:47 nobody(99)/nobody(99) 640
 

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
thanks inside on /msa i already have this
SESSION.dir 0 02-Dec-2011 10:08:13 nobody(99)/nobody(99) 640
SESSION.pag 0 02-Dec-2011 10:08:13 nobody(99)/nobody(99) 640
USER.dir 0 02-Dec-2011 10:18:19 nobody(99)/nobody(99) 640
USER.pag 0 02-Dec-2011 10:18:19 nobody(99)/nobody(99) 640 move
default_SESSION.dir 4.0k 02-Dec-2011 11:22:54 nobody(99)/nobody(99) 640
default_SESSION.pag 333.0k 02-Dec-2011 11:23:13 nobody(99)/nobody(99) 640
global.dir 0 27-Sep-2011 22:37:26 nobody(99)/nobody(99) 640
global.pag 1.0k 03-Dec-2011 17:01:49 nobody(99)/nobody(99) 640
ip.dir 4.0k 02-Dec-2011 11:22:07 nobody(99)/nobody(99) 640
ip.pag 540.0k 02-Dec-2011 12:24:47 nobody(99)/nobody(99) 640
Try to create a new file, just called "global" and check if that fix the error.

On the other hand, what rules are you using? the free or the payed ones?

Sergio
 

k-planethost

Well-Known Member
Sep 22, 2009
199
10
68
Athens Greece
the specific server that returns this has the free rules + crs rules
even with a global file 640 permission this joomla sites returns this error again
that happens only with google bots on a specific joomla site
i should try 770 to see
 

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
Although I can't reproduce your issue, the VERSION file for the real time rules is update to date (see below), can you tell me how you are getting this error?
Today has the same error:
VERSION Dec 02 2011 09:12:16

Inside it has:
MODSEC_VERSION=201112020912

Files on the directory shows:
modsec-201112031045.tar.gz Dec 03 2011 10:45:37 463kb

As you can see, the info on the VERSION file doesn't correspond to the last file, nor the version file date is the same as the latest rules.

Regards,
Sergio
 

mikegotroot

Well-Known Member
Verifed Vendor
Apr 29, 2008
85
1
58
Which directory are you referring to, the real time or the delayed?

If the delayed, thats correct. The VERSION variable always shows the current real time version. asl-lite uses that variable to report that the system is using out of date rules when the user is only using the delayed feed.

If I misunderstood and you mean the real time directory, I'm not able to reproduce what you are reporting with asl-lite, its correctly reading the VERSION and downloading the latest real time rules. Thats how it knows which real time file to download. If you see something different with asl-lite have you reported this issue to support?

Or did I totally misunderstand your issue?
 
Last edited:

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
Which directory are you referring to, the real time or the delayed?

If the delayed, thats correct. The VERSION variable always shows the current real time version. asl-lite uses that variable to report that the system is using out of date rules when the user is only using the delayed feed.

If I misunderstood and you mean the real time directory, I'm not able to reproduce what you are reporting with asl-lite, its correctly reading the VERSION and downloading the latest real time rules. Thats how it knows which real time file to download. If you see something different with asl-lite have you reported this issue to support?

Or did I totally misunderstand your issue?
Hello Michael,
sorry for answering so late, the file that I mention is at the payed rules, it has been very irregular on the ModSec Rules version, today 12/23/2011 it shows the following:
VERSION Dec 20 2011 14:35:28 (out dated from 3 days ago)

Inside that file, it has the following:
MODSEC_VERSION=201112201314

and the last version on the page is:
modsec-201112231653.tar.gz

It seems that the VERSION file is not updated anymore as it was on the past.

Regards,

Sergio
 

mikegotroot

Well-Known Member
Verifed Vendor
Apr 29, 2008
85
1
58
I'm still not sure what you are seeing, the variable looks fine to me, perhaps you are caching something?

-rw-r--r-- 1 foo bar 293947 Dec 27 13:16 modsec-201112271316.tar.gz
-rw-r--r-- 1 foo bar 235 Dec 27 13:16 VERSION
[[email protected] rules]$ cat VERSION
ASL_VERSION=3.0.16
APPINV_VERSION=201112201435
CLAMAV_VERSION=201112271316
GEOMAP_VERSION=201112271119
GRSEC_VERSION=0
KERNEL_VERSION=2.6.32.43-6
MODSEC_VERSION=201112271316
OSSEC_VERSION=201112221921

Also, if you have any issues with our product please contact support or post on our forums. We certainly don't want to miss a request for assistance, and as busy as we are these days its not that often we get around to the other forums, so posting here isn't likely to get a response as quickly as just contacting us for support. Afterall, you're paying for the support so why not use it! :)
 
Last edited:

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
I'm still not sure what you are seeing, the variable looks fine to me, perhaps you are caching something?

-rw-r--r-- 1 foo bar 293947 Dec 27 13:16 modsec-201112271316.tar.gz
-rw-r--r-- 1 foo bar 235 Dec 27 13:16 VERSION
[[email protected] rules]$ cat VERSION
ASL_VERSION=3.0.16
APPINV_VERSION=201112201435
CLAMAV_VERSION=201112271316
GEOMAP_VERSION=201112271119
GRSEC_VERSION=0
KERNEL_VERSION=2.6.32.43-6
MODSEC_VERSION=201112271316
OSSEC_VERSION=201112221921

Also, if you have any issues with our product please contact support or post on our forums. We certainly don't want to miss a request for assistance, and as busy as we are these days its not that often we get around to the other forums, so posting here isn't likely to get a response as quickly as just contacting us for support. Afterall, you're paying for the support so why not use it! :)
Thank you for fixing the file, next time I will do what you say.

Sergio
 

gkgcpanel

Well-Known Member
Jun 6, 2007
214
1
166
cPanel Access Level
DataCenter Provider
These directories will not be used for saving any data at all, they only are used because some of the rules checks for them.

Best Regards,

Sergio
Sergio,

You mentioned that these directories are not used, but that the rules simply check to make sure they exist. This isn't true. My /var/ partition ran out of disk space because inside the /var/asl/data/audit/ folder were tons of files (over 19GB) of log entries. I had to delete them to get disk space back. So they are apparently used somehow by ModSec.

Any idea if these log files are needed and should be just delete them daily or weekly?

Thanks.
 

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
Sergio,

You mentioned that these directories are not used, but that the rules simply check to make sure they exist. This isn't true. My /var/ partition ran out of disk space because inside the /var/asl/data/audit/ folder were tons of files (over 19GB) of log entries. I had to delete them to get disk space back. So they are apparently used somehow by ModSec.

Any idea if these log files are needed and should be just delete them daily or weekly?

Thanks.
That directories will retain your data depending on what you have declared on your modsec2.user.conf command: SecAuditLogParts

If you set it to:
SecAuditLogParts ABIFHZ
then your server will not retain any logs in that directories.

On the other hand, if you use CSF the info in that directories are not needed and can be deleted as CSF, LFD and CMC gather the info from /usr/local/apache/logs. If you use something different as CSF then you need to check what do you need.

Sergio
 

gkgcpanel

Well-Known Member
Jun 6, 2007
214
1
166
cPanel Access Level
DataCenter Provider
That directories will retain your data depending on what you have declared on your modsec2.user.conf command: SecAuditLogParts

If you set it to:

then your server will not retain any logs in that directories.

On the other hand, if you use CSF the info in that directories are not needed and can be deleted as CSF, LFD and CMC gather the info from /usr/local/apache/logs. If you use something different as CSF then you need to check what do you need.

Sergio
I do have that line in my modsec2.user.conf file...

SecAuditLogParts ABIFHZ

In fact, here's my entire modsec2.user.conf file...


SecRequestBodyAccess On
SecAuditLogType Concurrent
SecResponseBodyAccess On
SecResponseBodyMimeType (null) text/html text/plain text/xml
SecResponseBodyLimit 2621440
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecServerSignature Apache
SecUploadDir /var/asl/data/suspicious
SecUploadKeepFiles Off
SecAuditLogParts ABIFHZ
SecArgumentSeparator "&"
SecCookieFormat 0
SecRequestBodyInMemoryLimit 131072
SecDataDir /var/asl/data/msa
SecTmpDir /tmp
SecAuditLogStorageDir /var/asl/data/audit
SecResponseBodyLimitAction ProcessPartial
SecPcreMatchLimit 50000
SecPcreMatchLimitRecursion 50000

# ConfigServer ModSecurity whitelist file remove the mark if you are using ConfigServer CMC.
Include /usr/local/apache/conf/modsec2.whitelist.conf

#ASL Rules
Include /usr/local/apache/conf/modsec_rules/modsec/10_asl_antimalware.conf
Include /usr/local/apache/conf/modsec_rules/modsec/10_asl_rules.conf
Include /usr/local/apache/conf/modsec_rules/modsec/20_asl_useragents.conf
Include /usr/local/apache/conf/modsec_rules/modsec/30_asl_antispam.conf
Include /usr/local/apache/conf/modsec_rules/modsec/50_asl_rootkits.conf
Include /usr/local/apache/conf/modsec_rules/modsec/60_asl_recons.conf
Include /usr/local/apache/conf/modsec_rules/modsec/99_asl_jitp.conf


So, with the SecAuditLogParts already set to ABIFHZ, why are those directories still filling up?
 

Secmas

Well-Known Member
Feb 18, 2005
376
19
168
I do have that line in my modsec2.user.conf file...

SecAuditLogParts ABIFHZ

In fact, here's my entire modsec2.user.conf file...


SecRequestBodyAccess On
SecAuditLogType Concurrent
SecResponseBodyAccess On
SecResponseBodyMimeType (null) text/html text/plain text/xml
SecResponseBodyLimit 2621440
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecServerSignature Apache
SecUploadDir /var/asl/data/suspicious
SecUploadKeepFiles Off
SecAuditLogParts ABIFHZ
SecArgumentSeparator "&"
SecCookieFormat 0
SecRequestBodyInMemoryLimit 131072
SecDataDir /var/asl/data/msa
SecTmpDir /tmp
SecAuditLogStorageDir /var/asl/data/audit
SecResponseBodyLimitAction ProcessPartial
SecPcreMatchLimit 50000
SecPcreMatchLimitRecursion 50000

# ConfigServer ModSecurity whitelist file remove the mark if you are using ConfigServer CMC.
Include /usr/local/apache/conf/modsec2.whitelist.conf

#ASL Rules
Include /usr/local/apache/conf/modsec_rules/modsec/10_asl_antimalware.conf
Include /usr/local/apache/conf/modsec_rules/modsec/10_asl_rules.conf
Include /usr/local/apache/conf/modsec_rules/modsec/20_asl_useragents.conf
Include /usr/local/apache/conf/modsec_rules/modsec/30_asl_antispam.conf
Include /usr/local/apache/conf/modsec_rules/modsec/50_asl_rootkits.conf
Include /usr/local/apache/conf/modsec_rules/modsec/60_asl_recons.conf
Include /usr/local/apache/conf/modsec_rules/modsec/99_asl_jitp.conf


So, with the SecAuditLogParts already set to ABIFHZ, why are those directories still filling up?
Ok, I see in your modsec2.user.conf that you have this command line:
SecAuditLogRelevantStatus "^(?:5|4(?!04))"

REM that line and that will do the trick.

If you see in my web page ModSec Taylor Made and Tweaks by Sergio that command line is not declared.

Sergio
 

gkgcpanel

Well-Known Member
Jun 6, 2007
214
1
166
cPanel Access Level
DataCenter Provider
Ok, I see in your modsec2.user.conf that you have this command line:
SecAuditLogRelevantStatus "^(?:5|4(?!04))"

REM that line and that will do the trick.

If you see in my web page ModSec Taylor Made and Tweaks by Sergio that command line is not declared.

Sergio
Sorry, didn't work. I did rem that line out and restarted apache, and cleared out the /var/asl/data/audit directory again. A few seconds later, I saw this:

drwxrwx--- 3 nobody nobody 4096 Dec 29 11:49 ./
drwxr-xr-x 5 root root 4096 Mar 1 2011 ../
drwxr-x--- 3 nobody nobody 4096 Dec 29 11:49 20111229/
[email protected] [/var/asl/data/audit]#

inside the 20111229/ folder is another folder: 20111229-1149/
Inside that folder is: 20111229-114933-TvyoLdC0HLYAAAgyxeUAAAAa

This file is a log file that looks like this:

--6e10773e-B--
HEAD /v1site_images/splash3.jpg?var=0.21215300%201294395324 HTTP/1.1
Host: domainname.tld
Connection: close
User-Agent: Baiduspider-image+(+???????????Baiduspider)
Accept: */*

--6e10773e-F--
HTTP/1.1 403 Forbidden
Connection: close
Content-Type: text/html; charset=iso-8859-1

--6e10773e-H--
Message: Access denied with code 403 (phase 2). Match of "rx (/tags/|/shop/images/exclusive/)" against "REQUEST_URI" required. [file "/usr/local/apache/conf/modsec_rules/modsec/50_asl_rootkits.conf"] [line "89"] [id "390147"] [rev "10"] [msg "Atomicorp.com UNSUPPORTED DELAYED Rules: Malware attack: Known malware or remote shell"] [data "sh3.jpg?"] [severity "CRITICAL"]
Apache-Error: [file "core.c"] [line 3706] [level 3] File does not exist: /home/xxxxxxx/public_html/403.shtml
Action: Intercepted (phase 2)
Stopwatch: 1325180973376755 19915 (- - -)
Stopwatch2: 1325180973376755 19915; combined=1186, p1=123, p2=1045, p3=0, p4=0, p5=17, sr=43, sw=1, l=0, gc=0
Producer: ModSecurity for Apache/2.6.0 (ModSecurity: Open Source Web Application Firewall).
Server: Apache

--6e10773e-Z--


So basically every block by mod security is getting logged in here and that will very quickly fill up the directories..
 

gkgcpanel

Well-Known Member
Jun 6, 2007
214
1
166
cPanel Access Level
DataCenter Provider
Update: I went to your site and also noticed you didn't have:

SecAuditLogType Concurrent

So I commented it out and restarted apache and cleared the files/directories in /var/asl/data/audit and watched...

so far no more files. So that is apparently the trick.

Thanks