ModSecurity Auto Updater

lbeachmike

Well-Known Member
Dec 27, 2001
307
4
318
Long Beach, NY
cPanel Access Level
Root Administrator
Mike,
what is the name of the file that you have downloaded? Where did you download that file?

Sergio
Hi Sergio -

This should help answer your questions -

Code:
Time: Sat Apr 23 03:00:02 EDT 2011
The rules to install are from /root/modsec/modsec-2.5-free-latest.tar.gz
Backing up current directory
Mod_Sec Directory re-created successfully
Unpacking the rules TAR file that was downloaded...
Getting rid of 'asl directories' and Moving files into their positions... please wait.
Testing rules to be sure there are no errors
Syntax OK
Apache configuration PASSED!
Restarting Apache. (may take up to 2 minutes to restart)
Cleaning up
UPDATE RULES completed...
 

lbeachmike

Well-Known Member
Dec 27, 2001
307
4
318
Long Beach, NY
cPanel Access Level
Root Administrator
Also, here is the modified script which emails an update and also would revert back if apache failed to start. The admin who installed this for me made these modifications -

Code:
#!/bin/bash
#
# Mod_security rules updater for cPanel V.1.04
#
# Update rules for mod_security 2.5 from
# GotRoot.com onto a cPanel server
#
# Rules available from http://www.gotroot.com, it works with free
# or payed subscriptions.
#
# Author: Sergio Cabrera Country: Guatemala
# This file was downloaded from Web Page: http://www.puntapirata.com
# Contact [email protected] with any bugs or enhancement suggestions.
#
# Free to use as long as you leave my name and Country intact.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
####         USE IT AT YOUR OWN RISK, NO GUARANTEES OF ANY KIND          ####

clear
echo "This will automatically update Mod_Security rules on a cPanel server."

#change name of file if it changed on gotroot.com
TARFILE="modsec-2.5-free-latest.tar.gz"

# MODIFY THE FOLLOWING LINE WITH YOUR OWN DIRECTORY.
# DO NOT INCLUDE A TRAILING SLASH.
WORKNDIR="/root/modsec"

#MAIL OPTIONS
EMAILTO="[email protected]"
MESSAGE="/tmp/message.txt"

# DO NOT MODIFY THE FOLLOWING THREE LINES
RULEFILE=$WORKNDIR/$TARFILE
RULEDIR="/usr/local/apache/conf/modsec_rules"
BADOMAIN=$WORKNDIR"/brh1-badomains.txt"

#DO NOT CHANGE
echo "Time: `date`" >> $MESSAGE
SUBJECT="Mod_Sec Rule Auto Updates Success"

#DOWNLOAD THE LATEST RULE FILE
wget -q -O $WORKNDIR/$TARFILE http://updates.atomicorp.com/channels/rules/delayed/$TARFILE

if [ -e $RULEFILE ]; then
        echo "The rules to install are from" $RULEFILE >> $MESSAGE
else
        echo "GetRoot.com file not found." >> $MESSAGE
        echo "Make sure the file name has not changed and" >> $MESSAGE
        echo "edit updaterules.sh to match the new file name." >> $MESSAGE
        SUBJECT="Mode_Sec Rule Auto Updater Failed"
        exit 0
fi

echo "Backing up current directory" >> $MESSAGE
mv $RULEDIR $WORKNDIR/backups/`date +%m%d%Y`

mkdir $RULEDIR

if [ -d $RULEDIR ]; then
      echo "Mod_Sec Directory re-created successfully" >> $MESSAGE
else
      echo "Directory creation failed.  Check path to ensure it was" >> $MESSAGE
      echo "entered properly.  Installation aborted" >> $MESSAGE
      SUBJECT="Mode_Sec Rule Auto Updater Failed"
      exit 0
fi

echo "Unpacking the rules TAR file that was downloaded..." >> $MESSAGE
tar -xzf $RULEFILE -C $RULEDIR
echo "Getting rid of 'asl directories' and Moving files into their positions... please wait." >> $MESSAGE

LORIODIR=$RULEDIR"/modsec"

if [ -d $LORIODIR ]; then
      cd $LORIODIR
else
      cd $RULEDIR
fi

sed -i 's/\/etc\/asl\/whitelist/whitelist.txt/g' 00_asl_whitelist.conf
sed -i 's/\/etc\/asl\/whitelist/whitelist.txt/g' 00_asl_rbl.conf

##########
# ADD YOUR OWN BLACKLISTED DOMAINS INTO THE RULES
##########
# dos2unix $WORKNDIR/puntapirata-badomain.txt
# cat $WORKNDIR/puntapirata-badomain.txt malware-blacklist.txt > malware
# cat $WORKNDIR/puntapirata-badomain.txt domain-blacklist.txt > domain
# sort -u malware > malware-blacklist.txt
# sort -u domain > domain-blacklist.txt
##########

##########
# IF YOU DON'T LIKE THE FOLLOWING RULES IN YOUR SERVER, MAKE THE FOLLOWING LINES ACTIVE:
##########
rm -f 00_asl_rbl.conf
rm -f 05_asl_scanner.conf
rm -f 11_asl_data_loss.conf
##########

if [ -d $LORIODIR ]; then
      mv * $RULEDIR
      rmdir $LORIODIR
fi

cd $RULEDIR

chown -R root:root $RULEDIR

echo "Testing rules to be sure there are no errors" >> $MESSAGE
echo
/usr/local/apache/bin/httpd -t 2>/tmp/apachechk
APACHECHK=`grep "Syntax OK" /tmp/apachechk`
if [[ $APACHECHK = "Syntax OK" ]]; then
      echo $APACHECHK >> $MESSAGE
      echo "Apache configuration PASSED!" >> $MESSAGE
      echo "Restarting Apache. (may take up to 2 minutes to restart)" >> $MESSAGE
              /scripts/restartsrv_httpd
else
      echo "Apache Configuration FAILED!" >> $MESSAGE
      echo $APACHECHK >> $MESSAGE
      echo "Check the error above and resolve any conflicts" >> $MESSAGE
      echo "Previous rules were restored" >> $MESSAGE
      rm -rf $RULEDIR
      mv $WORKNDIR/backups/`date +%m%d%Y` $RULEDIR
      SUBJECT="Mode_Sec Rule Auto Updater Failed"
fi

echo "Cleaning up" >> $MESSAGE
rm -f $RULEFILE
rm -f /tmp/apachechk

echo "UPDATE RULES completed..." >> $MESSAGE
/bin/mail -s "$SUBJECT" "$EMAILTO" < $MESSAGE

rm -f $MESSAGE
exit 0
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
Also, here is the modified script which emails an update and also would revert back if apache failed to start. The admin who installed this for me made these modifications -

Code:
#!/bin/bash
#
# Mod_security rules updater for cPanel V.1.04
#
# Update rules for mod_security 2.5 from
# GotRoot.com onto a cPanel server
#
# Rules available from http://www.gotroot.com, it works with free
# or payed subscriptions.
#
# Author: Sergio Cabrera Country: Guatemala
# This file was downloaded from Web Page: http://www.puntapirata.com
# Contact [email protected] with any bugs or enhancement suggestions.
#
# Free to use as long as you leave my name and Country intact.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
####         USE IT AT YOUR OWN RISK, NO GUARANTEES OF ANY KIND          ####

clear
echo "This will automatically update Mod_Security rules on a cPanel server."

#change name of file if it changed on gotroot.com
TARFILE="modsec-2.5-free-latest.tar.gz"

# MODIFY THE FOLLOWING LINE WITH YOUR OWN DIRECTORY.
# DO NOT INCLUDE A TRAILING SLASH.
WORKNDIR="/root/modsec"

#MAIL OPTIONS
EMAILTO="[email protected]"
MESSAGE="/tmp/message.txt"

# DO NOT MODIFY THE FOLLOWING THREE LINES
RULEFILE=$WORKNDIR/$TARFILE
RULEDIR="/usr/local/apache/conf/modsec_rules"
BADOMAIN=$WORKNDIR"/brh1-badomains.txt"

#DO NOT CHANGE
echo "Time: `date`" >> $MESSAGE
SUBJECT="Mod_Sec Rule Auto Updates Success"

#DOWNLOAD THE LATEST RULE FILE
wget -q -O $WORKNDIR/$TARFILE http://updates.atomicorp.com/channels/rules/delayed/$TARFILE

if [ -e $RULEFILE ]; then
        echo "The rules to install are from" $RULEFILE >> $MESSAGE
else
        echo "GetRoot.com file not found." >> $MESSAGE
        echo "Make sure the file name has not changed and" >> $MESSAGE
        echo "edit updaterules.sh to match the new file name." >> $MESSAGE
        SUBJECT="Mode_Sec Rule Auto Updater Failed"
        exit 0
fi

echo "Backing up current directory" >> $MESSAGE
mv $RULEDIR $WORKNDIR/backups/`date +%m%d%Y`

mkdir $RULEDIR

if [ -d $RULEDIR ]; then
      echo "Mod_Sec Directory re-created successfully" >> $MESSAGE
else
      echo "Directory creation failed.  Check path to ensure it was" >> $MESSAGE
      echo "entered properly.  Installation aborted" >> $MESSAGE
      SUBJECT="Mode_Sec Rule Auto Updater Failed"
      exit 0
fi

echo "Unpacking the rules TAR file that was downloaded..." >> $MESSAGE
tar -xzf $RULEFILE -C $RULEDIR
echo "Getting rid of 'asl directories' and Moving files into their positions... please wait." >> $MESSAGE

LORIODIR=$RULEDIR"/modsec"

if [ -d $LORIODIR ]; then
      cd $LORIODIR
else
      cd $RULEDIR
fi

sed -i 's/\/etc\/asl\/whitelist/whitelist.txt/g' 00_asl_whitelist.conf
sed -i 's/\/etc\/asl\/whitelist/whitelist.txt/g' 00_asl_rbl.conf

##########
# ADD YOUR OWN BLACKLISTED DOMAINS INTO THE RULES
##########
# dos2unix $WORKNDIR/puntapirata-badomain.txt
# cat $WORKNDIR/puntapirata-badomain.txt malware-blacklist.txt > malware
# cat $WORKNDIR/puntapirata-badomain.txt domain-blacklist.txt > domain
# sort -u malware > malware-blacklist.txt
# sort -u domain > domain-blacklist.txt
##########

##########
# IF YOU DON'T LIKE THE FOLLOWING RULES IN YOUR SERVER, MAKE THE FOLLOWING LINES ACTIVE:
##########
rm -f 00_asl_rbl.conf
rm -f 05_asl_scanner.conf
rm -f 11_asl_data_loss.conf
##########

if [ -d $LORIODIR ]; then
      mv * $RULEDIR
      rmdir $LORIODIR
fi

cd $RULEDIR

chown -R root:root $RULEDIR

echo "Testing rules to be sure there are no errors" >> $MESSAGE
echo
/usr/local/apache/bin/httpd -t 2>/tmp/apachechk
APACHECHK=`grep "Syntax OK" /tmp/apachechk`
if [[ $APACHECHK = "Syntax OK" ]]; then
      echo $APACHECHK >> $MESSAGE
      echo "Apache configuration PASSED!" >> $MESSAGE
      echo "Restarting Apache. (may take up to 2 minutes to restart)" >> $MESSAGE
              /scripts/restartsrv_httpd
else
      echo "Apache Configuration FAILED!" >> $MESSAGE
      echo $APACHECHK >> $MESSAGE
      echo "Check the error above and resolve any conflicts" >> $MESSAGE
      echo "Previous rules were restored" >> $MESSAGE
      rm -rf $RULEDIR
      mv $WORKNDIR/backups/`date +%m%d%Y` $RULEDIR
      SUBJECT="Mode_Sec Rule Auto Updater Failed"
fi

echo "Cleaning up" >> $MESSAGE
rm -f $RULEFILE
rm -f /tmp/apachechk

echo "UPDATE RULES completed..." >> $MESSAGE
/bin/mail -s "$SUBJECT" "$EMAILTO" < $MESSAGE

rm -f $MESSAGE
exit 0
Sorry Mike, my fault.

I wanted to know about the free rules, not my script.

What is the free modsec rule version that you have downloaded?

Sergio
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
@ Mike,
I didn't knew that my script was modified :) that is nice.

Well, after reading the modified script I didn't find anything wrong, but try the following, enter as root:
cp > /var/asl/data/msa/global.dir
cp > /var/asl/data/msa/global.pag
cp > /var/asl/data/msa/ip.dir
cp > /var/asl/data/msa/ip.pag
If there is another file that GotRoot needs, create it the same way.

I am sure this will fix the error that you have.

Sergio
 

lbeachmike

Well-Known Member
Dec 27, 2001
307
4
318
Long Beach, NY
cPanel Access Level
Root Administrator
Hi Sergio -

Rules version was in my previous post (before the script post) - modsec-2.5-free-latest.tar.gz

Can you tell me why these additional files need to be created as shown in your last post? Did we miss something during installation? This was not in the installation directions.

Thanks for all of your help.

Mike
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
Hi Sergio -

Rules version was in my previous post (before the script post) - modsec-2.5-free-latest.tar.gz

Can you tell me why these additional files need to be created as shown in your last post? Did we miss something during installation? This was not in the installation directions.

Thanks for all of your help.

Mike
ModSec rules from GotRoot comes from an ASL package that was not designed for CPanel. The ASL security package includes a lot of futures that we don't use and this directories are used by ASL to store some info that is found on the modesec rules.

If you look in the ModSec2.user.conf file that I use for my autoupdate script, it includes the following command:

SecAuditLogParts ABIFHZ

If you modify this line, then the directories created for ASL are used, but they consume a lot of space and your /VAR directory trends to be filled so quickly. On the other hand, all the info saved in there is useless if you don't have the ASL Security package and that is why I only left the info that CSF uses to block IPs using ModSec.

Answering your question about why this files are not in the installation instructions, I really don't know. The only thing that I could think of is because you chmod 700 the directories instead of 770 as my post http://forums.cpanel.net/f185/modsecurity-auto-updater-147745.html#post646729 saids.

Sergio
 

k-planethost

Well-Known Member
Sep 22, 2009
199
11
68
Athens Greece
mike
grep "modsecurity" /usr/local/apache/logs/error_log
will show you which version of mod sec you have on the box
if its less than 2.5.12 you can update through easy apache update
Atomicorp

modsec-201101200807.tar.gz
http://www.atomicorp.com/wiki/index.php/Atomic_ModSecurity_Rules
here is how to install the rules on a cpanel /centos server
check if yo have pcre and APR installed
 
Last edited:

k-planethost

Well-Known Member
Sep 22, 2009
199
11
68
Athens Greece
This is the list of the rules that I use:
00_asl_0_global.conf
00_asl_whitelist.conf
05_asl_exclude.conf
10_asl_antimalware.conf
10_asl_antimalware_output.conf
10_asl_rules.conf
11_asl_adv_rules.conf
20_asl_useragents.conf
30_asl_antimalware.conf
30_asl_antispam.conf
30_asl_antispam_referrer.conf
40_asl_apache2-rules.conf
50_asl_rootkits.conf
60_asl_recons.conf
61_asl_recons_dlp.conf
98_asl_jitp.conf
99_asl_exclude.conf
99_asl_jitp.conf
99_asl_redactor.conf
99_asl_redactor_post.conf

Among my own set of rules.

Sergio
good i use all of these as well
what about the following is there any use of this files


domain-blacklist-local.txt
domain-blacklist.txt
domain-spam-whitelist.conf
domain-spam-whitelist.txt
malware-blacklist-high.txt
malware-blacklist-local.txt
malware-blacklist-low.txt
malware-blacklist.txt
sql.txt
trusted-domains.conf
trusted-domains.txt
whitelist.txt
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
good i use all of these as well
what about the following is there any use of this files


domain-blacklist-local.txt
domain-blacklist.txt
domain-spam-whitelist.conf
domain-spam-whitelist.txt
malware-blacklist-high.txt
malware-blacklist-local.txt
malware-blacklist-low.txt
malware-blacklist.txt
sql.txt
trusted-domains.conf
trusted-domains.txt
whitelist.txt
All of this files are needed, modsec rules need this files in order to work, inside this files are data that some modsec rules compares with what is attacking your server and acts accordingly.

I suggest don't delete any of this files. If you want, open them with a text editor and look what is inside of them you can edit them to suit your needs. In my site there is an example on how to tweak one particular file, look at this address:
How to Tailor Made ModeSecurity

Sergio
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
Yes, you can set them on the white list. Look for the file called whitelist.txt and set there the domains that you want not to block. Remember to restart Apache after you modify the file.
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
Kind of hard to say its a reliable list if google bots are making it into the blacklist =/
Hello MACscr,
what blacklist are you referring to? Can you give us the name of the blacklist file?

Sergio
 

MACscr

Well-Known Member
Sep 30, 2003
198
5
168
cPanel Access Level
Root Administrator
Sergio something last
i observe on the logs that are banned as well useful bots such as google bot msn bot etc
is there any way to avoid this thanks
I havent used it. My statement was based off the above observation. Google and others add legitimate bots all the time, would be to much of a hassle to keep track of their ip's and have to keep adding them to a white list. The rule managers (gotroot.com?) should be making sure their rules are clean of such things.
 

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
I havent used it. My statement was based off the above observation. Google and others add legitimate bots all the time, would be to much of a hassle to keep track of their ip's and have to keep adding them to a white list. The rule managers (gotroot.com?) should be making sure their rules are clean of such things.
Well, that is something that GotRoot has to answer as we are not related in any way to them.

I know that Mike from GotRoot some times reads my thread, so, may be he could answer this for us. Or, you can send him a PM.

On the other hand, there is an easy way to add google to the whitelist, instead of using IPs you can add domain names in the whitelist.txt file.

In the field, I have seen reported attacks by ModSecurity rules that comes from Google, but that is an small price to pay compared to the hundred of attacks that a server receives daily and that ModSecurity stops on the fly.

Sergio
 

k-planethost

Well-Known Member
Sep 22, 2009
199
11
68
Athens Greece
i think on whitelist.txt you place only ips
on the june free rules that they releashed i observe there is a file trusted-domains.txt with Google entry on it.
maybe was from before releases and i didnt see it
anyway
do you have to add any entries for this file on modsec2.user.conf in order to load it on apache?
 

mikegotroot

Well-Known Member
Verifed Vendor
Apr 29, 2008
85
1
58
The rule managers (gotroot.com?) should be making sure their rules are clean of such things.
We are not aware of any issues involving any search engine bots, could you please provide an example of the rules stopping something legitimate from a search engine? We would be happy to fix it the same day its reported. I personally check all our support cases everyday, and I have not seen any reports of the rules blocking a search bot, so please let us know.

Regardless of what kind of a false positive (or negative), if you run into any problems with our modsecurity rules please report them using the process linked to below, which will provide us all the information we need:

https://www.atomicorp.com/wiki/index.php/Reporting_False_Positives

We release fixes the same day they are reported.
 
Last edited:

Secmas

Well-Known Member
Feb 18, 2005
388
21
168
i think on whitelist.txt you place only ips
on the june free rules that they releashed i observe there is a file trusted-domains.txt with Google entry on it.
maybe was from before releases and i didnt see it
anyway
do you have to add any entries for this file on modsec2.user.conf in order to load it on apache?
If you already have defined the asl rules, then you don't need to add any entries on modsec2.user.conf, but any time that you modify any of the files including .TXT files you need to restar apache for the changes to work.

Regarding about what Mike saids, here as an example of what ModSec is blocking from Google:
Time: Sun Jul 3 09:53:33 2011 -0500
IP: 66.249.72.226 (US/United States/crawl-66-249-72-226.googlebot.com)
Failures: 4 (mod_security)
Interval: 3600 seconds
Blocked: Temporary Block

Log entries:

[Sun Jul 03 09:14:24 2011] [error] [client 66.249.72.226] ModSecurity: Access denied with code 403 (phase 2). Match of "rx (^-?[0-9]+$|^-?[0-9]+\\\\:([a-z0-9- ]+|[0-9a-z- ]+)$|^$|^[-0-9:a-z \\\\.\\\\!]+$)" against "ARGS:id" required. [file "/usr/local/apache/conf/modsec_rules/99_asl_jitp.conf"] [line "4288"] [id "390605"] [rev "13"] [msg "Atomicorp.com WAF Rules - Virtual Patch: Joomla id ARG injection"] [severity "CRITICAL"]
May be in a future it will be fixed.

Sergio