The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

My MailScanner / Exim Tweaks

Discussion in 'E-mail Discussions' started by drose25, Feb 16, 2007.

  1. drose25

    drose25 Well-Known Member

    Joined:
    Apr 3, 2002
    Messages:
    50
    Likes Received:
    0
    Trophy Points:
    6
    I use Chirpy’s oft-maligned MailScanner package. Despite the complaints about it, no one has put together something better, and Chirpy’s package works well, is easy to administer, and the Cpanel client front-end that comes with their installation package makes it easy for clients to configure their personal settings, all pretty important things to consider in my book.

    So I thought I would share my various MailScanner and Exim tweaks that I use to make the two work better and reduce the load on the server. Some of them come from other posts in this forum, and some come the Exim and MailScanner sites. They have worked well for me, but your mileage may vary.

    If you have any others, please share them with us.

    These are in no particular order, but I’ve added my comments and experiences with each.

    1. Use a ramdisk for MailScanner’s work directory.

    This was very easy to do and has helped reduced both server load and queue processing time. This is especially noticeable when dealing with large mailing lists or messages with large attachments.

    First, create a ramdisk using tmpfs. It needs to be remounted at every boot, so adding it to /etc/fstab is the easiest. Use vi or your favorite editor to open /etc/fstab and add a line like this:

    none /var/mailscanner tmpfs size=100m 0 0

    /var/mailscanner is the mount point for the ramdisk, and the size option sets the amount of memory to use.

    Second, go ahead and mount your new ramdisk:

    #mount /var/mailscanner

    You can do df -vh to make sure it has mounted if you like.

    Third, tell MailScanner to use the new ramdisk as its working directory. If you use the WHM front-end, just click on the MailScanner Configuration button to edit the file. Otherwise, it is usually located at /usr/mailscanner/etc/MailScanner.conf

    In your MailScanner configuration file, find the line that begins:

    Incoming Work Dir =

    and set it to your ramdisk. For Example:

    Incoming Work Dir = /var/mailscanner

    Finally, save your changes to the MailScanner configuration, and restart the service manually if you aren’t using the WHM front-end to edit the file.

    Determining the right size to use for the ramdisk is tricky. I originally used 20m, and MailScanner ran happily for several days until it choked without warning. This happened a couple of times before I realized it was choking because it had run out of workspace on the ramdisk. My current setting of 100mb is probably not optimal, and is based on educated guessing as follows. If anyone can provide a better method for calculating this, I’d love to hear it.

    My suggestion would be to find these lines in your MailScanner configuration:

    Max Unscanned Bytes Per Scan = (Mine is set to 50m)
    Max Unsafe Bytes Per Scan = (Mine is set to 30m)

    I recommend twice the Max Unscanned Byters Per Scan unless your mail usage would suggest otherwise. Theoretically, your ramdisk size needs to be at least as big as Max Unscanned Bytes Per Scan, but in reality, if Max Unscanned Byters Per Scan determined the size of the every batch, your ramdisk would need to be at least two or three times that size to accommodate expanding compressed files. Fortunately, the next two lines under those also limit the number of messages per scan:

    Max Unscanned Messages Per Scan = (Mine are both at 30)
    Max Unsafe Messages Per Scan =

    So MailScanner will scan 30 messages per scan, unless 30 would put it over the size limits above in which case it would scan fewer messages. Most of the time, this means you probably wouldn’t need more than 2-3mb of space except when large attachments are being sent. Then the size limit in Max Unscanned Bytes Per Scan will come into play. Consider what your numbers for these options are, and make an educated guess. :) Using twice the Max Unscanned Bytes has not caused me any problems in the five or six months I’ve used it, but your mileage may vary.

    2. Offload blacklist checking to Exim

    I hated doing this because I believe clients should get every piece of mail so *they* can determine what’s spam and what’s not, and to protect that occasional mis-marked non-spam message out there, but the volume of spam the server received finally forced me to do it. This has made a *big* difference in terms of the loads put on the system by both Exim and MailScanner. Instead of processing 12-15K messages a day, MailScanner now only has to deal with ~4K because the rest are dropped before receipt.

    Place this in the second ACL box in the advanced Exim configuration editor:

    deny message = Message rejected - $sender_fullhost is in an RBL, see $dnslist_text
    !hosts = +relay_hosts
    !authenticated = *
    dnslists = bl.spamcop.net : sbl-xbl.spamhaus.org

    Where it goes in the box will depend on your configuration. I use Chirpy’s dictionary attack ACL, and place this right after it. That puts it above this line:

    # Accept bounces to lists even if callbacks or other checks would fail

    You may need to put it elsewhere.

    3. Keep the MailWatch database clean

    If you use Chirpy’s package, you probably have MailWatch installed. MailWatch is great for keeping an eye on what’s happening with mail and spam, and for training the Bayesian filter. However, the default install never seems to clean out the accumulated messages in the database (as opposed to the quarantine). After a while, accessing the database starts to get slow, and MailScanner performance suffers. I encountered this when loading a MailWatch page went from taking seconds to minutes.

    You can execute a simple cron job to keep it clean and moving swiftly.

    First, create a small shell script to trim the database. Use vi or your favorite editor to create a file named mailwatch_mysql_maintenance.sql with these lines:

    delete from maillog where timestamp < date_sub(curdate(), interval 14 day);
    optimize table maillog;

    Second, add the following line to your crontab, editing as appropriate:

    0 10 * * * /path/to/mysql mailwatch_database -u mailwatch_user --password=mailwatch_password < /path/to/mailwatch_mysql_maintenance.sql

    This will run nightly and delete messages that are over 14 days old, keeping your database manageable and speedy.


    4. Let Exim drop senders who only use IPs

    This came from another thread in this forum. Like moving RBL checking to Exim, it helps reduce MailScanner loads by reducing the number of messages MailScanner has to deal with.

    Place this in the first ACL box in the advanced Exim configuration editor:

    check_helo:

    ################################################## ###
    # IP Only is sent as the HELO
    deny condition = ${if match {$sender_helo_name}\
    {^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\$}\
    {yes}{no}}
    message = Your server announces itself ($sender_helo_name) with a plain IP address which is in breach of RFC2821.
    log_message = Bad HELO: IP Only Announce
    ################################################## ###

    accept
     
  2. drose25

    drose25 Well-Known Member

    Joined:
    Apr 3, 2002
    Messages:
    50
    Likes Received:
    0
    Trophy Points:
    6
    5. Eliminate the crap that accumulates in your Exim queue.

    The fewer messages that Exim and MailScanner have to process, the less they tax your server. My Exim queue used to grow by 500-1000 messages a day. Most of these were spam bounces to non-existent addresses. Changing the default address for all accounts to :fail: helped get rid of most of these. You can find a number of threads in this forum on how to do this, and the pros and cons of doing so.

    I haven’t encountered any trouble with it. My only suggestion would be to advise all of your clients of the change, and how they can set their own default in Cpanel, because you will find that one or two clients rely on the default address catching everything sent to their domain.

    Another source of crap in the queue are messages that are held because the recipient account is over its disk quota. You can configure Cpanel to discard messages for accounts that are over quota in WHM, under Tweak Settings. I don’t recommend this because accounts sometimes go over quota accidentally, and I hate deleting mail. :)

    I prefer to keep an eye on the accounts that are over-quota, and contact the account owner if it’s not a temporary issue. Oftentimes the problem stems from a mail account no longer being used that no one bothered to delete, and a message to the client resolves the problem. Sometimes clients don’t realize they don’t get mail when their account is over-quota (and they ignore the quota warnings!), so a little education helps as well. Make sure they know that if their site is over the quota, so are their mailboxes, even if they are empty. :)

    Do a search on Exim queues here for more suggestions on how to keep the crap out of your Exim queue.


    6. Optimize Max Children and Queue Scan Interval

    Both of these settings are in the MailScanner.conf file. The actual numbers you need to use will vary depending on your server and mail traffic. After some experimentation, I found the defaults to be very high for my server. I now have Max Children set to 2, with a 15 second Queue Scan Interval, and there is rarely any backlog in the queue.

    To find the numbers that are right for your server, I recommend installing Chirpy’s mailqueue addon so you can easily see how many messages are in the Exim queue and the MailScanner queue. Also, I would wait until you’ve done whatever steps above you’re going do, otherwise you’ll probably need to do this again afterwards. It’s also important to have your mail queue clean to better gauge performance.

    I started by reducing the Max Children to 3, and checking server performance. Check your mail queues (both Exim AND MailScanner, because it won’t make it into the Exim queue until MailScanner is done with it) every 10 minutes or so for an hour or two during the server’s busy time of day. If your MailScanner queue isn’t filling up, let it run there for a couple of days to allow for the occasional weird traffic day. If things are still moving smoothly, drop it down by 1 and start over.

    You can do this until messages start accumulating in the MailScanner queue, at which point you will want to go back up one or two. If you set the number too low, you will usually know it pretty quickly because things can start backing up in a hurry.

    Queue Scan Interval appears to be more a function of personal preference and the number of children than anything else.
     
  3. bornonline

    bornonline Well-Known Member

    Joined:
    Nov 19, 2004
    Messages:
    139
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Earth
    Good stuff... thanks for the info.
     
  4. mrcgiguy

    mrcgiguy Registered

    Joined:
    Mar 18, 2004
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    what about blank helo?

    I was getting a TON of emails with blank helo's...
    So I added this:
    deny condition = ${if !def:sender_helo_name {yes}{no}}
    message = If you aren't going to say hello first, then screw you
    log_message = If you aren't going to say hello first, then screw you
     
  5. verdon

    verdon Well-Known Member

    Joined:
    Nov 1, 2003
    Messages:
    836
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Northern Ontario, Canada
    cPanel Access Level:
    Root Administrator
    Just wanted to say that I worked my way through this and have implimented most things (except the ram disk) and have noticed a large improvement in avg server load.

    I think the thing that made the biggest single difference was off-loading the blacklist checking to Exim. There is a whole lot less mail going through mailscanner now. This technique is also discussed on chirpy's site in the faq area for optimizing mailscanner.

    Thanks for your excellant post here drose25
     
  6. sawbuck

    sawbuck Well-Known Member

    Joined:
    Jan 18, 2004
    Messages:
    1,367
    Likes Received:
    5
    Trophy Points:
    38
    cPanel Access Level:
    Root Administrator
  7. allwebnow

    allwebnow Well-Known Member

    Joined:
    Jan 8, 2007
    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    6
    Where is this discussed on chirpy's Site? I can't find a FAQ section for tweaking mailscanner at configserver.com. Anyone know of where that is?
     
  8. allwebnow

    allwebnow Well-Known Member

    Joined:
    Jan 8, 2007
    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    6
    Thanks! Wow.
     
  9. screege

    screege Well-Known Member

    Joined:
    Aug 11, 2004
    Messages:
    186
    Likes Received:
    0
    Trophy Points:
    16
    Please hekp I get the following error when executing the script:

    /usr/sbin/mysqld mailscanner -u mailwatch --password Qfr3Aw8X
    < /etc/mailwatch_mysql_maintenance.sql
    071208 10:37:46 [ERROR] /usr/sbin/mysqld: unknown option '--password'

    /usr/sbin/mysqld mailscanner -u mailwatch -password Qfr3Aw8X <
    /etc/mailwatch_mysql_maintenance.sql
    071208 10:37:34 [ERROR] /usr/sbin/mysqld: unknown option '-p'

    /usr/sbin/mysqld mailscanner -u mailwatch --p Qfr3Aw8X < /etc/
    mailwatch_mysql_maintenance.sql
    071208 10:37:56 [ERROR] /usr/sbin/mysqld: ambiguous option '--p' (pid-file, prel
    oad_buffer_size)

    I am using mysql v5

    Thanks
     
  10. kernow

    kernow Well-Known Member

    Joined:
    Jul 23, 2004
    Messages:
    865
    Likes Received:
    9
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    drose25 Thanks for the tips ! :)
     
  11. screege

    screege Well-Known Member

    Joined:
    Aug 11, 2004
    Messages:
    186
    Likes Received:
    0
    Trophy Points:
    16
    Anyone can help me?

    Thanks in advanced
     
Loading...

Share This Page