The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

High Exim Load: Suggestions to improve??

Discussion in 'General Discussion' started by bmcpanel, Feb 10, 2005.

  1. bmcpanel

    bmcpanel Well-Known Member

    Joined:
    Jun 1, 2002
    Messages:
    546
    Likes Received:
    0
    Trophy Points:
    16
    I use RH 7.3, CPanel latest edge.
    I use suexec and have disallowed nobody from sending mail.
    I use CLam AV with BFD.

    I have a lot of clients who use email during business hours. My server load average will shoot to 7-9 and averages around 3-4 during business hours. It then levels off to 0.7 - 1.3 during the evening.

    I watch the server with "top" and "pstree", but can only see that exim processes are open and cannot see who has opened them.

    I guess I am seeking suggestions about how to reduce the load average on the server that is being caused by exim. What would you guys do in this instance?
     
  2. dezignguy

    dezignguy Well-Known Member

    Joined:
    Sep 26, 2004
    Messages:
    534
    Likes Received:
    0
    Trophy Points:
    16
    tail your exim logs for a while... though it can be hard to extract any useful info from them... so use WHM's exim statistics page and see how your traffic looks.

    Are you using any spam or virus filter? Those can significantly increase loads because of the cpu/disk intensive scanning.

    set a limit on email size in exim... emails with huge attachments can easily bog down a mailserver... and the office drones really don't need to be sending 10 or 20 MB AVI movies to each other. Use an ftp server or a fileshare for large files.

    Consider getting a more powerful server...

    Consider moving half your clients onto another server.

    Reject :)fail:) unwanted email in the smtp conversation (spam, viruses, and email sent to non-existent accounts). Don't use catch-all addresses.
     
  3. bmcpanel

    bmcpanel Well-Known Member

    Joined:
    Jun 1, 2002
    Messages:
    546
    Likes Received:
    0
    Trophy Points:
    16
    I do this daily. I don't notice anything out of the ordinary, except for a large level of incoming spam, which I suppose most of us have a problem with. Also, the load is always higher during business hours (9am to 5pm).

    Yes, Clam AV with Brute Force Detection installed. When I first installed Clam AV, it reduced my server load because the mail server was not delivering all that junk mail. Now, I don't know... maybe it is the BFD that is causing the high load. Don't know.

    That's a good idea. Where is the global setting for this, exim.conf?

    Probably not an issue as this problem did not exist on this server a few months ago and the number of accounts is about the same as before.

    See above...

    I prefer to :blackhole: them so exim doesn't waste time sending a failure reply to the origin address.

    Thanks for the suggestions.
     
  4. junglecat

    junglecat Well-Known Member

    Joined:
    Jul 6, 2004
    Messages:
    168
    Likes Received:
    0
    Trophy Points:
    16
    I may be wrong, but I *think* :blackhole: makes them bounce around in the server for a while before they are deleted, and :fail: means they never enter the server.

    Therefore :fail: should reduce server resources.

    Someone correct me if my information is faulty.
     
  5. DavidR

    DavidR Well-Known Member

    Joined:
    Feb 25, 2003
    Messages:
    177
    Likes Received:
    0
    Trophy Points:
    16
    From everything that I read, you are correct - :fail: is less server intensive and much preferred over :blackhole:.

    David
     
  6. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
  7. dezignguy

    dezignguy Well-Known Member

    Joined:
    Sep 26, 2004
    Messages:
    534
    Likes Received:
    0
    Trophy Points:
    16
    heh, I got stalled looking for your link, chirpy.

    bmcpanel, I limit the size of messages with this near the top of exim.conf (you can put it in the first box in the WHM config editor)

    Code:
    message_size_limit = 11M
    That limits mails to 11M... anything coming in through SMTP, I believe... which means incoming and outgoing mail will be limited together. But 11 megs is quite a large enough email incoming or outgoing.
     
  8. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Indeed, IIRC, a default sendmail installation limits (or at least used to limit) mail to 5MB.
     
  9. eurorocco

    eurorocco Well-Known Member

    Joined:
    Jun 23, 2003
    Messages:
    99
    Likes Received:
    0
    Trophy Points:
    6
    In my experience... high load due to exim is because...

    Someone is poking at port 25 with several threads, probably from a host connected with broadband. Then I see the mail log and it is someone poking at exim.

    Also, it could be because one of your customers is sending out an email with a very long CC or BCC line, and heavy attachments, like many of the messages in the thread have already pointed out.

    I have several simple Perl scripts, since I haven't found a better solution.

    One is cpumonster, that goes in the crontab and checks /proc/loadavg every minute. If it is greater than 8, then service chkservd stop, service cpanel stop, service http stop, service exim stop. It does everything it can do to cool the server down. Then, a notifier in my console tells me http there is down, then I investigate. If the load is greater than 3, just shutdown http. It's an arbitrary choice I made to never never see the load skyrocket to 90. If it does 90 I cannot even ssh into the box, so I got these things in place. When the load drops below 1, service chkservd start, then service cpanel start , service http start, service exim start.

    Another is whatpain, for instance whatpain 80 grabs the output of netstat -n , parses the output and tells me how many connections there are open on port 80.

    Or whatpain 25, to see who is poking the SMTP port and how many threads.

    Or whatpain 110 for POP3

    Then another script like blockip X.Y.Z.M b to block the ip or the network.

    Also, another script scanmailog IP that checks the tails of all the /var/log/exim_ and /var/log/maillog files to tell me what this IP is doing with exim or cppop.

    Maybe it should be a single script doing the whole thing.

    Sometimes, when I just don't feel like dealing with attacks, I just block the whole Southeast Asia region and the server, whatever problem, cools off. :)

    Most likely, now, serious, if exim causes an overload it is either a customer of yours sending thousands of heavy emails, or some intruder attempting a brute force attack on Exim to see what it accepts. Try tail -f /var/log/exim_mainlog or tail -f /var/log/exim_rejectlog and you'll see.

    Then, serious too, what we did was to get twice the number of servers and twice the power per server, and systems became less sensitive to attacks. They do spike every now and then though, but much less often.

    Now, my question, is there a formal tool I can use to replace the script zoo I got?

    ER
     
  10. junglecat

    junglecat Well-Known Member

    Joined:
    Jul 6, 2004
    Messages:
    168
    Likes Received:
    0
    Trophy Points:
    16
    Where does one find cpumonster?
     
  11. eurorocco

    eurorocco Well-Known Member

    Joined:
    Jun 23, 2003
    Messages:
    99
    Likes Received:
    0
    Trophy Points:
    6
    cpumonster

    This script stops http, exim and cpanel, and chkservd if load average exceeds 8.

    It stops http if load exceeds 6 but is less than 8.

    If it stops anything it checks every 10 seconds to see how load drops until load drops below 3, then it restarts whatever it stopped.

    It is a bit drastic, but has saved us from losing control of servers when a nasty script takes over or someone from the outside is pounding on the server. Our idea was to drop services before losing server control.

    Place this in /usr/local/sbin and then add a line into the crontab (ie crontab -e to edit crontab).

    */3 * * * * /usr/local/sbin/cpumonster >>/var/log/cpumonster.log 2>&1

    This is what I use. Use at your own risk. Improvements are welcome.

    Code:
    #!/usr/local/bin/perl
    
    $deathlimit = 8.0;
    $deathstop = 0;
    
    $hardlimit = 6.0;
    $softlimit = 3.0;
    $timespan = 10;
    
    $debug = 0;
    $quiet = 0;
    
    $load = getload();
    
    if($load >= $hardlimit){
        do{
            track("load $load too hight!\n");
            track("stopping apache!\n");
            hookup("/sbin/service httpd stop");
            hookup("/sbin/service httpd stop");
            hookup("/sbin/service httpd stop");
    
            if($load >= $deathlimit){
                hookup("/sbin/service exim stop");
                hookup("/sbin/service cpanel stop");
                hookup("/sbin/service ftp stop");
                hookup("/sbin/service chkservd stop");
    
                track("deathstop stop\n");
    
                $deathstop = 1;
            }
    
            track("sleep $timespan seconds...\n");
            sleep($timespan);
    
            $load = getload();
        }while($load >= $softlimit);
    
        track("load $load is okay!\n");
        track("starting apache!\n");
        hookup("/sbin/service httpd start");
    
        if($deathstop){
            track("deathstop start\n");
            hookup("/sbin/service exim start");
            hookup("/sbin/service cpanel start");
            hookup("/sbin/service ftp start");
            hookup("/sbin/service chkservd start");
        }
    
        exit(0);
    }
    else{
        track("load $load is okay!\n");
        exit(0);
    }
    
    sub getload{
    
        my $what = `cat /proc/loadavg`;
        chomp($what);
    
    
        my @cells = split(/\s+/,$what);
        chomp(@cells);
    
        return($cells[0]);
    }
    
    sub track{
    
        my $what = $_[0];
        my $stamp = gettimestamp(time);
    
        $quiet == 0 || return;
        print "$stamp $what";
    }
    
    sub hookup{
    
        my $cmd = $_[0];
    
        track("$cmd\n");
    
        if($debug == 0){
            system($cmd);
        }
    }
    
    sub gettimestamp{
    
        my ($sec,$min,$hour,$mday,
            $mon,$year,$wday,$yday,$isdst) =
            localtime($_[0]);
        return(sprintf("%04d-%02d-%02d %02d:%02d:%02d ",
             1900+$year,$mon+1,$mday,$hour,$min,$sec));
    
    }
    Hope it helps!

    ER
     
    #11 eurorocco, Mar 9, 2005
    Last edited: Mar 9, 2005
  12. iconraul

    iconraul Member

    Joined:
    Mar 14, 2006
    Messages:
    8
    Likes Received:
    0
    Trophy Points:
    1
    How many minutes..?

    Hello,

    I'm wondering. How long will it take for the e-mail to stay in the mail queue (cPanel -> Email -> Mail Queue)? Is there a way to control this?

    The situation is, i have one user whose account is being bombarded by newsletters and spam. Yesterday, I got chirpy's MailScanner working and also the Exim Dictionary attach ACL (also from chirphy).. but today, I found 100+ e-mails addressed to the same person. I've cheched his e-mail stats and he's like using 100% of the allowed qouta.


    Some of the e-mails were from clamav(mailscanner) saying that the attachment is not acceptable... how can I disable this e-mail informing the user of any virus or unacceptable attachments? because I think it also contributes to the Mail queue being clogged by LOTS of e-mails.


    Thanks! Thanks!



    More power to everyone!

    jun
     
  13. eurorocco

    eurorocco Well-Known Member

    Joined:
    Jun 23, 2003
    Messages:
    99
    Likes Received:
    0
    Trophy Points:
    6
    I wish...

    Responding to the previous post, in Service Configuration > Exim Configuration Editor > Advanced , at the bottom of that page, between the last textboxes you can see something like

    * * F,2h,15m; G,16h,1h,1.5; F,4d,8h

    This is the line that controls number of retries, and the lifespan of a message in the queue. Cpane does not let you modify it, but you can manually edit /etc/exim.conf.

    Unless you are some kind of Exim guru I recommend that you don't play with these.

    Consider the Anti Spam and Virus solution in rvskin.com, which is part of what we use.

    I struggle with viruses and spam using Cpanel. I wish it were more robust out of the box, but it isn't. I guess it is better than other systems I have seen, but it has a long way to go.

    ER
     
  14. beehave

    beehave Well-Known Member

    Joined:
    Jun 26, 2005
    Messages:
    104
    Likes Received:
    0
    Trophy Points:
    16
    I find cpumonster very helpfull. The following amended code checks to make sure script is not already running. Hope it helps...

    Code:
    #!/usr/local/bin/perl
    
    $deathlimit = 8.0;
    $deathstop = 0;
    
    $hardlimit = 6.0;
    $softlimit = 3.0;
    $timespan = 10;
    
    $debug = 0;
    $quiet = 0;
    
    $load = getload();
    
    # Checking to see if we are already running
    $pid = `/sbin/pidof -s cpumonster`;
    if($pid >= 1) {
        print "CPUMonster is already running!\n";
        exit[0];
    } else {
    
    if($load >= $hardlimit){
        do{
            track("load $load too hight!\n");
            track("stopping apache!\n");
            hookup("/sbin/service httpd stop");
            hookup("/sbin/service httpd stop");
            hookup("/sbin/service httpd stop");
    
            if($load >= $deathlimit){
                hookup("/sbin/service exim stop");
                hookup("/sbin/service cpanel stop");
                hookup("/sbin/service ftp stop");
                hookup("/sbin/service chkservd stop");
    
                track("deathstop stop\n");
    
                $deathstop = 1;
            }
    
            track("sleep $timespan seconds...\n");
            sleep($timespan);
    
            $load = getload();
        }while($load >= $softlimit);
    
        track("load $load is okay!\n");
        track("starting apache!\n");
        hookup("/sbin/service httpd start");
    
        if($deathstop){
            track("deathstop start\n");
            hookup("/sbin/service exim start");
            hookup("/sbin/service cpanel start");
            hookup("/sbin/service ftp start");
            hookup("/sbin/service chkservd start");
        }
    
        exit(0);
    }
    else{
        track("load $load is okay!\n");
        exit(0);
    }
    
    sub getload{
    
        my $what = `cat /proc/loadavg`;
        chomp($what);
    
    
        my @cells = split(/\s+/,$what);
        chomp(@cells);
    
        return($cells[0]);
    }
    
    sub track{
    
        my $what = $_[0];
        my $stamp = gettimestamp(time);
    
        $quiet == 0 || return;
        print "$stamp $what";
    }
    
    sub hookup{
    
        my $cmd = $_[0];
    
        track("$cmd\n");
    
        if($debug == 0){
            system($cmd);
        }
    }
    
    sub gettimestamp{
    
        my ($sec,$min,$hour,$mday,
            $mon,$year,$wday,$yday,$isdst) =
            localtime($_[0]);
        return(sprintf("%04d-%02d-%02d %02d:%02d:%02d ",
             1900+$year,$mon+1,$mday,$hour,$min,$sec));
    
    }}
     
Loading...

Share This Page