The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

EXIM spooling issue

Discussion in 'General Discussion' started by bhd, Apr 4, 2005.

  1. bhd

    bhd Well-Known Member

    Joined:
    Sep 20, 2003
    Messages:
    149
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    JNB ZA
    cPanel Access Level:
    Root Administrator
    I'm sure others have had this problem before. I wonder if anyone has a solution:

    We have many customers running mail lists of various descriptions - some PHP, some Perl. Most are small with only a few hundred subscribers but some are a lot bigger and, as Murphy would have it, these customers generally send out HTML mailshots where each email is 60-100K big!

    In most cases the mail code will open the mail stream with somthing like this:

    Code:
    $sendmail = '/usr/sbin/sendmail -t';
    This is fine provided the number of outgoing emails is small since this method will deliver the mail immediately. However, when the mailing list is BIG and the emails are large, immediate delivery can hammer the server for hours. Often pushing up the load to 10 or 20 for several hours.

    What works is to spool the mail to the mailqueue like so

    Code:
    $sendmail = '/usr/sbin/sendmail -t -odq';
    The problem is, we had two customers send out mailshots of 10,000 emails each on Saturday at the same time ... 20,000 emails of 60K all trying to be delivered immediately killed the server!

    My question is this: Is there a [relatively simple] way to force exim to spool ALL mail regardless of whether sendmail is called with the -odq switch or not? If so, how do you do that?
     
  2. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    Tracked this down for you. In WHM > Exim Configuration Editor > Advanced Mode > in the first textarea add:

    queue_smtp_domains = *

    Now all email will be queued until the next queue run. When the queue run happens released emails intended for the same mail server will be delivered in one SMTP session (potentially saving resources).

    Be aware though, that this will give you a huge queue (especially with the default 1 hour queue flush) and if your server strays over a load average of 3, the queue run will be skipped.
     
  3. bhd

    bhd Well-Known Member

    Joined:
    Sep 20, 2003
    Messages:
    149
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    JNB ZA
    cPanel Access Level:
    Root Administrator
    Thanks for this! I was sure it was possible.
     
  4. bhd

    bhd Well-Known Member

    Joined:
    Sep 20, 2003
    Messages:
    149
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    JNB ZA
    cPanel Access Level:
    Root Administrator
    Update - I've found something that promises to work even better.

    It seems queue_only_load will actually solve the problem far more elegantly.
    Thanks again Chirpy.
     
  5. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    That does look indeed look nicer and kinder on the mail system if it is server load in particular that you want to avoid :)
     
  6. bhd

    bhd Well-Known Member

    Joined:
    Sep 20, 2003
    Messages:
    149
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    JNB ZA
    cPanel Access Level:
    Root Administrator
    Thought you might be interested in an update. It's been running on one server for about 24 hours now. It works like a bomb!

    Mailqueue is all over the place of couse - anything from 20-5,000, but the server load is kept below 2 at all times. (I set it to start spooling at 2) - that is WAY better than 47 which is what it was spiking at every frew hours. When it spools, the longest email hangs around before getting sent is about 45 minutes ... don't see that as a problem.

    This is a must have tweak for all our servers as far as I'm concerned!
     
  7. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    That's great. Can I ask you:

    1. What interval do you have the queue run running at (default is 60min)?
    2. Have you changed deliver_queue_load_max from the default of 3?
     
  8. bhd

    bhd Well-Known Member

    Joined:
    Sep 20, 2003
    Messages:
    149
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    JNB ZA
    cPanel Access Level:
    Root Administrator
    A presume it is still 60 minutes since I haven't changed that. Actually I looked for it but could not find it in the exim documentation. The closest thing I could find was

    Code:
    F,2h,15m; G,16h,1h,1.5; F,4d,8h
    It is the default but I'm not sure if this applies to the spool queue as well?

    No, it is still set to 3. I figured that since this flag governs whether the queue should be delivered at all, it should be higher than the queue_only_load setting which will govern whether the mail gets delivered immediately or dumped in the queue.

    btw, one little problem I did find in MRTG is that this line in check-email does not like a mailqueue of more than (what seems to be) 3,200 .. it croaks with an error and shows a mailqueue = 0
    Code:
    QUE=`ls /var/spool/exim/input/*-H | wc -l | sed -e "s/ //g"`
     
    #8 bhd, Apr 6, 2005
    Last edited: Apr 6, 2005
  9. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    The queue interval is set in WHM > Tweak Settings > Number of minutes between mail server queue runs ;)
     
  10. bhd

    bhd Well-Known Member

    Joined:
    Sep 20, 2003
    Messages:
    149
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    JNB ZA
    cPanel Access Level:
    Root Administrator
    duh .. I guess I should RTFM!

    Yes, it is set to 60. :)
     
Loading...

Share This Page