The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[How-To] Mailman + Exim Tweaks

Discussion in 'E-mail Discussions' started by JackDcrack, Nov 7, 2003.

  1. JackDcrack

    JackDcrack Well-Known Member

    Joined:
    Aug 9, 2002
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    After reading several complaints regarding mailman, and messages being stuck in the exim queue, I did a little bit of research and found some interesting tweaks, that have helped my servers overall performance.

    1st, you must understand that it is NOT mailman that is causing the messages to not be delivered..
    mailman simply creates the messages, and passes it to Exim.
    It is EXIM that handles the delivery of the messages.


    Most of what I am about to say comes straight from the exim mailing list:
    http://www.exim.org/mailman/htdig/exim-users/Week-of-Mon-20021014/044956.html
    • Add the following to your EXIM MAIN CONFIGURATION:
      and running exim's queuerunner in small cycles (exim -q1m or -q2m).
    • Mailman will send as many MAIL FROM/RCPT TO as it needs. It may result
      in more than 10 or 100 messages sent in one connection, which will exceed
      the default value of Exim's smtp_accept_queue_per_connection
      This is bad because it will cause Exim to switch into queue mode and
      severely delay delivery of your list messages.
      The way to fix this is to set mailman's SMTP_MAX_SESSIONS_PER_CONNECTION
      (in ~mailman/Mailman/mm_cfg.py) to a smaller value than Exim's
      smtp_accept_queue_per_connection
    • Mailman should ignore Exim delay warning messages, even though
      Exim should never send this to list messages. Mailman 2.1's
      general bounce detection and VERP support should greatly improve
      the bounce detector's hit rates.
    • List existence is determined by the existence of a config.pck file
      for a list. If you delete lists by foul means, be aware of this.
    • If you are getting Exim or Mailman complaining about user ids
      when you send mail to a list, check that the MAILMAN_UID and
      MAILMAN_GID match those of Mailman itself (i.e. what were used
      in the configure script). Also make sure you do not have
      aliases in the main alias file for the list.
     
    #1 JackDcrack, Nov 7, 2003
    Last edited: Nov 22, 2003
  2. GOT

    GOT Get Proactive!

    Joined:
    Apr 8, 2003
    Messages:
    900
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Norfolk, VA
    cPanel Access Level:
    DataCenter Provider
    Is there any chance that you can elaborate on these suggestions some? I would be very interested in this. but I am very wear with the exim configuration.

    Would would be great for the community is if you could share how to implement these utilizing the exim configuration editor built into whm!

    Thanks!
     
  3. JackDcrack

    JackDcrack Well-Known Member

    Joined:
    Aug 9, 2002
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    Well, I have yet to use the built in function as I dont yet rust it :)

    But here is a step by step:
    (make sure you are logged into SSH as root) su -

    backup these files before proceeding:
    /etc/exim.conf
    /usr/local/cpanel/3rdparty/mailman/Mailman/mm_cfg.py


    using your favorite editor (vim, pico., etc)

    > pico /etc/exim.conf

    under MAIN CONFIGURATION SETTINGS
    INSERT BELOW perl_startup = do '/etc/exim.pl'
    PHP:
    queue_smtp_domains = ${if and {\
                                      {
    eq {127.0.0.1}{$interface_address}}\
                                      {
    def::h_X-Mailman-Version::}\
                                      }{*}{}\
                              }


    smtp_accept_queue_per_connection 50
    Save and exit

    > chattr +i /etc/exim.conf
    >pico /usr/local/cpanel/3rdparty/mailman/Mailman/mm_cfg.py

    INSERT BELOW # Put YOUR site-specific settings below this line.
    PHP:
    SMTP_MAX_SESSIONS_PER_CONNECTION 20
    Save and exit

    >service cpanel restart
    >service exim restart

    And thats that!
     
  4. WestBend

    WestBend Well-Known Member

    Joined:
    Oct 12, 2003
    Messages:
    173
    Likes Received:
    0
    Trophy Points:
    16
    Seeing as your a God with this stuff Jack :) Can you code/figure out a way to add an acl that will auto deny any smtp relay connections from valid user accounts UNLESS they are in a whitelist?

    I.e. I dont want to offer smtp relay accounts to all of my end users. They can use their own ISP smtp server
     
  5. markie

    markie BANNED

    Joined:
    Oct 5, 2003
    Messages:
    143
    Likes Received:
    0
    Trophy Points:
    0


    Thats not entirely correct. While mailman passes messages to Exim, exim.conf is what has been heavily modified by Cpanel to integrate messages via Exim and the issue lies in the Exim.conf and other modifications by Cpanel. If you take a stock exim.conf you will see its not integrating mailman at all so to say its an exim delivery problem is to say that exim + mailman has a slight coding problem. The idea would be to get the mailman delivery routines up to snuff so Exim doesnt kick them into the queue.
     
  6. JackDcrack

    JackDcrack Well-Known Member

    Joined:
    Aug 9, 2002
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    Thanks for the compliment, but I have to say, I am completely undeserving!

    I am fairly certain this can be accomplished by editing the AUTHENTICATORS
    in your /etc/exim.conf

    http://www.exim.org/exim-html-4.20/doc/html/spec_33.html

    First create the file:
    /etc/exim_whitelist

    usernames of allowed smtp authentication
    CONTENTS EXAMPLE:
    username1
    username3


    And then create a server_condition that first checks to see if the login name is in the /etc/exim_whitelist file, if it is, then proceed with username/password authentication:

    my syntax is WAY off, please use the below as psuedo-code, DO NOT PASTE INTO YOUR exim.conf
    Hopefully an EXIM guru out there could give the correct syntax:
    PHP:
    fixed_plain:
    driver plaintext
    public_name 
    PLAIN
    server_condition 
    "${lookup{$2}lsearch{/etc/exim_whitelist}{yes}{${if ${perl{checkuserpass}{$1}{$2}{$3}}"
    server_set_id = $2

    fixed_login:
    driver = plaintext
    public_name = LOGIN
    server_prompts = "
    Username:: : Password::"
    server_condition = "
    ${lookup{$1}lsearch{/etc/exim_whitelist}{yes}{${if ${perl{checkuserpass}{$1}{$2}}"
    server_set_id = $1
     
  7. plastic

    plastic Member

    Joined:
    Sep 10, 2003
    Messages:
    10
    Likes Received:
    0
    Trophy Points:
    1
    Hey I'm having a problem with this,

    ######################################################################
    # MAIN CONFIGURATION SETTINGS #
    ######################################################################

    perl_startup = do '/etc/exim.pl'

    queue_smtp_domains = ${if and {
    {eq {127.0.0.1}{$interface_address}}
    {def::h_X-Mailman-Version::}
    }{*}{}
    }


    smtp_accept_queue_per_connection = 50

    #dns_retry = 1
    #dns_retrans = 1s

    After I restart Exim I get this in WHM:

    Nov 18 10:52:16 hulk exim: exim shutdown succeeded Nov 18 10:52:17 hulk exim: antirelayd shutdown failed Nov 18 10:52:17 hulk exim: spamd shutdown succeeded Nov 18 10:53:40 hulk exim: exim startup succeeded Nov 18 10:53:40 hulk exim: exim startup succeeded Nov 18 10:53:40 hulk exim: antirelayd startup succeeded Nov 18 10:53:47 hulk exim: spamd startup succeeded Nov 21 13:43:08 hulk exim: exim shutdown failed Nov 21 13:43:08 hulk exim: antirelayd shutdown failed Nov 21 13:43:08 hulk exim: spamd shutdown failed Nov 21 13:43:09 hulk exim: exim shutdown failed Nov 21 13:43:09 hulk exim: antirelayd shutdown failed Nov 21 13:43:09 hulk exim: spamd shutdown failed Nov 21 13:43:09 hulk exim: exim shutdown failed Nov 21 13:43:09 hulk exim: antirelayd shutdown failed Nov 21 13:43:09 hulk exim: spamd shutdown failed Nov 21 13:43:09 hulk exim: 2003-11-21 13:43:09 Exim configuration error in line 55: Nov 21 13:43:09 hulk exim: option setting expected: {eq {127.0.0.1}{$interface_address}} Nov 21 13:43:09 hulk exim: exim startup failed Nov 21 13:43:09 hulk exim: 2003-11-21 13:43:09 Exim configuration error in line 55: Nov 21 13:43:09 hulk exim: option setting expected: {eq {127.0.0.1}{$interface_address}} Nov 21 13:43:09 hulk exim: exim startup failed Nov 21 13:43:09 hulk exim: antirelayd startup succeeded Nov 21 13:43:11 hulk exim: spamd startup failed exim has failed, please contact the sysadmin.


    The Reason I'm following your how to is this:

    I have a user with a mailinglist with more than 900 contacts in.
    It worked before on my old server but since they are on the new one it's giving problems.

    When he tries to send the mail to a small nr of people it's working, but with a lot of recepients exim doesn't want to send this I think.

    This is what I found in the exim.conf

    smtp_connect_backlog = 50
    smtp_accept_max = 100

    means this that exim will only send it to 100 users?

    this is a copy of a mail in the queue (about 900 mails):

    1AMpK8-00004o-Px-H
    mailnull 47 12
    <>
    1069336700 0
    -ident mailnull
    -received_protocol local
    -body_linecount 35
    -frozen 1069401238
    -localerror
    XX
    1
    mailman-bounces@xxxxxxxx.com

    163P Received: from mailnull by hulk.be-hostedns.be with local (Exim 4.24)
    id 1AMpK8-00004o-Px
    for mailman-bounces@xxxxxx.com; Thu, 20 Nov 2003 14:58:20 +0100
    074 X-Failed-Recipients: xxxx.xxxxx@xxxxxx.be,
    xxxx@xxxxx.com
    031 Auto-Submitted: auto-generated
    063F From: Mail Delivery System <Mailer-Daemon@hulk.be-hostedns.be>
    038T To: mailman-bounces@xxxxxx.com
    059 Subject: Mail delivery failed: returning message to sender
    052I Message-Id: <E1AMpK8-00004o-Px@hulk.be-hostedns.be>
    038 Date: Thu, 20 Nov 2003 14:58:20 +0100


    1AMpK8-00004o-Px-D
    This message was created automatically by mail delivery software.

    A message that you sent could not be delivered to one or more of its
    recipients. This is a permanent error. The following address(es) failed:

    xxxxx.xxxxx@pandora.be
    (ultimately generated from webmaster@xxxxxx.com)
    unrouteable mail domain "pandora.be"
    xxxxx@xxxxxx.com
    unrouteable mail domain "xxxxx.com"

    ------ This is a copy of the message, including all the headers. ------

    Return-path: <mailman-bounces@xxxxxx.com>
    Received: from localhost ([127.0.0.1] helo=hulk.be-hostedns.be)
    by hulk.be-hostedns.be with esmtp (Exim 4.24)
    id 1AMpK8-0004mA-L4; Thu, 20 Nov 2003 14:58:20 +0100
    MIME-Version: 1.0
    Content-Type: text/plain; charset="us-ascii"
    Content-Transfer-Encoding: 7bit
    Subject: Mailinglist unsubscribe notification
    From: mailman-bounces@xxxxxx.com
    To: Mailinglist-owner@xxxxxx.com
    Message-ID: <mailman.0.1069336696.16185.mailinglist_xxxxxx.com@xxxxxxxxx.com>
    Date: Thu, 20 Nov 2003 14:58:16 +0100
    Precedence: bulk
    X-BeenThere: Mailinglist@xxxxxx.com
    X-Mailman-Version: 2.1.3
    List-Id: Ragnarok <mailinglist_xxxxx.com.xxxxxxx.com>
    X-List-Administrivia: yes
    Sender: mailman-bounces@xxxxxx.com
    Errors-To: mailman-bounces@xxxxxx.com

    xxxxx@xxxxx.se has been removed from Mailinglist.

    ---end---

    I replaced the domains with xxx, against spam and blabla

    So first of all, why does it give, unroutable domain? the domains exist, and 2nd of all, why can't exim or mailman send to more than 900 mailadresses?

    Thx for any help!

    ps I'm running WHM 8.5.4 cPanel 8.5.4-R72
    RedHat 8.0 - WHM X v2.1.1

    [edit] I just tried to send the mails in the mailqueue and I get this error
    LOG: MAIN
    Unfrozen by auto-thaw
    LOG: MAIN
    ** mailman@xxxxx.com R=mailman_virtual_router_nodns T=mailman_virtual_transport: Child process of mailman_virtual_transport transport returned 1 from command: /usr/local/cpanel/3rdparty/mailman/mail/mailman
     
  8. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    I did a copy and paste of the suggestion, but upon restarting exim, I get this error:

    Shutting down spamd: [FAILED]
    Starting exim: 2003-11-22 15:51:27 Exim configuration error in line 60:
    option setting expected: {eq {127.0.0.1}{$interface_address}}
    [FAILED]
    Starting exim-smtps: 2003-11-22 15:51:27 Exim configuration error in line 60:
    option setting expected: {eq {127.0.0.1}{$interface_address}}
    [FAILED]

    Help?
     
  9. JackDcrack

    JackDcrack Well-Known Member

    Joined:
    Aug 9, 2002
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    Make sure you insterted the following lines into your Main Configuration like so:

    (it seems some backslashes were removed by the message board:
     
  10. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    JackDcrack, thanks for the quick reply, and for helping out a newbie like myself. You were right, I'm a bonehead!

    Adding the backslashes solved the problem.

    Thank you, again, very much!!

    - Scott
     
  11. Valetia

    Valetia Well-Known Member

    Joined:
    Jun 20, 2002
    Messages:
    207
    Likes Received:
    1
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    JackDcrack - do you by any chance know how to solve the problem with CPanel re-writing the "Return-path" headers?

    At the moment Cpanel's Exim config forces all "Return-path" headers to be "username@server.name.com" no matter what you set as the "Return-path" via a script (both PHP and Perl).

    The server has both SuExec and PhpSuExec enabled, if that has anything to do with it (seems not)...

    We are trying to set the "Return-path" to the same email address as that of the "From" field, because we want all undeliverable email to bounce back to the correct address.

    This seems to have been a Cpanel-related Exim config issue for some time now and no one seems to want to post a solution to this.

    Seeing as you are an exim config guru - do you have one? :D
     
  12. JackDcrack

    JackDcrack Well-Known Member

    Joined:
    Aug 9, 2002
    Messages:
    52
    Likes Received:
    0
    Trophy Points:
    6
    Well i certainly dont consider myself a guru, but thanks!

    In your transports section you will need to add the line:
    return_path_add = false
    Here is an example of my address transport:
    Read more about how exim handles return-path here
     
  13. Valetia

    Valetia Well-Known Member

    Joined:
    Jun 20, 2002
    Messages:
    207
    Likes Received:
    1
    Trophy Points:
    18
    cPanel Access Level:
    Root Administrator
    I just tried that and it still doesn't work...I even tried setting all the other "return_path_add" lines I could find to "false", to no avail.

    It did remove the "Return-path: username@machine.name.com" headers, but bounces *still* go back to username@machine.name.com (!).

    Do you have SuExec enabled and do your CGI scripts send out email that are able to bounce correctly?
     
  14. atjeu

    atjeu Member
    PartnerNOC

    Joined:
    Feb 15, 2002
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    Phoenix, AZ
    jackdcrack - after following your suggestions we get this in the logs for each email

    2003-11-26 13:05:43 1AP5ux-0000zC-FO failed to expand "${if and {{eq {127.0.0.1}{$interface_address}}{def::h_X-Mailman-Vers
    ion::}}{*}{}}" while checking a list: variable name omitted after "def:"
    2

    we put this in exim.conf - with the backslashes -

    queue_smtp_domains = ${if and {\
    {eq {127.0.0.1}{$interface_address}}\
    {def::h_X-Mailman-Version::}\
    }{*}{}\
    }
    smtp_accept_queue_per_connection = 50


    any ideas?
     
  15. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    Plastic (or anyone), what did you do to resolve the "child process" error? My queue has a bunch of mailman bounce messages now. When I try to force delivery, I get:

    Message 1AQlyP-0007VM-LK is no longer frozen
    LOG: MAIN
    ** mailman@www3.xyz.com R=mailman_virtual_router_nodns T=mailman_virtual_transport: Child process of mailman_virtual_transport transport returned 1 from command: /usr/local/cpanel/3rdparty/mailman/mail/mailman
    LOG: MAIN
    Frozen (delivery error message)

    What does this mean?

    - Scott
     
  16. sneader

    sneader Well-Known Member

    Joined:
    Aug 21, 2003
    Messages:
    1,126
    Likes Received:
    21
    Trophy Points:
    38
    Location:
    La Crosse, WI
    cPanel Access Level:
    Root Administrator
    I"m getting the same errors in my exim panic log:

    2003-12-01 02:43:43 1AQjeg-0003TO-Df failed to expand "${if and {{eq {127.0.0.1}{$interface_address}}{def::h_X-Mailman-Version::}}{*}{}}" while checking a list: variable name omitted after "def:"

    I'm clueless, I'll admit it. Jack, any ideas?
     

Share This Page