The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

mailman/cpanel bug can lead to false unsubcriptions!

Discussion in 'E-mail Discussions' started by cyberspirit, Jun 24, 2004.

  1. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Here is what I found out so far:

    The latest mailman version 2.1.5. uses a new mechanism to test bouncing addresses. If an address has been bouncing and is marked to be unsubscribe it will be tested one last time with an email "probe". This probe is sent in VERP format, if it fails the email addresses is unsubscribed right away!
    Now here lies the problem. Cpanel's exim configuration is not fully VERP aware. So this means these bounces fail right away and will lead to false unsubscriptions!!!
    One of my customer has thousands of unsubcriptions in one day. We tested about 100 of these addresses and they are all ok!
    I have submitted this bug to the cpanel team and we will see what they will do with it.
    For the time being I suggest turning the bounce handling completely off if you do not want to scrue up lists completely.
     
  2. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    yep, mailman did not notify anyone about this little changed behaviour. so typical for the mailman developers, I do not like how they ignore the whole community anyway, it is not the first time they screw up things big time!
    thanks to Nick to look into this right away and fix it. There is a nice VERP aware config for exim from Nigel, that should help so Nick does not need to reinvent the wheel..... ;-)
     
  3. Osaka

    Osaka Registered

    Joined:
    Jun 24, 2004
    Messages:
    1
    Likes Received:
    0
    Trophy Points:
    1
    hello

    i am not seeing this behavior with any of my clients lists across 4 servers. does anyone know how to tell if mailmans VERP is causing users to be unsibscribed ?
    any guidance in replicating this problem will be most appreciated.

    very many thanks
     
  4. Darren

    Darren Well-Known Member
    Staff Member

    Joined:
    Dec 26, 2001
    Messages:
    1,957
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    Houston, TX
    Hi All,

    I have been looking into this as well. Looking at http://exim.mirror.anlx.net/howto/mailman21.html , it would appear the mailman_virtual_transport is currently set up to work with mailman+VERP ; according to Nigel's howto ::

    "VERP settings within Mailman are done on a per-installation basis - ie they affect all the lists within the installation. To configure VERP within Mailman read the information in ~mailman/Mailman/Default.py for the options that start with VERP. In a nutshell, all you need to do to enable VERP with Exim is to add these lines to ~mailman/Mailman/mm_cfg.py:
    ( on cpanel boxes: /usr/local/cpanel/3rdparty/mailman/Mailman/mm_cfg.py )

    VERP_PASSWORD_REMINDERS = 1
    VERP_PERSONALIZED_DELIVERIES = 1
    VERP_CONFIRMATIONS = 1
    VERP_DELIVERY_INTERVAL = 1

    (The router and ACLs above are smart enough to deal with VERP bounces.)"

    For those obviously suffering from this problem, do you have a support ticket opened with us ? I would like to be able to replicate the problem so we can get it taken care of asap.

    Thanks,
    Darren
     
  5. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Darren,
    If you read up on VERP you will find out that the MTA needs to be able to route VERP-addresses properly. Right now the way exim is set up to route mailman addresses VERP does not work. I contacted Nick months ago about the VERP problem and never heard back. Now, with the change in bounce probing in mailman 2.1.5. this issue obviously became very important. Why can't you guys act more proactively? VERP has been discussed for a long time now, not only on the mailman list. If you as cpanel integrate a product like mailman then I expect that you stay up-to-date on it and its developments and not just jump into action after everything broke.
    The fact alone that Nigel felt the need to post new VERP aware scripts examples for exim shows that this needs special handling. Again, something that you could have known about simply by following the mailman developers mailing list.
    Now, the big questions is, are you guys staying up-to-date on the planned 3.x mailman or will you once again just read up on it once it is already out?
     
  6. Marty

    Marty Well-Known Member

    Joined:
    Oct 10, 2001
    Messages:
    630
    Likes Received:
    1
    Trophy Points:
    18
    Has this patch been issued yet?
     
  7. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    no, there has been no patch yet. I guess the "24" hours that were promised for this patch will more turn into 24 days..........
     
  8. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Well thaphantom,
    It looks like your connection to Nick is not as good as you make it sound like. You mentioned that there would be a fix within 48 hours and nothing yet. In fact, the bugreport 618 has been out for a while and nothing. This is a critical issue and nothing is being done. And I have to correct you, mailman did properly document and report the change when 2.1.5 came out. And since cpanel was not really one of the early adopters of mailman 2.1.5 all the information was in the readme. Sometimes it helps to read readmes, lol.
    And in response to Darren, VERP can be turned off in mailman 2.1.5. but not the VERP bounce probes that create the problem. So there is no fix other than setting the bounce counts very high for a list and wait until cpanel finally realizes the severity of the issue and patches the exim.conf
     
  9. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Thaphantom, once again you seem out of touch with the facts and current developments. The bug has been verified by many and as you can see by the response of this cpanel tech it is a known issue:

    Hello,

    By defualt exim is not setup to use VERP and mailman was by default not using VERP.

    It seems recently mailman was changed to use VERP by default.

    The solution would be to turn VERP on in exim as well as mailman.

    That is on the to do list here: http://bugzilla.cpanel.net/show_bug.cgi?id=618

    You can subscribe yourself to the CC lust to be notified of any updates to the issue.

    Thanks
    --
    Daniel Muey
    Technical Support
    cPanel Inc
     
  10. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    A tickets was opened for this but closed by cpanel tech support because it is a known bug and will be addressed by the people working on solving reported bug 618 which was opened by cpanel tech support.
    It is actually very easy to see that current cpanel added mailman transports in exim.conf do NOT support VERP addresses. This issue only comes up when mailman tries the new default VERP bounce probes and is independent of any VERP settings in either Default.py or mm_cfg.py
    (How many times do I need to repeat this until thaphantom will get it?).
     
  11. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,426
    Likes Received:
    2
    Trophy Points:
    38
    cPanel Access Level:
    DataCenter Provider
    http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq02.002.htp

    exim -bt <listname>-bounces+r.barrett=openinfo.demon.co.uk@mailman.domain.com

    works fine for me

    # exim -bt zipper-bounces+r.barrett=openinfo.demon.co.uk@koston.org
    zipper-bounces+r.barrett=openinfo.demon.co.uk@koston.org
    router = mailman_virtual_router, transport = mailman_virtual_transport


    If it was broken it would have returned " Unrouteable address
    "
     
  12. cPanelNick

    cPanelNick Administrator
    Staff Member

    Joined:
    Mar 9, 2015
    Messages:
    3,426
    Likes Received:
    2
    Trophy Points:
    38
    cPanel Access Level:
    DataCenter Provider
    I've also enabled VERP

    VERP_PASSWORD_REMINDERS = 1
    VERP_PERSONALIZED_DELIVERIES = 1
    VERP_CONFIRMATIONS = 1
    VERP_DELIVERY_INTERVAL = 1

    (in Defaults.py)

    on the list zipper@koston.org I welcome you to find a problem with it.
     
  13. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Thanks for the invitation to use your list for my tests but that would not work unless you are willing to give me root access.
    Reason being that changes in Defaults.py or mm_cfg.py only become effective after the maiman thread has restarted (default in a cpanel setup is several hours).

    Anyway, I have done some further testing and have this excerpt here from the exim mailqueue (in WHM) as proof that VERP bounce probes are not handled correctly and end up in nowhere.
    I have changed the domain name and ip address of the mailman host for privacy and security reasons. But in a direct email to you Nick I have sent you the original message.

    Code:
    Message 1BgxIx-0002LY-7t is no longer frozen
    transport error EPIPE ignored
    LOG: MAIN
      ** [email]testlist_domain.net@domain.net[/email] <testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net> R=mailman_virtual_router_nodns T=mailman_virtual_transport: Child process of mailman_virtual_transport transport returned 1 from command: /usr/local/cpanel/3rdparty/mailman/mail/mailman
    LOG: MAIN
      Frozen (delivery error message)
    Return to Mail Queue
    
    
    
    1BgxIx-0002LY-7t-H
    mailnull 47 12
    <>
    1088910275 0
    -helo_name omr-m12.mx.aol.com
    -host_address 64.12.136.10.58402
    -interface_address 65.65.65.65.25
    -received_protocol esmtp
    -body_linecount 89
    -frozen 1088910814
    -manual_thaw
    XX
    1
    [email]testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net[/email]
    
    243P Received: from [64.12.136.10] (helo=omr-m12.mx.aol.com)
    	by smtp.domain.net with esmtp (Exim 4.34)
    	id 1BgxIx-0002LY-7t
    	for [email]testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net[/email]; Sat, 03 Jul 2004 20:04:35 -0700
    176  Received: from  rly-ya02.mx.aol.com (rly-ya02.mail.aol.com [172.18.141.34]) by omr-m12.mx.aol.com (v98.19) with ESMTP id RELAYIN7-840e773a71c6; Sat, 03 Jul 2004 23:04:07 -0400
    157  Received: from localhost (localhost)
    	  by rly-ya02.mx.aol.com (8.8.8/8.8.8/AOL-5.0.0)
    	  with internal id XAB24594;
    	  Sat, 3 Jul 2004 23:04:07 -0400 (EDT)
    043  Date: Sat, 3 Jul 2004 23:04:07 -0400 (EDT)
    054F From: Mail Delivery Subsystem <MAILER-DAEMON@aol.com>
    056I Message-Id: <200407040304.XAB24594@rly-ya02.mx.aol.com>
    088T To: <testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net>
    018  MIME-Version: 1.0
    113  Content-Type: multipart/report; report-type=delivery-status;
    	boundary="XAB24594.1088910247/rly-ya02.mx.aol.com"
    037  Subject: Returned mail: User unknown
    041  Auto-Submitted: auto-generated (failure)
    024  X-AOL-IP: 172.18.141.34
    041  X-WhitelistedRCPT-nohdrfromcallback: Yes
    071  X-MailScanner-Information: Please contact the ISP for more information
    033  X-MailScanner: Found to be clean
    027  X-MailScanner-SpamScore: s
    
     
    1BgxIx-0002LY-7t-D
    This is a MIME-encapsulated message
    
    --XAB24594.1088910247/rly-ya02.mx.aol.com
    
    The original message was received at Sat, 3 Jul 2004 23:03:47 -0400 (EDT)
    from [url=http://www.domain.net]Index of /[/url] [65.65.65.65]
    
    
    *** ATTENTION ***
    
    Your e-mail is being returned to you because there was a problem with its
    delivery.  The address which was undeliverable is listed in the section
    labeled: "----- The following addresses had permanent fatal errors -----".
    
    The reason your mail is being returned to you is listed in the section
    labeled: "----- Transcript of Session Follows -----".
    
    The line beginning with "<<<" describes the specific reason your e-mail could
    not be delivered.  The next line contains a second error message which is a
    general translation for other e-mail servers.
    
    Please direct further questions regarding this message to your e-mail
    administrator.
    
    --AOL Postmaster
    
    
    
       ----- The following addresses had permanent fatal errors -----
    <blah94523@aol.com>
    
       ----- Transcript of session follows -----
    ... while talking to air-ya03.mail.aol.com.:
    >>> RCPT To:<blah94523@aol.com>
    <<< 550 MAILBOX NOT FOUND
    550 <blah94523@aol.com>... User unknown
    
    --XAB24594.1088910247/rly-ya02.mx.aol.com
    Content-Type: message/delivery-status
    
    Reporting-MTA: dns; rly-ya02.mx.aol.com
    Arrival-Date: Sat, 3 Jul 2004 23:03:47 -0400 (EDT)
    
    Final-Recipient: RFC822; [email]blah94523@aol.com[/email]
    Action: failed
    Status: 5.1.1
    Remote-MTA: DNS; air-ya03.mail.aol.com
    Diagnostic-Code: SMTP; 550 MAILBOX NOT FOUND
    Last-Attempt-Date: Sat, 3 Jul 2004 23:04:07 -0400 (EDT)
    
    --XAB24594.1088910247/rly-ya02.mx.aol.com
    Content-Type: text/rfc822-headers
    
    Received: from  smtp.domain.net ([url=http://www.domain.net]Index of /[/url] [65.65.65.65]) by rly-ya02.mx.aol.com (v100.23) with ESMTP id MAILRELAYINYA210-14a40e773923d2; Sat, 03 Jul 2004 23:03:47 -0400
    Received: from localhost ([127.0.0.1] helo=smtp.domain.net)
    	by smtp.domain.net with esmtp (Exim 4.34)
    	id 1BgxI2-0002IT-Iv
    	for [email]blah94523@aol.com[/email]; Sat, 03 Jul 2004 20:03:38 -0700
    Subject: Testlist mailing list probe message
    From: [email]testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net[/email]
    To: [email]blah94523@aol.com[/email]
    MIME-Version: 1.0
    Content-Type: multipart/mixed; boundary="===============1346133609=="
    Message-ID: <mailman.1.1088910216.6801.testlist_domain.net@domain.net>
    Date: Sat, 03 Jul 2004 20:03:36 -0700
    Precedence: bulk
    X-BeenThere: [email]Testlist@domain.net[/email]
    X-Mailman-Version: 2.1.5
    List-Id: testlist_domain.net.domain.net
    X-List-Administrivia: yes
    Sender: [email]testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net[/email]
    Errors-To: [email]testlist_domain.net-bounces+f59acd63293cb9aa8a52e5ebfc11bf02f746b879@domain.net[/email]
    X-MailScanner-Information: Please contact the ISP for more information
    X-MailScanner: Found to be clean
    X-MailScanner-SpamScore: sss
    X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
    X-AntiAbuse: Primary Hostname - smtp.domain.net
    X-AntiAbuse: Original Domain - aol.com
    X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
    X-AntiAbuse: Sender Address Domain - domain.net
    X-Source: 
    X-Source-Args: 
    X-Source-Dir: 
    X-AOL-IP: 65.65.65.65
    X-AOL-SCOLL-SCORE: 0:XXX:XX
    X-AOL-SCOLL-URL_COUNT: 0
    
    --XAB24594.1088910247/rly-ya02.mx.aol.com--
     
  14. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Just to update everyone who is waiting for news on this bug (it seems nobody from the cpanel team feels the need to update the rest of us about this):

    Nick from cpanel has finally agreed that there is a problem in the combination of cpanel/exim/mailman. According to him it is a problem with the mailman integration. This problem only arrises with VERP bounce probe messages from mailman that are sent out but bounce back from the receiving smtp host. They then end up in the exim queue and for some reason are not handed off correctly to mailman.
    Nick has created a patch and once I hear what the details and changes are in this patch (always good to know before testing something) I will try it and see if it solves the problem.
    I just wonder now how thorough the testing of thaphantom and others was.... ;-):
    Thaphantom: "4 people have looked into the issue, 3 from cpanel plus me and wnnot find the error. It is working corretly on atleasta dozen servers."
     
  15. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    Well, it looks like now even the version history of cpanel recognizes this bug and has posted a fix for it:

    -----------
    9.4.1 (build 80) Sun Jul 4 09:29:20 2004

    patch mailman to make verp probes work correctly
    ------------

    Let's see what people report here, I will test it once I hear back from cpanel what the changes are they made.
     
  16. cyberspirit

    cyberspirit BANNED

    Joined:
    Jun 27, 2003
    Messages:
    293
    Likes Received:
    0
    Trophy Points:
    0
    I tried the patch that cpanel/Nick supplied and it did not solve the problem for me......
    So I guess the bug still exists.
     
  17. So_alegria

    So_alegria Member

    Joined:
    Jan 20, 2004
    Messages:
    8
    Likes Received:
    0
    Trophy Points:
    1
    Location:
    BRASIL
    I have a problem in the mailist ! wha is this?

    Upgrading from version 0x20105f0 to 0x20105f0
    getting rid of old source files
    Updating mailing list: mailman
    Updating the held requests database.
    - updating old private mbox file
    looks like you have a really recent CVS installation...
    you're either one brave soul, or you already ran me
    - updating old public mbox file
    Fixing language templates: mailman

    Updating Usenet watermarks
    - nothing to update here
    Nothing to do.
    updating old qfiles
    Starting Mailman's master qrunner.
    root@rsdhost02 [~]# Traceback (most recent call last):
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 547, in ?
    main()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 429, in main
    kids = start_all_runners()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 267, in start_all_runners
    pid = start_runner(qrname, slice, count)
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 256, in start_runner
    os.execl(mm_cfg.PYTHON, mm_cfg.PYTHON, exe, rswitch, '-s')
    Traceback (most recent call last):
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 547, in ?
    main()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 429, in main
    kids = start_all_runners()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 267, in start_all_runners
    pid = start_runner(qrname, slice, count)
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 256, in start_runner
    os.execl(mm_cfg.PYTHON, mm_cfg.PYTHON, exe, rswitch, '-s')
    Traceback (most recent call last):
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 547, in ?
    main()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 429, in main
    kids = start_all_runners()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 267, in start_all_runners
    pid = start_runner(qrname, slice, count)

    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 256, in start_runner
    os.execl(mm_cfg.PYTHON, mm_cfg.PYTHON, exe, rswitch, '-s')
    Traceback (most recent call last):
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 547, in ?
    main()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 429, in main
    kids = start_all_runners()
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 267, in start_all_runners
    pid = start_runner(qrname, slice, count)
    File "/usr/local/cpanel/3rdparty/mailman/bin/mailmanctl", line 256, in start_runner
    os.execl(mm_cfg.PYTHON, mm_cfg.PYTHON, exe, rswitch, '-s')
    File "/usr/lib/python2.2/os.py", line 266, in execl
    File "/usr/lib/python2.2/os.py", line 266, in execl
    execv(file, args)
    OSError: [Errno 13] Permission denied
    execv(file, args)
    OSError: [Errno 13] Permission denied
    File "/usr/lib/python2.2/os.py", line 266, in execl
    execv(file, args)
    OSError: [Errno 13] Permission denied
    File "/usr/lib/python2.2/os.py", line 266, in execl
    execv(file, args)
    OSError: [Errno 13] Permission denied

    help me!! please!
     
Loading...

Share This Page