bounced email receipts delayed 18+ hours

[rolinger]

I am writing scripts to process bounced emails on my GoDaddy VPS server. These scripts will help us keep our mailing lists clean by removing email addresses that are bad or have bounced. However, testing this is NOT easy because bounced email receipts are taking like 18-24 hours before being returned.. Even when we send emails to bad addresses at our own domain, the bounced email receipt shows up like 20 hours later - this makes testing nearly impossible because we need to see the processed results, script errors, etc in real time.

Because GoDaddy acts as a relay between our exim mail server and the world, I can't quite tell if the issue is with GoDaddy or on my exim mail server. My hunch is that its a GoDaddy relay server issue. But I needed come here first and ask. Is this issue happening at my server or at the godaddy relay level? How can verify or tell if its their servers or my sever causing this? Are there settings on the exim server that I can control that will eliminate this delay?

If it is a GoDaddy issue, then why in the world would GoDaddy queue bad email receipts for that long? What purpose does it serve - all it does is delay the original sender knowing that their email never made it to a specific email address?

 

[rolinger]

I have adjusted my exim mail settings hoping that this is an issue I can resolve from my end. Simply for testing purposes

I temporarily set the following config options:

ignore_bounces_after_frozen: 5m
timeout_frozen_after: 5m
retryblock:
* data_4xx F,4h,2m
* rcpt_4xx F,4h,2m
* timeout F,4h,5m
* refused F,1h,10m
* lost_connection F,1h,2m
* * F,6h,5m


I then tested emails to made-up, bad external addresses and got the bounce in just a few minutes. I got these settings from a best practices article. Yay! Though I am not certain which setting specifically got the bounce to bounce faster, it will at least allow me to test...and then move the ignore/timeout settings back to 1h/4hr respectively.

However, when I tested to bad email addresses on my own domain (handled by this exim this same exim server), the I received no bounce message. I would much rather prefer to send to bad addresses on my own domain so I an not jeopardizing getting my server on any RBLs How can I get this to work now?

 

[cPanelLauren]

The issue is depending on the type of error the mail server will attempt to resend the email a number of times before it bounces. This means that in those instances the GoDaddy mail server which is responsible for retrying those messages is making these attempts. So while you're handling message processing retry times on your own server it may not be the same for the relay server which I'd assume you don't have control over to manage.

Some errors bounce back immediately based on the status code and these should be bounced back to you from the relay really quickly. Bounces to your own domains hosted locally have an entirely different process and the results may not be the same.

Regardless of the time which it is taking to get the bounces, I'm curious why you're writing your own scripts to perform bounce processing, have you not been able to use mailman's automated bounce processing?

 

[rolinger]

@cPanelLauren - I am not too familiar with mailman and therefore didn't know it could do that. On WHM and cPanel - the only thing I find for 'mailman' is to restart the service. In Exim Advanced Editor I found references to 'mailman' settings but that all appears to be managed emails lists (or something). How/where would I go about configuring mailman to capture the original TO address to process in my DB?

To clarify what I am doing, during users registration process we are sending a confirmation link to them, if it bounces when then need to mark the email as bad in our registration DB. Digging a little deeper it seems mailman manages email lists or mass emails - and thats not what we are doing here.

 

[cPanelLauren]

Hi @rolinger

Ahh my fault, I saw that you noted this:

These scripts will help us keep our mailing lists clean by removing email addresses that are bad or have bounced.

and assume you were using the Mailing Lists feature in cPanel which is Mailman. These aren't necessarily just mass mailing but they can be configured for a number of other items. If you're not using them for a mailing list currently though it might be irrelevant.

To clarify what I am doing, during users registration process we are sending a confirmation link to them, if it bounces when then need to mark the email as bad in our registration DB.

Ok so I think this is related to your other threads as well and it sounds like the process is as follows:

• user registers
• confirmation link sent to users email
  • success user receives email with link registration complete
  • failure mail is bounced back to your server - registration is incomplete
    • If incomplete email needs to be removed from db

The problem with this is the fact that DSN's (bounces) vary widely in the content they provide you.

Now if you wanted to use mailman for the return path of emails sent to the registrations it would count bounces and remove from the list of people to send to based on the bounce threshold which isn't quite what you want since you also need to remove them from your DB

What I think you might need is called a variable envelope return path which is the same mechanism mailman and other mailing lists use to track bounces. It's discussed here: Variable envelope return path - Wikipedia essentially what a VERP mechanism does is use a unique return path for every message it sends. It's discussed in the exim documentation here: Variable envelope return path - Wikipedia

I would assume that you could use this method to identify the original intended recipient then pipe to a program to remove the email from the DB

Another thing you might be able to do is parse the exim logs for the original sender. Bounces are given a new transaction ID but are included in an exim transaction when using exigrep. Though there are issues with this that VERP addresses better.