Yahoo! Defers Mail from cPanel

Operating System & Version
CENTOS 7.8 kvm
cPanel & WHM Version
v88.0.12

Nirjonadda

Well-Known Member
May 8, 2013
724
27
78
cPanel Access Level
Root Administrator
How can fix Yahoo! Defers Mail from cPanel? All our email going to defer Event.

Code:
SMTP error from remote mail server after pipelined MAIL FROM:<[email protected]> SIZE=2193: 421 4.7.0 [TSS04] Messages from IP temporarily deferred due to user complaints - 4.16.55.1; see https://help.yahoo.com/kb/postmaster/SLN3434.html
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,252
313
Houston

kdean

Well-Known Member
Oct 19, 2012
365
56
78
Orlando, FL
cPanel Access Level
Root Administrator
The primary way I found that ever resolves this (if you're not sending mass emails or spam) has to do with fixing the Exim's configuration. Yahoo will defer you for hours if you ever send more than 20 messages in a connection.


Unfortunately as far as I know the cPanel GUI does not have a way to add the needed lines to the exim.conf.

So for advanced users you can do the following:

You would need to edit the /etc/exim.conf and add "connection_max_messages = 20" into both sections for "remote_smtp:" and "dkim_remote_smtp:" such as:

Code:
remote_smtp:
  driver = smtp
  connection_max_messages = 20
  …trimmed other lines…

  dkim_remote_smtp:
  driver = smtp
  connection_max_messages = 20
  …trimmed other lines…
After saving, restart exim and the defer problem should go away. Note if you're already being deferred you have to wait that period out.

This brings us to the next problem in that, the exim.conf file will replace those changes every time upcp runs… so nightly. So we have to add post hooks to add it back. Now there's likely some advanced methods to edit and insert the connection_max_messages lines to preserve the current state of exim.conf but I'm not familiar with how to do that. Instead I made a copy of my exim.conf with my change in it. Let's call it exim_custom.conf.

Go to /scripts/ and edit or create the following 2 files:

posteximup
postupcp

I've added the following to these files so they run whenever cpanel updates or exim updates and replaces the exim.conf file with the customized version.

Code:
cp -f /etc/exim_custom.conf /etc/exim.conf
Here's what else you need to consider. If you make changes to the Exim configuration inside WHM, you will lose the connection_max_messages lines, plus the next time upcp runs, you'll lose the changes made in WHM. So each time you use the GUI in WHM to make changes, you need to edit the exim.conf file to include your connection_max_messages lines and copy the new exim.conf to your exim_custom.conf file.

Also, When updating cPanel versions, changes may be made to support new or different features. If that happens, the next time you visit Exim in WHM, it will show a warning about the problem (exim.conf has version numbers so it can detect when they don't match). Simply save the exim config in WHM, then add the connection_max_messages lines again to the latest exim.conf and copy it over to your exim_custom.conf.

So, it's a little bit of a pain and really should be something added to the WHM GUI for the Exim Advanced configuration, but well worth it because I went from getting deferred by Yahoo often to never being deferred again. In my experience this should always be set to remove any grief from Yahoo and servers who may act like them.

If someone has advice on how to script just checking for the lines and adding them if needed rather than replacing the whole file, I'm sure that would be beneficial. Plus if there is a hook that triggers whenever the Exim configuration (and advanced) is saved, that would be good to know as well.
 

Nirjonadda

Well-Known Member
May 8, 2013
724
27
78
cPanel Access Level
Root Administrator
In conjunction with following their suggestions you should also ensure that you have a valid PTR, DKIM and SPF on domains that send mail
We have made all guide but we have still problem send email to @yahoo.com domain. All are other email domain (@gmail.com) sending are working without any issue. We have Valid DKIM, SPF, Reverse DNS (PTR) Record but still does not know that why Yahoo! Defers Mail from cPanel.

Unfortunately as far as I know the cPanel GUI does not have a way to add the needed lines to the exim.conf.
Yes, cPanel should add support this. Also found one more here but does not understanding how can do this.
 
Last edited by a moderator:

MindServer

Well-Known Member
Mar 18, 2020
176
27
28
Spain
cPanel Access Level
Root Administrator
We have made all guide but we have still problem send email to @yahoo.com domain. All are other email domain (@gmail.com) sending are working without any issue. We have Valid DKIM, SPF, Reverse DNS (PTR) Record but still does not know that why Yahoo! Defers Mail from cPanel.



Yes, cPanel should add support this. Also found one more here but does not understanding how can do this.
Too you can install DMARC, this helped me some months ago.

Have a nice day!
 

kdean

Well-Known Member
Oct 19, 2012
365
56
78
Orlando, FL
cPanel Access Level
Root Administrator
Yes, cPanel should add support this. Also found one more here but does not understanding how can do this.
That "hosts_max_try_hardlimit" addition requires the same method as the "connection_max_messages" one I mentioned although I was able to resolve the deferrals without changing the "hosts_max_try_hardlimit" .

Since Yahoo will also mess with your connection if you open too many connection within a period of time that they unfortunately do not specific the limits for I also make use of adding the queue_smtp_domains value in the Advanced Editor for the Exim Configuration Manager in WHM. I apply this for Yahoo and some other free email services that can raise issues.

Code:
queue_smtp_domains = yahoo.com : ymail.com : rocketmail.com : aol.com : gmail.com : outlook.com : msn.com : hotmail.com : live.com
So, while I have most email get instantly delivered I also run the mail queue every 5 minutes, so any queue_smtp_domains are delayed by "queue_smtp_domains" and batched together for the next queue run. So at most, 12 connections an hour are opened to limit the change of annoying Yahoo. This is where the "connection_max_messages" being set to 20 definitely comes into play since if you start batching you will increase the odds of multiple emails being sent since when the emails are queued due to "queue_smtp_domains" they are sent over one connection when the queue runs. When the limit of 20 is reached, they're delayed until the next queue run so they don't trigger Yahoo deferrals.
 

kdean

Well-Known Member
Oct 19, 2012
365
56
78
Orlando, FL
cPanel Access Level
Root Administrator
Too you can install DMARC, this helped me some months ago.
Yes, DMARC is useful in making sure your emails pass multiple validations.

Problem is the deferrals have nothing to do with validation and everything to do with how often the emails are sent. As I've described Yahoo imposes strict rules that other servers do not. It's also worth mentioning that the user complaints part of "temporarily deferred due to user complaints" is completely untrue since it's an automatic deferral that has nothing to do with their deferrals. If Yahoo's users complain by marking your emails as spam, then your more likely to get straight up blocked then deferred in my experience.

Another feature you can use to monitor if emails received by yahoo are getting marked as spam is to use their Complaint Feedback Loop to add your domains.


You would then receive an Abuse Report anytime an email is marked as spam by a Yahoo user. Although again in my experience id you're not sending spam, you just see people incorrectly marking emails as spam, but at least you can address it with them.
 

sparek-3

Well-Known Member
Aug 10, 2002
1,983
218
343
cPanel Access Level
Root Administrator
Unfortunately as far as I know the cPanel GUI does not have a way to add the needed lines to the exim.conf.
Try making these modification to the file - /usr/local/cpanel/etc/exim/replacecf/dkim/remote_smtp

No way of doing this from the WHM GUI, but might be beneficial to advanced users.

Then rebuild the exim configuration

/scripts/buildeximconf

I'm not going to guarantee that this will work, or that it will remain there through subsequent cPanel and Exim updates, but was just a thought.
 

kdean

Well-Known Member
Oct 19, 2012
365
56
78
Orlando, FL
cPanel Access Level
Root Administrator
Try making these modification to the file - /usr/local/cpanel/etc/exim/replacecf/dkim/remote_smtp
Thanks for that. This is promising. It provides more permanence in that it survives saving new settings in WHM Exim configuration plus it should survive UPCP as well. From what I've read, it only gets touched / replaced if a future cpanel update needs to modify it, so changes could be lost then, but otherwise it should survive.

So perhaps I can replace my exim.conf replacement hook with a script that runs diff on a copy compared to the remote_smtp file and alerts me when it changes or set up a cron to do the same. Although a script that can modify the existing file to add missing lines would still be best. I know it's possible just haven't dug into figuring that out.

Although one would think, seeing that it's already an isolated file, that cpanel could easily add a text area for inserting lines in the advanced config.
 

kdean

Well-Known Member
Oct 19, 2012
365
56
78
Orlando, FL
cPanel Access Level
Root Administrator
Ok, so I have a whole new improved method. The following will use a script triggered by the post hooks to check "/usr/local/cpanel/etc/exim/replacecf/dkim/remote_smtp" and add custom lines to it if they do not exist. It will not modify existing custom lines so if new parameters are added, they'll will need to be manually implemented. However as long as you update your new "exim_custom_remote_smtp.conf" file with the same changes, they will be re-added in the future if an update removes them. That's the primary point of this is to make the changes persist through updates.

In your /etc/ folder create a file called:

exim_custom_remote_smtp.conf

Add the following lines to it (you can insert additional lines if there's other parameters you want added:

Code:
  #remote_smtp_insert_start
  connection_max_messages = 20
  #remote_smtp_insert_end
Also in /etc/ crearte a file called:

exim_custom_remote_smtp.sh

Set the permissions to be executable.

Add the following code to it:

Bash:
#!/bin/sh

#This script will check your existing REMOTE_SMTP file and add your custom line if they do not already exist.
#This script is not meant to modify existing custom lines, only add them.
#if you add new lines to your CUSTOM_LINES file afer they already exist in your REMOTE_SMTP file, then you will need to manually add them there as well the first time

#set the file path for the file that contains just your custom lines
CUSTOM_LINES="/etc/exim_custom_remote_smtp.conf"

#set the file path for the file being modified /usr/local/cpanel/etc/exim/replacecf/dkim/remote_smtp
#you can copy this path to a test file such as /etc/exim_custom_remote_smtp_test and set it here
REMOTE_SMTP="/usr/local/cpanel/etc/exim/replacecf/dkim/remote_smtp"

#check if CUSTOM_LINES file exists or is empty.
if [ -s $CUSTOM_LINES ]; then

    #check if REMOTE_SMTP file exists or is empty.
    if [ -s $REMOTE_SMTP ]; then

        echo "Checking if remote_smtp needs custom lines added"

        #check if any of the lines already exist and if they do not add them and rebuild exim
        CUSTOM_LINES_EXIST="$(grep -f $CUSTOM_LINES $REMOTE_SMTP)"

        if [ -z "$CUSTOM_LINES_EXIST" ]; then
            sed -i '/driver = smtp/a \\' $REMOTE_SMTP; sed -i "/driver = smtp/r $CUSTOM_LINES" $REMOTE_SMTP
            echo "Custom lines were added for remote_smtp"
            /scripts/buildeximconf
        fi
    else
        echo "$REMOTE_SMTP NOT FOUND OR EMPTY"
    fi
else
    echo "$CUSTOM_LINES NOT FOUND OR EMPTY"
fi
Go to /scripts/ and edit or create the following 2 files:

posteximup
postupcp

I've added the following to these files so they run whenever cpanel or exim updates and if the remote_smtp file was modified removing the custom lines, they should get added back in without concern for any exim.conf version and other changes, including manual changes within WHM Exim Configuration.

Code:
/bin/sh /etc/exim_custom_remote_smtp.sh
If everything is setup correctly, you can run "/scripts/postupcp" to test. You should run it at least once to add the lines for you to the remote_smtp file and to make sure everything is set correctly.

If it's the first time adding the lines, You'll get output stating:

Checking if remote_smtp needs custom lines added
Custom lines were added for remote_smtp
(and then standard output triggered by Exim rebuilding)

If the lines already exist and you run postupcp again it will only output:

Checking if remote_smtp needs custom lines added

Comments in the script explain how to customize if you prefer to test with another file first. I've tested it on my system.

So, this should work much better than replacing the exim.conf as I was doing before and should persist as best as I can foresee now.
 
Last edited: