Email forwarders not working with smarthost configured

[RGBS IT]

Hi All,

Giving background before coming to the problem direct.

I migrated away from a typical webserver to Azure. I find port 25 blocked globally on Microsoft. Initially there was zero communication to and from exim. I read online and found workaround to configure Smarthost. My smarthost is google workspace and it seems its relaying emails originating from websites and emails coming from other domains to my cpanel mailboxes perfectly.

Until i recently discovered that email forwarders are not receiving any emails!

I see in exim logs the message id gets completed for each forwarder email

=========================================================

Paste from log:

2023-01-16 04:32:46 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1pHHAq-0003xK-Bi

2023-01-16 04:32:46 1pHHAq-0003xK-Bi SMTP connection identification D=domain.com O=[email protected] E=[email protected] M=1pHHAq-0003xK-Bi U=cpanelacc ID=1029 B=redirect_resolver
2023-01-16 04:32:46 1pHHAq-0003xK-Bi Sender identification U=cpanelacc D=domain.com S=[email protected]
2023-01-16 04:32:46 1pHHAq-0003xK-Bi SMTP connection outbound 1673843566 1pHHAq-0003xK-Bi domain.com [email protected]

2023-01-16 04:32:46 1pHHAq-0003xK-Bi SMTP connection identification D=domain.com O=[email protected] E=[email protected] M=1pHHAq-0003xK-Bi U=cpanelacc ID=1029 B=redirect_resolver
2023-01-16 04:32:46 1pHHAq-0003xK-Bi Sender identification U=cpanelacc D=domain.com S=[email protected]
2023-01-16 04:32:46 1pHHAq-0003xK-Bi SMTP connection outbound 1673843566 1pHHAq-0003xK-Bi domain.com [email protected]

2023-01-16 04:32:46 1pHHAq-0003xK-Bi SMTP connection identification D=domain.com O=[email protected] E=[email protected] M=1pHHAq-0003xK-Bi U=cpanelacc ID=1029 B=redirect_resolver
2023-01-16 04:32:46 1pHHAq-0003xK-Bi Sender identification U=cpanelacc D=domain.com S=[email protected]
2023-01-16 04:32:46 1pHHAq-0003xK-Bi SMTP connection outbound 1673843566 1pHHAq-0003xK-Bi domain.com [email protected]

2023-01-16 04:32:47 1pHHAq-0003xK-Bi => it <[email protected]> R=virtual_user T=dovecot_virtual_delivery C="250 2.0.0 <[email protected]> 8Df5AW/TxGOUOwAARBevbQ Saved"

2023-01-16 04:32:49 1pHHAq-0003xK-Bi => [email protected] ([email protected]) <[email protected]> R=lookuphost T=remote_smtp H=smtp-relay.gmail.com [64.233.188.28] TFO X=TLS1.3:TLS_AES_256_GCM_SHA384:256 CV=yes C="250 2.0.0 OK 1673843569 i5-20020a170902e48500b00192c4055e4asm1643242ple.90 - gsmtp"
2023-01-16 04:32:49 1pHHAq-0003xK-Bi => [email protected] ([email protected]) <[email protected]> R=lookuphost T=remote_smtp H=smtp-relay.gmail.com [64.233.187.28] TFO X=TLS1.3:TLS_AES_256_GCM_SHA384:256 CV=yes C="250 2.0.0 OK 1673843569 cl8-20020a0566383d0800b003a150b13a18sm296566jab.65 - gsmtp"
2023-01-16 04:32:49 1pHHAq-0003xK-Bi => [email protected] ([email protected]) <[email protected]> R=lookuphost T=remote_smtp H=smtp-relay.gmail.com [108.177.125.28] TFO X=TLS1.3:TLS_AES_256_GCM_SHA384:256 CV=yes C="250 2.0.0 OK 1673843569 y2-20020a4aea22000000b004f51dfdf754sm87791ood.17 - gsmtp"

2023-01-16 04:32:49 1pHHAq-0003xK-Bi Completed

=========================================================

My forwarders recepients are gmail, yahoo and few other common emails.

So coming to problem:
Emails from outside land to the main cpanel mailbox but not on forwarder mailboxes.

PS: I moved sites from old host to new host using Transfer Tool.
the domain is listed in
/etc/localdomain

Thanks,

 

[RGBS IT]

I noticed i did receive emails for sometime while i performed following commands in some sequence.
/usr/local/cpanel/scripts/suspendacct cpanelacc --force
/usr/local/cpanel/scripts/unsuspendacct cpanelacc --force

then

whmapi1 unset_manual_mx_redirects domain=domain.com
whmapi1 unset_all_service_proxy_backends username=cpanelacc

I dont recall in which sequence i performed these commands but infact forwarders started working after that for that particular domain and doing them again in all domains stopped for all domains.

Thanks,

 

[cPRex]

Hey there! The details in the specific log entries you provided don't tell me much, as it seems to indicate things are working well.

Can you search your /var/log/exim_mainlog file for the following string and let me know if you see it?

local_parts check lookup or other defer

If so, you could be experiencing the following issue: https://support.cpanel.net/hc/en-us...Generated-Return-Path-Can-Fail-Intermittently

 

[RGBS IT]

grepping for the string did not result anything, no occurrence in logs.

I am keep testing various combinations to narrow down exact situation. Here is one observation i made:

I have:
[email protected] with forwarder set to [email protected], [email protected], [email protected]
while i send an email from any outside email like [email protected] to [email protected], everything trigger fine in logs but none of the 3 gmails receive any email, only [email protected] receives it.

While i send from [email protected] (webmail) to [email protected], everything again triggers in logs same lines but all the 3 gmails receive it along with [email protected]!

this is super confusing to me.

 

[cPRex]

Do you happen to have mail archiving enabled on that domain?

 

[RGBS IT]

mail archiving is not enabled. its set to off in tweak settings.

 

[cPRex]

I've run out ideas, although I'm suspecting this is related to SRS changes, such as the issue mentioned here:

Forwarding / SRS Issues

Hello, A client is having some email forwarding issues. We use mailbaby/mailchannels, and they are saying that everything is fine on their end. They can see on the log that SRS is not working properly on the server. Forwarding works fine for most domains except Gmail and Yahoo. Yahoo...

forums.cpanel.net

Could you submit a ticket to our team so we can check this out?

 

[RGBS IT]

I have created a ticket and granted server access.

ticketid: 94522419

 

[cPRex]

Thanks for that - I'm following along with that ticket on my end now also.

 

[RGBS IT]

Out of curiosity, I wanted to know how do emails normally work on Azure or any other cloud platform like AWS that offer cPanel image. They all have port 25 outgoing blocked but they do allow port 25 incoming. I have read across major relay players like AWS SES, SMTP2Go, they all assure mailbox delivery in and out of server mailbox but none of them are welcoming on email forwarding.

I got SMTP2Go to test out if that can resolve. I faced the same issue with it, emails delivery till mailboxes went fine but not to its forwarders.

SMTP2Go support said this,
"It sounds like the plugin is working exactly as expected and that the cPanel forwarders are not functioning correctly. We have no influence on the forwarders, that is entirely a cPanel/WHM issue."

Forwarders are primary requirement for our websites.

 

[cPRex]

When our team checked the server, we found that messages are reaching Gmail normally. Are you still seeing issues with the forwarded addresses?

 

[RGBS IT]

I moved out all sites out of my cpanel server. I have kept a domain or two that doesn't have much email traffic on it. Email issues we couldn't afford.

I don't have any logs to confirm but I have a gut feeling that says gmail perhaps rate limits messages without error but again I get baffled as why emails originating from webmail always delivered out but forwarders recipients never delivered. Or gmail sees forwarder messages differently and rate-limits them?

I have more questions than answers so moving sites back to typical webserver where port 25 is open seemed practical.

I will keep searching for this and will see if something gives me any direction.

Also SRS angle hasnt been looked at still.

 

[hello2029]

I've run out ideas, although I'm suspecting this is related to SRS changes, such as the issue mentioned here:

Forwarding / SRS Issues

Hello, A client is having some email forwarding issues. We use mailbaby/mailchannels, and they are saying that everything is fine on their end. They can see on the log that SRS is not working properly on the server. Forwarding works fine for most domains except Gmail and Yahoo. Yahoo...

forums.cpanel.net

Could you submit a ticket to our team so we can check this out?

Dear cPanel Technical Support,

I'm writing in reference to ticket #95062534. I seem to be experiencing a similar issue as described in that ticket.

Despite my best efforts to follow the recommended steps and configure the smarthost properly for my domain, I'm still unable to have emails forwarded correctly using the SMTP of my domain.

I have tried different approaches to solve this problem, but regardless of the sender's email, forwarded emails are still not using the smarthost of my domain.

I would appreciate your help in reviewing my current configuration and identifying what might be causing this issue. I believe a fresh pair of eyes might see something I've missed.

Looking forward to your assistance.

Best Regards,

 

[cPRex]

@hello2029 - in general, if you have a ticket open, that's going to get you better answers since we'll have direct access to the server. I did see in this case we reviewed the log and found this entry showing the forwarding failure:

REDACTED

Details directly from Gmail about that specific error can be found here:

https://support.cpanel.net/hc/en-us/articles/360060888073-The-IP-you-re-using-to-send-mail-is-not-authorized-to-send-email-directly-to-our-servers-

but essentially, you're blocked by Gmail because of the smarthost configuration.

You'll want to check the smarthost side of things in order to get this working properly.

 

[hello2029]

Dear @cPRex and All,

1. Please kindly review and remove my domain name from your previous response. It is not advisable to disclose specific information like that publicly.
2. Currently, I am using WHM and cPanel. The reason my IP is being rejected by Google, Yahoo, and others is because it falls within the IP ranges offered for dial-up, DSL, or other broadband access.

Therefore, I have to use a smarthost, and I am using Postmarkapp as my smarthost.

I have configured the smarthost following the instructions provided here: How to send email from different domains using different smarthosts

The smarthost works well when I send emails directly. For example, my domain is "domain.com," and I have configured it with the Postmarkapp smarthost. I can send emails from [email protected] to [email protected] without any issues, and it is properly routed through the smarthost.

However, the problem arises when I configure automatic forwarding for all emails from [email protected] to [email protected], and when I send emails from [email protected] to [email protected] with the expectation that they will be forwarded to [email protected]. Unfortunately, this forwarding is not being sent through the smarthost but is instead being sent directly from my IP to [email protected]. As a result, Gmail rejects these emails.

This is the crux of the issue, and despite my numerous interactions with cPanel support, we have not been able to resolve it.

If I were a cPanel support staff member, I would seek assistance from my colleagues to identify the specific cause of this issue and find a concrete solution for my customer. I would even consider reaching out to Postmarkapp for collaborative troubleshooting and resolution.

Thank you for your attention to this matter.

Best regards,

 

[cPRex]

@hello2029 - I did anonymize the domain names - is there a portion I missed?

Unfortunately we aren't able to reach out to third-party tools to work with them to see if the issue could be on their end. You're welcome to try that, but that's outside the scope of our support.

 

[hello2029]

@hello2029 - I did anonymize the domain names - is there a portion I missed?

Unfortunately we aren't able to reach out to third-party tools to work with them to see if the issue could be on their end. You're welcome to try that, but that's outside the scope of our support.

The server log mentioned above still contains the domain name and server IP address.

 

[cPRex]

I've just removed that entire log now.