Why different HELOs? - Horde vs Roundcube

[EneTar]

Environment

[~]# grep '' /etc/redhat-release /usr/local/cpanel/version /                      var/cpanel/envtype ; grep CPANEL= /etc/cpupdate.conf ; httpd -v ; php -v ; mysql                       -V
/etc/redhat-release:CentOS release 6.9 (Final)
/usr/local/cpanel/version:11.68.0.12
/var/cpanel/envtype:kvm
CPANEL=release
Server version: Apache/2.4.29 (cPanel)
Server built:   Nov  7 2017 03:54:43
ea-php-cli Copyright 2017 cPanel, Inc.
PHP 7.0.25 (cli) (built: Nov  7 2017 04:14:18) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.25, Copyright (c) 1999-2017, by Zend Technologies
mysql  Ver 15.1 Distrib 10.1.28-MariaDB, for Linux (x86_64) using readline 5.1

my.hostname.eu is my server hostname
domain1.com and domain2.com are domains on the same ip

When sending from Horde I get

Return-Path: <[email protected]>
Delivered-To: [email protected]
Received: from my.hostname.eu
    by my.hostname.eu with LMTP id 6GnpAD01C1puJwAAnaC/hg
    for <[email protected]>; Tue, 14 Nov 2017 20:26:05 +0200
Return-path: <[email protected]>
Envelope-to: [email protected]
Delivery-date: Tue, 14 Nov 2017 20:26:05 +0200
Received: from [127.0.0.1] (port=52998 helo=my.hostname.eu)
    by my.hostname.eu with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128)
    (Exim 4.89)
    (envelope-from <[email protected]>)
    id 1eEfuO-0002cd-Ou
    for [email protected]; Tue, 14 Nov 2017 20:26:04 +0200
Received: from 85.72.177.50 ([85.72.177.50]) by domain1.com (Horde Framework)
 with HTTPS; Tue, 14 Nov 2017 20:26:04 +0200
Date: Tue, 14 Nov 2017 20:26:04 +0200
Message-ID: <[email protected]>
From: John <[email protected]>
To: [email protected]
Subject: Horde test
User-Agent: Horde Application Framework 5
Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes
MIME-Version: 1.0
Content-Disposition: inline

When sending from Roundcube I get

Return-Path: <[email protected]>
Delivered-To: [email protected]
Received: from my.hostname.eu
    by my.hostname.eu with LMTP id IOB6NMQ2C1rbKAAAnaC/hg
    for <[email protected]>; Tue, 14 Nov 2017 20:32:36 +0200
Return-path: <[email protected]>
Envelope-to: [email protected]
Delivery-date: Tue, 14 Nov 2017 20:32:36 +0200
Received: from [127.0.0.1] (port=41690 helo=domain1.com)
    by my.hostname.eu with esmtpa (Exim 4.89)
    (envelope-from <[email protected]>)
    id 1eEg0i-0002kR-Cx
    for [email protected]; Tue, 14 Nov 2017 20:32:36 +0200
MIME-Version: 1.0
Date: Tue, 14 Nov 2017 20:32:36 +0200
From: John <[email protected]>
To: [email protected]
Subject: Sending Plain from Roundcube
Message-ID: <[email protected]>
X-Sender: [email protected]
User-Agent: Roundcube Webmail/1.2.4

The issue is that when sending from Horde AND [email protected] has configured GMAIL to fetch his messages. all messages from [email protected] go to spam. One difference I can see is that the HELO is different. Where should I start seeking for this problem?

 

[cPanelMichael]

Hello,

This is answered on the following thread:

Prevention client IP to appear in mail headers Horde webmail

Thank you.

 

[EneTar]

Hi Michael thank you for your answer.
So how to set the proper HELO in Horde or Roundcube
It has to be the active domain instead of server hostname

Form the thread you mentioned how is this done:

It's important to keep in mind that a client must send it's own Domain in the Helo/EHLO command per RFC requirements.

Currently all emails sent from Horde are going to spam for situations I described in the first post

 

[cPanelMichael]

Currently all emails sent from Horde are going to spam for situations I described in the first post

Are you sure it's from the HELO name and not because the client's IP address is added to a spam blacklist?

Thank you.

 

[EneTar]

Totally sure the client ip is not in a spam list because the client ip is dynamic and we have tried several IPs so far from different networks. Furthermore Gmail states this in the headers

1st case from my first post (Horde)

Authentication-Results: mx.google.com;
       spf=softfail (google.com: domain of transitioning [email protected] does not designate 85.75.xxx.xxx as permitted sender) [email protected]
Received-SPF: softfail (google.com: domain of transitioning [email protected] does not designate 85.75.xxx.xxx as permitted sender) client-ip=85.75.xxx.xxx;

second case from my first post (Roundcube)

Authentication-Results: mx.google.com;
       spf=pass (google.com: found no external ips, assuming domain of [email protected] as permitted sender) [email protected]
Received-SPF: pass (google.com: found no external ips, assuming domain of [email protected] as permitted sender)

In the second case (Roundcube) Gmail assumes the original message is sent from the same domain. Neither this should happen. Gmail should know that it is indeed from the specific domain.

One more thing I noticed is that when sending from @domain1.com to @domain2.com there are no SPF and DKIM headers. However when sending from @domain1.com to @gmail.com or any other external domain then DKIM and SPF and DMARC are setup correctly. If I could solve this then I think there would be no issues.

 

[cPanelMichael]

One more thing I noticed is that when sending from @domain1.com to @domain2.com there are no SPF and DKIM headers. However when sending from @domain1.com to @gmail.com or any other external domain then DKIM and SPF and DMARC are setup correctly. If I could solve this then I think there would be no issues.

Hello,

Could you open a support ticket using the link in my signature so we can take a closer look?

Thank you.

 

[EneTar]

Sorry can't see the link or any signature at all. I don't know why. Can you post the link in a reply?

 

[cPanelMichael]

Hi,

Sure, it's:

Create Support Ticket - Documentation - cPanel Documentation

Thank you.

 

[EneTar]

Michael I'm trying to understand a few things first so before opening a ticket could you please tell me

1)By default when sending from [email protected] to [email protected] using Horde or Roundcube is DKIM and SPF included in the headers? Please note that domain1.com and domain2.com belong to accounts on the same server and ip.

2)Is there any walkthrough to set the HELO when using Horde or Roundcube?I would like it to have the active domain on HELO rather than the server hostname

3)Is this above related to the values of

• $smtp_banner
• $smtp_active_hostname
• $message_id_header_domain

If it is related, how should those 3 values be?

4)on my Exim Configuration I have set "Send mail from account’s dedicated IP address" to On. In this case does the content of /etc/mailhelo and /etc/mailips affect the configuration?

Thank you.

 

[EneTar]

To answer some of my questions after some research,

1)In this case LMTP is used and DKIM and SPF do not exist. That's a default behavior. I'm not sure if this can be changed somehow but I would like to know.

3)I couldn't alter the HELO with those options. This needs further research though

4) If "Send mail from account’s dedicated IP address" is set then the content of /etc/mailhelo and /etc/mailips are ignored.

No need to open a ticket, as that's how things work by default. So if anyone uses POP3 through Gmail to fetch emails from a server powered by cPanel then those messages which are delivered originally with the LMTP protocol and fetched later from Gmail POP3 would probably end in the spam folder. So a custom filter in Gmail to not mark as spam all domains of the server is the way to go, at least for now... Even if they not end into the spam folder the question mark icon will be there for sure which links to Check if your Gmail message is authenticated - Android - Gmail Help

I think cPanel should investigate this. More headers should be added through Exim in case of LMTP so that Gmail knows that emails are already authenticated.

 

[cPanelMichael]

I think cPanel should investigate this. More headers should be added through Exim in case of LMTP so that Gmail knows that emails are already authenticated.

Would you mind opening a support ticket so we can review an affected system and rule out any bugs with the cPanel & WHM software itself? It's much easier to investigate this type of issue when reviewing the issue on a system affected by the issue. Note that you may find this post helpful for modifying the Roundcube HELO setting:

HELO name problem on roundcube

Thank you.

 

[EneTar]

@cPanelMichael there is not any affected system. What I described above is standard functionality in cPanel/WHM when using LMTP. What you need to test is

1) Two domains domain1.com and domain2.com on the same WHM/cPanel server. It can be only one domain if you wish. I just want to show that it affects all email messages transferred through LMTP meaning all email accounts of all domains in the same server.

2) Two email accounts [email protected] and [email protected]. It can be [email protected] as well. It is the same.

3) Send from the first account to the second one message. In case of Horde the external user IP is included. In case of roundcube it is hidden (<- This is what the title of the current thread describes. It is standard functionality and it is how Roundcube and Horde work. no worries so far. but the real problem lies elsewhere)

4) Add the second account ([email protected]) to Gmail by going to Gmail -> Settings -> Accounts and Import -> Check mail from other accounts

5)Wait for Gmail to fetch messages from [email protected] and notice that our message goes to spam because there is no DKIM neither SPF validation from step 3. Furthermore if in step 3 Roundcube has been used then Gmail finds no external ips and assumes the domain1.com as permitted sender and passes SPF. Please check the headers. My opinion is that Gmail shouldn't do that and should fail SPF too but that's another story. In case of Horde it finds an external IP which is not valid, it doesn't find any DKIM or SPF records and the message goes to spam.

I know that Gmail is a third party software but I'm sure that other similar services will run into the same issue sooner or later. They won't find any DKIM/SPF when LMTP is used in step 3 and they will forward the message to spam.

This can be avoided only if DKIM and SPF headers are added even at LMTP through Exim. So do you see it now?

 

[cPanelMichael]

4) Add the second account ([email protected]) to Gmail by going to Gmail -> Settings -> Accounts and Import -> Check mail from other accounts

5)Wait for Gmail to fetch messages from [email protected] and notice that our message goes to spam because there is no DKIM neither SPF validation from step 3. Furthermore if in step 3 Roundcube has been used then Gmail finds no external ips and assumes the domain1.com as permitted sender and passes SPF. Please check the headers. My opinion is that Gmail shouldn't do that and should fail SPF too but that's another story. In case of Horde it finds an external IP which is not valid, it doesn't find any DKIM or SPF records and the message goes to spam.

I know that Gmail is a third party software but I'm sure that other similar services will run into the same issue sooner or later. They won't find any DKIM/SPF when LMTP is used in step 3 and they will forward the message to spam.

This can be avoided only if DKIM and SPF headers are added even at LMTP through Exim. So do you see it now?

Hello,

Thank you for the additional clarification. Would you mind opening a bug report for this issue so we can investigate this further? It won't require access to your server, but will allow us to complete additional testing:

Submit A Bug Report

Thank you.

 

[EneTar]

Thank you Michael I just did
Request ID is: 9073349