Autoconfig mail.domain.tld vs. domain.tld

lorio

Well-Known Member
Feb 25, 2004
313
21
168
cPanel Access Level
Root Administrator
What is the intended setting for Autoconfig and Autodiscover?

On a fresh WHM setup, a user will get displayed mail.userdomain.tld as incoming and outgoing server.
When using e.g. Thunderbird the autoconfig info will set incoming and outgoing server to userdomain.tld.

mail.userdomain.tld vs. userdomain.tld.
I would expect, that autoconfig will also advertise mail.userdomain.tld and NOT userdomain.tld.
Any reason why that is still the case? Any way to change that?

Autodiscover (Outlook) seems to advertise mail.userdomain.tld.
All checked with v74.0.8 (CentOS7).
 
Last edited:

lorio

Well-Known Member
Feb 25, 2004
313
21
168
cPanel Access Level
Root Administrator
Thanks for remembering me about that post ;-)

But I'm not sure why you point me to the manual.
Perhaps my post was not clear enough.
I'm pointing out a inconsistent behavior.

cPanel user mail section informs user to choose mail.userdomain.tld as mailserver (SMTP,IMAP,POP3).
Autodiscover (Outlook) advertise mail.userdomain.tld as mailserver (SMTP,IMAP,POP3).
Autoconfig (Thunderbird) advertise userdomain.tld as mailserver (SMTP,IMAP,POP3).(SMTP,IMAP,POP3).

Reason #1: Glitch in cpanel/WHM
Reason #2: I made a mistake somewhere.

Allowing to edit the XML files directly or choosing a different one, would be a nice feature too.
But currently I don't think I need to go the route of providing custom config files to get the behavior consistent.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
HI @lorio

In most cases mail.userdomain.tld and userdomain.tld point to the same place - most commonly folks choose to use mail.userdomain.tld but the difference between the two is actually more related to the mail client and not the autodiscover/autoconfig.

In regard to customizing the XML - You may want to read through the documentation as well:
Notes:

  • For more information about how to use a custom XML file, visit Mozilla's autoconfiguration page, or the Exchange's Autodiscover page.
  • You must enable the Thunderbird and Outlook autodiscover and autoconfig support (enables service subdomain and SRV record creation) option in order to configure this setting.

Thanks!
 

Andy M-B

Member
Jul 30, 2013
10
1
3
cPanel Access Level
Root Administrator
I've been testing this for two days in various ways and there is definitely an inconsistency with the mail server name returned by cPanel. It is a big problem for anyone running mail on one server and a website on another which I don't think is an edge case.

Forgive the length of this post, but during my research I've seen this issue brushed off in a number of forum posts and it really does need someone to take in the situation, confirm or correct my findings (the best outcome would be if I'm wrong!) and if necessary get this fixed.


Single cPanel combined web/mail server

For a domain myclient.tld, querying Autodiscover on host myclient.tld responds with mail.myclient.tld:


<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
[...snip...]
<Server>mail.myclient.tld</Server>
[...snip...]
</Autodiscover>

But, querying Autodiscover on host autodiscover.myclient.tld responds with myclient.tld:

curl -d '<EMailAddress>[email protected]</EMailAddress>' https://autodiscover.myclient.tld/Autodiscover/Autodiscover.xml

<Server>myclient.tld</Server>

As far as I can see, this inconsistent response is cPanel's standard when Autodiscover is correctly set up on a server.

This does confuse clients who always see the hostname mail.myclient.tld if they look in Webmail at settings or use the configuration scripts for Macs, iPhones and other platforms, but see myclient.tld in any account configured with Autodiscover. Querying autodiscover.myclient.tld ought to return mail.myclient.tld.


Separate cPanel web & mail servers

So consider this sort of setup, I wouldn't actually use cnames like this, but it makes the situation clear. Here's a pseudo DNS setup for a typical client domain:

myclient.tld cname webhost.tld
www.myclient.tld cname webhost.tld

myclient.tld mx mailhost.tld

mail.myclient.tld cname mailhost.tld
autodiscover.myclient.tld cname mailhost.tld

where webhost.tld and mailhost.tld are two cPanel based servers that I manage.

The account myclient.tld has correctly configured SSL certificates for the services on each server.

When Outlook tries to configure an email account it will first look to myclient.tld. When this hostname is a separate (cPanel) web server, with the domain's cPanel MX correctly set to Remote, or with Autodiscover turned off, then the server correctly responds with a 400 error code.

Outlook will then try autodiscover.myclient.tld. On a single combined mail & web server, the response of mail server name of myclient.tld would technically work. But in the two server setup, autodiscover.myclient.tld is mailhost.tld and the SSL certificate for myclient does not contain the hostname myclient.tld as that hostname points to webhost.tld. cPanel is clever enough not to return the hostname myclient.tld. But what it does instead is a disaster for anyone reselling hosting: it returns the server's hostname instead!

curl -d '<EMailAddress>[email protected]</EMailAddress>' https://autodiscover.myclient.tld/Autodiscover/Autodiscover.xml

<Server>clients-should-not-see.hosting-company.tld</Server>

Again, none of this would happen if querying autodiscover.myclient.tld returned a mail server name of mail.myclient.tld.


Customizing the XML

A number of forum posts assert that things will be fine if you have turned on Autoconfig and Autodiscover properly. This clearly is not the case (please please prove me wrong: I want this to work!)

A number of forum posts then go on to suggest that we can customise the XML and refer to guides from Microsoft and Mozilla.

While these external guides explain nicely how the XML works, what no-one points out is how you can install custom XML on your cPanel server. cPanel's Apache has a ScriptAlias for the path /Autodiscover/Autodiscover.xml. The "script" it goes to looks like a binary, so I can't try to read it and see what the underlying logic is. Turning off autodiscover just makes the script return a 400 error code, it doesn't remove the ScriptAlias, so there doesn't actually appear to be a way to use our own XML.


Suitable Solutions

1 - make autodiscover.myclient.tld return a mail server name of mail.myclient.tld. This would just make all of this pain go away.

2 - even better, provide us with a setting (in Tweaks maybe) to choose whether autodiscover should return mail.myclient.tld or mail.%this-clients-reseller%.tld or a serverwide default hostname of our choice.

3 - give us an easy way to change the ScriptAlias and use our own code. This isn't as good as the other options, but at least gives us a workaround.


Thanks.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @Andy M-B

Interesting, I'm not able to replicate the inconsistency you are having. I get the same results whether or not I query mydomain.tld or autodiscover.mydomain.tld:

Code:
curl -d '<EMailAddress>[email protected]</EMailAddress>' https://autodiscover.mydomain.tech/Autodiscover/Autodiscover.xml
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
    <Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
        <User>
            <DisplayName>[email protected]</DisplayName>
        </User>
        <Account>
            <AccountType>email</AccountType>
            <Action>settings</Action>
            <Protocol>
                <Type>IMAP</Type>
                <Server>mail.mydomain.tech</Server>
                <Port>993</Port>
                <DomainRequired>off</DomainRequired>
                <SPA>off</SPA>
                <SSL>on</SSL>
                <AuthRequired>on</AuthRequired>
                <LoginName>[email protected]</LoginName>
            </Protocol>
            <Protocol>
                <Type>SMTP</Type>
                <Server>mail.mydomain.tech</Server>
                <Port>465</Port>
                <DomainRequired>off</DomainRequired>
                <SPA>off</SPA>
                <SSL>on</SSL>
                <AuthRequired>on</AuthRequired>
                <LoginName>[email protected]</LoginName>
            </Protocol>
        </Account>
    </Response>
</Autodiscover>
Code:
curl -d '<EMailAddress>[email protected]</EMailAddress>' https://mydomain.tech/Autodiscover/Autodiscover.xml
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
    <Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
        <User>
            <DisplayName>[email protected]</DisplayName>
        </User>
        <Account>
            <AccountType>email</AccountType>
            <Action>settings</Action>
            <Protocol>
                <Type>IMAP</Type>
                <Server>mail.mydomain.tech</Server>
                <Port>993</Port>
                <DomainRequired>off</DomainRequired>
                <SPA>off</SPA>
                <SSL>on</SSL>
                <AuthRequired>on</AuthRequired>
                <LoginName>[email protected]</LoginName>
            </Protocol>
            <Protocol>
                <Type>SMTP</Type>
                <Server>mail.mydomain.tech</Server>
                <Port>465</Port>
                <DomainRequired>off</DomainRequired>
                <SPA>off</SPA>
                <SSL>on</SSL>
                <AuthRequired>on</AuthRequired>
                <LoginName>[email protected]</LoginName>
            </Protocol>
        </Account>
    </Response>
</Autodiscover>
All I've done on this test server is enable Autodiscover and AutoConfig support in tweak settings ( I had it off for testing something else)

Code:
Updating “Thunderbird and Outlook autodiscover and autoconfig support (enables service subdomain and SRV record creation)” from “Off” to “On”.
“Thunderbird and Outlook autodiscover and autoconfig support (enables service subdomain and SRV record creation)” was updated.
Updating “Service subdomains” from “Off” to “On”.
“Service subdomains” was updated.
Updating “Service subdomain creation” from “Off” to “On”.
“Service subdomain creation” was updated.
Processing post action for Thunderbird and Outlook autodiscover and autoconfig support (enables service subdomain and SRV record creation):
The master proxysubdomains setting changed state so we do not need to update the autodiscover domains.
“Thunderbird and Outlook autodiscover and autoconfig support (enables service subdomain and SRV record creation)” was updated.
Processing post action for Service subdomains:
Creating service subdomain DNS entries in background. This process can take several minutes to complete.
“Service subdomains” was updated.

I'm still looking at the rest of your response, please allow me a bit of time to address the rest of it appropriately, I just wanted to show you the response I get when testing this.
 

lorio

Well-Known Member
Feb 25, 2004
313
21
168
cPanel Access Level
Root Administrator
Nice post Andy.

But as long as we have to read this:
but the difference between the two is actually more related to the mail client and not the autodiscover/autoconfig.
the cPanel-Forum support is not recognizing the bug. Let's see what answer you get.
 

lorio

Well-Known Member
Feb 25, 2004
313
21
168
cPanel Access Level
Root Administrator
Interesting, I'm not able to replicate the inconsistency you are having. I get the same results whether or not I query mydomain.tld or autodiscover.mydomain.tld:
What cPanel version did you use for testing?
 

Andy M-B

Member
Jul 30, 2013
10
1
3
cPanel Access Level
Root Administrator
Hi @Andy M-B

Interesting, I'm not able to replicate the inconsistency you are having. I get the same results whether or not I query mydomain.tld or autodiscover.mydomain.tld:
Hi @cPanelLauren, thanks for that. Interesting. Let me check my work: two days of switching things around and testing might mean I've transcribed a result incorrectly somewhere.

I'm still looking at the rest of your response, please allow me a bit of time to address the rest of it appropriately, I just wanted to show you the response I get when testing this.
Yep, sure. Thanks for the early response on that part.
 

Andy M-B

Member
Jul 30, 2013
10
1
3
cPanel Access Level
Root Administrator
Hi @Andy M-B

Interesting, I'm not able to replicate the inconsistency you are having. I get the same results whether or not I query mydomain.tld or autodiscover.mydomain.tld:
I've just checked with three different accounts and yes I consistently get the discrepancy in my original post.

So if you don't get the issue then there is some hope! I can't for the life of me think where my problem might be though, and from seeing other forum posts, I'm not alone.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Outlook will then try autodiscover.myclient.tld. On a single combined mail & web server, the response of mail server name of myclient.tld would technically work. But in the two server setup, autodiscover.myclient.tld is mailhost.tld and the SSL certificate for myclient does not contain the hostname myclient.tld as that hostname points to webhost.tld. cPanel is clever enough not to return the hostname myclient.tld. But what it does instead is a disaster for anyone reselling hosting: it returns the server's hostname instead!
I believe this is why you have the ability to customize this setting. The way this data returned is pretty simple and most likely not able to determine differences in mail servers like you're referencing.

Customizing the XML

A number of forum posts assert that things will be fine if you have turned on Autoconfig and Autodiscover properly. This clearly is not the case (please please prove me wrong: I want this to work!)

A number of forum posts then go on to suggest that we can customise the XML and refer to guides from Microsoft and Mozilla.

While these external guides explain nicely how the XML works, what no-one points out is how you can install custom XML on your cPanel server. cPanel's Apache has a ScriptAlias for the path /Autodiscover/Autodiscover.xml. The "script" it goes to looks like a binary, so I can't try to read it and see what the underlying logic is. Turning off autodiscover just makes the script return a 400 error code, it doesn't remove the ScriptAlias, so there doesn't actually appear to be a way to use our own XML.
So by default we use cpandemaildiscovery.cpanel.net but that xml can be added anywhere you point autodiscovery to with WHM>>Server Configuration>>Tweak Settings -
Host to publish in the SRV records for Outlook autodiscover support.
If you have an SSL enabled host with a CA signed SSL certificate on this server and want to use it instead of the cPanel provided server, enter the FQDN that has a CA signed SSL certificate here.

This is all done through DNS using SRV records. Microsoft gives instructions on how to set up their AutoDiscover - https://support.microsoft.com/en-us...e-that-enables-outlook-2007-to-use-dns-servic
White Paper: Exchange 2007 Autodiscover Service

What cPanel version did you use for testing?
I'm on 74.0.9 on CentOS 6.10
I'm using v76 on CentOS7 to test curious if that's the discrepancy.
 

lorio

Well-Known Member
Feb 25, 2004
313
21
168
cPanel Access Level
Root Administrator
I believe this is why you have the ability to customize this setting. The way this data returned is pretty simple and most likely not able to determine differences in mail servers like you're referencing.
The cpandemaildiscovery.cpanel.net is no longer needed, since AutoSSL can ensure that all setups have a valid (not selfsigned) SSL cert.
When you change the value of cpandemaildiscovery.cpanel.net to the hostname of your server, your local cPanel installtion is serving the XML stuff of autoconfig/autodiscover without any additional, manually setup config files. Allowing WHM users to access and modify these templates would be the best move.

That we still can point to different config server is fine (and sometimes needed, when you have a separate exchange server in place too), but the bug is there (or was there).

I'm using v76 on CentOS7 to test curious if that's the discrepancy.
I used v74 too. So this bug might be fixed in v76. Will have to check.

But we still need get the same understanding of the concept of offering autodiscover/autoconfig via WHM.
Whoever implemented autoconfig/autodiscover inside WHM should be able to point us to the XML templates, which are inside WHM/cpanel somewhere. cpandemaildiscovery.cpanel.net is no longer needed. There was a time, when changing it to the whm hostname didn't work at all. It should be possible, to track down people at cPanel which touched the code and let them provide a few lines for the manual.
 

Andy M-B

Member
Jul 30, 2013
10
1
3
cPanel Access Level
Root Administrator
I believe this is why you have the ability to customize this setting. The way this data returned is pretty simple and most likely not able to determine differences in mail servers like you're referencing.
If I change DNS so that all the hostnames point to the one server, renew the SSL cert and retest then things are fine. As soon as I set @ & www hosts off-server and renew the cert, I get the fallback to the server hostname.

And I can see why this happens: It makes sense that you'd only want to serve up a hostname that is on this server and has a valid certificate.

I guess that if I can I just want to find some way that I can serve up mail.myclient.tld (or better yet my choice of hostname) on autodiscover.myclient.tld without giving myself too much extra stuff to set up whenever I add a hosting account.

So by default we use cpandemaildiscovery.cpanel.net but that xml can be added anywhere you point autodiscovery to with WHM>>Server Configuration>>Tweak Settings -
Host to publish in the SRV records for Outlook autodiscover support.
If you have an SSL enabled host with a CA signed SSL certificate on this server and want to use it instead of the cPanel provided server, enter the FQDN that has a CA signed SSL certificate here.

This is all done through DNS using SRV records. Microsoft gives instructions on how to set up their AutoDiscover - https://support.microsoft.com/en-us...e-that-enables-outlook-2007-to-use-dns-servic
White Paper: Exchange 2007 Autodiscover Service
Thanks, yes. I could get rid of my DNS entries for autodiscover.myclient.tld, but there are two reasons I'd prefer to "fix" the problem that I am currently seeing.

1 - Microsoft says that using the SRV record, Outlook prompts the user for permission to use the username and password and I'd prefer not to worry clients with that extra question if I can help it.

2 - I haven't covered autoconfig here, but I get the same thing happen:

curl autoconfig.myclient.tld

<?xml version="1.0"?>
<clientConfig version="1.1">
[...snip...]
<hostname>myclient.tld</hostname>
[...snip...]
</clientConfig>

And it looks like I would have to run another server if I use custom XML because cPanel has a ScriptAlias for the path /Autodiscover/Autodiscover.xml

Turning off autodiscover doesn't seem to remove this ScriptAlias. Again if there's a recommended way of doing this then that gives me a path to achieving what I need.


I'm using v76 on CentOS7 to test curious if that's the discrepancy.
Ah, yes. I wonder if the new work on Service Subdomains has led to a change here?
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
The version difference between 74 and 76 shouldn't make any difference looking at the change logs/release notes:
Code:
Rename proxy subdomains to service subdomains
In cPanel & WHM version 76, we renamed proxy subdomains to service subdomains. This name more clearly describes the subdomains’ purpose rather than a specific implementation. This updates the following settings in the Domains section of WHM's Tweak Settings interface (WHM >> Home >> Server Configuration >> Tweak Settings):

Service subdomains
Service subdomain creation
Thunderbird and Outlook autodiscover and autoconfig support (enables service subdomain and SRV record creation)
Overwrite custom A records used for service subdomains
Service subdomain override
There are some changes to AutoDiscover/AutoConfig noted in v76 ChangeLogs 76 Change Log - Change Logs - cPanel Documentation but they don't appear to be related:
Code:
Fixed case CPANEL-22657: Prevent error while access autodiscover under cphttpd.
Fixed case CPANEL-22385: Rebuild httpd.conf and restart httpd after adjusting the autodiscover proxy subdomains tweak setting.
That we still can point to different config server is fine (and sometimes needed, when you have a separate exchange server in place too), but the bug is there (or was there).
I'm not sure if this specific case falls into this but there is an open case from v74

CPANEL-23635 - AutoConfig failure if main domain is pointed to a remote system.

If a domain's A record is pointed offsite or the website is entirely being hosted on another machine it seems that AutoConfig doesn't work. There seems to be sections in the code to suggest that it should use "mail." but even when the SSL supports mail. it isn't used.

When a domain is in localdomains and the SSL supports the "mail." domain it seems like we should also make sure AutoConfig consistently picks up "mail." as the incoming and outgoing server name.
And I'm thinking this could be what you're experiencing. The thing is there's a comment in the case that this is fixed/not able to be replicated in v76 - the only real changes done were the changes from proxy subdomains to service subdomains and the only thing I can think of is this resolved some issue we weren't aware of.


@Andy M-B I'm going to get you instructions on creating and deploying your own autodiscover.xml - though I'm wondering if you'd need if this is resolved in v76
 

lorio

Well-Known Member
Feb 25, 2004
313
21
168
cPanel Access Level
Root Administrator
Under v76.0.1 (CentOS7) autodiscover is correctly using mail.accountdomain.tld.
Looks like someone fixed the issue.
 

Andy M-B

Member
Jul 30, 2013
10
1
3
cPanel Access Level
Root Administrator
Fixed case CPANEL-22385: Rebuild httpd.conf and restart httpd after adjusting the autodiscover proxy subdomains tweak setting.
Hmm, I haven't been rebuilding httpd.conf and restarting httpd in any of my testing, so I suppose it may have affected some of my results. If anything, maybe turning off autodiscover and rebuilding httpd.conf would get rid of the ScriptAlias directive that is blocking me from using custom XML.


CPANEL-23635 - AutoConfig failure if main domain is pointed to a remote system.
That does sound like an exact match for autoconfig, I wonder if that fixes the odd autodiscover discrepancy too.

@Andy M-B I'm going to get you instructions on creating and deploying your own autodiscover.xml - though I'm wondering if you'd need if this is resolved in v76
I haven't checked yet when v76 will be on general release. I've had this server since Jan 2015 with all the upgrades along the way, as well as config changes, so there is still a chance it's a problem with my server that wouldn't be fixed by v76, though it does sound like v76 will do the trick.

So, yes please, it would be great to know I have the power to get this working, if you can get me that information.

Thanks very much for having a good look at this too!
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hmm, I haven't been rebuilding httpd.conf and restarting httpd in any of my testing, so I suppose it may have affected some of my results. If anything, maybe turning off autodiscover and rebuilding httpd.conf would get rid of the ScriptAlias directive that is blocking me from using custom XML.




That does sound like an exact match for autoconfig, I wonder if that fixes the odd autodiscover discrepancy too.



I haven't checked yet when v76 will be on general release. I've had this server since Jan 2015 with all the upgrades along the way, as well as config changes, so there is still a chance it's a problem with my server that wouldn't be fixed by v76, though it does sound like v76 will do the trick.

So, yes please, it would be great to know I have the power to get this working, if you can get me that information.

Thanks very much for having a good look at this too!
We're probably looking at release in the couple of weeks for 76 - this might be before I can get you a guide - I'll do my best though