Email Deliverability DNS config issue

[krembo99]

I just took over a recently installed server that one of my colleagues installed.
I immediately noticed that there is a problem with mails staying too long in the queue.

After some investigations of logs I saw that big networks ( yahoo, qq , gmail ) are rejecting SOME mails temporarily while receiving others. Generally speaking mails going out from set accounts from within the webmail interface are ok. scripted (php) mails not so much - and also cpanel/whm notifications arrive with few hours delay - sometimes also blocked.

I checked the Home>>Email>>Email Deliverability page and in fact there was problem reported .

Domain
host.domain.gLTD
DKIM PROBLEMS EXIST
A “DKIM” record does not exist for this domain.

This system does not control DNS for the “host.mydomain.gLTD” domain. Contact the person responsible for the “ns1.vultr.com” and “ns2.vultr.com” nameservers and request that they update the “DKIM” record with the following:

and :

SPF  PROBLEMS EXIST

A “SPF” record does not exist for this domain.

This system does not control DNS for the “host.mydomain.gLTD” domain. Contact the person responsible for the “ns1.vultr.com” and “ns2.vultr.com” nameservers and request that they update the “SPF” record with the following:
.....

And a very similar one for PTR :

REVERSE DNS (PTR) PROBLEMS EXIST
The system sends “host.mydomain.gLTD”’s outgoing email from the “xxx.180.xxx.xx” IP address. The only PTR value for this IP address must be “host.mydomain.gLTD”. This is the name that this server sends with SMTP’s “HELO” command to send “host.mydomain.gLTD”’s outgoing email.

1 unexpected PTR value exists for this IP address:

[LIST]
[*]xxx.180.xxx.xx.vultr.com
[/LIST]
To fix this problem, replace all PTR records for “10.140.180.139.in-addr.arpa” with the following record at “reversedns.vultr.com”:
....

[[ the xxx.180.xxx.xx.vultr.com is actually my.ip.address.vultr.com ]]


At this point I started suspect that it is a DNS problem, and as it turned out, during the initial setup my colleague has gave it the hostname of host.mydomain.gLTD but has input ns1.vultr.com and ns2.vultr.com as the name server ( vultr is a VPS host where the server host.mydomain.gLTD is installed ). He also added an addon domain with the same mydomain.gLTD as the main host.mydomain.gLTD (which by itself should not be a problem i think )

The server has it's own ns1.mydomain.gLTD, and ns1.mydomain.gLTD, that I assume were setup by WHM upon install because my colleague did not know nothing about them - but it is my opinion that these need to be the server's main resolver / DNS and NOT those of vultr...

I have searched here on the forums and in the documentation, but I am still not 100% clear on how to resolve this problem and what are the ramifications / consequences of it. I am not even 100% sure that in fact the DNS is the problem that is causing mails to be labeled as spam.

Everything else seem to work fine for now ( the server have circa 30~40 domains, parked, and addons)

I am a bit worried about fiddling with DKIM and SPF etc. because they are not super simple and I know next to nothing about the syntax or how they work.

Is wrong DNS in fact the main problem ? ..and if so - What are the correct steps to fix it?
What is my correct course of action here? what should I do first and how ?

( I am not sure if this qualifies as mail problem, DNS problem, or general configuration problem - So I just posted it under general discussion. sorry if it is the wrong place. )

Edit I:

I found a webpage describing some possible steps to change name servers - are those steps correct ? will it get me closer towards resolving the problem?

 

[cPanelLauren]

The server has it's own ns1.mydomain.gLTD, and ns1.mydomain.gLTD, that I assume were setup by WHM upon install because my colleague did not know nothing about them - but it is my opinion that these need to be the server's main resolver / DNS and NOT those of vultr...

If you want to manage DNS locally on the server then yes you would want to login at the registrar and change the nameservers to point to your server's custom DNS the documentation here should be useful for this How to Set Up Nameservers in a cPanel Environment - cPanel Knowledge Base - cPanel Documentation

I am a bit worried about fiddling with DKIM and SPF etc. because they are not super simple and I know next to nothing about the syntax or how they work.

Because DNS is not managed locally if you were to leave the configuration as is you'd need to manually add these records where DNS is hosted. Otherwise, they are automatically created and the DNS zone file updated with them automatically

 

[krembo99]

@cPanelLauren

Thank you for your reply -

If you want to manage DNS locally on the server then yes you would want to login at the registrar and change the nameservers to point to your server's custom DNS the documentation here should be useful for this How to Set Up Nameservers in a cPanel Environment - cPanel Knowledge Base - cPanel Documentation

Maybe I described the issue in an unclear way :

I know how to set up the name server. Currently it is already set up and apperarntly working ( using BIND )

DNS is already managed locally, apparently working and all registered domains are pointing to ns1.mydomain.gLTD and ns1.mydomain.gLTDat the different registers. ( server own NS ). All domains ( including the main one ) are resolved apparently correctly. ( ping, trace route, normal browser access etc.. even mails)

Because DNS is not managed locally if you were to leave the configuration as is you'd need to manually add these records where DNS is hosted. Otherwise, they are automatically created and the DNS zone file updated with them automatically

I understand the problem and what you wrote is exactly the issue as I (tried) to described it in my question.

I do not want to fiddle with these records manually. I do want an automatic handling - but how to correct the settings now ?

Still my issue remains :

what are the steps and ORDER of the steps to reverse the problem without creating new problems?

Can I just change the NS in WHM >> Home >> Server Configuration >> Basic WebHost Manager Setup ?
( already have A entry for hostname )

Is there a way to "reset" the DKIM/SPF records and automatically create new correct ones ?


Will following all the steps in your linked page : How to Set Up Nameservers in a cPanel Environment - cPanel Knowledge Base - cPanel Documentation will correct the DKIM / SPF records on all domains and resolve the problem AFTER there are numerous domains already configured and active ? Will it correct the entries for the main host ?

It is becoming a bit urgent after so long because the result will be that the IP will get blacklisted .
I am still not so clear on the exact steps - Should I open a ticket ?

 

[cPanelLauren]

You noted that your nameservers are ns1/ns2.vultr.com:

At this point I started suspect that it is a DNS problem, and as it turned out, during the initial setup my colleague has gave it the hostname of host.mydomain.gLTD but has input ns1.vultr.com and ns2.vultr.com as the name server

This tells me immediately that regardless of what is set in your WHM the domain's DNS is not hosted locally on the server. The only way to resolve this is to point DNS to the nameservers you created within WHM at the registrar.

To further identify whether or not this is in fact the case a simple dig query should give you the nameservers of the domains:

dig ns yourdomain.tld

Will following all the steps in your linked page : How to Set Up Nameservers in a cPanel Environment - cPanel Knowledge Base - cPanel Documentation will correct the DKIM / SPF records on all domains and resolve the problem AFTER there are numerous domains already configured and active ? Will it correct the entries for the main host ?

It should pending all domains are hosting their DNS locally and not using vultr's nameservers.

 

[krembo99]

@cPanelLauren Thank you for your patience reply.

You noted that your nameservers are ns1/ns2.vultr.com:

This tells me immediately that regardless of what is set in your WHM the domain's DNS is not hosted locally on the server. The only way to resolve this is to point DNS to the nameservers you created within WHM at the registrar.

To further identify whether or not this is in fact the case a simple dig query should give you the nameservers of the domains:

dig ns yourdomain.tld

Apparently the whole issue began when the ns1/ns2.vultr.com were used during initial installazion wizard of WHM.

In fact you are absolutely right., results of dig clearly shows that all domains are pointing to ns1/ns2.vultr.com.

The main domain :

[[email protected] ~]# dig ns host.mydomain.gTLD

; <<>> DiG 9.9.4-RedHat-9.9.4-73.el7_6 <<>> ns host.mydomain.gTLD
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16170
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;host.mydomain.gTLD.                   IN      NS

;; ANSWER SECTION:
host.mydomain.gTLD.            300     IN      CNAME   mydomain.gTLD.
mydomain.gTLD.                 272     IN      NS      ns1.vultr.com.
mydomain.gTLD.                 272     IN      NS      ns2.vultr.com.

;; Query time: 2 msec
;; SERVER: 108.61.10.10#53(108.61.10.10)
;; WHEN: Thu May 30 05:56:22 +08 2019
;; MSG SIZE  rcvd: 99

( note CNAME )

...and anotherdomain.com

[[email protected] ~]# dig ns anotherdomain.com.

; <<>> DiG 9.9.4-RedHat-9.9.4-73.el7_6 <<>> ns anotherdomain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4832
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;anotherdomain.com.                      IN      NS

;; ANSWER SECTION:
anotherdomain.com.              86400   IN      NS      ns1.vultr.com.
anotherdomain.com.              86400   IN      NS      ns2.vultr.com.

;; Query time: 72 msec
;; SERVER: 108.61.10.10#53(108.61.10.10)
;; WHEN: Thu May 30 06:04:59 +08 2019
;; MSG SIZE  rcvd: 79

Other Observations :

Under WHM >> Resolver configuration ( see attached image ) I can find the same IP from the `dig` result

SERVER: 108.61.10.10#53(108.61.10.10)

Under WHM >> Nameserver Record Report I have ns1/ns2.vultr.com but with a different ip which is not mine ( see image attached )

Under WHM >> Edit DNS zone all the domains in fact point to ns1/ns2.vultr

Under WHM >> Basic Web host Manager Setup I have nameservers ns1/ns2.vultr.com ( which is probably what started the whole issue as all accounts were created by root ..

When I ping ns1/ns2.maindomain.gTLD I get my own correct IP ..

It should pending all domains are hosting their DNS locally and not using vultr's nameservers.

So my question is again :

What are the steps ( and ORDER ) needed to correct the issue ?

Which one should I change first, Which parts will be automatically corrected ? and which ones I should not touch ?

If I change WHM >> Basic Web host Manager Setup nameservers to my own, will I need to edit every existing DNS zone manually ? In WHM ? In Cpanel ? In Both ?

If I change WHM >> Resolver configuration will it change / fix some other problems ?

Please guide me to the right order of steps that will ensure less manual fiddling. Which should I correct first ?

I believe that the order of the actions is paramount here.

I must confirm again that all the domain names at the registers are pointing to either ns1/ns2.mydomain.gTLD or directly to my IP.

Also: Since the server was not initially set by me - I just confirmed that the vultr control panel has a DNS record for maindomain.gTLD. ( I know you guys are not vultr but I just thought this fact might be of some importantce - as well as the fact thaat the maindomain.gTLD is a geographical TLD)

Attached some images for clarification of the above issues / settings.

 

[cPanelLauren]

If I change WHM >> Basic Web host Manager Setup nameservers to my own, will I need to edit every existing DNS zone manually ? In WHM ? In Cpanel ? In Both ?

If the nameservers in the DNS zone file aren't ns1/ns2.yournameservers.tld then yes, you would need to manually modify this.

If I change WHM >> Resolver configuration will it change / fix some other problems ?

This shouldn't be related to the nameservers I'm not sure I understand why the DNS resolvers are using this IP - the resolvers contain nameservers which your server uses to query to resolve domains - this isn't to be confused with the nameservers you're using on the server. The documentation on this might be useful: Resolver Configuration - Version 80 Documentation - cPanel Documentation
Commonly people use something like google or cloudflare's dns resolvers:

Public DNS  |  Google Developers
Introducing DNS Resolver, 1.1.1.1 (not a joke)

Sometimes providers may have DNS resolvers you can use as well.

I must confirm again that all the domain names at the registers are pointing to either ns1/ns2.mydomain.gTLD or directly to my IP.

There's no way for the domain to be using vultr's nameservers and the nameservers also point to your custom nameservers so some portion of this must be confused. You noted when you executed the dig query that you got vultr's nameservers in response, are you saying that when you go to the registrar for that domain and look at the settings you see the custom nameservers you created as where the domain is pointed?


As far as the order of operations I would suggest following the order set out by the documentation I sent previously.

 

[krembo99]

@cPanelLauren

Thank you for your reply .

If the nameservers in the DNS zone file aren't ns1/ns2.yournameservers.tld then yes, you would need to manually modify this.

There's no way for the domain to be using vultr's nameservers and the nameservers also point to your custom nameservers so some portion of this must be confused. You noted when you executed the dig query that you got vultr's nameservers in response, are you saying that when you go to the registrar for that domain and look at the settings you see the custom nameservers you created as where the domain is pointed?

Yes this is exactly what I am saying.
The domains are registered in 3 different registers.
All of them are either pointing to ns1/ns2.maindomain.gTLD OR directly to the IP ( with A record )

As far as the order of operations I would suggest following the order set out by the documentation I sent previously.

Are you referring to the " How to set Up name servers in a cPanel Environment " documentation ?
I did not see there a document as to how to change the setup of already existing domains or fix DKIM and SPF records.
Should I assume these, and other problems would be automatically fixed if I follow the document in this link ?

 

[cPanelLauren]

Can you please open a ticket using the link in my signature as something must be amiss in this instance? Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved.


Thanks!

 

[krembo99]

@cPanelLauren

Thanks ( yet again ) for your reply ..

Once open please reply with the Ticket ID here so that we can update this thread with the resolution once the ticket is resolved.

I have opened a ticket through the WHM interface - Request ID is presumably #12480401

 

[cPanelLauren]

Great, thanks I'm watching that thread and I'll update you here when more information is available.