Adding domain to server not possible due to authoritative nameserver issue

[Steini Petur]

Hey,

One of our servers is now all of the sudden starting to malfunction, but before I get the usual let me tell you I have already tried the

• Install Jwhois - Does not work
• Run UPCP force - Does not work
• Rebooting the server - Does not work

This started to randomly happen, here is the error

The system failed to find the IPv4 addresses for “a.gtld-servers.net”, “b.gtld-servers.net”, “c.gtld-servers.net”, “d.gtld-servers.net”, “e.gtld-servers.net”, “f.gtld-servers.net”, “g.gtld-servers.net”, “h.gtld-servers.net”, “i.gtld-servers.net”, “j.gtld-servers.net”, “k.gtld-servers.net”, “l.gtld-servers.net”, and “m.gtld-servers.net”. Because of this, the system cannot find “<domain>.com”’s authoritative nameservers. See the cPanel &amp; WHM error log for more details.

Before i am asked about the log, this is the part of it (It repeats for a while before failing

[2020-12-01 07:35:32 +0000] info [park] Creating Addon domain '<domain>.com' on '<domain>.<domain>.tld'.
        Cpanel::DNS::GetNameservers::get_nameservers(Cpanel::DNS::Unbound=HASH(0x31ebe70), "<domain>.com", undef, 1) called at /usr/local/cpanel/Cpanel/DnsRoots.pm line 27
        Cpanel::DnsRoots::fetchnameservers("<domain>.com") called at /usr/local/cpanel/Cpanel/DnsUtils/Add.pm line 233
        Cpanel::DnsUtils::Add::doadddns("domain", "<domain>.com", "IP", "<IP>", "has_ipv6", 0, "ipv6", "n/a", ...) called at /usr/local/cpanel/Cpanel/ParkAdmin.pm line 532
        Cpanel::ParkAdmin::_local_park("newdomain_exists", 0, "skip_ssl_setup", 0, "user", "<user>", "domain", "<domain>.<domain>.tld", ...) called at /usr/local/cpanel/Cpanel/ParkAdmin.pm line 306
        Cpanel::DNS::GetNameservers::get_nameservers(Cpanel::DNS::Unbound=HASH(0x31ebe70), "<domain>.com", undef, 1) called at /usr/local/cpanel/Cpanel/DnsRoots.pm line 27

I've removed some information irrelevant to this post, but It attempts and fails and then says "Couldn't find IPv4", my resolvers are google 8.8.8.8 and 8.8.4.4 and I can confirm I can ping those so called root servers

root@hekla [/usr/local/cpanel/logs]# ping a.gtld-servers.net
PING a.gtld-servers.net (192.5.6.30) 56(84) bytes of data.
64 bytes from a.gtld-servers.net (192.5.6.30): icmp_seq=1 ttl=54 time=51.9 ms
64 bytes from a.gtld-servers.net (192.5.6.30): icmp_seq=2 ttl=54 time=52.3 ms
64 bytes from a.gtld-servers.net (192.5.6.30): icmp_seq=3 ttl=54 time=52.6 ms

So I can't agree with the fact there is no IPv4 for these addresses, only cPanel appears to have some issue with this, I've searched these threads but they seem to come up empty or ends with "please open a ticket". In few cases JWhois fixed this but for me it is not.

I also have enabled

Allow Unregistered Domains
Allow Remote Domains

In the Tweak Settings so it's not there, it just randomly started to occur. Anyone have any idea what is going on because the server doesn't seem to have issue finding the IP of those root nameservers, but cPanel does appear to have some issue.

Also it should be noted that port 53 TCP and UDP is open and the other one as well, so its not a port issue.

 

[andrew.n]

I think Submit a Ticket directly at cPanel would be the best as it can be a few things which could cause this or alternatively get in touch with a cPanel certified sys admin to investigate this.

A few questions:

1. Are you behind NAT?
2. Do you have a public IP address?
2. Is the server in a DC?
3. Is it hosted at Amazon, Google Cloud etc..? (they utilise NAT)
4. How does your /etc/hosts file looks?
5. Can you make sure there is no extra space etc in /etc/resolv.conf file

 

[Steini Petur]

Hi, @andrew.n

I forgot to mention the NAT


1. Are you behind NAT?
No
2. Do you have a public IP address?
Yes
2. Is the server in a DC?
Yes
3. Is it hosted at Amazon, Google Cloud etc..? (they utilise NAT)
No, Im the server administrator, this is a server hosted in a DC in Iceland, I set this up it's no NAT setup
4. How does your /etc/hosts file looks?
Standard first line localhost, second is <IP> <FQDN> <Hostname>
5. Can you make sure there is no extra space etc in /etc/resolv.conf file
No extra space, few days ago everything worked fine, now it doesn't this is the only node of 12 currently that is experiencing this, 8 of which are on same C-class so no firewall issues, no block by Voxility, have already checked that. It just suddenly happened.

I will eventually open a ticket but I felt that if we can figure this out here, it could help some others because I've seen so far 0 solutions to few people having this problem, but I might as well open a ticket, since we're cPanel certified partners. Will open a ticket and then if they do figure it out I ask them to share the solution with me.

 

[andrew.n]

hah! Voxility! they can have issues like this at least back then especially with their fancy DDoS protection we had...so check this with them and be persuasive

 

[Steini Petur]

hah! Voxility! they can have issues like this at least back then especially with their fancy DDoS protection we had...so check this with them and be persuasive

Tell me about it, I have asked them to verify this, the only thing that is going on right now is that the ICMP/TRACE is being dropped due to consistent attacking to the server, but again with ICMP and TRACE ignore, It is just about asking the server to respond, not out from the server, so besides that there shouldn't be an issue and another server is currently under consistent UDP attack and Create/Addon domains works fine there, but yes I did ask them few times ;D ask them to be very very sure.. I have opened a ticket now with the cPanel team because I can ping all those servers, they all return an IP but this "Unbound" thing that runs with this script doesn't return correctly the IPv4.

But yes, Voxility can but I have not noticed that much with Voxility but I did endlessly experience this with Staminus once they became stackpath, their protection was driving me nuts.

 

[andrew.n]

ya crazy. Keep us updated what the cPanel team figures.

 

[Steini Petur]

Wish I could say the issue was resolved by cpanel, and not "it resolved itself automatically", I guess in my case something outside was causing it but I don't have proof of that so I will just have to accept that it resolved itself.

Hello XXXXXX,

Today something happened on one of our servers X it was not able to reach the root nameservers this morning 07:00am Icelandic time

[2020-12-01 07:36:33 +0000] warn [cpanel] Encountered error in AddonDomain::addaddondomain: The system failed to find the IPv4 addresses for “a.gtld-servers.net”, “b.gtld-servers.net”, “c.gtld-servers.net”, “d.gtld-servers.net”, “e.gtld-servers.net”, “f.gtld-servers.net”, “g.gtld-servers.net”, “h.gtld-servers.net”, “i.gtld-servers.net”, “j.gtld-servers.net”, “k.gtld-servers.net”, “l.gtld-servers.net”, and “m.gtld-servers.net”. Because of this, the system cannot find “x.com”’s authoritative nameservers. See the cPanel & WHM error log for more details. at /usr/local/cpanel/Cpanel/EventHandler.pm line 47.
Adding secondary domain 'x.com' to the Antispam filter... Failed!

[2020-12-01 15:02:17 +0000] info [park] Creating Addon domain x.com' on 'x.y.com'.

Then as of 03:00pm afternoon the IP X is able to reach again the root nameservers such as a.gtld-servers.net etc to m.gtld-servers.net, please confirm if any changes occurred to the IP X between 07am this morning to 03pm afternoon Iceland time that is GMT.

If any filter was removed, Any protection removed, Adjusted, anything at all that automatically changed because right now everything magically went back to normal. Could you look at this for me.

Sadly it appears that the issue was elsewhere, I want to figure out if there were any adjustments.

 

[cPRex]

Hey there, @Steini Petur

When I see the errors you provided initially, I immediately think "network problem outside the server," especially with all the tests you already performed. Something happened during the time of the error that kept the network tools from properly reaching the root nameservers, which will break AutoSSL.

Since the issue likely wasn't on your machine, it's hard to say what was happening though.

 

[Steini Petur]

You are right @cPRex, it looks a lot like it was an outside disruption, most likely related to the DDOS mitigation, although they aren't super keen on admitting such so, for now it's all good but the weird thing is ping worked to resolve the hostname etc so I would have hoped the network tools used for this procedure would have some sort of fallback if one tool doesn't work, run a secondary option like retrieve the IPv4 from the hostname using different method.

 

[cPRex]

The problem with having a fallback option in place for this type of work is that the primary way other systems will be looking up your domain name may not be successful. We'd rather be cautious and have a failure, to ensure this gets looked into by the admin, than try and workaround the issue and allow things to work that may not be successful.