Careful when migrating between servers with different DNS service

[Steini Petur]

Hello,

I had migrated about 6 servers from servers with Bind and Cloudlinux 6-7 to that of AlmaLinux 8.5/Cloudlinux 8.5 with PowerDNS I found that when doing so I would get these random instability occurrences such as

RESTORE: Account “xxx”: Warnings

Sorry, An error occurred while adding dns zone: Could not store zonedata for xxx.com on x Could not remove xxx.com from the Bind configuration (named.conf) on x Configuration may be in an inconsistent state on x

Sorry, An error occurred while adding dns zone: Could not store zonedata for yyy.com on x Could not remove yyy.com from the Bind configuration (named.conf) on Configuration may be in an inconsistent state on x

I took out the data as it's irrelevant, but It had to do with an Inconsistent state, now I can't be 100% sure that this is the reason but this time on the 8th server, I had NO such thing when I made sure that both servers matched to start meaning that both are BIND! the Cloudlinux is 6 to 8.5 but both are BIND and as such during the transfer of 800 accounts there was not a single incident with an inconsistent state.

Only post-migration did I go to Nameserver Selection and install PowerDNS again as the leading Nameserver.

It could have been a coincidence but seeing how the previous 6 servers always had at least 3-5 of these where we had to actually delete all the addon/subdomain links and recreate them to their folders post-migration to resolve the issue and this one had 0 incident, I am semi-wondering if its caused by inconsistency between migrations between BIND and PowerDNS for these zones for whatever reason.

It kept happening to addon-based domains and their respective links.

I have yet one more or two more servers to migrate from C6-7 to C8.5 to finish the upgrades, and should that also reveal 0 then that will be enough for me, I just thought to share that with others who might be considering large scale user migrations, out of 800-1000 you would get this for maybe a handful of accounts 3-5 accounts, just remember if it says "Sorry, An error occurred while adding DNS zone" you NEED to re-do it for the client or his site will not resolve properly, it's not a "warning" this really is an "ERROR", it should be RED ERROR for the account but it really returns YELLOW WARNING as it doesn't matter which is false, it really does matter for the client you are moving.

Just my two cents.

 

[cPRex]

Hey there! We do a lot of migrations, and I haven't come across this particular issue just yet. If you're still seeing that problem, and you have the affected systems online, it would definitely be worth submitting a ticket to our team so we could do some testing.

 

[Steini Petur]

Hey there! We do a lot of migrations, and I haven't come across this particular issue just yet. If you're still seeing that problem, and you have the affected systems online, it would definitely be worth submitting a ticket to our team so we could do some testing.

I'd love to but these are production servers and we've fixed the accounts that this occurred. It is hard to replicate because out of 800+ accounts only 3-4 were affected.. I can't really replicate this enough so I can report this to the support, they would have to run transfers,.

Ill get back to this ticket if this does not come up when I migrate my 7th server in about a week or two, and Ill report here if out of the 800+ I migrate if there are any fallout Bind to Bind.. as that caused no chaos last time but Bind to PowerDNS did.

 

[Steini Petur]

If there is any verbose logging that I can enable for when I run my migration next time something that I could then give access to cPanel to investigate I'd love to know about it,I can easily set that on before running the migrations when the time comes.

 

[Steini Petur]

@cPRex I will be migrating tomorrow 800 account server BIND to BIND, I will let you know if there is any inconsistent state, if not then that will solidify that the issue was caused by Bind to a PowerDNS server as most of our servers host 800 each, out of 8 first 6 were done Bind to PowerDNS with always handful of them inconsistent needing to be re-done, and tomorrow will be server 8 after 7 was successful with Bind to Bind, we do again Bind to Bind tomorrow and ill tell you right after if it yields inconsistent state or just 100%..

 

[Steini Petur]

@cPRex I can confirm Bind to Bind yielded no issues but Bind to PowerDNS on the second server test was

RESTORE: Account x
Failed to restore the domain “x.com”: Sorry, An error occurred while adding dns zone: Could not store zonedata for x.com on 82-221-136-5 Could not remove x.com from the Bind configuration (named.conf) on Configuration may be in an inconsistent state on x

Out of the so far restored 32% / 100% of 200 account based server 1 issue has already come up, I can definitely confirm that the issue with this is Bind to PowerDNS, there was 0 incident with the 800 server.

I can see other threads error with transfer tool - I see that others have this issue, I can confirm just that "BIND TO BIND = 100%" whilst "BIND TO POWERDNS = causes these inconsistencies.."

 

[cPRex]

Thanks for that - I've made myself a note to check this out, but I know I won't get to it today. I'll be sure to post back once I have some more details.

 

[cPRex]

I've been poking this for a bit now and I'm struggling to reproduce this problem in my testing environments. Is there any way you could create a ticket with our team so we can see this in action?

 

[Steini Petur]

I've been poking this for a bit now and I'm struggling to reproduce this problem in my testing environments. Is there any way you could create a ticket with our team so we can see this in action?

Sorry I neglected this so long @cPRex I don't know how to reproduce this, you sort of need a LOAD of accounts with bind settings, you would probably need to create some sort of localized zone with 800-1000 bind zone files, even if its just .local fake zones, and then another server to MIGRATE all those accounts over, but again It's a pure guess because this happened like 4-8 times in 800 users, it's such a small needle.

 

[cPRex]

If you find another server where this is happening, feel free to make a ticket so we can check it out!