Zone Serial Has Gone Backwards

[crwilliams]

I'm losing my mind on this one. I found a couple references to this in past posts, but haven't seen a solution.

My NS1 server in my cluster is reverting back to old DNS entries, and I can't figure out where in the world it's even getting the old information. The other servers in the cluster all have the correct entries, but every hour, the primary server's zones revert back to old data (for anything that has been updated in the last few weeks, which is about 20 records), and I have to resync to the other servers in order to get the correct info loaded in the zone files. I see the zone serial has gone backwards message in the messages log.

I've tried the fixrndc, fixbuggynamed, rebuilddnsconfig, rebuildnsdzones, and I restart named afterward. I've tried these scripts individually, and in sequences, too. I've even tried disabling clustering long enough to see what would happen the next hour, and it still reverts back to old entries-- so it must be getting the old data locally.

I've also tried manually editing some entries, keeping the old serial number it's reverting back to, just to see, and sure enough, after the hour, when it reloads and syncs, the old information is back in the zone files.

When running the rebuilddnsconfig, I did see this little notification:
warn [rebuilddnsconfig] cache_dump does not exist in named.conf

But there is a path to a cache_dump.db in named.conf, so I'm not sure what that error really means.

Any thoughts on what I seem to be missing, here? Is there a dns zone cache somewhere on that server that it's reverting to?

Thanks for any tips, help, suggestions!

[cpanelkenneth]

It sounds like you have an old request stuck in one of the cluster queues. Check the contents of /var/cpanel/clusterqueue/requests/ on each server.

[crwilliams]

That was it! Thank you!

Don't know which one it was that was causing the problem, but I just cleared out the clusterqueue requests and restarted named on all the servers in the cluster, and that did the trick. The next hour the dns synced, it kept the correct data for the dns zones.

The help is much appreciated!

[cpanelkenneth]

This should be resolved once cPanel 11.25 is available in all branches ( including DNSONLY ). Currently one "bad" member of a cluster will cause queued requests to keep replaying. cPanel 11.25 will:

1. Allow the admin to set a failure threshold. Consecutive failures by a single cluster member over the threshold will cause the local server to disable that cluster member.

2. More intelligent queue handling

Among other bug fixes and performance improvements.