Filed with Developers [Case 44619] Mass TTL Change

[anzenketh]

It would be a nice feature to do a Mass Change on a TTL. This would be extremely useful for server moves.

[Ishware]

As I'm facing such a prospect (changing a bunch of TTL), I must heartily agree!

[cPanelDavidG]

If you're doing a cPanel-to-cPanel transfer, the Express Transfer functionality negates the need for low TTLs as DNS on your server. With Express Transfer, DNS entries on the old server will automatically point to the new server after the transfer. Therefore low TTLs are not necessary for migrating accounts among cPanel/WHM servers.

As a result, I'm curious about what is causing a need for mass changing of TTLs?

[JawadArshad]

If you're doing a cPanel-to-cPanel transfer, the Express Transfer functionality negates the need for low TTLs as DNS on your server. With Express Transfer, DNS entries on the old server will automatically point to the new server after the transfer. Therefore low TTLs are not necessary for migrating accounts among cPanel/WHM servers.

As a result, I'm curious about what is causing a need for mass changing of TTLs?

Doesn't TTL determine the amount of time a caching name sever (like ISP) keeps the record before retrieving a fresh record from the authoritative name server (cPanel server where named is running).
If so, wouldn't end clients whose ISPs have the records already cached see the sites from the source server till TTL expires, even though the source server zone starts pointing sites to the new IP (express transfer).

[cPanelDavidG]

Doesn't TTL determine the amount of time a caching name sever (like ISP) keeps the record before retrieving a fresh record from the authoritative name server (cPanel server where named is running).
If so, wouldn't end clients whose ISPs have the records already cached see the sites from the source server till TTL expires, even though the source server zone starts pointing sites to the new IP (express transfer).

The ISP is still looking at the cached record, which stores which nameserver to go to for the latest DNS information. Let's call the old server server1 and the new server server2 for this example. So, your ISP (and everyone inbetween) has an outdated cached entry which still points to the nameserver on server1 because we didn't shorten the TTLs.

So your computer goes to the nameserver on server1, essentially retrieves the entry on server1 for the domain/subdomain you are trying to visit. This entry contains an IP that resides on server2, allowing you to get to the server you needed to get to for the newly relocated website.

[anzenketh]

The ISP is still looking at the cached record,

The point of changing the TTL is to reduce the time that the ISP keeps this cached record. This is done 24-48 hours before the server move. That way when the server move is preformed and NS records are changed the DNS changing has a possibility to happen faster as long as the ISP is obeying the TTL.

[JawadArshad]

The ISP is still looking at the cached record, which stores which nameserver to go to for the latest DNS information. Let's call the old server server1 and the new server server2 for this example. So, your ISP (and everyone inbetween) has an outdated cached entry which still points to the nameserver on server1 because we didn't shorten the TTLs.

So your computer goes to the nameserver on server1, essentially retrieves the entry on server1 for the domain/subdomain you are trying to visit. This entry contains an IP that resides on server2, allowing you to get to the server you needed to get to for the newly relocated website.

David, I believe ISP will cache DNS records as well that were retrieved from server1 for the period determined by TTL and may not go to the cached name server (server1) to retrieve fresh records for each DNS query. I may be wrong but what would be the purpose of TTL records in zones if only name server TTL was valid. Even if server1 starts pointing records to server2 IP, clients whose ISP has cached records will still be pointed to server1.
Source

The point of changing the TTL is to reduce the time that the ISP keeps this cached record. This is done 24-48 hours before the server move. That way when the server move is preformed and NS records are changed the DNS changing has a possibility to happen faster as long as the ISP is obeying the TTL.

Reducing TTL, TTL seconds before the migration should suffice.

[dwykofka]

I agree, changing the TTL for a thousand domain names is a real pain. Right now I have 20% of my dns cluster with low ttl's and the rest with normal ttl's because I can't find the time to go change them back after a server migration.

[ljesh]

+1.

David, in that scenario, what would happen if you simply turn off the server1?
From my experience (I had to move websites to another server and turn off the server1 using the migration tool) the websites will fail.

[anzenketh]

Reducing TTL, TTL seconds before the migration should suffice.

This is not the case due to you have to wait for the existing ttl to expire and update with the new TTL. The point of the TTL change is to tell the External DNS servers that follow the TTL explicitly to inquire more often. Often times servers can update the records before the TTL expires however this is not always the case.

[JawadArshad]

This is not the case due to you have to wait for the existing ttl to expire and update with the new TTL. The point of the TTL change is to tell the External DNS servers that follow the TTL explicitly to inquire more often. Often times servers can update the records before the TTL expires however this is not always the case.

Wasn't I saying the same thing . If current TTL is 24 hours, you need to change TTL atleast 24+ hours before transfer, if its 4 hours, you need to change TTL 4 hours to be on the safe side. This should ensure all cached records are refreshed across board and then check with NS for fresh records more often due to reduced TTL.

[cPanelDavidG]

Another scenario where mass changing TTLs would be useful is in the event of IP migration. For example, when a RIR forces a data center onto a new IP allocation and a cPanel/WHM server receives a new batch of IPs, mass changing the TTLs would be useful to minimize downtime caused by an IP migration.

[dwykofka]

Another situation that would be useful would be to mass change the TTL's of a particular reseller, I think it would be smart to have filters like the other mass change functions have.

We need to filter based on:
IP
Reseller
Server
Plan
domain string
(might be more that I am not covering off the top of my head)

[BStanley346]

I had the same issue and I thought I would just put down the bash script that I used to update the TTL's manually (all at once). I am sure it could be improved upon greatly but it does work.

#!/bin/bash

## Variables ##
stanTTLorig=14400
stanTTLnew=5
nsTTLorig=86400
nsTTLnew=5
soaTTLorig=86400
soaTTLnew=5



## Update the serial to +1 so that it refreshes ##
for ZONE in `ls /tmp/named/ | grep ".db"`
	do
	for SERIAL in `cat $ZONE | grep serial | awk '{print $1}'`
		do
		let "NEWSERIAL=$SERIAL+1"
		sed -i "s/$SERIAL/$NEWSERIAL/g" /var/named/$ZONE

## Now we need to update the standard TTL ##
		sed -i "s/TTL $stanTTLorig/TTL $stanTTLnew/g" /var/named/$ZONE
		
## Next we update the SOA records main TTL ##
		sed -i "s/ $soaTTLorig	IN      SOA/ $soaTTLnew	IN      SOA/g" /var/named/$ZONE

## Now we update the NS TTL records and we should be done ##
		sed -i "s/ $nsTTLorig IN NS/ $nsTTLnew IN NS/g" /var/named/$ZONE
	done
done

The TTL times can be updated in the variables section so that you can easily use the script to change the TTL back once you have made the necessary changes.

[Messiah]

This should do the trick:

NEWTTL=300
sed -i 's/^\$TTL\s.*$/$TTL '$NEWTTL'/' /var/named/*.db
sed -i 's/\s\s*[0-9][0-9]*\s\s*IN\s\s*/\t'$NEWTTL'\tIN\t/' /var/named/*.db

Just set NEWTTL in that first line to whatever you like, then copy/paste the second and third lines. You'll need to update the serial numbers too, unfortunately since the serial is often (but not always) on an otherwise-empty line by itself, I can't guarantee a regex that will match it. Generally though, the serial is the only 10-digit number in the zone file, which means this line is pretty reliable:

sed -i s/'[0-9]\{10\}'/`date +%Y%m%d%H`/ /var/named/*.db