TIP: Zero downtime transition between servers for multiple accounts...

[hostww]

First: 72-48 hours before the transition, change all dns zones from TTL 14400 to TTL 120 and 86400 to TTL 180...

perl -pi.bak -e "s/14400/120/g" *.db
perl -pi.bak -e "s/86400/180/g" *.db
rndc reload

After 48 hours: Copy all accounts to the new server, and on the old server, use one of the following fowarding techniques for EACH ip... (I recommend the script!)

================================
"Manual IP Forwarding"
================================
echo "1" >/proc/sys/net/ipv4/ip_forward

iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to newip:80
iptables -t nat -A PREROUTING -p tcp --dport 25 -j DNAT --to newip:25
iptables -t nat -A PREROUTING -p tcp --dport 20 -j DNAT --to newip:20
iptables -t nat -A PREROUTING -p tcp --dport 21 -j DNAT --to newip:21
iptables -t nat -A PREROUTING -p tcp --dport 110 -j DNAT --to newip:110

iptables -t nat -A PREROUTING -p tcp --dport 53 -j DNAT --to newip:53
iptables -t nat -A PREROUTING -p udp --dport 53 -j DNAT --to newip:53

iptables -t nat -A PREROUTING -p tcp --dport 1024:65535 -j DNAT --to newip
iptables -t nat -A PREROUTING -p udp --dport 1024:65535 -j DNAT --to newip

iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 80 -j SNAT --to-source oldip
iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 25 -j SNAT --to-source oldip
iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 20 -j SNAT --to-source oldip
iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 21 -j SNAT --to-source oldip
iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 110 -j SNAT --to-source oldip

iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 53 -j SNAT --to-source oldip
iptables -t nat -A POSTROUTING -p udp --dst newip --dport 53 -j SNAT --to-source oldip

iptables -t nat -A POSTROUTING -p udp --dst newip --dport 1024:65535 -j SNAT --to-source oldip
iptables -t nat -A POSTROUTING -p tcp --dst newip --dport 1024:65535 -j SNAT --to-source oldip

/sbin/modprobe ip_conntrack_ftp
/sbin/modprobe ip_nat_ftp

This will foward dns, smtp, pop, web, mysql and ftp to the new server...

================================
"Scripted IP Forwarding"
================================
#!/bin/bash

#Forwarding By Paul Fleming
#Shouts to #cpanel on efnet

if [ "$UID" != "0" ]; then
echo "You must be root to use this tool"
exit 1
fi

DOWHAT=$1

case $DOWHAT in
'on')
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A PREROUTING -i eth0 --protocol tcp --destination-port ! 22 -j DNAT --to-destination $2
iptables -t nat -A PREROUTING -i eth0 --protocol udp -j DNAT --to-destination $2
echo "All Ports Excluding Port 22 Forwarded"
echo "Shutting Down Local Services"
service httpd stop
service pure-ftpd stop
service proftpd stop
service cpanel stop
;;
'off')
iptables -F
echo "Forwarding Off"
echo "Restarting Services"
service httpd restart
service cpanel restart

;;

*)
# how to use this thing.
echo "example: forward on 12.12.12.12 will enable forwarding to 12.12.12.12"
echo "forward off to disable"
echo "forward by Paul Fleming"
;;
================

Last: Change your dns on the registrar, and tell you clients about the new ips...

 

[chirpy]

First: 72-48 hours before the transition, change all dns zones from TTL 14400 to TTL 120 and 86400 to TTL 180...

perl -pi.bak -e "s/14400/120/g" *.db
perl -pi.bak -e "s/86400/180/g" *.db
rndc reload

You need to increment the serial numbers as well or you can have problems doing that. Any change to a zone file should always include an increment of the serial number.

You could also look to using rinetd instead of iptables for rerouting IP addresses. It's much simpler and cleaner to use:
http://www.boutell.com/rinetd/

Don't forget to also reroute ports 143 993 995 443

 

[hostww]

Thanks for the tips... we did realize the issue with serial numbers but since we're altering TTL on all domains on all servers it doesn't much matter. Once the move is done then they'll all be moved back so that is a null point for our particular move, but would cause problems with some other peoples.

Have any links or ideas on simple ways to mass update serial numbers?

We are going to route all ports for each IP over instead of doing port by port (using the script)... but I"m going to take a look at that rinetd beforehand as well. Thanks again for that.

 

[sparek-3]

This looks good. Its given me some ideas on some changes to our transitional procedure.

One question though. How do you accomodate for changes made to files between the time of the account backup and the server switch.

For example, say you create a backup copy of an account at 6am (or use the cpanel move stuff), the process for that on an entire server for each account can take some time, but say it takes you 8 hours. So at 2pm you have copied over all accounts and then you reroute traffic.

What happens if Domain A receive an e-mail at 7am? Its on the old server, but not copied over to the new server.

 

[hostww]

We don't make a backup and then restore the backup on the new server when doing transitions... we make use of the "Copy multiple accounts from another server" function in WHM.

By doing it this way you only have the chance of missing posts in forums, data changes, and emails during the time that particular account is packaging/downloading/restoring and until you complete the ip forwarding after its restored.

The above usually only gives a window of 5-20mins of actual downtime for the account depending on the site size, number of accounts etc... Unlike the way you mentioned with can lead to hours of possibly missed data changes, emails, and forums posts etc...

 

[lloyd_tennison]

Update Serial numbers:

http://www.cplicensing.net/files/scripts/updateserial

 

[hostww]

Nice one.