How to upgrade server from CentOS 5 to 6 without much downtime and pain?

I know year ago Bluehost upgraded their centos5 servers to centos6, and they announced that work is done in 2 hours (per server). Does anybody know how they did it? I doubt this (as is usually recommended) is migration to other server/hw via pkgacct/restorepkg, or multiple accounts copy, because moving so much users/gigabytes (per server) could take much more hours.

I just want on same hardware server to have centos6 after (most painless way preferrable) upgrade anybody could suggest. Main requirement is that cpanel does not change dns records of all clients in crazy way (i.e. ruin) and most important is that IP addresses of clients do not change and process does not take whole day (few hours is ok). If php/apache/etc change version this is ok too.

I tried mass move clients from older centos server to other centos6 server but it had many problems which I want to avoid next time. That is what I did before:

1. useradd all client accounts to the new server.
2. rsync each client homedir from old server to the new. I thought this will be much faster that pkgacct zip/copy/unzip every client homedir.
3. pkgacct --skiphomedir for each client account on old server.
4. ip addr del for each ip in the old server ip pool.
5. Add old IPs to the new server.
6. scp cpmove-*.tar.gz to the new server.
7. Then I attempt to restorepkg --force --ip=x.x.x.x cpmove-$client.tar.gz on the new server.
This is failed because restorepkg didn't liked --ip=x.x.x.x
8. restorepkg --force cpmove-$client.tar.gz for each $client username.
This of course assigned wrong IPs to clients, but I planned to reassign them later. This somehow led to mess in dns records.
9. Disable DNS clustering on old server.
10. Change Multiple Sites’ IP Addresses onnew server to proper (old) IP.
There I forced to play with main shared IP address to finally set correct IPs for these clients, otherwise it didn't work.
Then I found that some domains are empty (as of no client's php/html), it seems that restorepkg cleared subdomain directories. To fix that I rsynced homedirs again.
And tons of other mess, like fixing dns records (it somehow did not updated them correctly so many clients still got wrong IPs), and fixing soa records.

This was very painful, plus this time I need to stay on the same hardware.

So, how is recommended (or someone succeeded) to upgrade to centos6 w/o change client IPs and too much downtime?

 

Thanks dalem and cPanelMichael for answers.

cPanelMichael, you probably meant 'Copy Multiple Accounts/Packages From Another Server', but this is much harder to use than scripted pkgacct/restorepkg, because it requires a lot of attentive mouse clicking. First I should plan transfer in waves, becasue of difference or shared IPs (many shared IPs is also possible becasue older server could have users transferred from even older server). First I need to transfer accounts from one IP (maybe hundred(s)), then for second IP, etc. In multiple copy I need carefully first click hundreds of packages from the list, then hundreds correct account names from thsi wave, for each account name I need to click also 'Dedicated IP' or 'Skip Homedir' when needed. So with just 100 users migration I should click like 300 times and each username should be carefully checked from the list (so I don't click user from other IP). This is very hard, requires a lot of attention, and error prone. That's why I prefer to use pkgacct/restorepkg.

What I would like intead is option to transfer IP from one server to another, with all linked to it users automatically, and/or ability to really force IP in restorepkg. And I like to really be sure it never touch DNS. Maybe cPanel could add more transfer friendly options into restorepkg? (Like --force IP really force it no matter what, and subdomain creation to not clean its directory, and protect dns from changing old records).

Anyway, I (preferrably) need to transfer users to upgraded system on the same hardware (after os upgrade). And I really want to replace all these older rhel/centoses with centos 6. Would be good if cPanel devise checked intructions/plan for painless transfer like this (w/o touching DNS and IPs). I think many people will want to migrate from centos 5 as it became 'final'. But I have even older, centos4 (these even with unsupported version of cPanel, sigh). So long wanted to upgrade all this, but can't...

I also don't want to migrate (i.e. copy in usual way) users from server A to server B, and then back to A after A's os is upgraded, because people don't like these movements (downtimes and mess with dns updates, which not always even possible if dns is external). Ideally all this migration should be done in one night per ~200 users server (I dont even believe in 2 hours like Bluehost could promise).

ps. Why need to 'protect dns from changing old records' -- some domains could have changed its IP (migrated to other hosting, but dns left here), when I copy(ed) account cPanel assign(ed) new IP to these domains which is very wrong. Then I search manually which IPs is changed wrongly and revert them manually, mess.

 

cPanelMichael, thanks, I posted feature request, but I don't see it appearing in the feature requests list. So I post its text here:

Maybe someone can re-post it or support it by voting up (if it appears).

- - - Updated - - -

cPanelMichael, btw link to feature request system on your page is not working.
You gave link to page http://forums.cpanel.net/cpfeatures.php which links to http://go.cpanel.net/iwant which does not work.
Correct link seems to be http://features.cpanel.net/

 

It seems that feature request is not accepted...

 

No, I didn't get any message. I don't rememebr exact request title (except it had word centos in it) and I forgot to save it either. (As I didnt expect there will be problems with it.) But its text is fully quoted in #6.

 

cPanelMichael, Infopro, Thanks! I see that feature request is appeared.

If anyone else want to support it - please vote or comment!