The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Consolidating multiple servers into one during upgrade/migration

Discussion in 'General Discussion' started by porcupine, Apr 1, 2016.

  1. porcupine

    porcupine Well-Known Member
    PartnerNOC

    Joined:
    Apr 18, 2002
    Messages:
    74
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Toronto, Ontario
    cPanel Access Level:
    DataCenter Provider
    Ok, so I figured I'd ask the forum on this one, in case anybody has run into this.

    We've got some old CPanel shared/reseller hosting servers. They're nearing the end of their useful lives (CentOS 5.11 going EOL and SSL support issues, etc.), and we're doing a forklift upgrade to CentOS 7.1, new hardware, etc.

    The population on these servers has dropped since their original deployments, and the new hardware we've spec'd is many multiples more powerful (much due to SSD's vs HDD's). We're looking at a 3:1 consolidation, and still having oodles of space/processor/etc. left over. We don't really want to drop VM's into play, as it'd be nice to reduce management overhead.

    Typically when doing moves like this in the past, we would deploy the new hardware, install OS/CPanel/perform basic setup, then rsync all of the content over (using the old cpanel backup script as a reference for what to fetch). We'd rsync a number of times in advance, then on the day it came to actually move, we'd shut down services on the old (namely MySQL and mail), and do one final sync to catch everything before shifting IP's/routing/etc. as necessary. We had several big moves with < 5 minutes of downtime from this method.

    In this case, since we want to do a 3:1 consolidation, that's not going to work. The problem is, we don't plan to migrate to new IP's (which seems to be what WHM centers around last I looked), as a slew of customers have custom NS's, setup, and there will literally be a good number of IP's per server once the dust settles (from the SSL, private shared IP's, nameservers, etc.)

    What have you folks found works best for this sort of stuff? I realize we can use the pkgacct and restorepkg scripts to pull accounts over [and even set IP's], etc. (but then we'd have some work cut out to ensure every account restored, or got migrated back to their proper IP's [IE: what happens with addon/parked/sub domains in that scenario?], etc.). We've already scanned for conflicts in terms of identical users across the machines we plan to collapse together, etc.

    The least amount of downtime possible is preferable, though I'd be keep to do 3 servers a night (aka 1 finished server per night), and hopefully be as clean as possible.... Wouldn't we all?
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Hello :)

    The "Transfer Tool" option in Web Host Manager is the suggested method of copying accounts to the destination server:

    Transfer Tool - Documentation - cPanel Documentation

    Addon/Parked/Subdomains are included in transfers and backup archives by default. Do your addon/subdomains have their own assigned IP addresses, separate from the IP address assigned to the account they are associated with?

    Thank you.
     
  3. porcupine

    porcupine Well-Known Member
    PartnerNOC

    Joined:
    Apr 18, 2002
    Messages:
    74
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Toronto, Ontario
    cPanel Access Level:
    DataCenter Provider
    Hard to say, that depends entirely on what the customers setup I suppose, though I recall it was difficult (if not impossible) to assign different IP's to subdomains/addon domains/parked domains VIA CPanel awhile back (as we had a customer trying to setup SSL on an addon domain, and there was no practicable manner to separate it off to a dedicated IP).

    My concern is we don't intend to do an IP migration here (the new server will take over the IP's from the old server), and IIRC, the transfer tool cannot directly accommodate that (IE: it'll renumber into new IP space arbitrarily on transfer), correct?

    I realize we can do a mass "change multiple sites IP addresses" after the migration, but if there's an easier way/cut down on steps/errors/etc., it's preferable obviously.
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    The "IP" flag is available if you use the "/scripts/restorepkg" command to restore the account from backup archives. EX:

    Code:
    /scripts/restorepkg --ip=192.0.30.10 cpmove-testaccount.tar.gz
    However, it's only helpful if that IP address is already added to the destination server. Are you able to add those IP addresses to the destination server beforehand without negative effects to IP propagation on that network?

    Thank you.
     
  5. porcupine

    porcupine Well-Known Member
    PartnerNOC

    Joined:
    Apr 18, 2002
    Messages:
    74
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Toronto, Ontario
    cPanel Access Level:
    DataCenter Provider
    I had run across this, and it was high up on my list of potentially good candidates for the move (going by the restorepkg tool VIA CLI instead of WHM).

    I can operate the primary interface for the new server on a temporary IP, then add the actual/proper IP's onto the server as additional IP's, having the server in a VLAN that simply cannot route the additional IP's (so it can add accounts, but the IP's will not have conflicts, etc. until it's swung over).

    The problem is, all of the servers are currently on the same VLAN (they share a /24 to allow IP's to allow swapping accounts/IP's among 'em, etc.). This means with a 3:1 consolidation, I can do this for the first source server (to shuffle everything over, and bring it fully online), but after this, source servers #2-3 need to be down before the replacement server starts restoring the accounts (as the replacement server will need to be in the same VLAN for the IP's from the first source server to be usable/changes to be effective/validation to occur, which will create a conflict).

    This also means since I'd be doing several hundred accounts VIA the CLI as fast as possible, it's entirely likely we'd miss any errors/etc. (as it'd be a simple bash script with a few hundred lines for the accounts/IP's/etc.). This could be caught by reviewing the logs (in real time or after the moves)... But that's a slippery slope in most scenario's, when dealing with massive logs, etc.

    The other potential is to write out the entire script for all 3 source servers, gather all of the backups from all of the servers, and blast out the restorepkg action blind/all at once (IIRC: the cpmove files are identical to the regular CPanel backup files). I assume doing multiple restores in parallel (IE: to expedite) is a horrible idea and should be avoided? And/or the restore process creates a lock file as it goes to prevent this sort of shenanigans?

    Obviously I want there to be minimum time between when the backup was made, and when it was restored, to ensure DB entries, emails, etc. are not duplicated/lost/abandoned on the old servers/all the fun you get when doing moves like this.
     
  6. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    Based on the information you have provided, my advice is to transfer the accounts to new IP addresses and then handle the IP change post-migration once you are ready to activate the accounts on the destination server. The "IP Migration Wizard" is likely the fastest way to do this:

    "WHM Home » IP Functions » IP Migration Wizard"

    I've seen instances where users update the main shared IP address via "WHM Home » Server Configuration » Basic cPanel & WHM Setup" to a different IP address each time they transfer from a different source server. This way, when using the "IP Migration Wizard", it's easy to categorize which accounts came from a specific server.

    Thank you.
     
  7. porcupine

    porcupine Well-Known Member
    PartnerNOC

    Joined:
    Apr 18, 2002
    Messages:
    74
    Likes Received:
    0
    Trophy Points:
    6
    Location:
    Toronto, Ontario
    cPanel Access Level:
    DataCenter Provider
    I haven't used that tool in many many years. The last time we ran it (again, *many* years back, nearly 8-10 I believe), there was substantial leave-behind of old junk on the IP's which were no longer in use/bound to the system/etc.

    I assume it's improved since then?

    How is this different than the "change multiple sites IP addresses" tool? I remember the "IP Migration Wizard" was setup to stage the move, old/new IP's in parallel/etc. however that's not a desirable feature in this instance.
     
  8. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,854
    Likes Received:
    676
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
    You may want to use the "Change Multiple Sites' IP Addresses" instead of the "IP Migration Wizard" if converting accounts from certain IP addresses to different ones in parallel is not a required feature. Both options will update the IP address of the account, but the "IP Migration Wizard" will update server configuration files, which is not a feature you will need based on the previous response.

    Thank you.
     
Loading...

Share This Page