Mirror accounts to a different server

b0072k1

Well-Known Member
Dec 30, 2004
132
0
166
Glasgow
Hi, AM just wondering would it be possible to mirror all accounts from 1 or 2 servers to a seperate server. So if one of the servers go down a recent back up of the server will auto matically kick in on the mirror server.

Would it be possible to do and if so how?
 

boylizard

Registered
Jul 25, 2005
1
0
151
Like an automated cpanel backup system, with an ip failover (i think thats what its called) method?
 

tanfwc

Well-Known Member
Mar 12, 2004
94
0
156
b0072k1 said:
Hi, AM just wondering would it be possible to mirror all accounts from 1 or 2 servers to a seperate server. So if one of the servers go down a recent back up of the server will auto matically kick in on the mirror server.

Would it be possible to do and if so how?
With cPanel, I think it is not possible. If you have custom webserver & database, you can use hardware load-balancing
 

chirpy

Well-Known Member
Verifed Vendor
Jun 15, 2002
13,437
31
473
Go on, have a guess
Indeed. A hardware solution was posted a while backup, but you're talking thousands of dollars of investment to achieve that. You may be able to script something, but it would not be straightforward and would probably require significant work.
 

expedio

Active Member
Jun 30, 2007
36
0
56
You can now have servers that do the same. We have designed clusters that mirror data across multiple servers and switchover traffic to other server in array when first server is down. This process does not involve any IP change or DNS propagation.
 

freedman

Well-Known Member
Feb 13, 2005
314
5
168
Hi, AM just wondering would it be possible to mirror all accounts from 1 or 2 servers to a seperate server. So if one of the servers go down a recent back up of the server will auto matically kick in on the mirror server.

Would it be possible to do and if so how?
We've got all our systems in a round-robin mirrored pair configuration.

the quick answer is:
run mysql using master-master replication... this will address the issue of the database being synced... it is plenty fast and we've run into no problems.

now, you have the issue of filesystem synchronization.
if you are managing your own servers you have some options.. one is to use data (/home) from an NFS server. one is to use shared storage and linux cluster file system (this is pretty risky in my opinion)... or.
the option we're currently using, is to use Unison to sync the filesystems. Start by using rsync or scp to get a copy of the filesystem on the new machine.. then use unison to sync the filesystems. we do this every 15 minutes which seems to be fast enough for most uses but not so frequent as it consumes lots of ressources (it does take quite a while the first run, however).

put both server IP's in DNS and it will serve the ips round-robin and both servers will be accessed.. if one goes down, the users browser will just try the other IP.

now, there are a few problems:
in general, filesystem data doesn't change all that much on most web servers.. but.. when people update their web-pages, the changes aren't propogated to the other server for up to 15 minutes... this may cause some confusion.
Also, if they're updating files through the web, they may be updating on both servers at the same time.. this causes problems.
we add an 'admin.domain.com' which points to only one of the servers, and recommend people do their ftp updates or other filesystem related updating through that, so that it's always done on one server (ftp is a cname for admin).

admin is also the lowest mx while the other server is a secondary mx.

unison also syncs the appropriate files in /etc/ which relate to ftp, email, etc.

our solution isn't really viable yet for large scale cpanel environments since we havn't addressed adding/removing accounts.. as it is, the httpd.conf has to be updated manually when adding or changing an account.
removing accounts isn't so bad, just have to run the rmove script on both servers and it cleans everything up, but we're waiting for cpanel 11 and apache2 to stabilize before we bother moving ahead.

so, since we dont host thousands of hosts with people signing up all day long, this works for us, it may work for you, but, as has been stated.. really robust solutions cost money.
 

utropic

Well-Known Member
Aug 6, 2005
60
0
156
put everything from:

/home
/usr/local
/etc
/var

on a SAN - we are planning on using iSCSI.

You need to write some "check" scripts that are basically heartbeats to see if a server dies. You can use a software loadbalancer to handle failover.

Clustering cPanel is really not that hard - you don't focus so much on changing any of the software, but making the infrastructure resiliant without changing how it appears to the software.
 

freedman

Well-Known Member
Feb 13, 2005
314
5
168
put everything from:

/home
/usr/local
/etc
/var

on a SAN - we are planning on using iSCSI.

You need to write some "check" scripts that are basically heartbeats to see if a server dies. You can use a software loadbalancer to handle failover.

Clustering cPanel is really not that hard - you don't focus so much on changing any of the software, but making the infrastructure resiliant without changing how it appears to the software.
legally, I think you have to license cpanel on the 2nd server, so if it's just a 'hot failover' you're wasting a license.. might as well use the load balancer to load balance rather than failover.
 

bigdessert

Active Member
Jul 21, 2006
37
0
156
We've got all our systems in a round-robin mirrored pair configuration.

the quick answer is:
run mysql using master-master replication... this will address the issue of the database being synced... it is plenty fast and we've run into no problems.

now, you have the issue of filesystem synchronization.
if you are managing your own servers you have some options.. one is to use data (/home) from an NFS server. one is to use shared storage and linux cluster file system (this is pretty risky in my opinion)... or.
the option we're currently using, is to use Unison to sync the filesystems. Start by using rsync or scp to get a copy of the filesystem on the new machine.. then use unison to sync the filesystems. we do this every 15 minutes which seems to be fast enough for most uses but not so frequent as it consumes lots of ressources (it does take quite a while the first run, however).

put both server IP's in DNS and it will serve the ips round-robin and both servers will be accessed.. if one goes down, the users browser will just try the other IP.

now, there are a few problems:
in general, filesystem data doesn't change all that much on most web servers.. but.. when people update their web-pages, the changes aren't propogated to the other server for up to 15 minutes... this may cause some confusion.
Also, if they're updating files through the web, they may be updating on both servers at the same time.. this causes problems.
we add an 'admin.domain.com' which points to only one of the servers, and recommend people do their ftp updates or other filesystem related updating through that, so that it's always done on one server (ftp is a cname for admin).

admin is also the lowest mx while the other server is a secondary mx.

unison also syncs the appropriate files in /etc/ which relate to ftp, email, etc.

our solution isn't really viable yet for large scale cpanel environments since we havn't addressed adding/removing accounts.. as it is, the httpd.conf has to be updated manually when adding or changing an account.
removing accounts isn't so bad, just have to run the rmove script on both servers and it cleans everything up, but we're waiting for cpanel 11 and apache2 to stabilize before we bother moving ahead.

so, since we dont host thousands of hosts with people signing up all day long, this works for us, it may work for you, but, as has been stated.. really robust solutions cost money.


what files in /etc do you copy that relate to mail,ftp, etc.?
 

freedman

Well-Known Member
Feb 13, 2005
314
5
168
what files in /etc do you copy that relate to mail,ftp, etc.?
here's the unison command to sync the necessary /etc files.
Unfortunately, passwd,group,shadow have to be handled manually -- it's not safe synching them in an automated fashion and we havn't gotten around to writing a script to sync the updates.

unison /etc ssh://$remote//etc -owner -group -perms -1 -path valiases -path vdomainaliases -path vfilters -path vftp -path vmail -path proftpd -path localdomains -path userdomains -path remotedomains -path secondarymx