Auto Restore Backups

[Shane3673]

I am setting up a CPanel server for my company. We have been using InMotion to host our websites and email for other businesses. Due to network related issues and since we are an internet service provider it only made sense to bring the server in house. I have two servers in which I am trying to make the second one a fail-over. They both use the same public IP's and have different private IPs. This allows me to switch to the 2nd server using routing protocols which is almost instant vs using DNS to switch to the other server. I had read a lot on mirror 2 CPanel servers without much luck. I have tried using Rsync and it halfway works. It will transfer stuff, but if also messed up some things because of files being transferred that should not be.

I option I am looking into now is having server1 auto backup daily which I have setup already, using a cron job to Rsync the backups to server2 daily, and having server2 auto restore the backups daily. I haven't found any option in WHM to auto restore so I would probably have to do a cron job for that also.

I am not fluent with Linux, just able to figure it out as I go. Can somebody help me with a auto restore solution?

 

[cPanelMichael]

Hello

You could setup a cron job that utilizes the following script:

/scripts/restorepkg

This is the script that is used to restore accounts. Note that you may need to adjust the command so that it restores all of the accounts. EX:

for x in $(cat /root/list_of_users); do /scripts/restorepkg --force $x ;done

Thank you.

 

[Shane3673]

Okay, so how I have it setup right now is I have a cron that says
30 01 * * * /scripts/cpbackup - This triggers the backup at 1:30am everyday on server 1 (it seems to use the legacy backup which works for me because I had issue with the new backup not including account when I told it to)
30 02 * * * rsync -v -r --delete /backup/ (server2ip)sitename-/)::/backup - This syncs the backups from server 1 to server 2 and puts the backups in backup. SInce it uses the legacy the backups are actually in cpbackup and then daily and monthly.

I ran the code you gave me and it seems to have went trough with out any errors. I just want to make sure that this would be backing up from /backup/cpbackup/daily so that way it gets the correct information restored

 

[Shane3673]

Sorry if this question comes through twice. It didn't show up the first time.
Using this script where does it look for the backups?
I need it to resotre the backup from /backup/cpbackup/daily/.
I also need it using the legacy restore with selections:
recreate account-false
give ip address-false
restore subdomains-true
restore mail config-true
restore mysql-true

 

[Shane3673]

Okay I fixed the new backup system so that is also transferring, but I would still like for it to auto restore the newest backups somehow. As far as I can tell this cron job searches the home folder and restored a cpmove backup that was created when I was first putting the account on server2.

 

[cPanelMichael]

The following thread should explain the /scripts/restorepkg script so that you are familiar with which flags are available to use:

Full Explain /scripts/restorepkg

Thank you.

 

[Shane3673]

Okay I made a script off of the info you gave me. It works, but one major issue. Here is the script:

Year=$(date +"%Y")
Month=$(date +"%m")
Day=$(date +"%d")

/scripts/restorepkg --force /backup/$Year-$Month-$Day/accounts/user1.tar
/scripts/restorepkg --force --ip=(dedicatedIP) /backup/$Year-$Month-$Day/accounts/user2.tar

I added in the --ip because both users were being set to the shared ip and user2 has a dedicated. When I run this it does not restore user2 because it errors saying the ip address is not available because it recognizes that user2 is already using that ip. I could add line to the script to switch all users to the shared ip before it runs the restores.

Any ideas?

 

[Shane3673]

Nobody has any ideas for this?
I tried looking for a solution for using a script to switch the IPs to the shared IP before doing the restore, but have found that this is not an easy task and can really screw everything up.
Is there a way to trigger the script or code that the WHM uses to switch accounts IPs or restore backups using the new backup system? Restoring with the new backup system does not affect the IP and leaves it on the IP that is already set for the account.
Maybe I am off on my thinking, but wouldn't the WHM have to have some script or set of code to perform these tasks? And if yes, why would we not be able to use these scripts or code through the command line?
The restorepkg script seems to be only for the task of migrating an account from one server to another using a cpmove file. In order to me to use this as an option I would have to write a script to delete all accounts on the server before restoring the backups and I'm sure I would run into quite a bit of issues with that.
It doesn't seem like there is a good way of having a fail-over server using CPanel other than using RSync which I am told is not supported by CPanel and there is alot to configuring this because of having to figure out exactly which folders/files would need to transfer without affecting necessary differences between the servers.
Right now I am forced to manually restore the packages every morning and I also have Rsync syncing the email every 5 minutes to ensure the email is up to date by 5 min intervals in case a switch over needs to happen.

 

[cPanelMichael]

It seems like you may be more interested in a mirroring solution. Currently, there are no native options that allow for mirroring of your websites. I recommend searching our forums for "mirroring" and "high availability" for custom solutions that other users may have implemented. There are lots of discussions about the best ways to do this. EX:

Load Balancing Between 2 cPanel Servers
Load Balancing
Load Balancing Servers
cPanel Load Balanced Servers

Also, feel free to add your input to the following active feature requests:

Active Redundancy or High Availability
Build in Load Balancing

Thank you.

 

[Shane3673]

Rsync and Email

I have been trying to sync email from one server to another server with the same domains on it using rsync. The problem I am having is, if I use -a it tells me 'no cur dir' in the mail client after running, if I use different options I always get -ERR Can't open message file - it's gone!. I am not using --delete so whatever file "is gone" shouldn't actually be gone. I am not copying permissions or owner. The options I have been testing with are -vrlHKu. Also I am syncing the whole /home/user/mail/ folder. Has anyone been able to sync mail successfully using rsync? If so, how?

 

[cPanelMichael]

Re: Rsync and Email

Hello

Are you attempting to copy the files one time only, or are you attempting to copy the files over via Rsync on a regular basis? Have you checked the permission/ownership values on the destination server to ensure they match the standard values?

Thank you.

 

[Shane3673]

Re: Rsync and Email

I am trying to do it on a regualar basis and it worked for a bit, but after adding domains that are more in use it started this. Currently for testing, I am not doing it on a regular basis and just trying to find something that at least works once so I can implemet that into something on a regular basis.

I have narrowed it down to the 'cur' folders. Transferring the contents in these folders is what causes the error.

How do I test the perrmisson/ownership values and what should they be?

Also if I rsync using -a or -o to transfer the ownership from server 1 I get the "no 'cur' directory" error.

- - - Updated - - -

Also, in order to fix the accounts after trying the only way I have been able to is to terminate the account and restore it with a cpmove file

 

[cPanelMichael]

Re: Rsync and Email

You can review the contents of the "cur" directory and it's files within for a working email account using the "ls -al" or "stat" command. A list of options available for rsync and their description is available at websites such as Linux - Rsync Manual or by typing:

man rsync

Thank you.

 

[Shane3673]

Re: Rsync and Email

I got it. I used a similar script that runs on the second server using chown to give ownership the the right account. Thanks for the help.

 

[Shane3673]

Just in case anyone is interested in how I finally got as close as I could.

I have backups daily running on server 1.

cron job on server 1 to transfer the backups daily
30 03 * * * rsync -v -r --delete /backup/ xx.xx.xx.xx::{site}/backup > /dev/null 2>&1

cron job on server1 to transfer mail every 5 mins
*/5 * * * * for x in $(cat /userlist.txt); do rsync -vrlptgDuo --delete /home/$x/mail/ xx.xx.xx.xx::{site}/home/$x/mail/ ;done > /dev/null 2>&1

cronjob on server 2 to corrrect permissions after every mail transfer
2-59/5 * * * * for x in $(cat /userlist.txt); do chown -R $x /home/$x/mail/ ;done > /dev/null 2>&1

This gives me backups on both servers with any changes made to CPanel settings on the active server on both servers so it can be restore when changes have been made. It also transfers mail every 5 mins so if I need to quickly switch over to the other server using OSPF routing (2 min switch over) the mail on both servers will be 5 mins up to date. I had to do the chown because I was running into error with mail client because of the permissions of the emails that were rsynced so it gives it the correct owner every 5 mins on the 1s and 6s to give a min for the transfer.

This is the closest I have come to a full fail over Cpanel.