RSYNC to update files between old and new server?

Arkaic

Active Member
Jun 23, 2015
34
5
58
United Kingdom
cPanel Access Level
Root Administrator
Hey there!
We're looking to migrate an account which is 2TB to a new server, due to the size of the account it takes approx 6 hours to complete.

We had planned to use the Transfer Tool to migrate the account during the day and then do a final rsync at an off-peak time to catch any latest files or changes that might've been made since the initial Transfer started.
We can take care of MySQL by re-running the Transfer Tool excluding the home directory at the off-peak time before the IP address switch over to ensure database changes are up to date.

Has anybody done something like this before?

What sort of rsync command should we be using to recursively catch and carry over and overwrite if applicable NEW / MODIFIED files within a users /home/*cpanelaccount* directory whilst retaining correct permissions + ownership?
Or any alternative method of replication would be appreciated.

Thanks in advance.
 
Last edited:

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,297
1,259
313
Houston
This is not unusual for people to do, especially with large accounts. Something like the following may be helpful:

Code:
rsync -auvH -e “ssh -p $port” /home/cpmove-$username.tar.gz [email protected]$sourceIP:/home/cpmove-$username.tar.gz
Explanation of the flags noted in the table below
  • -a is the equivalent of using -rlptgoD so I included those flags as well for full disclosure

Flag

Fulltext
Description
-v--verboseIncrease verbosity
-r--recursiverecurse into directories
-p--permspreserve permissions
-l--linksWhen symlinks are encountered, recreate symlink on the destination
-o--ownerpreserve owner
-g--grouppreserve group
-D--devices --specialspreserve device files and special files
-t--timespreserve modification times
-H--hard-linksthis tells rsync to look for hardlinked files in the transfer and link together the corresponding files on the receiving side. Without this option, hard-linked files in the transfer are treated as though they were separate files.
-e--rsh=COMMANDspecify the remote shell to use (if the remote host is listening on something other than 22)
-a--archive archive mode (equivalent to -rlptgoD. This tells rysnc to sync directories recursively, transfer special and block devices, preserve symlinks, modification times, groups, ownership, and perms.
-u--updateskip files that are newer on the receiver (destination server)

Some other options you may want to look at using depending on your situation:

FlagFulltextDescription
-z--compressThis option forces rsync to compress the data as it is sent to the destination machine. Use this option only if the connection to the remote machine is slow.
-P--partial --progressWhen this is used rsync shows a progress bar during the transfer and keeps the partially transferred files. Useful for transferring large files over a slow or unstable network connection.
 
Last edited:
  • Like
Reactions: Arkaic

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
3,186
398
243
cPanel Access Level
Root Administrator
@AzeDK

Here's the examples I always provide for rsync, with various possibilities:

From a remote server to the local server:

Code:
rsync -avh [email protected]:/path/on/remote/server /path/on/local/server
If you need to use a custom SSH port:

Code:
rsync -avz -e "ssh -p $portNumber" [email protected]:/path/to/files/ /local/path/
From a local server to a remote server:

Code:
rsync -avh /full/path/to/directory [email protected]:/destination/directory/on/remote/server
Following those examples, Lauren's is correct for the example she was providing, as that matches my custom port example as well.
 
  • Like
Reactions: AzeDK