Transfer tool: Time out issues

ASG

Member
Jun 8, 2016
16
0
1
India
cPanel Access Level
Root Administrator
Source:
Centos 6.8
WHM 56.0

Destination
Centos 7.2
WHM 56.0

I am moving a 30GB website to a new server.

At about 12% through, received the following errors, after which the transfer aborted.
Code:
___________________________________________
[INDENT]Verifying cpmove file checksum …
[  3065][TRANSFER:1][A:home]: Checksum Failed: The file transfer was not successf  ul!
[  3065][TRANSFER:1][A:home]: Expected Checksum: a9500d8170931512c3f2ee60839f5cc1
[  3065][TRANSFER:1][A:home]: Actual Checksum: f7f3b7fec4aa4884add90b917cdf9c00
[  3065][TRANSFER:1][A:home]: Failed to validate cpmove file.
[  3065][TRANSFER:1][A:home]: Retrying transfer.
[  3065][TRANSFER:1][A:home]: Attempt #2 to transfer using “SCP” method.
[  3065][TRANSFER:1][A:home]: Copying account package file … …
[  3065][TRANSFER:1][A:home]: Progress 37%
[  3065][TRANSFER:1][A:home]: Done
[  3065][TRANSFER:1][A:home]: Verifying cpmove file checksum …
[  3065][TRANSFER:1][A:home]: Checksum Failed: The file transfer was not successf  ul!
[  3065][TRANSFER:1][A:home]: Expected Checksum: a9500d8170931512c3f2ee60839f5cc1
[  3065][TRANSFER:1][A:home]: Actual Checksum: f7f3b7fec4aa4884add90b917cdf9c00
[  3065][TRANSFER:1][A:home]: Failed to validate cpmove file.
[  3065][TRANSFER:1][A:home]: Retrying transfer.
[  3065][TRANSFER:1][A:home]: Multiple copy failures, switching to verbose mode a  nd trying one final attempt.
[  3065][TRANSFER:1][A:home]: Attempt #3 to transfer using “SCP” method.
[  3065][TRANSFER:1][A:home]: Copying account package file … …
[  3065][TRANSFER:1][A:home]: ERROR: The system experienced a timeout error while  it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Retrying …
[  3065][TRANSFER:1][A:home]: Progress 12%
[  3065][TRANSFER:1][A:home]: ERROR: The system experienced a timeout error while  it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Retrying …
[  3065][TRANSFER:1][A:home]: Progress 12%
[  3065][TRANSFER:1][A:home]: ERROR: The system experienced a timeout error while  it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Done
[  3065][TRANSFER:1][A:home]: [The system experienced a timeout error while it at  tempted to connect to “43.xxx.xx.xx”: timeout]
[  3065][TRANSFER:1][A:home]: Connecting to 43.xxx.xx.xx:22 as root...using metho  d password...Connection Success
[  3065][TRANSFER:1][A:home]: Downloading with method “SCP” failed: The system ex  perienced a timeout error while it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Removing copied archive on remote server. …
[  3065][TRANSFER:1][A:home]: ERROR: The system experienced a timeout error while  it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Retrying …
[  3065][TRANSFER:1][A:home]: ERROR: The system experienced a timeout error while  it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Retrying …
[  3065][TRANSFER:1][A:home]: ERROR: The system experienced a timeout error while  it attempted to connect to “43.xxx.xx.xx”: timeout
[  3065][TRANSFER:1][A:home]: Done
[  3065][TRANSFER:1][A:home]: [The system experienced a timeout error while it at  tempted to connect to “43.xxx.xx.xx”: timeout]
[  3065][TRANSFER:1][A:home]: Connecting to 43.xxx.xx.xx as root...using metho  d password...Connection Success
[  3065][TRANSFER:1][A:home]: Normalizing root shell....
[  3065][TRANSFER:1][A:home]: Done normalizing shell.
[  3065][TRANSFER:1][A:home]: zz c_17681105_P c_49000241P zz
[  3065][TRANSFER:1][A:home]: c_82944874_P__==sshcontroloutput==Cpanel::SSHContro  l::fqwq6dBALqCTI81e==
[  3065][TRANSFER:1][A:dressify  ]: __c_92243184P___c_26312687_P__c_40620623P___
[  3065][TRANSFER:1][A:home]: Progress: 100%
[  3065][TRANSFER:1][A:home]: Failed: Unable to download “/home/cpmove-xxx-s  plit/cpmove-dressify.tar.gz.part00011” from the remote server.
[  3065][TRANSFER:1]: Account “home”: Unable to download “/home/cpmove-xxx-split/cpm  ove-home.tar.gz.part00011” from the remote server.
[  3065][TRANSFER:1]: Progress: 100%
[  3065][RESTORE:1 ][A:home]: Starting “RESTORE” for “Account” “home”.
[  3065][RESTORE:1 ][A:home]: Failed: Unable to download “/home/cpmove-xxx-s  plit/cpmove-home.tar.gz.part00011” from the remote server.
[  3065][RESTORE:1 ]: Account “home”: Unable to download “/home/cpmove-xxx-split/cpm  ove-home.tar.gz.part00011” from the remote server.[/INDENT]
___________________________________________
I am able to login to root via SSH, so I am not sure what exactly has caused this error. Any ideas?

Is it possible to run this installation from where it left off, since it has already taken 3 hours to get to where it was.

Secondly, it looks like the transfer tool creates split archives of ~256 GB on the source server before these are moved to the new server. In my case, the number of these archives could very well be around 125. To save time which is used to creating these archives is it possible to tell the server to have archives which are larger in size, and hence more time is spent moving those, rather than processing and splitting the data.
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,884
2,250
463

ASG

Member
Jun 8, 2016
16
0
1
India
cPanel Access Level
Root Administrator
Hello,

Unfortunately I had to workaround by reducing the size of the movable files. There was a beta domain & its files which I removed, thereby reducing the entire size of movement by about 50%.

I am still not sure why the timeouts occured, because on the second attempt after the above post the transfer was completed in about 3 hours.

I would be kind of apprehensive on moving a larger site, if I need to do so in the future. Transfer resumption from where it left off/failed should really be an option. I cannot imagine my plight had this been a 1TB movement.

Also because a site heavy with images, usually does not benefit from compression and packaging since images are already web-optimized with almost no scope of size reduction on packaging. CSS,JS or any other code or content files contribute less than 10% of the total disk space for any site these days.
 

NOC_Serverpoint

Well-Known Member
Jul 3, 2016
103
7
18
cPanel Access Level
Website Owner
Hello :)

The better option to transfer a large website is to manually transfer the skeletal of account without home directory to the remote server and then deploy the package at the remote server to create a new account.

Once the new account is created at the other end, open screen in the ssh terminal in the source server and put the compressed rsync process or a normal rsync process over the source server to the destination server. This rsync process won't be failing unless and until the screen is terminated / the server is rebooted.

As long as you have space available (and it shouldn't take much depending on the size of the accounts databases and email accounts), you can do this:

/scripts/pkgacct --skiphomedir username

This will backup the cpanel account settings, email, mysql dbs, etc, etc - and create an archive at /home/cpmove-username.tar.gz (as the --skiphomedir option suggests, this leaves out all files under /home/username)

Then copy the /home/cpmove-username.tar.gz file to the new server, and run (on the new server) /scripts/restorepkg username

Once that finishes, you need only to use rsync, scp or similar to copy over all the files under /home/username/ to /home/username on the new server & update the DNS.

This seems to be the most efficient way to transfer large (5 GB +) accounts to another server. And it is MUCH faster than doing it via WHM. :cool:
 
  • Like
Reactions: ASG