Transferring large sites

dc2447

Well-Known Member
Apr 18, 2003
49
0
156
Can the transfer accounts function tranfer largish sites (20GB of images + 3 GB Mysq)?
 

aby

Well-Known Member
May 31, 2005
638
0
166
India
dc2447 said:
Can the transfer accounts function tranfer largish sites (20GB of images + 3 GB Mysq)?

I doubt the apache may not be able to keep the session for that long.

-----
Aby,
 

webignition

Well-Known Member
Jan 22, 2005
1,876
2
166
I can't see any reason why the account transfer process would fail, however the larger a site is the longer it will take and the longer it takes the greater the risk of some unwanted factors entering the equation.

It all really depends on the speed of the connections to the two servers. If both servers are in the same datacenter on the same local network you may be able to transfer the 23GB or so quite straightforwardly. If the servers are in different datacenters then you'll be limited by the servers' respective Internet connections.

If the servers are in different data centers you might be better off moving the images manually and then kick in the account transfer process. As far as I know you should be able to do the following:

1. Download, however you like, the images from server 1
2. Create an account on server two that mirrors that on server 1 with respect to the domain name, username and password.
3. Pop the images into the correct directory in the new account on server 2.
4. Remove the images from the account on server 1 (probably don't delete them, just move them outside of the relevant account's home directory)
5. Use the WHM account transfer process to move the account properly from server 1 to server 2

With the images moved beforehand there will be considerably less data for the account transfer process to handle and 3GB of mysql data should compress quite nicely.
 

dc2447

Well-Known Member
Apr 18, 2003
49
0
156
webignition said:
I can't see any reason why the account transfer process would fail, however the larger a site is the longer it will take and the longer it takes the greater the risk of some unwanted factors entering the equation.

It all really depends on the speed of the connections to the two servers. If both servers are in the same datacenter on the same local network you may be able to transfer the 23GB or so quite straightforwardly. If the servers are in different datacenters then you'll be limited by the servers' respective Internet connections.

If the servers are in different data centers you might be better off moving the images manually and then kick in the account transfer process. As far as I know you should be able to do the following:

1. Download, however you like, the images from server 1
2. Create an account on server two that mirrors that on server 1 with respect to the domain name, username and password.
3. Pop the images into the correct directory in the new account on server 2.
4. Remove the images from the account on server 1 (probably don't delete them, just move them outside of the relevant account's home directory)
5. Use the WHM account transfer process to move the account properly from server 1 to server 2

With the images moved beforehand there will be considerably less data for the account transfer process to handle and 3GB of mysql data should compress quite nicely.
This is intresting - sounds like an excellent way to go however if I have laready created ab account on thye new server will I be able to transfer in from the old server? I kind of assumed that you would be transferring into a 'new' account, if you transfer into an existing accounjt that would be very cool.

Are we sure it transfers over HTTP? Surely it would use rsync or similar?
 

aby

Well-Known Member
May 31, 2005
638
0
166
India
dc2447 said:
Apache way? You don't mean it transfers over HTTP do you?

Oops I meant the server. :eek:
In my experience sites greater than 20 GB using the WHM transfer functin is not easy , unless you go for indirect means lke the one like copying the images separately.

----
Aby,
 

romanus

Well-Known Member
Jul 17, 2004
68
0
156
I have been using the /scripts/pkgacct username with a bit of success, then I restore the account with a cprestore. I had a problem trying to transfer with the account copy function
 

brianoz

Well-Known Member
Mar 13, 2004
1,146
7
168
Melbourne, Australia
cPanel Access Level
Root Administrator
I'm fairly sure the transfer process is limited by the size of /tmp on the source (from, or old) machine.

Unfortunate, but I don't think there's a way around it. Many hosts seem to have small /tmp filesystems - 200-500Mb instead of a more decent 1Gb.

Wget works wonders for large accounts!

Or, another possible solution: What I've done in the past is to transfer all the large files manually to the new server (FXP or SCP or login to the target machine and use ftp to pull files down). Once you've checked all the large files are transferred delete them from the source machine, continuing this copy-check-delete cycle until what's left falls under 100-200Mb then use WHM 'copy account' to transfer the rest of the account.

Remember copyaccount doesn't copy mailman lists ....
 

romanus

Well-Known Member
Jul 17, 2004
68
0
156
I have split up large files and then used cat to put them back together

I think the transfer limit in linux/pureftp is less than 2gb

I had to split a 7 gb file into 6 files before I could transfer it.

/scripts/pkgacct

man split

man cat

or search on google for exact syntax
 

amal

Well-Known Member
Nov 22, 2003
155
0
166
India
cPanel Access Level
Root Administrator
Support the split option..

Yeah, Split is very special, especially, when we do not have root access on the destination server.. :)

if you want to split a file - backup.tar.gz into smaller files of max size of 1000000 bytes, just use the command

split -b 1000000 backup.tar.gz

This will create files like xaa, xab, etc..

bash-2.05b$ ls d3DNA_Loft_110.exe xab xad xaf xah xaj xal xan xap xar xat xav xaa xac xae xag xai xak xam xao xaq xas xau xaw
After moving these files, one by one, you can use the cat command to join them back...

You can use

cat x?? > backup.tar.gz

to join them again, after transferring each of them... :)