Any SCP heroes here?

[Data 1]

I'm transferring about 250G of files from a non cpanel server to a cpanel server.
I'm doing a SCP server to server transfer, going well.

My question is... can you open multiple sessions and transfer different folders during an open session?

If you can... do you gain anything?

Thanks-

Jim

[cPanelEric]

Howdy,

Sure you can have more than one transfer going, but if you have large files the data transfer rates will be split between them.

When you have small files scp can take some time starting files and checking them after it's done with them. So you might gain some by doing a multi transfer there. But the way around this would be to tar/compress the files into a single file and unzip on the other side.

Also multi-session helps when you're doing bi-directional transfers. Say you're downloading and uploading at the same time. Two connections is the way to go.

[Data 1]

Thank you cPanelEricE for the response.

You really think I should try and compress 250G of info? I'll try it with a folder I know is about 9G and see what I net.

Thank you again, I have never started 2 SCP sessions in my life and didn't know if I was going to kill the existing one or not.

Jim

[Data 1]

Wow I have 5 sessions started and no noticeable drop in transfer speed on any of them.

Thanks for the tips- I'm going to keep opening them till I'm out of folders!

Jim

[cPanelEric]

Howdy,

I'd try using rar or split and make several moderate sized files that you can move over while keeping an eye on the md5sum hash to make sure the data is move across accurately.

[Data 1]

This ended up being much deeper than it looked, still working on it. If I don't die of old age before I finish the project I'll post the quite interesting resulting saga.

Tens of thousands of error log files. So large I couldn't view them. After 3 days SCP running 4 windows I gave up and started asking why.

Thanks-

[Data 1]

If anyone doesn't have a life and is following this...

The files ended up being not needed. It was from a misfiring phpBB3 install that put 4 cached sql queries in the folder EVERY TIME ANYONE VISITED. So you can imagine the exponential gain in files. I never did determine exactly how many files were actually there but it was so many that "rm" did not work. Returned an error.

I ended up using "find . -name "sql_*" -exec rm -rf {} \;" and after about a half day it did delete them all.

[Lyttek]

If anyone doesn't have a life and is following this...

Great.... another post confirming that I don't have a life