FTP over SSL fails with timeout (but Plain FTP connects)

[foolonthehill]

I am currently having a nightmare trying to connect to a cPanel server over FTP. The client connects perfectly happily over plain FTP, but fails as soon as a data channel is opened when using SSL.

Logs from lftp with/without SSL below. I seem to get sent to a similar port range from PASV, but the MLSD (or LIST) packet never seems to get a response when sent in the encrypted session. I have tried FileZilla and lftp without success from a number of different clients on different networks, so I can't see it being a firewall issue.

Can anyone suggest where I might be going wrong, or whether there could be a config error on the host. (The only complication, as you will see, is that I do not have a certificate specifically for this hostname, so the verification fails unless I force its acceptance - could this be at fault here?)

Thanks for your help.

[email protected] ~ $ lftp -d
lftp :~> set ftp:ssl-allow no
lftp :~> open -u user,password -p 21 server.co.uk
---- Resolving host address...
---- 1 address found: xx.xx.xx.xx
lftp [email protected]:~> ls
---- Connecting to server.co.uk (xx.xx.xx.xx) port 21
<--- 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
<--- 220-You are user number 8 of 50 allowed.
<--- 220-Local time is now 23:32. Server port: 21.
<--- 220-This is a private system - No anonymous login
<--- 220 You will be disconnected after 15 minutes of inactivity.
---> FEAT
<--- 211-Extensions supported:
<--- EPRT
<--- IDLE
<--- MDTM
<--- SIZE
<--- MFMT
<--- REST STREAM
<--- MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
<--- MLSD
<--- AUTH TLS
<--- PBSZ
<--- PROT
<--- ESTA
<--- PASV
<--- EPSV
<--- SPSV
<--- ESTP
<--- 211 End.
---> OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;
<--- 200 MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
---> USER user
<--- 331 User user OK. Password required
---> PASS password
<--- 230 OK. Current restricted directory is /
---> PWD
<--- 257 "/" is your current location
---> PASV
<--- 227 Entering Passive Mode (xx.xx.xx.xx,160,126)
---- Connecting data socket to (xx.xx.xx.xx) port 41086
---- Data connection established
---> LIST
<--- 150 Accepted data connection
<--- 226-Options: -a -l
<--- 226 32 matches total
---- Got EOF on data connection
---- Closing data socket
drwx--x--x 19 user user 4096 Feb 26 11:52 .
drwx--x--x 19 user user 4096 Feb 26 11:52 ..
drwx------ 2 user user 4096 Feb 12 05:54 .files

[email protected] ~ $ lftp -d
lftp :~> set ssl:verify-certificate no
lftp :~> open -u user,password -p 21 server.co.uk
---- Resolving host address...
---- 1 address found: x.x.x.x
lftp [email protected]:~> ls
---- Connecting to server.co.uk (x.x.x.x) port 21
<--- 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
<--- 220-You are user number 7 of 50 allowed.
<--- 220-Local time is now 23:35. Server port: 21.
<--- 220-This is a private system - No anonymous login
<--- 220 You will be disconnected after 15 minutes of inactivity.
---> FEAT
<--- 211-Extensions supported:
<--- EPRT
<--- IDLE
<--- MDTM
<--- SIZE
<--- MFMT
<--- REST STREAM
<--- MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
<--- MLSD
<--- AUTH TLS
<--- PBSZ
<--- PROT
<--- ESTA
<--- PASV
<--- EPSV
<--- SPSV
<--- ESTP
<--- 211 End.
---> AUTH TLS
<--- 234 AUTH TLS OK.
---> OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;
Certificate: OU=Domain Control Validated,CN=*.differentserver.com
Issued by: O=AlphaSSL,CN=AlphaSSL CA - G2
Checking against: O=AlphaSSL,CN=AlphaSSL CA - G2
Trusted
Certificate: O=AlphaSSL,CN=AlphaSSL CA - G2
Issued by: C=BE,O=GlobalSign nv-sa,OU=Root CA,CN=GlobalSign Root CA
Trusted
WARNING: Certificate verification: certificate common name doesn't match requested host name ‘server.co.uk’
<--- 200 MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
---> USER user
<--- 331 User user OK. Password required
---> PASS password
<--- 230 OK. Current restricted directory is /
---> PWD
<--- 257 "/" is your current location
---> PBSZ 0
<--- 200 PBSZ=0
---> PROT P
<--- 200 Data protection level set to "private"
---> PASV
<--- 227 Entering Passive Mode (x,x,x,x,36,214)
---- Connecting data socket to (x.x.x.x) port 9430
**** Socket error (Connection timed out) - reconnecting
---> LIST
---> ABOR
---- Closing aborted data socket
---- Closing control socket

 

[PeterJBishop]

Have you tried connect via sFTP via the host? (To which the certificate is issued for) - If SSL isn't activated on your server, it's unlikely you'll get through hence the connection timeout.

Sorry if I'm wrong, it's not my area! Hope this helps though.

 

[foolonthehill]

SSL is active on the server as far as I can tell. A certificate certainly is issued, though it does not cover this hostname, so it is just considered "not valid". (I assume this is right, though I do not know much about SSL certificate processes)

Once manually verified (or with verification disabled as above), then the SSL session seems to establish happily on port 21, as shown by the fact that the login goes through ok. It's only when a data connection is opened on another port that the server seems to shut up shop.

 

[PeterJBishop]

This is what I am worrying about:

**** Socket error (Connection timed out) - reconnecting

I get the feeling that connecting via the server hostname will solve it. I think to go sFTP via your own domain, you must install an SSL certificate or configure the hostname SSL to work with said domain.

 

[foolonthehill]

I get the feeling that connecting via the server hostname will solve it.

Unfortunately, I am not able to connect via the server which the certificate is valid for. The site is on a server from a hosting company (whose certificate is the one issued, as there is nothing installed for my site), and there is no certificated hostname (that I know of) which would redirect to the server where my ftp account username is valid.

(Just to be clear, I am talking about FTPS here, not SFTP).

I think to go sFTP via your own domain, you must install an SSL certificate or configure the hostname SSL to work with said domain.

I can only think you are right that the certificate is causing the problem - I have tried everything else!
It just seems so odd that the (forced verify) certificate is valid enough for me to get through login successfully, but that it won't allow a data transfer.

Thanks for your help anyway.

 

[PeterJBishop]

Sorry, I meant FTPs. Its been a tiring few days. Speak to your webhost, they should be able to provide the hostname to which they have SSL installed on. Use that to connect. Fingers crossed that will work.

 

[foolonthehill]

Sorry, I meant FTPs.

No worries - I assume you meant FTPS, but just didn't want to confuse others who might come here.

Speak to your webhost, they should be able to provide the hostname to which they have SSL installed on. Use that to connect. Fingers crossed that will work.

I managed to get hold of the hostname which is covered under the SSL certificate. Still no cigar.

[email protected] ~ $ lftp -d
lftp :~> open -u user,password -p 21 ftp.anotherserver.com
---- Resolving host address...
---- 1 address found: x.x.x.x
lftp [email protected]:~> ls
---- Connecting to ftp.anotherserver.com (x.x.x.x) port 21
<--- 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
<--- 220-You are user number 3 of 50 allowed.
<--- 220-Local time is now 16:38. Server port: 21.
<--- 220-This is a private system - No anonymous login
<--- 220 You will be disconnected after 15 minutes of inactivity.
---> FEAT
<--- 211-Extensions supported:
<--- EPRT
<--- IDLE
<--- MDTM
<--- SIZE
<--- MFMT
<--- REST STREAM
<--- MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
<--- MLSD
<--- AUTH TLS
<--- PBSZ
<--- PROT
<--- ESTA
<--- PASV
<--- EPSV
<--- SPSV
<--- ESTP
<--- 211 End.
---> AUTH TLS
<--- 234 AUTH TLS OK.
---> OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;
Certificate: OU=Domain Control Validated,CN=*.anotherserver.com
Issued by: O=AlphaSSL,CN=AlphaSSL CA - G2
Checking against: O=AlphaSSL,CN=AlphaSSL CA - G2
Trusted
Certificate: O=AlphaSSL,CN=AlphaSSL CA - G2
Issued by: C=BE,O=GlobalSign nv-sa,OU=Root CA,CN=GlobalSign Root CA
Trusted
<--- 200 MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
---> USER user
<--- 331 User user OK. Password required
---> PASS password
<--- 230 OK. Current restricted directory is /
---> PWD
<--- 257 "/" is your current location
---> PBSZ 0
<--- 200 PBSZ=0
---> PROT P
<--- 200 Data protection level set to "private"
---> PASV
<--- 227 Entering Passive Mode (x,x,x,x,228,175)
---- Connecting data socket to (x.x.x.x) port 58543
**** Socket error (Connection timed out) - reconnecting
---> LIST
---> ABOR
---- Closing aborted data socket
---- Closing control socket

So it can't be a validation issue. I can only think that my webhost has a firewall somewhere which is dynamically opening ports, and failing do so as it can't read the encrypted packet. Though I can't beleive that I'm the only one unable to connect!

Thanks for your help anyway.

 

[foolonthehill]

I wan't going to name my webhost, on the basis that it may well have been a problem with my client config.

But after two weeks of "support" messages, I have now received a conclusive answer from HostPapa:

"FTP over SSL/TLS wont be supported on our servers."

This is despite their recommendation to use FTPS in their documentation. Add that to an earlier message:

"We do not provide with SFTP at HostPapa anymore."

and, other than manual file management through the cPanel interface, the only method of access to HostPapa sites is apparently insecure FTP:

"It appears the only option is to use plain FTP."

Wow! I am in awe that a webhost can run a service in this way. The idea that such a host might be used to run sites containing customer data is seriously scary (and by the way, their main FTP login is the same for cPanel - which they also default to HTTP rather than HTTPS....)

If you are thinking of purchasing any packages from HostPapa, I'd have to strongly recommend you try somewhere that at least offers you some level of security.

 

[quranreading1]

FTP over SSL:

A little tool that converts an Excel file to HTML and then uploads it to a remote host via FTPS (that's right, FTP over SSL). There are a few good Java libraries out there for FTP, but I spent hours and hours finding one that supported TSL/SSL and was free. And worked. There are several out there that claim to have this ability, but I've tried most of them, and most of them don't work right out of the box, or at all.

Finally, I found a library called ftp4che. It's free, well-documented, and just works. I'll give you a glimpse of my uploadFile() method:

private boolean uploadFile() {
txtStatus.append("Setting connection properties..." + newLine);

String host = properties.getProperty("host");
String port = properties.getProperty("port");
String user = txtUserName.getText();
String pass = new String(txtPassword.getPassword());
String path = properties.getProperty("path");

Properties pt = new Properties();
pt.setProperty("connection.host", host);
pt.setProperty("connection.port", port);
pt.setProperty("user.login", user);
pt.setProperty("user.password", pass);
pt.setProperty("connection.type", "AUTH_SSL_FTP_CONNECTION");
pt.setProperty("connection.timeout", "10000");
pt.setProperty("connection.passive", "true");
FTPConnection connection = null;
try {

FTPFile fromFile = new FTPFile(htmlFile);
FTPFile toFile = new FTPFile(path, htmlFile.getName());

txtStatus.append("Connecting to " + host + " on port " + port + "..." + newLine);

connection = FTPConnectionFactory.getInstance(pt);

connection.connect();
connection.noOperation();
txtStatus.append("Connected..." + newLine);

try {
txtStatus.append("Deleting old file..." + newLine);
connection.deleteFile(toFile);
} catch (FtpWorkflowException ex) {
connection.noOperation();
}

txtStatus.append("Uploading new file..." + newLine);
connection.uploadFile(fromFile, toFile);
connection.disconnect();

return true;

} catch (Exception ex) {
ex.printStackTrace();
if (connection != null) {
connection.disconnect();
}
txtStatus.append(ex.getMessage() + newLine);
txtStatus.append("Cannot continue..." + newLine);
return false;
}
}

Pretty straightforward, as you can see. You just set up a Properties object with a set of key/value pairs defining the parameters for your connection, including the authentication type, and then just use an FTPConnectionFactory to create a connection. After that, I try deleting the remote file if it already exists, and then upload the new one, with appropriate exception handling of course. Simple and easy, the way it should be.