SOLVED Delayed response on ssh connection

[Simon Greenwood]

In the last week we have had a couple of reports from our clients of slow SSH responses. I can reproduce this with verbose logging turned on:

[[email protected] ~]$ sudo ssh -p xxx -i xxx -v xxx.xxx.xxx
OpenSSH_5.2p1, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to xxx.xxx.xxx [xxx.xxx.xxx.xxx] port xxx.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file xxx type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.2
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-ripemd160 none
debug1: kex: client->server aes128-ctr hmac-ripemd160 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<2048<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'xxx:xxx' is known and matches the RSA host key.
debug1: Found key in xxx
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: xxx
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Requesting [EMAIL][email protected][/EMAIL]
debug1: Entering interactive session.

[20 second wait here]

debug1: client_input_global_request: rtype [EMAIL][email protected][/EMAIL] want_reply 0
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Last login: xxxxxxx
CentOS Linux release 7.3.1611 (Core)

---

I have tried setting UseDNS to no and also disabling GSSAPI but this doesn't seem to work. It seems to have only started this week.

 

[sparek-3]

Did you restart SSH after setting UseDNS to no?

 

[cPanelLauren]

I'm also not able to replicate this nor have I heard any reports internally of an issue with SSH logins. I've got a few 70 test servers with no issues personally as well. I am curious to know if the issue persists after a restart of the service as well.


Thanks!

 

[Simon Greenwood]

I'm also not able to replicate this nor have I heard any reports internally of an issue with SSH logins. I've got a few 70 test servers with no issues personally as well. I am curious to know if the issue persists after a restart of the service as well.


Thanks!

Disabling GSSAPIStrictAcceptorCheck as well as GSSAPIAuthentication seems to have fixed it, which I'm sure has occurred before but not recently so I'm not sure why an update would cause it.

 

[Simon Greenwood]

Ah, spoke too soon - it did, once and then started doing it again. There does seem to be an issue with keys that have a 'from=' component but I can't track it down.

 

[sparek-3]

Is the from listing an IP address or a hostname?

What DNS resolvers are you using on the server? Are those DNS resolvers working correctly?

This sounds like a DNS issue. One of the DNS resolvers you are using is not responding to requests, but it is being tried anyway and those requests have to timeout before the next resolver is tried. Or perhaps none of the resolvers are working. Or perhaps whatever you are looking up isn't responding.

 

[Simon Greenwood]

This was due to an issue with systemd-logind. Restarting it made everything work again - this is a common solution but obviously not an obvious one.

 

[cPanelLauren]

HI @Simon Greenwood

Thank you for updating the thread with the solution. I'm glad you got it worked out!