Unable to SSH - Error: Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

[Cityhues]

I recently setup a new install and have been making adjustments to secure the install. I was perviously able to SSH in (using Private/Public keys) with no issue. Today I received the error, Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

This is happening for all user accounts. I created and installed new keys (via cPanel login) .. but the new keys yield the same result.

I used a verbose ssh command to get more info (readout below - sensitive info removed),

ssh -p 1017 [email protected] -vvv

From reading the results, it appears that the private key which is named "id_rsa" is not being found on my local machine. I can confirm that it is there. I'm thinking that the sshd_config file may have an issue, but I can't spot anything there.

If I turn on "SSH Password Authorization Tweak" in WHM, I am able to SSH using a password, but would prefer to use the key pairs option.

Here is the verbose readout of an attempt to ssh in:

Last login: Thu May 14 13:38:41 on ttys000
xxxx.xxxx@King-Cobra ~ % ssh -p 1017 [email protected] -vvv
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug1: Connecting to thesite.com port 1017.
debug1: Connection established.
debug1: identity file /Users/xxxx.xxxx/.ssh/id_rsa type 0
debug1: identity file /Users/xxxx.xxxx/.ssh/id_rsa-cert type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_dsa type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_dsa-cert type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_ecdsa type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_ed25519 type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_xmss type -1
debug1: identity file /Users/xxxx.xxxx/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug3: fd 5 is O_NONBLOCK
debug1: Authenticating to thesite.com:1017 as 'user_xxx'
debug3: put_host_port: [thesite.com]:1017
debug3: hostkeys_foreach: reading file "/Users/xxxx.xxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxx.xxxx/.ssh/known_hosts:11
debug3: load_hostkeys: loaded 1 keys from [thesite.com]:1017
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],[email protected],[email protected],ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],hmac-sha2-[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: MACs ctos: [email protected],umac[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:GNjB8V3O8cImNT2UKzB1z8HayA7SnHApGvI3CoWinFo
debug3: put_host_port: [54.189.9.177]:1017
debug3: put_host_port: [thesite.com]:1017
debug3: hostkeys_foreach: reading file "/Users/xxxx.xxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxx.xxxx/.ssh/known_hosts:11
debug3: load_hostkeys: loaded 1 keys from [thesite.com]:1017
debug3: hostkeys_foreach: reading file "/Users/xxxx.xxxx/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/xxxx.xxxx/.ssh/known_hosts:12
debug3: load_hostkeys: loaded 1 keys from [54.189.9.177]:1017
debug1: Host '[thesite.com]:1017' is known and matches the ECDSA host key.
debug1: Found key in /Users/xxxx.xxxx/.ssh/known_hosts:11
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/xxxx.xxxx/.ssh/id_rsa RSA SHA256:9EXKCd88EJZbgv4MBH+EJzM6A39hdSBN6L4/ILpvxt0
debug1: Will attempt key: /Users/xxxx.xxxx/.ssh/id_dsa
debug1: Will attempt key: /Users/xxxx.xxxx/.ssh/id_ecdsa
debug1: Will attempt key: /Users/xxxx.xxxx/.ssh/id_ed25519
debug1: Will attempt key: /Users/xxxx.xxxx/.ssh/id_xmss
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /Users/xxxx.xxxx/.ssh/id_rsa RSA SHA256:9EXKCd88EJZbgv4MBH+EJzM6A39hdSBN6L4/ILpvxt0
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_dsa
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_ecdsa
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_ed25519
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_xmss
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
xxxx.xxxx@King-Cobra ~ %

 

[cPanelLauren]

The issue is here:

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_dsa
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_ecdsa
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_ed25519
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /Users/xxxx.xxxx/.ssh/id_xmss
debug3: no such identity: /Users/xxxx.xxxx/.ssh/id_xmss: No such file or directory

The private key matching the public one you provided is unable to be found. Does /Users/xxxx.xxxx/.ssh/id_xmss or any of the other noted files exist? And if so what are their permissions?

 

[Cityhues]

Hi cPanelLauren - thanks for the response.

Actually - none of those Private keys exist in my local .ssh folder. I expected the system to be asking for the private key at /Users/xxxx.xxxx/.ssh/id_rsa (which does exist with permissions 600 for user xxxx.xxxx) .

I had created a new id_rsa keyset and installed the private key locally ... but when I ssh, the system is not looking for that private key.

 

[ItsMattSon]

Hi @Cityhues - Just thinking out loud here.

Have you added the Public key to your authorized_keys file in ~/.ssh ?

The permissions of ~/.ssh on the server should be 0700 and the file ~/.ssh/authorized_keys on the server should be 0600. Best you also double-check the owner and group are set to your user also, xxxx.xxxx

 

[Cityhues]

Hi @Cityhues - Just thinking out loud here.

Have you added the Public key to your authorized_keys file in ~/.ssh ?

The permissions of ~/.ssh on the server should be 0700 and the file ~/.ssh/authorized_keys on the server should be 0600. Best you also double-check the owner and group are set to your user also, xxxx.xxxx

Thanks @ItsMattSon for your reply. I reset the pub/priv key pairs and rechecked all the permissions. That seemed to be the solution.