SSHD fails with AuthorizedKeysCommandUser unknown error

[adamreece.webbox]

I've had 14 servers report that the SSHD service appears to be down.

> be me
> 14 servers report sshd is down
> realise I can't ssh in to look why
> ssh restart from whm fails -- "Starting sshd: /etc/ssh/sshd_config: line 49: Bad configuration option: AuthorizedKeysCommandUser"
> open terminal from whm
> `nano /etc/ssh/sshd_config`
> ctrl+w to search for "AuthorizedKeysCommandUser"
> terminal tab closes
> bamboozled again (╯°□°）╯︵ ┻━┻
> open whm and terminal
> `nano /etc/ssh/sshd_config`
> manually find "AuthorizedKeysCommandUser" resisting the urge to ctrl+w search for it
> replace it with "AuthorizedKeysCommandRunAs"
> `/scripts/restartsrv_sshd`
> not profit, time was wasted!

We have a mixture of CentOS 6.10 and 7.4 due to age. (2 run CloudLinux.) Only 3 of the CentOS 6.10 servers that do NOT run CloudLinux were impacted by this though.

Looks like this option has suddenly been replaced for "AuthorizedKeysCommandRunAs"? Not sure why "AuthorizedKeysCommandUser" would have been accepted for the past 3-5 years though, or why it still works fine on the 2 CloudLinux servers also running on top of CentOS 6.10.

--

Anyone else suddenly having this, you can quickly fix it in the WHM terminal with this command:

sed -e 's/AuthorizedKeysCommandUser/AuthorizedKeysCommandRunAs/g' -i /etc/ssh/sshd_config && /scripts/restartsrv_sshd​

 

[cPanelLauren]

Hi @adamreece.webbox

I'm glad that you were able to identify the issue and thank you for sharing with us, it's much appreciated.

> bamboozled again (╯°□°）╯︵ ┻━┻

This may have been the funniest thing I read all day as well.

 

[adamreece.webbox]

Hi Lauren,

No problem. -- I had a talk about this further with Alex on Discord #horrorstories. It appears that the two CloudLinux 6 instances we have are also impacted by this. As soon as I restarted the SSH service just to try this out they encountered the same problem as the three CentOS 6 instances. The same command I posted above resolved it right away though.

What I'm not sure of is:

• Why the `upcp` cron only just restarted the SSH services on the CentOS 6 instances yesterday as it noticed the running instances were outdated, though the yum logs show the last update installed was way back on 1st September. (I'd have expected this issue to occur the next time `upcp` ran overnight.)
• Why the `upcp` cron didn't notice the SSH services were outdated on the CL6 instances at all.
• Why `AuthorizedKeysCommandUser` worked for so long despite OpenSSH on RHEL/CentOS/CL 6 never supporting this option. (Perhaps it was just gracefully ignored in previous builds.)

Oh well, not a major issue. Resolved very quickly.

Glad my story gave you a giggl too.

 

[cPanelLauren]

Hi @adamreece.webbox

I just read over that this morning - I found OpenSSH: Release Notes which indicates they did make some changes but it really doesn't explain to me why it only took effect recently except to wonder if you only received the OpenSSH update recently - which it appears to have done so.