記一次詭異的 ssh 互信免密碼登陸失敗

0、背景

由於 hadoop 環境須要 master 能免密碼 ssh localhost，因此咱們須要創建與本機 localhost 的互信，方法很簡單：

1. ssh-keygen -t rsa
   #Press enter for each line 
2. cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
3. chmod og-wx ~/.ssh/authorized_keys

#################
### 簡單粗暴這麼來：
#################
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

這三步執行下來就能順利 ssh localhost 免密碼登陸了，可是昨天剛建好的互信，今天下午忽然不能用了，ssh localhost 須要密碼，第一反應是可能哪裏設置和配置被改動了，看了下文件版本、配置修改時間都無變化，然而登陸時的提示信息又過於簡單，這個時候排查陷入僵局了。

work@test_zz_Master 192.168.187.213 18:45:18 ~ >
ssh localhost            
work@localhost's password: 

work@test_zz_Master 192.168.187.213 18:45:24 ~ >

一、怎麼排查？

1.1 debug 日誌

首先仍是要拿到明細 debug 日誌，看看卡在哪裏了。linux 下的很多命令都自帶調試功能，好比 ssh 就自帶 debug 功能：

ssh -vvv localhost
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to localhost [127.0.0.1] port 22.
debug1: Connection established.
debug1: identity file /home/work/.ssh/identity type -1
debug1: identity file /home/work/.ssh/identity-cert type -1
...
debug3: remaining preferred: keyboard-interactive,password
// 啓用公鑰登陸
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/work/.ssh/identity
debug3: no such identity: /home/work/.ssh/identity
debug1: Offering public key: /home/work/.ssh/id_rsa
debug3: send_pubkey_test
// 發送公鑰包，等待服務器認證響應
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 368 bytes for a total of 1741
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/work/.ssh/id_dsa
debug3: no such identity: /home/work/.ssh/id_dsa
debug1: Trying private key: /home/work/.ssh/id_ecdsa
debug3: no such identity: /home/work/.ssh/id_ecdsa
// 沒經過認證，禁用該認證方法
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
// 下一個認證方法：啓用密碼登陸
debug1: Next authentication method: password
work@localhost's password:

能夠看到，確實是認證失敗了，可是僅憑一句 we did not send a packet, disable method，我們仍是沒法看到失敗的深層次緣由，那我們再對比下正常的認證流程應該是怎樣的：

能夠看到右邊正常的會接受公鑰，左邊的則沒有獲得響應，繼續走別的認證方式。

1.2 檢查配置

打開服務器的 /etc/ssh/sshd_config

確認下面幾行是這樣的:

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys

#GSSAPIAuthentication yes
#GSSAPICleanupCredentials yes

配置沒問題，此路仍是不通。

1.3 Debugging SSH public key 

在B機器上，we sent a public key packet, wait for reply 以後則是緊跟着"debug1: Server accepts key: pkalg ssh-rsa blen 277"。由此能夠看出，是A機器的sshd不承認publickey。

至於爲何不承認，在google上查了許多，毫無頭緒，直到使用相似「ssh publickey ignore debug diagnose」這樣的關鍵詞，發現這個頁面，其中的第二條和第六條給出瞭解答：

2. Debugging on the remote host by running sshd in debug mode: Run ‘/usr/sbin/sshd -d -p 2222′ on the remote host and connect to it. ’2222′ here is the port number of the sshd process you started on the remote host.

6. Check the permissions on your home directory, .ssh directory, and the authorized_keys file: If your ssh server is running with ‘StrictModes on’, it will refuse to use your public keys in the ~/.ssh/authorized_keys file. Your home directory should be writable only by you, ~/.ssh should be 700, and authorized_keys should be 600.

經過執行 /usr/sbin/sshd -d -p 2222 （在2222端口啓動一個帶debug輸出的sshd） ，

而後 ssh -vv localhost -p 2222 ，能夠看到 sshd 的輸出：

[root(hostname)@bjdhj-187-213 ~]# /usr/sbin/sshd -d -p 2222
debug1: sshd version OpenSSH_5.3p1
debug1: read PEM private key done: type RSA
...
debug1: trying public key file /home/work/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
Authentication refused: bad ownership or modes for directory /home/work
debug1: restore_uid: 0/0
debug1: temporarily_use_uid: 500/500 (e=0/0)
debug1: trying public key file /home/work/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
Authentication refused: bad ownership or modes for directory /home/work
debug1: restore_uid: 0/0
Failed publickey for work from 127.0.0.1 port 45548 ssh2

能夠看到倒數第三行：Authentication refused: bad ownership or modes for directory /home/work，

正好與那第六條相對應，再檢查一下 /home/work ，其權限是不是其餘組可讀寫。

同時，我們也能從 /var/log/secure 看到明細的 debug 日誌：

[root(hostname)@bjdhj-187-213 ~]# tail -f /var/log/secure
Sep  1 18:52:20 bjdhj-187-213 sshd[30936]: Server listening on 0.0.0.0 port 22.
Sep  1 18:52:23 bjdhj-187-213 sshd[30944]: Authentication refused: bad ownership or modes for directory /home/work
Sep  1 18:52:23 bjdhj-187-213 sshd[30944]: Authentication refused: bad ownership or modes for directory /home/work
Sep  1 18:52:25 bjdhj-187-213 sshd[30948]: Connection closed by 127.0.0.1

二、最終解決方案

ssh 爲了保證通訊安全，防止 key 被篡改或竊取，對目錄和文件的權限要求至關嚴格，

我們最終須要確保相關目錄權限與下述一致：

chmod 0755 ~               # 或 chmod g-w ~   
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

sudo service sshd restart

後記：

固然了，這篇文章所反映的問題雖然很小，最後的答案也很簡單，可是其展示的排查思路和方法卻很獨特，值得借鑑，畢竟不少時候我們不能像平時同樣，直接 debug 源碼。

Refer：

[1] 記一次sshd異常：沒法經過創建信任關係登陸

https://www.felix021.com/blog/read.php?2085

[2] ssh公鑰登陸無效

https://my.oschina.net/sukai/blog/686981

[3] Debugging SSH public key authentication problems

https://blog.codefront.net/2007/02/28/debugging-ssh-public-key-authentication-problems/