ceph節點系統盤更換

此版本集羣的系統盤目前並未採用raid1, 因此出現任何故障, 都是一個頭疼的問題.

1，安裝系統
2，安裝軟件
請留意版本號. 要和集羣保持一致
在目前的架構裏, 系統盤不存放數據, 可是須要負責服務及配置的存放. 並且不一樣類型的配置至少位於3個節點.
咱們能夠從其餘節點恢復必要的軟件及配置, 並人工啓用對應的服務啓動項.

參考服務啓動項
systemctl status
這是一個運行有mon和osd的節點, systemctl status命令的部分結果

.├─system-ceph\x2dmon.slice
│ └─ceph-mon@ceph-4.service
│   └─1701 /usr/bin/ceph-mon -f --cluster ceph --id ceph-4 --setuser ceph --setgroup ceph
├─system-ceph\x2dosd.slice
│ ├─ceph-osd@22.service
│ │ └─35523 /usr/bin/ceph-osd -f --cluster ceph --id 22 --setuser ceph --setgroup ceph
│ ├─ceph-osd@23.service
│ │ └─35553 /usr/bin/ceph-osd -f --cluster ceph --id 23 --setuser ceph --setgroup ceph
│ ├─ceph-osd@26.service
│ │ └─35482 /usr/bin/ceph-osd -f --cluster ceph --id 26 --setuser ceph --setgroup ceph
│ ├─ceph-osd@24.service
│ │ └─35545 /usr/bin/ceph-osd -f --cluster ceph --id 24 --setuser ceph --setgroup ceph
│ ├─ceph-osd@21.service
│ │ └─35488 /usr/bin/ceph-osd -f --cluster ceph --id 21 --setuser ceph --setgroup ceph
│ └─ceph-osd@25.service
│   └─35356 /usr/bin/ceph-osd -f --cluster ceph --id 25 --setuser ceph --setgroup ceph

systemd目錄

[root@ceph-4 ~]# ls -l /etc/systemd/system/ceph*
/etc/systemd/system/ceph-mon.target.wants:
total 0
lrwxrwxrwx 1 root root 41 Jun 21  2016 ceph-mon@ceph-4.service -> /usr/lib/systemd/system/ceph-mon@.service

/etc/systemd/system/ceph-osd.target.wants:
total 0
lrwxrwxrwx 1 root root 41 Jul  6  2018 ceph-osd@21.service -> /usr/lib/systemd/system/ceph-osd@.service
lrwxrwxrwx 1 root root 41 Jul  6  2018 ceph-osd@22.service -> /usr/lib/systemd/system/ceph-osd@.service
lrwxrwxrwx 1 root root 41 Jul  6  2018 ceph-osd@23.service -> /usr/lib/systemd/system/ceph-osd@.service
lrwxrwxrwx 1 root root 41 Jul  6  2018 ceph-osd@24.service -> /usr/lib/systemd/system/ceph-osd@.service
lrwxrwxrwx 1 root root 41 Jul  6  2018 ceph-osd@25.service -> /usr/lib/systemd/system/ceph-osd@.service
lrwxrwxrwx 1 root root 41 Jul  6  2018 ceph-osd@26.service -> /usr/lib/systemd/system/ceph-osd@.service

/etc/systemd/system/ceph.target.wants:
total 0
lrwxrwxrwx 1 root root 39 May 24  2016 ceph-mds.target -> /usr/lib/systemd/system/ceph-mds.target
lrwxrwxrwx 1 root root 39 May 24  2016 ceph-mon.target -> /usr/lib/systemd/system/ceph-mon.target
lrwxrwxrwx 1 root root 39 May 24  2016 ceph-osd.target -> /usr/lib/systemd/system/ceph-osd.target
lrwxrwxrwx 1 root root 43 May 24  2016 ceph-radosgw.target -> /usr/lib/systemd/system/ceph-radosgw.target

參考其餘ceph節點配置好這些服務。
啓用mon
system enable ceph-mon@(這裏的ID參考其餘的mon節點, 相似格式)
報錯：

Jan 29 12:19:33 ceph-4.os ceph-create-keys[41389]: admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
Jan 29 12:19:33 ceph-4.os ceph-create-keys[41389]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Jan 29 12:19:34 ceph-4.os ceph-create-keys[41389]: admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
Jan 29 12:19:34 ceph-4.os ceph-create-keys[41389]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Jan 29 12:19:35 ceph-4.os ceph-create-keys[41389]: admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
Jan 29 12:19:35 ceph-4.os ceph-create-keys[41389]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Jan 29 12:19:36 ceph-4.os ceph-create-keys[41389]: admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
Jan 29 12:19:36 ceph-4.os ceph-create-keys[41389]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Jan 29 12:19:37 ceph-4.os ceph-create-keys[41389]: admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
Jan 29 12:19:37 ceph-4.os ceph-create-keys[41389]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Jan 29 12:19:38 ceph-4.os ceph-create-keys[41389]: admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
Jan 29 12:19:38 ceph-4.os ceph-create-keys[41389]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet

解決：
1，在/var/lib/ceph/mon下建立目錄ceph-ceph- （若是是ceph-4，號就爲4）
2，將其餘節點上對應目錄下全部文件複製過來（）

[root@ceph-5 ceph-ceph-5]# ll
total 60
-rw-r--r-- 1 ceph ceph 0 Jun 21 2016 done
-rw------- 1 ceph ceph 77 Jun 21 2016 keyring
drwxr-xr-x 2 ceph ceph 53248 Jan 29 12:31 store.db
-rw-r--r-- 1 ceph ceph 0 Jun 21 2016 systemd
[root@ceph-5 ceph-ceph-5]# pwd
/var/lib/ceph/mon/ceph-ceph-5

啓用mgr
這一版尚未mgr. 就先不寫了.
後期對比其餘mgr節點看吧.

啓用ceph-osd
這個看節點數量而而定
至少掛載點的一批目錄是須要人工建立的.
system enable ceph-osd@(這裏的ID參考其餘的osd節點, 相似格式, 所有osd可以使用ceph osd tree看出)