kubernetes 之 二進制安裝(三)安裝Etcd集羣

前言

Kubernetes 系統使用 Etcd 存儲集羣配置和一些狀態信息。若是單一部署，etcd若是掛了會致使集羣異常甚至集羣崩潰沒法恢復。因此單點部署是很風險的。因此咱們要部署Etcd集羣

項目地址：https://github.com/etcd-io/etcd

Etcd集羣詳解

環境說明：

etcd1 192.168.214.200

etcd2 192.168.214.201

etcd3 192.168.214.202

三臺節點建立相關目錄

[root@etcd1 ~]# mkdir -p /data/etcd
[root@etcd1 ~]# mkdir -p /opt/kubernetes/{bin,conf,ssl}

將咱們以前生成的證書分發到節點的/opt/kubernetes/ssl/目錄下，詳見上篇文檔《證書詳解》

etcd集羣用到的證書爲ca.pem、kubernetes-key.pem、kubernetes.pem

[root@master1 ssl]# scp *.pem etcd1:/opt/kubernetes/ssl
[root@master1 ssl]# scp *.pem etcd2:/opt/kubernetes/ssl
[root@master1 ssl]# scp *.pem etcd3:/opt/kubernetes/ssl

三臺etcd節點下載並解壓二進制文件

[root@etcd1 ~]# wget  https://github.com/coreos/etcd/releases/download/v3.2.11/etcd-v3.2.11-linux-amd64.tar.gz
[root@etcd1 ~]# tar xzvf etcd-v3.2.11-linux-amd64.tar.gz
[root@etcd1 ~]# cp etcd-v3.2.11-linux-amd64/etcd* /opt/kubernetes/bin/

建立etcd.service文件

在/usr/lib/systemd/system/下建立etcd.service

[root@etcd1 system]# vim etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
Type=notify
WorkingDirectory=/data/etcd/
EnvironmentFile=-/opt/kubernetes/conf/etcd.conf
ExecStart=/opt/kubernetes/bin/etcd \
--name=etcd1 \
--cert-file=/opt/kubernetes/ssl/kubernetes.pem \
--key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
--peer-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
--peer-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
--trusted-ca-file=/opt/kubernetes/ssl/ca.pem \
--peer-trusted-ca-file=/opt/kubernetes/ssl/ca.pem \
--initial-advertise-peer-urls=https://192.168.214.200:2380 \
--listen-peer-urls=https://192.168.214.200:2380 \
--listen-client-urls=https://192.168.214.200:2379,http://127.0.0.1:2379 \
--advertise-client-urls=https://192.168.214.200:2379 \
--initial-cluster-token=etcd-cluster-0 \
--initial-cluster=etcd1=https://192.168.214.200:2380,etcd2=https://192.168.214.201:2380,etcd3=https://192.168.214.202:2380 \
--initial-cluster-state=new \
--data-dir=/data/etcd
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

說明：

自行根據節點IP設置其餘節點

指定 etcd 的工做目錄和數據目錄爲 /data/etcd，需在啓動服務前建立這個目錄；

--initial-cluster-state 值爲 new 時，--name 的參數值必須位於 --initial-cluster 列表中；

爲了保證通訊安全，須要指定 etcd 的公私鑰(cert-file和key-file)、Peers 通訊的公私鑰和 CA 證書(peer-cert-file、peer-key-file、peer-trusted-ca-file)、客戶端的CA證書（trusted-ca-file）；

增長環境變量

[root@etcd1 ~]# vim .bash_profile 
export PATH=/opt/kubernetes/bin:$PATH
[root@etcd1 ~]# source .bash_profile

啓動etcd服務

[root@etcd1 ~]# systemctl daemon-reload
[root@etcd1 ~]# systemctl enable etcd
[root@etcd1 ~]# systemctl start etcd

集羣驗證

（1）驗證集羣狀態

[root@etcd1 ssl]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem cluster-health
member 31a87df7577ee4e3 is healthy: got healthy result from https://192.168.214.201:2379
member d009f1b31e51b9c3 is healthy: got healthy result from https://192.168.214.202:2379
member fb94879e6d597fdf is healthy: got healthy result from https://192.168.214.200:2379
cluster is healthy

（2）經過集羣節點etcd1設置一個值，到etcd2和etcd3獲取，若是獲取正常，則存儲也沒問題

[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem set dong "diss and peace"
diss and peace
[root@etcd2 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem get dong
diss and peace
[root@etcd3 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem get dong
diss and peace

（3）能夠經過etcdctl member list查看誰是當前集羣中選舉出的leader

[root@etcd2 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem member list
31a87df7577ee4e3: name=etcd2 peerURLs=https://192.168.214.201:2380 clientURLs=https://192.168.214.201:2379 isLeader=false
d009f1b31e51b9c3: name=etcd3 peerURLs=https://192.168.214.202:2380 clientURLs=https://192.168.214.202:2379 isLeader=false
fb94879e6d597fdf: name=etcd1 peerURLs=https://192.168.214.200:2380 clientURLs=https://192.168.214.200:2379 isLeader=true

（4）驗證leader切換。上步能夠看出此時的leader爲etcd1，停掉etcd1後，再次查看，leader變爲了etcd2

[root@etcd1 ~]# systemctl stop etcd
[root@etcd2 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem member list
31a87df7577ee4e3: name=etcd2 peerURLs=https://192.168.214.201:2380 clientURLs=https://192.168.214.201:2379 isLeader=true
d009f1b31e51b9c3: name=etcd3 peerURLs=https://192.168.214.202:2380 clientURLs=https://192.168.214.202:2379 isLeader=false
fb94879e6d597fdf: name=etcd1 peerURLs=https://192.168.214.200:2380 clientURLs=https://192.168.214.200:2379 isLeader=false

建立kubernetes網段，用於後續的flannel請求調用，經過其餘etcd集羣節點若是能夠查看到這個網段，說明網段建立成功。

[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem     mk /kubernetes/network/config '{ "Network": "172.20.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}'
{ "Network": "172.20.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}           #mk建立一個鍵值存儲，同時會自動建立鍵值目錄
[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem   ls -r
/dong             #ls -r列出鍵值存儲的目錄
/kubernetes
/kubernetes/network
/kubernetes/network/config
[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem   get /kubernetes/network/config
{ "Network": "172.20.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}    #get查看鍵值存儲

kubernetes二進制安裝(一)環境說明與準備

kubernetes二進制安裝(二)證書詳解