分佈式鍵值高可用存儲集羣架構搭建（ETCD)

ETCD 是一個高可用的分佈式鍵值數據庫，可用於服務發現。ETCD 採用 raft 一致性算法，基於 Go 語言實現，隨着CoreOS和Kubernetes等項目在開源社區日益火熱，它們項目中都用到的etcd組件做爲一個高可用強一致性的服務發現存儲倉庫，漸漸爲開發人員所關注。在雲計算時代，如何讓服務快速透明地接入到計算集羣中，如何讓共享配置信息快速被集羣中的全部機器發現，更爲重要的是，如何構建這樣一套高可用、安全、易於部署以及響應快速的服務集羣，已經成爲了迫切須要解決的問題。etcd爲解決這類問題帶來了福音，本文將從etcd的應用場景開始，深刻解讀etcd的實現方式，以供開發者們更爲充分地享用etcd所帶來的便利。

特色
簡單：安裝配置使用簡單，提供 HTTP API
安全：支持 SSL 證書
可靠：採用 raft 算法，實現分佈式系統數據的可用性和一致性
下載安裝包：
[root@master ~]# wget https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz

複製安裝包到各接點：
[root@master ~]# ansible k8s -m copy -a'src=/root/etcd-v3.3.2-linux-amd64.tar.gz dest=/root/'

部署環境三臺機子：IP--主機名--集羣結點名
192.168.27.211 client1 (etcd1)
192.168.27.212 client2 (etcd2)
192.168.27.213 client3 (etcd3)

1. 解壓版本包
   tar -zxvf etcd-v3.3.2-linux-amd64.tar.gz 或 [root@master ~]# ansible k8s -m shell -a'tar -zxf etcd-v3.3.2-linux-amd64.tar.gz'
   
   
2. 將解壓文件重命名並移到/etc/目錄下
   mv etcd-v3.3.2-linux-amd64 /etc/etcd-v3.3.2 或
   [root@master ~]# ansible k8s -m shell -a'mv etcd-v3.3.2-linux-amd64 /etc/etcd-v3.3.2'
   
3. 將etcd和etcdctl加入系統變量PATH中(能夠省略)
   [root@master ~]# ansible k8s -m shell -a'echo export PATH=/etc/etcd-v3.3.2:$PATH >> ~/.bashrc'
   [root@master ~]# ansible k8s -m shell -a'source ~/.bashrc '
   [root@master ~]# ansible k8s -m shell -a'source /root/.bashrc ' #靠普些
   
   
   4.進入/etc/etcd-v3.3.2，並建立配置文件etcd.conf，內容：
   
   
   name: etcd1
   data-dir: /etc/etcd-v3.3.2/data
   listen-client-urls: http://0.0.0.0:2379
   advertise-client-urls: http://192.168.27.211:2379
   listen-peer-urls: http://0.0.0.0:2380
   initial-advertise-peer-urls: http://192.168.27.211:2380
   initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
   initial-cluster-token: etcd-cluster-my
   initial-cluster-state: new
   5.設置etcd版本號，默認爲2，咱們安裝的版本爲3.3.2，因此此處設置爲3
   
   在 文件~/.bashrc中加入一行改成3
   export ETCDCTL_API=3
   
   
   
4. 設置開機啓動，新建文件/usr/lib/systemd/system/etcd.service，設置方便管理配置內容：
   [root@client1 etcd-v3.3.2]# cat /usr/lib/systemd/system/etcd.service
   [Unit]
   Description=Etcd Server
   After=network.target
   After=network-online.target
   Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/etc/etcd-v3.3.2
#User=etcd
ExecStart=/etc/etcd-v3.3.2/etcd --config-file /etc/etcd-v3.3.2/etcd.conf
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

5. 啓動etcd，啓動第一個member的時候，此member等待其它member初始化完畢，處於watting狀態，此時緊接着啓動第二個就好了
   systemctl daemon-reload
   systemctl enable etcd
   systemctl start etcd
   systemctl restart etcd

注意：192.168.27.212，192.168.27.213部署如同上面步驟，只須要將步驟4裏面接點名字、ip改成本身ip便可。

name: etcd2
data-dir: /etc/etcd-v3.3.2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.27.212:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.27.212:2380
initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
initial-cluster-token: etcd-cluster-my
initial-cluster-state: new

name: etcd3
data-dir: /etc/etcd-v3.3.2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.27.213:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.27.213:2380
initial-cluster: etcd1=http://192.168.27.211:2380,etcd2=http://192.168.27.212:2380,etcd3=http://192.168.27.213:2380
initial-cluster-token: etcd-cluster-my
initial-cluster-state: new
六、啓動成功後查看集羣成員及狀態:
[root@client3 etcd-v3.3.2]# etcdctl member list

etcdctl endpoint health --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379

[root@client3 etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379

[root@client3 etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379

經過非ETCD集羣成員訪問測試：
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
[root@master etcd-v3.3.2]# etcdctl endpoint status --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
192.168.27.211:2379, f4e909b85dbd820b, 3.3.2, 25 kB, false, 69, 71
192.168.27.212:2379, 265ebeb3298fa51f, 3.3.2, 25 kB, false, 69, 71
192.168.27.213:2379, 510ff85ed64c7436, 3.3.2, 25 kB, true, 69, 71
[root@master etcd-v3.3.2]# etcdctl put name jerry --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
OK
[root@master etcd-v3.3.2]# etcdctl put topic "this is a etcd client test" --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
OK
[root@master etcd-v3.3.2]# etcdctl get topic --endpoints=192.168.27.211:2379,192.168.27.212:2379,192.168.27.213:2379
topic
this is a etcd client test
[root@master etcd-v3.3.2]# etcdctl get topic --endpoints=192.168.27.211:2379
topic
this is a etcd client test

Etcd使用：
讀寫鍵值：
[root@client1 etcd-v3.3.2]# etcdctl put key1 value1
OK
[root@client1 etcd-v3.3.2]# etcdctl get key1
key1
value1

前綴讀（將全部前綴是key的鍵值對都讀出來）

鍵值刪除：
[root@client1 etcd-v3.3.2]# etcdctl del key
0
[root@client1 etcd-v3.3.2]# etcdctl get key
[root@client1 etcd-v3.3.2]# etcdctl get key1
key1
value1
[root@client1 etcd-v3.3.2]# etcdctl del key1
1
[root@client1 etcd-v3.3.2]# etcdctl get key1

觀察已無KEY1鍵及值的存在。

按前綴所有刪除

觀察已無鍵值及數據

從新建立三個鍵值：

其它ETCD集羣成員上訪問鍵值測試：

監聽某個鍵，每次鍵值有變化
[root@client1 etcd-v3.3.2]# etcdctl watch key --prefix
PUT
key1
value
PUT
key2
jerry

設置過時時間TTL


[root@client2 etcd-v3.3.2]# etcdctl lease grant 30
lease 251f6cf538287411 granted with TTL(30s)
[root@client2 etcd-v3.3.2]# etcdctl put key5 value5 --lease=251f6cf538287411
Error: etcdserver: requested lease not found
[root@client2 etcd-v3.3.2]# etcdctl lease grant 30
lease 251f6cf538287414 granted with TTL(30s)
[root@client2 etcd-v3.3.2]# etcdctl put key5 value5 --lease=251f6cf538287414
OK
[root@client2 etcd-v3.3.2]# etcdctl lease keep-alive 251f6cf538287414
lease 251f6cf538287414 expired or revoked.
[root@client2 etcd-v3.3.2]# etcdctl get key5
[root@client2 etcd-v3.3.2]# etcdctl get key5