Kubernetes集羣搭建之Etcd集羣配置篇

時間 2019-11-18

原文原文鏈接

介紹linux

etcd 是一個分佈式一致性k-v存儲系統，可用於服務註冊發現與共享配置，具備如下優勢。web

簡單：相比於晦澀難懂的paxos算法，etcd基於相對簡單且易實現的raft算法實現一致性，並經過gRPC提供接口調用算法

安全：支持TLS通訊，並能夠針對不一樣的用戶進行對key的讀寫控制json

高性能：10,000 /秒的寫性能安全

本次系列使用的所需部署包版本都使用的目前最新的或最新穩定版，安裝包地址請到公衆號內回覆【K8s實戰】獲取bash

注: K8s的集羣相關數據是存放在Etcd中的,全部咱們得先部署架構

架構分佈式

cfssl安裝工具

Kubernetes 系統的各組件須要使用 TLS 證書對通訊進行加密，使用 CloudFlare 的 PKI 工具集 cfssl 來生成 Certificate Authority (CA) 和其它證書。性能

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64mv cfssl_linux-amd64 /usr/local/bin/cfsslmv cfssljson_linux-amd64 /usr/local/bin/cfssljsonmv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

建立證書目錄

[root@master-01 ssl]#mkdir /etc/etcd/ssl -p[root@master-01 ssl]#mkdir /etc/kubernetes/ssl -p[root@master-01 ssl]##cd /etc/etcd/ssl

生成證書

1. Etcd ca配置

[root@master-01 ssl]# cat << EOF | tee ca-config.json{"signing": {"default": {"expiry": "87600h"},"profiles": {"etcd": {"expiry": "87600h","usages": ["signing","key encipherment","server auth","client auth"]}}}}EOF

2. Etcd ca證書

[root@master-01 ssl]# cat << EOF | tee ca-csr.json{"CN": "etcd CA","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "Hangzhou","ST": "Hangzhou"}]}EOF

3. etcd server證書

[root@master-01 ssl]# cat << EOF | tee server-csr.json{"CN": "etcd","hosts": ["192.168.209.130","192.168.209.131","192.168.209.132"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "Hangzhou","ST": "Hangzhou"}]}EOF

3. 生成etcd ca證書和私鑰初始化ca

[root@master-01 ssl]# lsca-config.json ca-csr.json server-csr.json[root@master-01 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca2019/03/09 14:49:51 [INFO] generating a new CA key and certificate from CSR2019/03/09 14:49:51 [INFO] generate received request2019/03/09 14:49:51 [INFO] received CSR2019/03/09 14:49:51 [INFO] generating key: rsa-20482019/03/09 14:49:51 [INFO] encoded CSR2019/03/09 14:49:51 [INFO] signed certificate with serial number 131013203369168241950883398321469825148900357407[root@master-01 ssl]# lsca-config.json ca.csr ca-csr.json ca-key.pem ca.pem server-csr.json

****4. 生成server 證書

[root@master-01 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server2019/03/09 14:53:18 [INFO] generate received request2019/03/09 14:53:18 [INFO] received CSR2019/03/09 14:53:18 [INFO] generating key: rsa-20482019/03/09 14:53:18 [INFO] encoded CSR2019/03/09 14:53:18 [INFO] signed certificate with serial number 1346689003013978487615203135781755822261954464372019/03/09 14:53:18 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable forwebsites. For more information see the Baseline Requirements for the Issuance and Managementof Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);specifically, section 10.2.3 ("Information Requirements").[root@master-01 ssl]# lsca-config.json ca.csr ca-csr.json ca-key.pem ca.pem server.csr server-csr.json server-key.pem server.pem

以上生成etcd證書完成，只須要會使用便可，不用過多深刻研究

同步證書

拷貝證書 master-0二、master-03上

[root@master-01 ssl]# scp * 192.168.209.131:/etc/etcd/ssl/[root@master-01 ssl]# scp * 192.168.209.132:/etc/etcd/ssl/

etcd 部署

1. 解壓縮並同步二進制文件到 master-0二、master-03

[root@master-01 ~]# tar xf etcd-v3.3.12-linux-amd64.tar.gz [root@master-01 ~]# cd etcd-v3.3.12-linux-amd64[root@master-01 etcd-v3.3.12-linux-amd64]# cp etcd etcdctl /usr/bin/[root@master-01 etcd-v3.3.12-linux-amd64]# scp etcd etcdctl 192.168.209.131:/usr/bin/[root@master-01 etcd-v3.3.12-linux-amd64]# scp etcd etcdctl 192.168.209.132:/usr/bin/

2. 建立etcd數據目錄

[root@master-01 ~]# mkdir /var/lib/etcd

3. 配置etcd文件

[root@master-01 ~]# cat /etc/etcd/etcd.conf # configure file for etcd.service#[Member]ETCD_NAME="infra1" ETCD_DATA_DIR="/var/lib/etcd/infra1.etcd"ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"#[Clustering]ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.209.130:2380"ETCD_ADVERTISE_CLIENT_URLS="https://192.168.209.130:2379"ETCD_INITIAL_CLUSTER="infra1=https://192.168.209.130:2380,infra2=https://192.168.209.131:2380,infra3=https://192.168.209.132:2380"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"ETCD_INITIAL_CLUSTER_STATE="new"#[Security]ETCD_CERT_FILE="/etc/etcd/ssl/server.pem"ETCD_KEY_FILE="/etc/etcd/ssl/server-key.pem"ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"ETCD_CLIENT_CERT_AUTH="true"ETCD_PEER_CERT_FILE="/etc/etcd/ssl/server.pem"ETCD_PEER_KEY_FILE="/etc/etcd/ssl/server-key.pem"ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"ETCD_PEER_CLIENT_CERT_AUTH="true"ETCD_PEER_CERT_FILE="/etc/etcd/ssl/server.pem"ETCD_PEER_KEY_FILE="/etc/etcd/ssl/server-key.pem"ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"ETCD_PEER_CLIENT_CERT_AUTH="true"

ETCD_NAME 節點名稱

ETCD_DATA_DIR 數據目錄

ETCD_LISTEN_PEER_URLS 集羣通訊監聽地址

ETCD_LISTEN_CLIENT_URLS 客戶端訪問監聽地址

ETCD_INITIAL_ADVERTISE_PEER_URLS 集羣通告地址

ETCD_ADVERTISE_CLIENT_URLS 客戶端通告地址

ETCD_INITIAL_CLUSTER 集羣節點地址

ETCD_INITIAL_CLUSTER_TOKEN 集羣Token

ETCD_INITIAL_CLUSTER_STATE 加入集羣的當前狀態，new是新集羣，existing表示加入已有集羣

注意: master-0二、master-03節點上須要修改ETCD_NAME 和對應IP地址

4.. 配置etcd啓動文件

[root@master-01 ~]# cat /usr/lib/systemd/system/etcd.service

[Unit]Description=Etcd ServerAfter=network.targetAfter=network-online.targetWants=network-online.target[Service]Type=notifyWorkingDirectory=/var/lib/etcd/EnvironmentFile=-/etc/etcd/etcd.conf# set GOMAXPROCS to number of processorsExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" --listen-peer-urls=\"${ETCD_LISTEN_PEER_URLS}\" --advertise-client-urls=\"${ETCD_ADVERTISE_CLIENT_URLS}\" --initial-cluster-token=\"${ETCD_INITIAL_CLUSTER_TOKEN}\" --initial-cluster=\"${ETCD_INITIAL_CLUSTER}\" --initial-cluster-state=\"${ETCD_INITIAL_CLUSTER_STATE}\" --cert-file=\"${ETCD_CERT_FILE}\" --key-file=\"${ETCD_KEY_FILE}\" --trusted-ca-file=\"${ETCD_TRUSTED_CA_FILE}\" --client-cert-auth=\"${ETCD_CLIENT_CERT_AUTH}\" --peer-cert-file=\"${ETCD_PEER_CERT_FILE}\" --peer-key-file=\"${ETCD_PEER_KEY_FILE}\" --peer-trusted-ca-file=\"${ETCD_PEER_TRUSTED_CA_FILE}\" --peer-client-cert-auth=\"${ETCD_PEER_CLIENT_CERT_AUTH}\""Restart=on-failureLimitNOFILE=65536[Install]WantedBy=multi-user.target

5. 同步配置及啓動文件到 master-0二、master-03

[root@master-01 ~]# scp /etc/etcd/etcd.conf 192.168.209.132:/etc/etcd/[root@master-01 ~]# scp /etc/etcd/etcd.conf 192.168.209.131:/etc/etcd/[root@master-01 ~]# scp /usr/lib/systemd/system/etcd.service 192.168.209.131:/usr/lib/systemd/system/[root@master-01 ~]# scp /usr/lib/systemd/system/etcd.service 192.168.209.132:/usr/lib/systemd/system/

6. 啓動etcd，注意02 03 要一樣配置下

[root@master-01 ssl]# systemctl daemon-reload[root@master-01 ssl]# systemctl enable etcd[root@master-01 ssl]# systemctl start etcd

7. 驗證集羣狀態

[root@master-01 ~]# etcdctl --ca-file=/etc/etcd/ssl/ca.pem --cert-file=/etc/etcd/ssl/server.pem --key-file=/etc/etcd/ssl/server-key.pem --endpoints="https://192.168.209.130:2379,https://192.168.209.131:2379,https://192.168.209.132:2379" cluster-healthmember 586a62bfa8ca5f8a is healthy: got healthy result from https://192.168.209.130:2379member 5f55d6687099cd91 is healthy: got healthy result from https://192.168.209.131:2379member ee591e661050ac9d is healthy: got healthy result from https://192.168.209.132:2379cluster is healthy

進行到這一步，說明etcd集羣部署完成啦，下一章介紹部署高可用K8s集羣，敬請期待，謝謝！

END

若是你以爲文章還不錯，請你們點『好看』分享下。你的確定是我最大的鼓勵和支持。