Kubernetes學習之路（26）之kubeasz+ansible部署集羣

目錄

• 一、環境說明
• 二、準備工做
• 三、分步驟安裝
  • 3.一、建立證書和安裝準備
  • 3.二、安裝etcd集羣
  • 3.三、安裝docker
  • 3.四、安裝master節點
  • 3.五、安裝node節點
  • 3.六、部署集羣網絡
  • 3.七、部署集羣插件（dns，dashboard）
  • 3.八、測試DNS解析

學習文檔：https://github.com/gjmzj/kubeasz/

一、環境說明

IP	主機名	角色	虛擬機配置
192.168.56.11	k8s-master	deploy、master一、lb一、etcd	4c4g
192.168.56.12	k8s-master2	master二、lb2	4c4g
192.168.56.13	k8s-node01	etcd、node	2c2g
192.168.56.14	k8s-node02	etcd、node	2c2g
192.168.56.110		vip
系統內核	3.10	docker版本	18.09
k8s版本	1.13	etcd版本	3.0

二、準備工做

• 四臺機器，所有執行：

yum install -y epel-release
yum update -y 
yum install python -y

• deploy節點安裝ansible並配置密鑰認證

yum install -y ansible
ssh-keygen
for ip in 11 12 13 14;do ssh-copy-id 192.168.56.$ip;done

• deploy節點編排K8S

[root@k8s-master ~]# git clone https://github.com/gjmzj/kubeasz.git
[root@k8s-master ~]# mv kubeasz/* /etc/ansible/

從百度雲網盤下載二進制文件 https://pan.baidu.com/s/1c4RFaA#list/path=%2F
能夠根據本身所需版本，下載對應的tar包，這裏我下載1.13
通過一番折騰，最終把k8s.1-13-5.tar.gz的tar包放到了depoly上

[root@k8s-master ~]# tar -zxf k8s.1-13-5.tar.gz 
[root@k8s-master ~]# mv bin/* /etc/ansible/bin/

• 配置集羣參數

[root@k8s-master ~]# cd /etc/ansible/
[root@k8s-master ansible]# cp example/hosts.m-masters.example hosts
cp: overwrite ‘hosts’? y
[root@k8s-master ansible]# vim hosts  #根據實際狀況的ip進行更改
[deploy]
192.168.56.11 NTP_ENABLED=no    #設置集羣是否安裝 chrony 時間同步

[etcd]  #etcd集羣請提供以下NODE_NAME，注意etcd集羣必須是1,3,5,7...奇數個節點
192.168.56.11 NODE_NAME=etcd1
192.168.56.13 NODE_NAME=etcd2
192.168.56.14 NODE_NAME=etcd3

[kube-master]
192.168.56.11
192.168.56.12

[kube-node]
192.168.56.13
192.168.56.14

[lb]    # 負載均衡(目前已支持多於2節點，通常2節點就夠了) 安裝 haproxy+keepalived
192.168.56.12 LB_ROLE=backup
192.168.56.11 LB_ROLE=master

## 集羣 MASTER IP即 LB節點VIP地址，爲區別與默認apiserver端口，設置VIP監聽的服務端口8443
# 公有云上請使用雲負載均衡內網地址和監聽端口
[all:vars]
DEPLOY_MODE=multi-master
MASTER_IP="192.168.56.110"  #設置vip
KUBE_APISERVER="https://{{ MASTER_IP }}:8443"
CLUSTER_NETWORK="flannel"
SERVICE_CIDR="10.68.0.0/16"
CLUSTER_CIDR="172.20.0.0/16"
NODE_PORT_RANGE="20000-40000"
CLUSTER_KUBERNETES_SVC_IP="10.68.0.1"
CLUSTER_DNS_SVC_IP="10.68.0.2"
CLUSTER_DNS_DOMAIN="cluster.local."
bin_dir="/opt/kube/bin"
ca_dir="/etc/kubernetes/ssl"
base_dir="/etc/ansible"

#修改完成後，測試hosts
[root@k8s-master ansible]# ansible all -m ping
192.168.56.12 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
192.168.56.13 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
192.168.56.14 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
192.168.56.11 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

三、分步驟安裝

3.一、建立證書和安裝準備

[root@k8s-master ansible]# ansible-playbook 01.prepare.yml

3.二、安裝etcd集羣

[root@k8s-master ansible]# ansible-playbook 02.etcd.yml
[root@k8s-master ansible]# bash

#驗證etcd集羣狀態
[root@k8s-master ansible]# systemctl status etcd

#在任一 etcd 集羣節點上執行以下命令
[root@k8s-master ansible]# for ip in 11 13 14;do ETCDCTL_API=3 etcdctl --endpoints=https://192.168.56.$ip:2379 --cacert=/etc/kubernetes/ssl/ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem endpoint health;done
https://192.168.56.11:2379 is healthy: successfully committed proposal: took = 7.967375ms
https://192.168.56.13:2379 is healthy: successfully committed proposal: took = 12.557643ms
https://192.168.56.14:2379 is healthy: successfully committed proposal: took = 9.70078ms

3.三、安裝docker

[root@k8s-master ansible]# ansible-playbook 03.docker.yml

3.四、安裝master節點

[root@k8s-master ansible]# ansible-playbook 04.kube-master.yml 

#查看進程狀態
[root@k8s-master ansible]# systemctl status kube-apiserver
[root@k8s-master ansible]# systemctl status kube-controller-manager
[root@k8s-master ansible]# systemctl status kube-scheduler
[root@k8s-master ansible]# kubectl get componentstatus  #查看集羣狀態
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok                  
controller-manager   Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   
etcd-1               Healthy   {"health":"true"}   
etcd-2               Healthy   {"health":"true"}

3.五、安裝node節點

[root@k8s-master ansible]# ansible-playbook 05.kube-node.yml
[root@k8s-master ansible]# systemctl status kubelet
[root@k8s-master ansible]# systemctl status kube-proxy
[root@k8s-master ansible]# kubectl get nodes
NAME            STATUS                     ROLES    AGE     VERSION
192.168.56.11   Ready,SchedulingDisabled   master   6m56s   v1.13.5
192.168.56.12   Ready,SchedulingDisabled   master   6m57s   v1.13.5
192.168.56.13   Ready                      node     40s     v1.13.5
192.168.56.14   Ready                      node     40s     v1.13.5

3.六、部署集羣網絡

[root@k8s-master ansible]# ansible-playbook 06.network.yml 
[root@k8s-master ansible]# kubectl get pod -n kube-system   #查看flannel相關pod
NAME                          READY   STATUS    RESTARTS   AGE
kube-flannel-ds-amd64-856rg   1/1     Running   0          115s
kube-flannel-ds-amd64-j4542   1/1     Running   0          115s
kube-flannel-ds-amd64-q9cmh   1/1     Running   0          115s
kube-flannel-ds-amd64-rhg66   1/1     Running   0          115s

3.七、部署集羣插件（dns，dashboard）

[root@k8s-master ansible]# ansible-playbook 07.cluster-addon.yml 

[root@k8s-master ansible]# kubectl get svc -n kube-system   #查看服務
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
heapster               ClusterIP   10.68.29.48    <none>        80/TCP                   64s
kube-dns               ClusterIP   10.68.0.2      <none>        53/UDP,53/TCP,9153/TCP   71s
kubernetes-dashboard   NodePort    10.68.117.7    <none>        443:24190/TCP            64s
metrics-server         ClusterIP   10.68.107.56   <none>        443/TCP                  69s

[root@k8s-master ansible]# kubectl cluster-info #查看集羣信息
Kubernetes master is running at https://192.168.56.110:8443
CoreDNS is running at https://192.168.56.110:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
kubernetes-dashboard is running at https://192.168.56.110:8443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

[root@k8s-master ansible]# kubectl top node     #查看節點資源使用率
NAME            CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
192.168.56.11   523m         13%    2345Mi          76%       
192.168.56.12   582m         15%    1355Mi          44%       
192.168.56.13   182m         10%    791Mi           70%       
192.168.56.14   205m         11%    804Mi           71%

一步ansible安裝k8s集羣命令以下：

ansible-playbook 90.setup.yml

3.八、測試DNS解析

[root@k8s-master ansible]# kubectl run nginx --image=nginx --expose --port=80
[root@k8s-master ansible]# kubectl run busybox --rm -it --image=busybox /bin/sh
/ # nslookup nginx.default.svc.cluster.local
Server:     10.68.0.2
Address:    10.68.0.2:53

Name:   nginx.default.svc.cluster.local
Address: 10.68.149.79