使用saltstack自動部署K8S

使用saltstack自動部署K8S

1、環境準備

1.1 規劃

1. 操做系統 CentOS-7.x-x86_64。 2. 關閉 iptables 和 SELinux。 3. 全部節點的主機名和 IP 地址，使用/etc/hosts 作好主機名解析。
主機名	IP地址（NAT）	CPU	內存
k8s-master	eth0 : 10.0.0.25	1VCPU	2G
k8s-node-1	eth0 : 10.0.0.26	1VCPU	2G
k8s-node-1	eth0 : 10.0.0.27	1VCPU	2G

1.2 網絡設置

 

1.3 配置靜態IP地址

#將 UUID 和 MAC 地址已經其它配置刪除掉，3個節點除了IP和主機名不一樣其餘相同。
[root@k8s-master ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=static
NAME=eth0
DEVICE=eth0
ONBOOT=yes
IPADDR=10.0.0.25
NETMASK=255.255.255.0
GATEWAY=10.0.0.254
DNS=223.5.5.5

#重啓網絡服務
[root@k8s-master ~]# systemctl restart network

#設置 DNS 解析
[root@k8s-master ~]# vi /etc/resolv.conf
nameserver 223.5.5.5

1.4 關閉selinux、防火牆、networkmanage

setenforce 0
sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config
systemctl disable firewalld.service
systemctl stop firewalld.service
systemctl stop NetworkManager
systemctl disable NetworkManager

1.5 設置主機名解析

3個節點都作

cat >>/etc/hosts<<EOF
10.0.0.25 k8s-master
10.0.0.26 k8s-node-1
10.0.0.27 k8s-node-2
EOF

1.6 配置epel源

3個節點都作

rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
#下載經常使用命令
yum install -y net-tools vim lrzsz tree screen lsof tcpdump nc mtr nmap

#保證能上網
[root@k8s-master ~]# ping www.baidu.com -c3
PING www.a.shifen.com (61.135.169.121) 56(84) bytes of data.
64 bytes from 61.135.169.121: icmp_seq=1 ttl=128 time=5.41 ms
64 bytes from 61.135.169.121: icmp_seq=2 ttl=128 time=6.55 ms
64 bytes from 61.135.169.121: icmp_seq=3 ttl=128 time=8.97 ms

--- www.a.shifen.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2023ms
rtt min/avg/max/mdev = 5.418/6.981/8.974/1.486 ms

1.7 配置免祕鑰登陸

只在master節點作

[root@k8s-master ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
b1:a0:5b:02:57:0e:8f:1e:25:bf:46:1f:d1:f3:24:c4 root@k8s-master
The key's randomart image is:
+--[ RSA 2048]----+
|    o o .+.      |
|     X   .E .    |
|  . + * o  =     |
|   + + + +  .    |
|    + + S        |
|     =           |
|    .            |
|                 |
|                 |
+-----------------+ [root@k8s-master ~]# ssh-copy-id k8s-master
The authenticity of host 'k8s-master (10.0.0.25)' can't be established.
ECDSA key fingerprint is 75:5c:83:a1:b4:cc:bf:28:71:a5:d5:d1:94:35:3c:9a.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@k8s-master's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'k8s-master'"
and check to make sure that only the key(s) you wanted were added.

[root@k8s-master ~]# ssh-copy-id k8s-node-1
The authenticity of host 'k8s-node-1 (10.0.0.26)' can't be established.
ECDSA key fingerprint is 75:5c:83:a1:b4:cc:bf:28:71:a5:d5:d1:94:35:3c:9a.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@k8s-node-1's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'k8s-node-1'"
and check to make sure that only the key(s) you wanted were added.

[root@k8s-master ~]# ssh-copy-id k8s-node-2
The authenticity of host 'k8s-node-2 (10.0.0.27)' can't be established.
ECDSA key fingerprint is 75:5c:83:a1:b4:cc:bf:28:71:a5:d5:d1:94:35:3c:9a.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@k8s-node-2's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'k8s-node-2'"
and check to make sure that only the key(s) you wanted were added.

2、安裝Salt-SSH並克隆本項目代碼

master節點作

 

2.1 安裝Salt SSH
[root@k8s-master ~]# yum install -y https://repo.saltstack.com/yum/redhat/salt-repo-latest-2.el7.noarch.rpm
[root@k8s-master ~]# yum install -y salt-ssh git

2.2 獲取項目代碼放置在/srv目錄
[root@k8s-master ~]# git clone https://github.com/unixhot/salt-kubernetes.git
[root@k8s-master ~]# cd salt-kubernetes/
[root@k8s-master salt-kubernetes]# mv * /srv/
[root@k8s-master salt-kubernetes]# cd /srv/
[root@k8s-master srv]# cp master /etc/salt/master
[root@k8s-master srv]# cp roster /etc/salt/roster

2.3 下載二進制文件
連接: https://pan.baidu.com/s/1kJmvR9wzleHGHnSVHaNpMg 
密碼: 4taa
下載完成後，將文件移動到/srv/salt/k8s/目錄下。
[root@linux-node1 ~]# cd /srv/salt/k8s/
[root@k8s-master k8s]# unzip k8s-v1.9.3-auto.zip 
[root@k8s-master k8s]# ls -l files/
總用量 4
drwxr-xr-x 2 root root   91 3月  28 00:33 cfssl-1.2
drwxrwxr-x 2 root root 4096 3月  27 23:15 cni-plugins-amd64-v0.7.0
drwxr-xr-x 2 root root   31 3月  28 00:33 etcd-v3.3.1-linux-amd64
drwxr-xr-x 2 root root   45 3月  28 12:05 flannel-v0.10.0-linux-amd64
drwxr-xr-x 3 root root   16 3月  28 00:47 k8s-v1.9.3

 

3、Salt SSH管理的機器以及角色分配

• k8s-role: 用來設置K8S的角色
• etcd-role: 用來設置etcd的角色，若是隻須要部署一個etcd，只須要在一臺機器上設置便可
• etcd-name: 若是對一臺機器設置了etcd-role就必須設置etcd-name

master節點作

[root@k8s-master ~]# vim /etc/salt/roster
k8s-master:
  host: 10.0.0.25
  user: root
  priv: /root/.ssh/id_rsa
  minion_opts:
    grains:
      k8s-role: master
      etcd-role: node
      etcd-name: etcd-node1

k8s-node-1:
  host: 10.0.0.26
  user: root
  priv: /root/.ssh/id_rsa
  minion_opts:
    grains:
      k8s-role: node
      etcd-role: node
      etcd-name: etcd-node2

k8s-node-2:
  host: 10.0.0.27
  user: root
  priv: /root/.ssh/id_rsa
  minion_opts:
    grains:
      k8s-role: node
      etcd-role: node
      etcd-name: etcd-node3

4、配置參數

master節點作

[root@k8s-master ~]# vim /srv/pillar/k8s.sls
# -*- coding: utf-8 -*- #設置Master的IP地址(必須修改)
MASTER_IP: "10.0.0.25" #設置ETCD集羣訪問地址（必須修改）
ETCD_ENDPOINTS: "https://10.0.0.25:2379,https://10.0.0.26:2379,https://10.0.0.27:2379" #設置ETCD集羣初始化列表（必須修改）
ETCD_CLUSTER: "etcd-node1=https://10.0.0.25:2380,etcd-node2=https://10.0.0.26:2380,etcd-node3=https://10.0.0.27:2380" #經過Grains FQDN自動獲取本機IP地址，請注意保證主機名解析到本機IP地址
NODE_IP: {{ grains['fqdn_ip4'][0] }}

#設置BOOTSTARP的TOKEN，能夠本身生成
BOOTSTRAP_TOKEN: "ad6d5bb607a186796d8861557df0d17f" #配置Service IP地址段
SERVICE_CIDR: "10.1.0.0/16" #Kubernetes服務 IP (從 SERVICE_CIDR 中預分配)
CLUSTER_KUBERNETES_SVC_IP: "10.1.0.1"

#Kubernetes DNS 服務 IP (從 SERVICE_CIDR 中預分配)
CLUSTER_DNS_SVC_IP: "10.1.0.2" #設置Node Port的端口範圍
NODE_PORT_RANGE: "20000-40000" #設置POD的IP地址段
POD_CIDR: "10.2.0.0/16" #設置集羣的DNS域名
CLUSTER_DNS_DOMAIN: "cluster.local."

5、執行SaltStack狀態

master節點作

5.1 測試Salt SSH聯通性
[root@k8s-master ~]# salt-ssh '*' test.ping

5.2 部署Etcd，因爲Etcd是基礎組建，須要先部署，目標爲部署etcd的節點。
[root@k8s-master ~]#  salt-ssh -L 'k8s-master,k8s-node-1,k8s-node-2' state.sls k8s.etcd

5.3 部署K8S集羣
因爲包比較大，這裏執行時間較長，5分鐘+，若是執行有失敗能夠再次執行便可！
[root@k8s-master ~]#  salt-ssh '*' state.highstate

 6、測試Kubernetes安裝

master節點作

[root@k8s-master ~]# source /etc/profile
[root@k8s-master ~]# kubectl get cs
[root@k8s-master ~]# kubectl get node

7、測試Kubernetes集羣和Flannel網絡

 master節點作

[root@k8s-master ~]#  kubectl run net-test --image=alpine --replicas=2 sleep 360000
須要等待拉取鏡像，可能稍有的慢，請等待。
[root@linux-node1 ~]# kubectl get pod -o wide

測試聯通性，若是都能ping通，說明Kubernetes集羣部署完畢，。
[root@k8s-master ~]#  ping -c 1 10.2.12.2
PING 10.2.12.2 (10.2.12.2) 56(84) bytes of data.
64 bytes from 10.2.12.2: icmp_seq=1 ttl=61 time=8.72 ms

--- 10.2.12.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 8.729/8.729/8.729/0.000 ms

[root@k8s-master ~]#  ping -c 1 10.2.24.2
PING 10.2.24.2 (10.2.24.2) 56(84) bytes of data.
64 bytes from 10.2.24.2: icmp_seq=1 ttl=61 time=22.9 ms

--- 10.2.24.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 22.960/22.960/22.960/0.000 ms

8、如何新增Kubernetes節點

1. 設置主機名解析

2. 設置SSH無密碼登陸

3. 在/etc/salt/roster裏面，增長對應的機器IP

4. 執行SaltStack狀態：salt-ssh '*' state.highstate

5. [root@k8s-node-2 ~]# vim /etc/salt/roster 
k8s-node-3:
  host: 10.0.0.28
  user: root
  priv: /root/.ssh/id_rsa
  minion_opts:
    grains:
      k8s-role: node

6. [root@linux-node1 ~]# salt-ssh '*' state.highstate