經過kubeadm快速部署K8S集羣

時間 2021-03-06

標籤 node linux nginx git github docker json bootstrap windows centos 欄目負載均衡简体版

原文原文鏈接

kubeadm是官方社區推出的一個用於快速部署kubernetes集羣的工具。node

這個工具能經過兩條指令完成一個kubernetes集羣的部署：linux

# 建立一個 Master 節點nginx

$ kubeadm initgit

# 將一個 Node 節點加入到當前集羣中
$ kubeadm join <Master節點的IP和端口 >github

1. 安裝要求

在開始以前，部署Kubernetes集羣機器須要知足如下幾個條件：docker

一臺或多臺機器，操做系統 CentOS7.x-86_x64json
硬件配置：2GB或更多RAM，2個CPU或更多CPU，硬盤30GB或更多bootstrap
能夠訪問外網，須要拉取鏡像，若是服務器不能上網，須要提早下載鏡像並導入節點windows
禁止swap分區centos

2. 準備環境

角色	IP
master	192.168.1.1
node1	192.168.1.2
node2	192.168.1.5

# 關閉防火牆 systemctl stop firewalld systemctl disable firewalld # 關閉selinux sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久 setenforce 0 # 臨時 # 關閉swap swapoff -a # 臨時 sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久 # 根據規劃設置主機名 hostnamectl set-hostname <hostname> # 在master添加hosts cat >> /etc/hosts << EOF 192.168.44.146 k8smaster 192.168.44.145 k8snode1 192.168.44.144 k8snode2 EOF  # 將橋接的IPv4流量傳遞到iptables的鏈 cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl --system # 生效  # 時間同步 yum install ntpdate -y ntpdate time.windows.com

3. 全部節點安裝Docker/kubeadm/kubelet

Kubernetes默認CRI（容器運行時）爲Docker，所以先安裝Docker。

3.1 安裝Docker、配置加速器

#添加docker官方的repo

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

#安裝docker

yum install -y docker-ce-18.09.9-3.el7

#配置鏡像加速器
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://lcs5rvt6.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
#查看docker信息檢查加速器地址
sudo docker info 
 Registry Mirrors:
  https://lcs5rvt6.mirror.aliyuncs.com/
 Live Restore Enabled: false
 Product License: Community Engine

3.2 添加阿里雲YUM軟件源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF

3.3 安裝kubeadm，kubelet和kubectl

因爲版本更新頻繁，這裏指定版本號部署：

$ yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0
$ systemctl enable kubelet && systemctl start kubelet

4. 部署Kubernetes Master

在192.168.1.1（Master）執行。

$ kubeadm init \ --apiserver-advertise-address=192.168.1.1 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.18.0 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=10.244.0.0/16 #192.168.1.1是apiserver的地址
#上述這步驟解釋：
下載k8s的組件images，稍等，能夠再開一個終端查看images變化。

 1 [root@k8s-master ~]# kubeadm init \
 2 >   --apiserver-advertise-address=192.168.1.1 \
 3 >   --image-repository registry.aliyuncs.com/google_containers \
 4 >   --kubernetes-version v1.18.0 \
 5 >   --service-cidr=10.96.0.0/12 \
 6 >   --pod-network-cidr=10.244.0.0/16
 7 W0304 10:07:21.552884   67589 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
 8 [init] Using Kubernetes version: v1.18.0
 9 [preflight] Running pre-flight checks
10     [WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
11     [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
12 [preflight] Pulling images required for setting up a Kubernetes cluster
13 [preflight] This might take a minute or two, depending on the speed of your internet connection
14 [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
15 [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
16 [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
17 [kubelet-start] Starting the kubelet
18 [certs] Using certificateDir folder "/etc/kubernetes/pki"
19 [certs] Generating "ca" certificate and key
20 [certs] Generating "apiserver" certificate and key
21 [certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.1]
22 [certs] Generating "apiserver-kubelet-client" certificate and key
23 [certs] Generating "front-proxy-ca" certificate and key
24 [certs] Generating "front-proxy-client" certificate and key
25 [certs] Generating "etcd/ca" certificate and key
26 [certs] Generating "etcd/server" certificate and key
27 [certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.1.1 127.0.0.1 ::1]
28 [certs] Generating "etcd/peer" certificate and key
29 [certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.1.1 127.0.0.1 ::1]
30 [certs] Generating "etcd/healthcheck-client" certificate and key
31 [certs] Generating "apiserver-etcd-client" certificate and key
32 [certs] Generating "sa" key and public key
33 [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
34 [kubeconfig] Writing "admin.conf" kubeconfig file
35 [kubeconfig] Writing "kubelet.conf" kubeconfig file
36 [kubeconfig] Writing "controller-manager.conf" kubeconfig file
37 [kubeconfig] Writing "scheduler.conf" kubeconfig file
38 [control-plane] Using manifest folder "/etc/kubernetes/manifests"
39 [control-plane] Creating static Pod manifest for "kube-apiserver"
40 [control-plane] Creating static Pod manifest for "kube-controller-manager"
41 W0304 10:08:28.041239   67589 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
42 [control-plane] Creating static Pod manifest for "kube-scheduler"
43 W0304 10:08:28.041875   67589 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
44 [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
45 [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
46 [apiclient] All control plane components are healthy after 15.002231 seconds
47 [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
48 [kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
49 [upload-certs] Skipping phase. Please see --upload-certs
50 [mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
51 [mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
52 [bootstrap-token] Using token: 39sbs2.rnezmmzq6k3nfazh
53 [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
54 [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
55 [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
56 [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
57 [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
58 [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
59 [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
60 [addons] Applied essential addon: CoreDNS
61 [addons] Applied essential addon: kube-proxy
62 
63 Your Kubernetes control-plane has initialized successfully!
64 
65 To start using your cluster, you need to run the following as a regular user:
66 
67   mkdir -p $HOME/.kube
68   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
69   sudo chown $(id -u):$(id -g) $HOME/.kube/config
70 
71 You should now deploy a pod network to the cluster.
72 Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
73   https://kubernetes.io/docs/concepts/cluster-administration/addons/
74 
75 Then you can join any number of worker nodes by running the following on each as root:
76 
77 kubeadm join 192.168.1.1:6443 --token 39sbs2.rnezmmzq6k3nfazh \
78     --discovery-token-ca-cert-hash sha256:5618c53639da0a3aebe95209614612a200ae084c04678737b9e18f2c523531f2

init初始化過程

#在初始的時候尾部會生成一個祕鑰，做用就是將各個node節點加入k8s羣集當中
 kubeadm join 192.168.1.1:6443 --token 39sbs2.rnezmmzq6k3nfazh 
     --discovery-token-ca-cert-hash sha256:5618c53639da0a3aebe95209614612a200ae084c04678737b9e18f2c523531f2

#images的變化，下載的各個組件
[root@k8s-master ~]# docker images
REPOSITORY                                                        TAG       IMAGE ID       CREATED         SIZE
registry.aliyuncs.com/google_containers/kube-proxy                v1.18.0   43940c34f24f   11 months ago   117MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.18.0   74060cea7f70   11 months ago   173MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.18.0   d3e55153f52f   11 months ago   162MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.18.0   a31f78c7c8ce   11 months ago   95.3MB
registry.aliyuncs.com/google_containers/pause                     3.2       80d28bedfe5d   12 months ago   683kB
registry.aliyuncs.com/google_containers/coredns                   1.6.7     67da37a9a360   13 months ago   43.8MB
registry.aliyuncs.com/google_containers/etcd                      3.4.3-0   303ce5db0e90   16 months ago   288MB

因爲默認拉取鏡像地址k8s.gcr.io國內沒法訪問，這裏指定阿里雲鏡像倉庫地址。

使用kubectl工具：

#在初始的完成的時候會有這三行命令，做用是開啓kubectl工具
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

5. 加入Kubernetes Node

在192.168.1.2/5（Node）執行。

向集羣添加新節點，執行在kubeadm init輸出的kubeadm join命令：

kubeadm join 192.168.1.1:6443 --token 39sbs2.rnezmmzq6k3nfazh
--discovery-token-ca-cert-hash sha256:5618c53639da0a3aebe95209614612a200ae084c04678737b9e18f2c523531f2

默認token有效期爲24小時，當過時以後，該token就不可用了。這時就須要從新建立token，操做以下：

kubeadm token create --print-join-command

#查看一下集羣信息
[root@k8s-master ~]# kubectl get nodes
NAME           STATUS     ROLES    AGE     VERSION
k8s-master     NotReady   master   14m     v1.18.0
k8s-worker01   NotReady   <none>   103s    v1.18.0
k8s-worker02   NotReady   <none>   9m46s   v1.18.0

6. 部署CNI網絡插件master操做

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

默認鏡像地址沒法訪問，sed命令修改成docker hub鏡像倉庫。

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

[root@k8s-master ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7ff77c879f-8kf97 1/1 Running 0 20m
coredns-7ff77c879f-qbmqm 1/1 Running 0 20m
etcd-k8s-master 1/1 Running 0 20m
kube-apiserver-k8s-master 1/1 Running 0 20m
kube-controller-manager-k8s-master 1/1 Running 0 20m
kube-flannel-ds-68ptv 1/1 Running 0 2m9s
kube-flannel-ds-7p6d9 1/1 Running 0 2m9s
kube-flannel-ds-zm5wh 1/1 Running 0 2m9s
kube-proxy-4nzft 1/1 Running 0 7m41s
kube-proxy-bc5rl 1/1 Running 0 20m
kube-proxy-g6msv 1/1 Running 0 15m
kube-scheduler-k8s-master 1/1 Running 0 20m

[root@k8s-master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 21m v1.18.0
k8s-worker01 Ready <none> 8m34s v1.18.0
k8s-worker02 Ready <none> 16m v1.18.0

#這是全部的節點都處於ready準備狀態

7. 測試kubernetes集羣

在Kubernetes集羣中建立一個pod，驗證是否正常運行：

#經過kubectl建立部署一個名稱nginx鏡像是nginx的pod

[root@k8s-master ~]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created

#經過kubectl部署nginx暴露80端口，類型爲節點端口
[root@k8s-master ~]# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed

#查看rc和service的列表
[root@k8s-master ~]# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-f89759699-24xd5 1/1 Running 0 36s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23m
service/nginx NodePort 10.101.245.166 <none> 80:30067/TCP 12s