ubuntu 使用阿里雲鏡像源快速搭建kubernetes 1.15.2集羣

時間 2019-12-19

標籤 ubuntu 使用阿里鏡像快速搭建 kubernetes 1.15.2 集羣欄目 Ubuntu 简体版

原文原文鏈接

1、概述

搭建k8s集羣時，須要訪問google，下載相關鏡像以及安裝軟件，很是麻煩。html

正好阿里雲提供了k8s的更新源，國內用戶就能夠直接使用了。node

2、環境介紹

操做系統	主機名	IP地址	功能	配置
ubuntu-16.04.5-server-amd64	k8s-master	192.168.10.130	主節點	2核4G
ubuntu-16.04.5-server-amd64	k8s-node1	192.168.10.131	從節點	2核4G
ubuntu-16.04.5-server-amd64	k8s-node2	192.168.10.132	從節點	2核4G

注意：請確保CPU至少2核，內存2Glinux

3、安裝前準備

主機名

確保3臺主機的 /etc/hostname 已經修改成正確的主機名，修改後，請重啓系統。git

時間

務必保證3臺服務器的時區是同樣的，強制更改時區爲上海，執行如下命令github

ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
bash -c "echo 'Asia/Shanghai' > /etc/timezone"

安裝ntpdatedocker

apt-get install -y ntpdate

若是出現如下錯誤flask

E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

執行2個命令解決bootstrap

sudo rm /var/cache/apt/archives/lock
sudo rm /var/lib/dpkg/lock

使用阿里雲的時間服務器更新ubuntu

ntpdate ntp1.aliyun.com

3臺服務器都執行一下，確保時間一致！vim

請確保防火牆都關閉了！

4、正式開始

禁用swap

全部主機

sudo sed -i '/swap/ s/^/#/' /etc/fstab
sudo swapoff -a

安裝Docker

更新apt源,並添加https支持（全部主機）

sudo apt-get update && sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y

使用utc源添加GPG Key (全部主機)

curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add

添加Docker-ce穩定版源地址（全部主機）

sudo add-apt-repository "deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu $(lsb_release -cs) stable"

安裝docker-ce（全部主機）

安裝最新版docker

sudo apt-get update
sudo apt-get install -y docker-ce=5:19.03.1~3-0~ubuntu-xenial

安裝kubelet，kubeadm，kubectl

添加apt key以及源（全部主機）

sudo apt update && sudo apt install -y apt-transport-https curl
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >>/etc/apt/sources.list.d/kubernetes.list

安裝（全部主機）

最新版kubelet

sudo apt update
sudo apt install -y kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00
sudo apt-mark hold kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00

安裝kubernetes集羣（僅master）

sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.15.2 --pod-network-cidr=192.169.0.0/16 | tee /etc/kube-server-key

參數解釋：

--image-repository 指定鏡像源，指定爲阿里雲的源，這樣就會避免在拉取鏡像超時，若是沒問題，過幾分鐘就能看到成功的日誌輸入

--kubernetes-version 指定版本

--pod-network-cidr 指定pod網絡地址。設置爲內網網段！

三大內網網絡爲：

C類：192.168.0.0-192.168.255.255
B類：172.16.0.0-172.31.255.255
A類：10.0.0.0-10.255.255.255

輸出：

[init] Using Kubernetes version: v1.15.2
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.10.130]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.10.130 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.10.130 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 41.507944 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: bz16uu.olqxoh5q5bnt50sd
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.10.130:6443 --token bz16uu.olqxoh5q5bnt50sd \
    --discovery-token-ca-cert-hash sha256:9177017ff3016dbb2aadf7484f7823f8b963c989fe9ecdccbe601c9305ce000f

View Code

出現WARNING信息，能夠忽略。

輸出信息，會保存到 /etc/kube-server-key 文件中

拷貝kubeconfig文件到家目錄的.kube目錄 （僅master）

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

安裝網絡插件，讓pod之間通訊（僅master）

使用最新版的

kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml

查看kube-system命名空間下的pod狀態（僅master）

kubectl get pod -n kube-system

等待1分鐘，效果以下：

NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7bd78b474d-lpfvf   0/1     Running   0          67s
calico-node-vfm28                          1/1     Running   0          67s
coredns-bccdc95cf-dm4pb                    1/1     Running   0          111s
coredns-bccdc95cf-lvhcg                    1/1     Running   0          111s
etcd-k8s-master                            1/1     Running   0          69s
kube-apiserver-k8s-master                  1/1     Running   0          67s
kube-controller-manager-k8s-master         1/1     Running   0          59s
kube-proxy-jpqsq                           1/1     Running   0          111s
kube-scheduler-k8s-master                  1/1     Running   0          56s

查看加入節點命令（僅master）

cat /etc/kube-server-key | tail -2

輸出：

kubeadm join 192.168.10.130:6443 --token bz16uu.olqxoh5q5bnt50sd \
    --discovery-token-ca-cert-hash sha256:9177017ff3016dbb2aadf7484f7823f8b963c989fe9ecdccbe601c9305ce000f

加入node節點 (僅node)

在每一個node上執行

kubeadm join 192.168.10.130:6443 --token bz16uu.olqxoh5q5bnt50sd \
    --discovery-token-ca-cert-hash sha256:9177017ff3016dbb2aadf7484f7823f8b963c989fe9ecdccbe601c9305ce000f

等待5分鐘，查看集羣狀態（僅master）

root@k8s-master:~# kubectl get nodes
NAME         STATUS   ROLES    AGE     VERSION
k8s-master   Ready    master   5m54s   v1.15.2
k8s-node1    Ready    <none>   73s     v1.15.2
k8s-node2    Ready    <none>   71s     v1.15.2

命令補全

（僅master）

apt-get install bash-completion

source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
source  ~/.bashrc

4、部署應用

（僅master）

這裏以flask爲例：

vim flask.yaml

內容以下：

apiVersion: extensions/v1beta1
kind: Deployment 
metadata: 
  name: flaskapp-1
spec: 
  replicas: 1
  template: 
    metadata: 
      labels: 
        name: flaskapp-1
    spec: 
      containers: 
        - name: flaskapp-1
          image: jcdemo/flaskapp
          ports:
          - containerPort: 5000

---
apiVersion: v1
kind: Service
metadata:
  name: flaskapp-1
  labels:
    name: flaskapp-1
spec:
  type: NodePort
  ports:
  - port: 5000
    name: flaskapp-port
    targetPort: 5000
    protocol: TCP
    nodePort: 30005
  selector:
    name: flaskapp-1

啓動應用

kubectl apply -f flask.yaml

查看應用狀態

root@k8s-master:~# kubectl get pods -o wide
NAME                          READY   STATUS    RESTARTS   AGE   IP              NODE        NOMINATED NODE   READINESS GATES
flaskapp-1-59698bc97d-2xnhb   1/1     Running   0          24s   192.168.36.65   k8s-node1   <none>           <none>

以上信息能夠說明，這個pod運行在k8s-node1這臺主機上面

ping pod ip地址，若是能夠ping通，說明calico插件運行正常。

root@k8s-master:~# ping 192.168.36.65 -c 1
PING 192.168.36.65 (192.168.36.65) 56(84) bytes of data.
64 bytes from 192.168.36.65: icmp_seq=1 ttl=63 time=6.77 ms

--- 192.168.36.65 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 6.777/6.777/6.777/0.000 ms

測試 pod 是否能夠上網

root@k8s-master:~# kubectl exec -it flaskapp-1-59698bc97d-2xnhb -- ping www.baidu.com -c 1
PING www.baidu.com (61.135.169.125): 56 data bytes
64 bytes from 61.135.169.125: seq=0 ttl=53 time=27.079 ms

--- www.baidu.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 27.079/27.079/27.079 ms

使用curl訪問pod ip的服務

root@k8s-master:~# curl 192.168.36.65:5000
<html><head><title>Docker + Flask Demo</title></head><body><table><tr><td> Start Time </td> <td>2019-Aug-11 04:52:17</td> </tr><tr><td> Hostname </td> <td>flaskapp-1-59698bc97d-2xnhb</td> </tr><tr><td> Local Address </td> <td>192.168.36.65</td> </tr><tr><td> Remote Address </td> <td>192.168.235.192</td> </tr><tr><td> Server Hit </td> <td>1</td> </tr></table></body></html>

查看svc端口

root@k8s-master:~# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
flaskapp-1   NodePort    10.107.181.43   <none>        5000:30005/TCP   3m40s
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          10m

直接網頁訪問k8s-node1的30005端口

http://192.168.10.131:30005/

效果以下：

5、部署dashboard可視化插件

概述

在 Kubernetes Dashboard 中能夠查看集羣中應用的運行狀態，也可以建立和修改各類 Kubernetes 資源，好比 Deployment、Job、DaemonSet 等。用戶能夠 Scale Up/Down Deployment、執行 Rolling Update、重啓某個 Pod 或者經過嚮導部署新的應用。Dashboard 能顯示集羣中各類資源的狀態以及日誌信息。

能夠說，Kubernetes Dashboard 提供了 kubectl 的絕大部分功能，你們能夠根據狀況進行選擇。

github地址：

https://github.com/kubernetes/dashboard

安裝

Kubernetes 默認沒有部署 Dashboard，可經過以下命令安裝：

kubectl apply -f http://mirror.faasx.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

查看service

root@k8s-master:~# kubectl --namespace=kube-system get deployment kubernetes-dashboard
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-dashboard   1/1     1            1           5m23s
root@k8s-master:~# kubectl --namespace=kube-system get service kubernetes-dashboard
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes-dashboard   ClusterIP   10.100.111.103   <none>        443/TCP   5m28s

查看pod

確保狀態是Running

root@k8s-master:~# kubectl get pod --namespace=kube-system -o wide | grep dashboard
kubernetes-dashboard-8594bd9565-t78bj   1/1     Running   0          8m41s   192.169.2.7      k8s-node2    <none>           <none>

容許外部訪問

注意：會佔用終端

kubectl proxy --address='0.0.0.0'  --accept-hosts='^*$'

配置登陸權限

Dashboard 支持 Kubeconfig 和 Token 兩種認證方式，爲了簡化配置，咱們經過配置文件 dashboard-admin.yaml 爲 Dashboard 默認用戶賦予 admin 權限。

vim dashboard-admin.yml

內容以下：

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels: 
     k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

執行kubectl apply使之生效

kubectl apply -f dashboard-admin.yml

經過瀏覽器訪問

注意：192.168.10.130爲master ip

http://192.168.10.130:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.

如今直接點擊登陸頁面的跳過

就能夠進入 Dashboard 了，效果以下：

關於dashboard界面結構介紹，請參考連接：

http://www.javashuo.com/article/p-xnldwumu-gm.html

6、解決k8s 外網沒法訪問nodePort問題

上面flask的例子，沒法經過master ip+nodeport訪問。

是由於防火牆阻止了

root@k8s-master:~# iptables -xnL|grep FORWARD 
Chain FORWARD (policy DROP)
cali-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* cali:wUHhoiAYhphO9Mso */
KUBE-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
Chain KUBE-FORWARD (1 references)
Chain cali-FORWARD (1 references)