Ubuntu18使用kubeadm安裝kubernetes1.12

安裝前準備

virtualbox安裝三個Ubuntu虛擬機

k8s-master 和 k8s-slave一、k8s-slave2 爲了上網，都是雙網卡（NAT和Host-only）

關閉防火牆

sudo ufw disable

關閉swap

sudo swapoff -a

安裝Docker

卸載舊docker

sudo apt-get remove docker docker-engine docker.io

完全刪除docker-ce和docker-cli

sudo apt-get autoremove docker-ce

安裝依賴

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

添加docker的gpg key

爲了使用https訪問，使用阿里雲加速

curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -

設置docker鏡像源

sudo add-apt-repository \
   "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

更新源

sudo apt update && sudo apt-get update

安裝docker 18.06

查看docker-ce版本

apt-cache madison docker-ce

安裝指定版本

sudo apt-get install -y docker-ce=18.06.3~ce~3-0~ubuntu

若是由於libltdl版本低，能夠安裝新版本 wget http://launchpadlibrarian.net/236916213/libltdl7_2.4.6-0.1_amd64.deb && sudo dkpg -i libltdl7_2.4.6-0.1_amd64.deb

默認啓動

sudo systemctl enable docker && sudo systemctl start docker

修改docker啓動參數，進行加速

$ sudo tee /etc/docker/daemon.json <<-'EOF'
{
 "registry-mirrors": ["https://xxxxxxxx.mirror.aliyuncs.com"]
}
EOF

將xxxxxx替換爲在阿里雲申請的路徑

安裝kubernetes

更新apt源

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

sudo curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

sudo tee /etc/apt/sources.list.d/kubernetes.list <<-'EOF'
deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main
EOF

sudo apt-get update

安裝kubectl kubeadm kubelet

在master和node都安裝

sudo apt-get install -y kubelet=1.12.8-00 kubeadm=1.12.8-00 kubectl=1.12.8-00
sudo apt-mark hold kubelet=1.12.8-00 kubeadm=1.12.8-00 kubectl=1.12.8-00

啓動kubelet

sudo systemctl enable kubelet && sudo systemctl start kubelet

初始化集羣

sudo kubeadm init --apiserver-advertise-address 192.168.56.112 --pod-network-cidr=10.244.0.0/16 --kubernetes-version=1.12.8

安裝網絡插件

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

加入node

sudo kubeadm join 192.168.56.112:6443 --token xxxxxx --discovery-token-ca-cert-hash sha256:xxxxxxxx

將以前kubeadm init後，輸出的加入node指令拷貝到node執行。

遇到的問題

沒法啓動kubelet

journalctl -u kubelet 看到報錯： failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/ku 這個不影響，不用關係，啓動了kubernetes後就會正常

kubectl沒法連上服務

執行kubectl看到報錯 The connection to the server localhost:8080 was refused - did you specify the right host or port? 這是因爲kubernetes默認不開放http 8080訪問。

能夠在/etc/kubernetes/manifests/kube-apiserver.yaml看到開啓了https 6443訪問。 也能夠經過sudo netstat -apn | grep tcp | grep LISTEN看到開的的端口

能夠使用sudo kubectl --kubeconfig=/etc/kubernetes/admin.conf api-versions訪問，admin.conf中有端口以及key

使用阿里雲apt源致使沒法安裝

備份/etc/apt/sources.list後替換爲如下內容

deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

執行sudo apt update && sudo apt-get update更新apt源。可是致使了 package xxx has no installation candidate，使用Ubuntu官方源就OK

pod狀態異常

安裝完集羣后，生成一個pod，pod一直是pending狀態。查看pod的描述

sudo kubectl describe pod helloworld

pod描述信息

.......
Events:
  Type     Reason            Age                From               Message
  ----     ------            ----               ----               -------
  Warning  FailedScheduling  32s (x2 over 32s)  default-scheduler  0/3 nodes are available: 3 node(s) had taints that the pod didn't tolerate.

執行kubectl taint nodes --all node-role.kubernetes.io/master-，node正常使用參與調度

node狀態異常

service沒法訪問，通查看node狀態

sudo kubectl get pod -o yaml

存儲node網絡異常狀況

......
conditions:
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has sufficient disk space available
      reason: KubeletHasSufficientDisk
      status: "False"
      type: OutOfDisk
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has sufficient memory available
      reason: KubeletHasSufficientMemory
      status: "False"
      type: MemoryPressure
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has no disk pressure
      reason: KubeletHasNoDiskPressure
      status: "False"
      type: DiskPressure
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has sufficient PID available
      reason: KubeletHasSufficientPID
      status: "False"
      type: PIDPressure
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: 'runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
        message:docker: network plugin is not ready: cni config uninitialized'
      reason: KubeletNotReady
      status: "False"
      type: Ready
......

第一次出現問題是由於沒有安裝flannel網絡插件。

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

安裝完畢網絡插件後，提示cni config uninitialized。

先關閉node上的kubelet； 而後使用kubeadm reset|| kubeadm join ..., 將node從新加入集羣

系統宕機後重啓服務

1. 關閉防火牆
2. 關閉swap
3. 重啓docker
4. 重啓kubelet
5. 而後使用docker先重啓pause容器，而後在重啓其餘容器
6. 也能夠使用kubeadm reset重置kubernetes，而後從新初始化集羣
7. 注意使用journal查看docker和kubelet日誌，另外要主要docker和kubelet的systemd啓動參數

參考資料

基於 Ubuntu 18.04 安裝部署K8s集羣 Ubuntu-18.04使用kubeadm安裝kubernetes-1.12.0 kubeadm安裝k8s集羣 apt-get 軟件源文件格式以及ubuntu 18.04更換阿里源 [cni config uninitialized解決方案](cni config uninitialized "cni config uninitialized解決方案") cni config uninitialized