10分鐘搭建Kubernetes容器集羣平臺（kubeadm）

官方提供Kubernetes部署3種方式

• minikube

Minikube是一個工具，能夠在本地快速運行一個單點的Kubernetes，嘗試Kubernetes或平常開發的用戶使用。不能用於生產環境。

官方文檔：https://kubernetes.io/docs/setup/minikube/

• kubeadm

kubeadm可幫助你快速部署一套kubernetes集羣。kubeadm設計目的爲新用戶開始嘗試kubernetes提供一種簡單的方法。目前是Beta版。

官方文檔：https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
https://kubernetes.io/docs/setup/independent/install-kubeadm/

• 二進制包

從官方下載發行版的二進制包，手動部署每一個組件，組成Kubernetes集羣。目前企業生產環境中主要使用該方式。
下載地址：https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1113

1. 安裝要求

• 操做系統
  • Ubuntu 16.04+
  • Debian 9
  • CentOS 7
  • RHEL 7
  • Fedora 25/26 (best-effort)
  • 其餘
• 內存2GB + ，2核CPU +
• 集羣節點之間能夠通訊

• 每一個節點惟一主機名，MAC地址和product_uuid

  • 檢查MAC地址：使用ip link或者ifconfig -a
  • 檢查product_uuid：cat /sys/class/dmi/id/product_uuid
• 禁止swap分區。這樣才能使kubelet正常工做

2. 準備環境

關閉防火牆：
# systemctl stop firewalld
# systemctl disable firewalld

關閉selinux：
# sed -i 's/enforcing/disabled/' /etc/selinux/config 
# setenforce 0

關閉swap：
# swapoff -a  # 臨時
# vim /etc/fstab  # 永久

添加主機名與IP對應關係：
# cat /etc/hosts
192.168.0.11 k8s-master
192.168.0.12 k8s-node1
192.168.0.13 k8s-node2

同步時間：
# yum install ntpdate -y
# ntpdate  ntp.api.bz

3. 安裝Docker

Docker與Kubernetes關係圖：

# yum install -y yum-utils device-mapper-persistent-data lvm2 

# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# yum install docker-ce-17.03.3.ce -y   #目前kubeadm最大支持docker-ce-17.03，因此要指定該版本安裝

# systemctl enable docker && systemctl start docker

若是提示container-selinux依賴問題，先安裝ce-17.03匹配版本：
# yum localinstall https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm

4. 安裝kubeadm，kubelet和kubectl

• kubeadm： 引導集羣的命令
• kubelet：集羣中運行任務的代理程序
• kubectl：命令行管理工具

4.1 添加阿里雲YUM軟件源

# cat << EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

4.2 安裝kubeadm，kubelet和kubectl

# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
# systemctl enable kubelet && systemctl start kubelet

注意：使用Docker時，kubeadm會自動檢查kubelet的cgroup驅動程序，並/var/lib/kubelet/kubeadm-flags.env在運行時將其設置在文件中。若是使用的其餘CRI，則必須在/etc/default/kubelet中cgroup-driver值修改成cgroupfs：

# cat /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni
# systemctl daemon-reload
# systemctl restart kubelet

5. 使用kubeadm建立單個Master集羣

5.1 默認下載鏡像地址在國外沒法訪問，先從準備好所需鏡像

保存到腳本之間運行：

K8S_VERSION=v1.11.2
ETCD_VERSION=3.2.18
DASHBOARD_VERSION=v1.8.3
FLANNEL_VERSION=v0.10.0-amd64
DNS_VERSION=1.1.3
PAUSE_VERSION=3.1
# 基本組件
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy-amd64:$K8S_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd-amd64:$ETCD_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:$PAUSE_VERSION
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:$DNS_VERSION
# 網絡組件
docker pull quay.io/coreos/flannel:$FLANNEL_VERSION
# 修改tag
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver-amd64:$K8S_VERSION k8s.gcr.io/kube-apiserver-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager-amd64:$K8S_VERSION k8s.gcr.io/kube-controller-manager-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler-amd64:$K8S_VERSION k8s.gcr.io/kube-scheduler-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy-amd64:$K8S_VERSION k8s.gcr.io/kube-proxy-amd64:$K8S_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd-amd64:$ETCD_VERSION k8s.gcr.io/etcd-amd64:$ETCD_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:$PAUSE_VERSION k8s.gcr.io/pause:$PAUSE_VERSION
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:$DNS_VERSION k8s.gcr.io/coredns:$DNS_VERSION

5.2 初始化Master

# kubeadm init --kubernetes-version=1.11.2 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.0.11

...

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run (as a regular user):

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the addon options listed at:
  http://kubernetes.io/docs/admin/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

5.3 安裝Pod網絡 - 插件

# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml

5.4 加入工做節點

在Node節點切換到root帳號執行：

# kubeadm join 192.168.0.11:6443 --token 6hk68y.0rdz1wdjyh85ntkr --discovery-token-ca-cert-hash sha256:d1d3f59ae37fbd632707cbeb9b095d0d0b19af535078091993c4bc4d9d2a7782

格式：kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>

6. kubernetes dashboard

先將yaml文件下載下來，修改裏面鏡像地址和Service NodePort類型。

# wget https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

修改鏡像地址：

# registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.0

修改Service：

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30001
  selector:
    k8s-app: kubernetes-dashboard

# kubectl apply -f kubernetes-dashboard.yaml

建立一個管理員角色：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: dashboard-admin
  namespace: kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: dashboard-admin
subjects:
  - kind: ServiceAccount
    name: dashboard-admin
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

# kubectl apply -f k8s-admin.yaml

使用上述建立帳號的token登陸Kubernetes Dashboard：

# kubectl get secret -n kube-system
# kubectl describe secret dashboard-admin-token-bwdjj  -n kube-system
...
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tYndkamoiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNzIyOTRmNTUtYjc1OC0xMWU4LThkY2UtMDAwYzI5ZGUyNWVhIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.0hQU5Di_P1OX1DcnW2AYzjDAED66EOrqhF5iupv39wvB8wE-aLRSQyp0twX2M8u1KMZ67n6LxbH17VwEQkMDRVXs7ZlUCyAAD6kHDz3k-f7PAzH5vcuyO4veQ9ooVjk3DKjrP4zXQChHllBB1wmD_oyLjoWxK3Z5MBTlVGzSixVwuQNpFPbuS6Z7iLGwUOgjI0cGZ9Tt6cXzcK81KfAEpDIP_CtFV_Jw4s98EgBex9mZh6vq1dcxr03qfuK--udd_8GWZctu_p_P15hZZLoKEm5GCbs6JGvKL2aao_DEHfLp3XYEnApojI91vU4qAqdkvMZ2qWQNGYv4KNi2yPOOJQ

技術彩蛋

在2018/2019年Docker/Kubernetes容器技術無疑是業內最火的技術。根據招聘簡介狀況來看，容器技術已成爲運維工程師、架構師必備技能。
爲幫助你們快速掌握這門主流技術，少走彎路，提升核心競爭力。決定寫《基於Kubernetes企業容器雲平臺落地與實踐》文章專欄，給朋友在企業落地容器雲平臺提供一些企業實踐性指導，但願本身所學所思的東西可以幫助到你們，可以有所啓發。

傳送門：基於Kubernetes企業容器雲平臺落地與實踐

若你在容器運維中，遇到容器方面的問題，能夠給我微信↓。一樣，若發現文章有任何紕漏，還請隨時指正，相互學習，共同進步！