Kubernetes : 多節點 k8s 集羣實踐

須要說明的是, 這本書裏有不少描述的東西和實踐的案例不吻合. python

Kubernets 集羣架構

架構圖linux

Server Listdocker

節點名稱	節點 IP
k8s-master	10.10.10.10
k8s-node1	10.10.10.11
k8s-node2	10.10.10.12

shell

配置主機 hostsapi

須要說明的是, 這個不是必須的. 可是若是不配置主機 hosts, 後續配置 kubelet 參數KUBELET_API_SERVER 以及 proxy 中參數 KUBE_PROXY_ARGS 的時候指定 api-server 的具體 IP 地址便可. bash

在三臺節點上分別配置 hosts 文件:網絡

/etc/hosts架構

10.10.10.10 k8s-master
10.10.10.11 k8s-node1
10.10.10.12 k8s-nodes

dom

節點服務說明

Master : etcd , kube-apiserver , kube-controller-manager, kube-scheduler.

Node : docker, kubelet, kube-proxy

配置 Master

安裝 kubernetes

yum install etcd kubernetes

關閉防火牆

systemctl stop firewalld
ssytemctl disable firewalld

sed -i "s/SELINUX=enforcing/SELINUX=disabled/1" /etc/selinux/config 
setenforce  0

配置文件

/etc/kubernetes/apiserver

KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,,ResourceQuota"
KUBE_API_ARGS="--insecure-port=8080 --service-node-port-range=1-65535"

/etc/kubernetes/config

KUBE_LOGTOSTDERR="--logtostderr=false"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://127.0.0.1:8080"

/etc/kubernetes/controller-manager

KUBE_CONTROLLER_MANAGER_ARGS="--master=http://127.0.0.1:8080"

/etc/kubernetes/scheduler

KUBE_SCHEDULER_ARGS="--master=http://127.0.0.1:8080"

啓動服務

kube-controller-manager 和 kube-scheduler 依賴 kube-apiserver , kube-apiserver 依賴 etcd. 因此啓動順序以下:

systemctl start etcd
systemctl start kube-apiserver
systemctl start kube-controller-manager
systemctl start kube-scheduler

systemctl enable etcd
systemctl enable kube-apiserver
systemctl enable kube-controller-manager
systemctl enable kube-scheduler

或者在 shell CLI 以下啓動:

for svc in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl start $svc; systemctl enable $svc;done

至此, master 節點已經配置完成了.

配置 Node

安裝 kubernetes 和 Docker

yum install kubernetes docker

關閉防火牆

systemctl stop firewalld
ssytemctl disable firewalld

sed -i "s/SELINUX=enforcing/SELINUX=disabled/1" /etc/selinux/config
setenforce  0

配置文件

/etc/sysconfig/docker

OPTIONS='--selinux-enabled=false --log-driver=journald --signature-verification=false --insecure-registry gcr.io'
if [ -z "${DOCKER_CERT_PATH}" ]; then
    DOCKER_CERT_PATH=/etc/docker
fi

/etc/kubernetes/kubelet (k8s-node1)

KUBELET_ADDRESS="--address=10.10.10.11"
KUBELET_HOSTNAME="--hostname-override=k8s-node1"
KUBELET_API_SERVER="--api-servers=http://k8s-master:8080"
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
KUBELET_ARGS="--cluster-dns=8.8.8.8 --cluster-domain=kubelet-node1"

ps: k8s-node2 配置不一樣之處在於 KUBELET_ADDRESS & KUBELET_HOSTNAME

/etc/kubetnetes/proxy

KUBE_PROXY_ARGS="--master=http://k8s-master:8080"

啓動服務

kubelet 依賴 docker 服務,因此先啓動 docker.

systemctl start docker
systemctl start kubelet
systemctl start kube-proxy

systemctl enable docker
systemctl enable kubelet
systemctl enable kube-proxy

或者在 shell CLI 執行:

for svc in docker kubelet kube-proxy; do systemctl start $svc; systemctl enable $svc;done

這樣一個 multi node k8s cluster 就搭建完了. 能夠在 master 節點執行: kubectl get nodes 來查看節點.

node 是註冊機制, 因此只要一個網絡中的 kubelet 的 apiserver 配置的是 master 節點, master 就會將其歸入.

搭建過程當中遇到問題, 多使用 describe 去查看具體信息.

好比建立 rc 一直處於 pending 狀態, 能夠執行: kubectl describe rc <rc name> .

Q&A

報錯信息均使用 kubelet describe 得到

no API token found for service account default/default, retry after the token is automatically created and added to the service account

/etc/kubernetes/apiserver 中 KUBE_ADMISSION_CONTROL 參數去掉 ServiceAccount. KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

<br>

****

```sh
Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request.  details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)"

yum -y install python-rhsm-certificates

建立文件: /etc/rhsm/ca/redhat-uep.pem