Kubernetes安裝之九：配置node節點之kubelet

配置node節點須要提早在node準備工做

1.docker 環境+kubectl客戶端配置+kubeadm二進制文件

2.flanneld跨主機通訊

能夠參考前面的文章完成安裝配置

3.準備初始化環境

yum install -y conntrack ipvsadm ipset jq iptables curl sysstat libseccomp
複製代碼

4.修改內核參數

cat > /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
EOF

sysctl -p /etc/sysctl.d/kubernetes.conf
複製代碼

5.開啓內核模塊，設置防火牆規則，建立工做目錄

[root@localhost ~]# modprobe br_netfilter
[root@localhost ~]# modprobe ip_vs
[root@localhost ~]# iptables -P FORWARD ACCEPT
[root@localhost ~]# mkdir -p /var/lib/{kubelet,kube-proxy}
[root@localhost ~]# 
複製代碼

6.啓動相關的服務

systemctl daemon-reload
systemctl enable flanneld docker
systemctl restart flanneld docker
systemctl status flanneld docker複製代碼

7.node節點建立token和kubeconfig文件

[root@localhost ~]# kubectl config set-cluster kubernetes \
>       --certificate-authority=/etc/ssl/ca.pem \
>       --embed-certs=true \
>       --server=https://192.168.1.43:8443 \
>       --kubeconfig=kubelet-bootstrap.kubeconfig
Cluster "kubernetes" set.
複製代碼

[root@localhost ~]# kubectl config set-credentials kubelet-bootstrap \
>       --token=pnk1m9.fvbnpd37bp35ij4w \
>       --kubeconfig=kubelet-bootstrap.kubeconfig
User "kubelet-bootstrap" set.
複製代碼

[root@localhost ~]# kubectl config set-context default \
>       --cluster=kubernetes \
>       --user=kubelet-bootstrap \
>       --kubeconfig=kubelet-bootstrap.kubeconfig
Context "default" created.
複製代碼

[root@localhost ~]# kubectl config use-context default --kubeconfig=kubelet-bootstrap.kubeconfig
Switched to context "default".
複製代碼

8.檢查token狀態

[root@localhost ~]# kubeadm token list --kubeconfig ~/.kube/config
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION               EXTRA GROUPS
pnk1m9.fvbnpd37bp35ij4w   23h       2019-02-15T15:02:05+08:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-node01
複製代碼

9.編寫kubelet的配置文件

cat > /opt/kubernetes/cfg/kubelet.config.json <<EOF
{
  "kind": "KubeletConfiguration",
  "apiVersion": "kubelet.config.k8s.io/v1beta1",
  "authentication": {
    "x509": {
      "clientCAFile": "/etc/ssl/ca.pem"
    },
    "webhook": {
      "enabled": true,
      "cacheTTL": "2m0s"
    },
    "anonymous": {
      "enabled": false
    }
  },
  "authorization": {
    "mode": "Webhook",
    "webhook": {
      "cacheAuthorizedTTL": "5m0s",
      "cacheUnauthorizedTTL": "30s"
    }
  },
  "address": "192.168.1.44",
  "port": 10250,
  "readOnlyPort": 0,
  "cgroupDriver": "cgroupfs",
  "hairpinMode": "promiscuous-bridge",
  "serializeImagePulls": false,
  "featureGates": {
    "RotateKubeletClientCertificate": true,
    "RotateKubeletServerCertificate": true
  },
  "clusterDomain": "cluster.local.",
  "clusterDNS": ["10.254.0.2"]
}
EOF
複製代碼

• 說明：
• address：API 監聽地址，不能爲0.0.1，不然 kube-apiserver、heapster 等不能調用 kubelet 的 API
• readOnlyPort=0：關閉只讀端口(默認 10255)，等效爲未指定
• anonymous.enabled：設置爲 false，不容許匿名訪問 10250 端口
• x509.clientCAFile：指定簽名客戶端證書的 CA 證書，開啓 HTTP 證書認證
• webhook.enabled=true：開啓 HTTPs bearer token 認證
• 對於未經過 x509 證書和 webhook 認證的請求(kube-apiserver 或其餘客戶端)，將被拒絕，提示 Unauthorized
• mode=Webhook：kubelet 使用 SubjectAccessReview API 查詢 kube-apiserver 某 user、group 是否具備操做資源的權限(RBAC)
• RotateKubeletClientCertificate、featureGates.RotateKubeletServerCertificate：自動 rotate 證書，證書的有效期取決於 kube-controller-manager 的 --experimental-cluster-signing-duration 參數
• 須要 root 帳戶運行

10.配置kuebelet服務

cat > /usr/lib/systemd/system/kubelet.service <<EOF
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
ExecStart=/usr/bin/kubelet \\
  --bootstrap-kubeconfig=/opt/kubernetes/cfg/kubelet-bootstrap.kubeconfig \\
  --cert-dir=/etc/ssl \\
  --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \\
  --config=/opt/kubernetes/cfg/kubelet.config.json \\
  --hostname-override=k8s-node01 \\
  --pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest \\
  --allow-privileged=true \\
  --alsologtostderr=true \\
  --logtostderr=false \\
  --log-dir=/var/log/kubernetes \\
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF
複製代碼

11.Bootstrap Token Auth 和授予權限

[root@localhost ~]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --group=system:bootstrappers
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created
複製代碼

12.啓動服務

systemctl daemon-reload
systemctl enable kubelet
systemctl restart kubelet
systemctl status kubelet複製代碼

13.建立kube證書

cat > /etc/ssl/kube-proxy/kube-proxy-csr.json <<EOF
{
  "CN": "system:kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "ChengDu",
      "L": "ChengDu",
      "O": "k8s",
      "OU": "dessler"
    }
  ]
}
EOF
複製代碼

cfssl gencert -ca=/etc/ssl/ca.pem \
  -ca-key=/etc/ssl/ca-key.pem \
  -config=/etc/ssl/ca-config.json \
  -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy
複製代碼

[root@host40 kube-proxy]# ls
kube-proxy.csr  kube-proxy-csr.json  kube-proxy-key.pem  kube-proxy.pem
複製代碼

14.分發證書，及二進制文件

15.配置kubeconfig文件

root@localhost kube-proxy]# kubectl config set-cluster kubernetes \
>   --certificate-authority=/etc/ssl/ca.pem \
>   --embed-certs=true \
>   --server=https://192.168.1.43:8443 \
>   --kubeconfig=kube-proxy.kubeconfig
Cluster "kubernetes" set.
複製代碼

[root@localhost kube-proxy]# kubectl config set-credentials kube-proxy \
>   --client-certificate=/etc/ssl/kube-proxy/kube-proxy.pem \
>   --client-key=/etc/ssl/kube-proxy/kube-proxy-key.pem \
>   --embed-certs=true \
>   --kubeconfig=kube-proxy.kubeconfig
User "kube-proxy" set.
複製代碼

[root@localhost kube-proxy]# kubectl config set-context default \
>   --cluster=kubernetes \
>   --user=kube-proxy \
>   --kubeconfig=kube-proxy.kubeconfig
Context "default" created.
複製代碼

[root@localhost kube-proxy]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
Switched to context "default".
複製代碼

16.準備配置文件

cat > /opt/kubernetes/cfg/kube-proxy.config.yaml <<EOF
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 192.168.1.44
clientConnection:
  kubeconfig: /opt/kubernetes/cfg/kube-proxy.kubeconfig
clusterCIDR: 172.30.0.0/16
healthzBindAddress: 192.168.1.44:10256
hostnameOverride: k8s-node01
kind: KubeProxyConfiguration
metricsBindAddress: 192.168.1.44:10249
mode: "ipvs"
EOF
複製代碼

17.準備kube-proxy systemctl啓動文件

• 說明：
• bindAddress: 監聽地址
• kubeconfig: 鏈接 apiserver 的 kubeconfig 文件
• clusterCIDR: 必須與 kube-controller-manager 的--cluster-cidr 選項值一致；kube-proxy 根據 --cluster-cidr 判斷集羣內部和外部流量，指定 --cluster-cidr 或 --masquerade-all 選項後 kube-proxy 纔會對訪問 Service IP 的請求作 SNAT
• hostnameOverride: 參數值必須與 kubelet 的值一致，不然 kube-proxy 啓動後會找不到該 Node，從而不會建立任何 ipvs 規則
• mode: 使用 ipvs 模式

18.啓動服務

systemctl daemon-reload
systemctl enable kube-proxy
systemctl restart kube-proxy
systemctl status kube-proxy
複製代碼