Kubernetes集羣部署之四Master節點部署

Kubernetes Master節點部署三個服務：kube-apiserver、kube-controller-manager、kube-scheduler和一個命令工具kubectl.

Master節點來負責整個集羣的管理和控制，其中

kube-apiserver 服務提供了HTTP Rest接口的關鍵服務進程，是Kuberneters裏全部資源的增刪改查等操做的惟一入口，也是集羣控制的入口進程.

kube-controller-manager 服務 是kubernetes裏面全部資源對象的自動化控制中心，能夠理解爲資源對象的」大總管」

kube-scheduler 服務負責資源調度(pod調度)的進程，至關於公交公司的」調度室」。

Kubernetes API服務部署：

1.準備軟件包,軟件包有兩種下載方式:

方式一(推薦使用):

從https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md 頁面下載 client 或 server tar包 文件.

[root@k8s-master ~]# cd /usr/local/src/
[root@k8s-master src]# wget https://dl.k8s.io/v1.10.1/kubernetes.tar.gz
[root@k8s-master src]# wget https://dl.k8s.io/v1.10.1/kubernetes-server-linux-amd64.tar.gz
[root@k8s-master src]# wget https://dl.k8s.io/v1.10.1/kubernetes-client-linux-amd64.tar.gz
[root@k8s-master src]# wget https://dl.k8s.io/v1.10.1/kubernetes-node-linux-amd64.tar.gz

方式二：

準備軟件包從github release 頁面下載發佈版tar包,解壓後再執行下載腳本.

[root@k8s-master ~]# cd /usr/local/src/
[root@k8s-master src]#wget https://github.com/kubernetes/kubernetes/releases/download/v1.10.3/kubernetes.tar.gz
[root@k8s-master src]# tar -zxvf kubernetes.tar.gz
[root@k8s-master src]# ll
total 2664
drwxr-xr-x 9 root root     156 May 21 18:16 kubernetes
-rw-r--r-- 1 root root 2726918 May 21 19:15 kubernetes.tar.gz
[root@k8s-master src]# cd kubernetes/cluster/
[root@k8s-master cluster]# ./get-kube-binaries.sh 
...

解壓安裝包,並添加可執行文件：

[root@k8s-master kubernetes]# cd /usr/local/src/kubernetes
[root@k8s-master kubernetes]#cp server/bin/kube-apiserver /opt/kubernetes/bin/
[root@k8s-master kubernetes]#cp server/bin/kube-controller-manager /opt/kubernetes/bin/
[root@k8s-master kubernetes]#cp server/bin/kube-scheduler /opt/kubernetes/bin/

 2.建立生成CSR的 JSON 配置文件:

[root@k8s-master ssl]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# cat > kubernetes-csr.json <<EOF
{
  "CN": "kubernetes",
  "hosts": [
    "127.0.0.1",
    "10.200.3.105",
    "10.1.0.1",
    "10.254.0.2",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

3.生成 kubernetes 證書和私鑰,並分發到因此節點.

[root@k8s-master src]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
   -ca-key=/opt/kubernetes/ssl/ca-key.pem \
   -config=/opt/kubernetes/ssl/ca-config.json \
   -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

[root@k8s-master ssl]# cp kubernetes*.pem /opt/kubernetes/ssl/
[root@k8s-master ssl]# scp kubernetes*.pem 10.200.3.106:/opt/kubernetes/ssl/
[root@k8s-master ssl]# scp kubernetes*.pem 10.200.3.107:/opt/kubernetes/ssl/

4.建立 kube-apiserver 使用的客戶端 token 文件

[root@k8s-master ~]# head -c 16 /dev/urandom | od -An -t x | tr -d ' '
10a89e49bc403bce8fb134e5a2ae82f1
[root@k8s-master ~]# vim /opt/kubernetes/ssl/bootstrap-token.csv
10a89e49bc403bce8fb134e5a2ae82f1,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

5.建立基礎用戶名/密碼認證配置(密碼、用戶名、uid爲後面建立dashborad後用戶認證)

[root@k8s-master ~]# vim /opt/kubernetes/ssl/basic-auth.csv
admin,admin,1
readonly,readonly,2

.6.部署Kubernetes API Server

[root@k8s-master ~]# cat > /usr/lib/systemd/system/kube-apiserver.service <<EOF
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
ExecStart=/opt/kubernetes/bin/kube-apiserver \
  --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \
  --bind-address=10.200.3.105 \
  --insecure-bind-address=127.0.0.1 \
  --authorization-mode=Node,RBAC \
  --runtime-config=rbac.authorization.k8s.io/v1 \
  --kubelet-https=true \
  --anonymous-auth=false \
  --basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \
  --enable-bootstrap-token-auth \
  --token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \
  --service-cluster-ip-range=10.1.0.0/16 \
  --service-node-port-range=20000-40000 \
  --tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
  --tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
  --client-ca-file=/opt/kubernetes/ssl/ca.pem \
  --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --etcd-cafile=/opt/kubernetes/ssl/ca.pem \
  --etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \
  --etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \
  --etcd-servers=https://10.200.3.105:2379,https://10.200.3.106:2379,https://10.200.3.107:2379 \
  --enable-swagger-ui=true \
  --allow-privileged=true \
  --audit-log-maxage=30 \
  --audit-log-maxbackup=3 \
  --audit-log-maxsize=100 \
  --audit-log-path=/opt/kubernetes/log/api-audit.log \
  --event-ttl=1h \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

EOF

7.啓動API Server服務

[root@k8s-master ~]# systemctl daemon-reload
[root@k8s-master ~]# systemctl enable kube-apiserver
[root@k8s-master ~]# systemctl start kube-apiserver
[root@k8s-master ~]# systemctl status kube-apiserver

部署Controller Manager服務

[root@k8s-master ~]# cat > /usr/lib/systemd/system/kube-controller-manager.service <<EOF
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/opt/kubernetes/bin/kube-controller-manager \
  --address=127.0.0.1 \
  --master=http://127.0.0.1:8080 \
  --allocate-node-cidrs=true \
  --service-cluster-ip-range=10.1.0.0/16 \
  --cluster-cidr=10.2.0.0/16 \
  --cluster-name=kubernetes \
  --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \
  --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --root-ca-file=/opt/kubernetes/ssl/ca.pem \
  --leader-elect=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

EOF

啓動Controller Manager

[root@k8s-master ~]# systemctl daemon-reload
[root@k8s-master ~]# systemctl enable kube-controller-manager
[root@k8s-master ~]# systemctl start kube-controller-manager
[root@k8s-master ~]# systemctl status kube-controller-manager

部署Kubernetes Scheduler:

[root@k8s-master ~]# cat > /usr/lib/systemd/system/kube-scheduler.service <<EOF
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/opt/kubernetes/bin/kube-scheduler \
  --address=127.0.0.1 \
  --master=http://127.0.0.1:8080 \
  --leader-elect=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

EOF

啓動Kubernetes Scheduler

[root@k8s-master ~]# systemctl daemon-reload
[root@k8s-master ~]# systemctl enable kube-scheduler
[root@k8s-master ~]# systemctl start kube-scheduler
[root@k8s-master ~]# systemctl status kube-scheduler

部署kubectl 命令行工具

1.準備二進制命令包

[root@k8s-master ~]# cd /usr/local/src/kubernetes/client/bin/
[root@k8s-master bin]# cp kubectl /opt/kubernetes/bin/

2.建立 admin 證書籤名請求

[root@k8s-master ~]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# cat > admin-csr.json <<EOF
{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}

EOF

3.生成 admin 證書和私鑰：

[root@k8s-master ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
   -ca-key=/opt/kubernetes/ssl/ca-key.pem \
   -config=/opt/kubernetes/ssl/ca-config.json \
   -profile=kubernetes admin-csr.json | cfssljson -bare admin
   
[root@k8s-master ssl]# ls -l admin*
-rw-r--r-- 1 root root 1009 May 31 10:29 admin.csr
-rw-r--r-- 1 root root  230 May 31 10:21 admin-csr.json
-rw------- 1 root root 1679 May 31 10:29 admin-key.pem
-rw-r--r-- 1 root root 1399 May 31 10:29 admin.pem
[root@k8s-master ssl]# cp  admin*.pem /opt/kubernetes/ssl/
[root@k8s-master ssl]# 

4.設置集羣參數

[root@k8s-master ssl]# kubectl config set-cluster kubernetes \
   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
   --embed-certs=true \
   --server=https://10.200.3.105:6443
Cluster "kubernetes" set.Cluster "kubernetes" set.

5.設置客戶端認證參數

[root@k8s-master ssl]# kubectl config set-credentials admin \
   --client-certificate=/opt/kubernetes/ssl/admin.pem \
   --embed-certs=true \
   --client-key=/opt/kubernetes/ssl/admin-key.pem
User "admin" set.

6.設置上下文參數

[root@k8s-master ssl]# kubectl config set-context kubernetes \
   --cluster=kubernetes \
   --user=admin
Context "kubernetes" created.

7.設置默認上下文

[root@k8s-master ssl]# kubectl config use-context kubernetes
Switched to context "kubernetes".

8.使用kubectl工具

[root@k8s-master ssl]# kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok                   
controller-manager   Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
etcd-2               Healthy   {"health": "true"}   
etcd-1               Healthy   {"health": "true"}   
[root@k8s-master ssl]# 

9.驗證master節點功能：

# kubectl get componentstatuses
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   
scheduler            Healthy   ok                   
etcd-1               Healthy   {"health": "true"}   
etcd-0               Healthy   {"health": "true"}   
etcd-2               Healthy   {"health": "true"}   

 

參考文檔：https://github.com/unixhot/salt-kubernetes