K8s集羣部署（二）------ Master節點部署

Master節點要部署三個服務：API Server、Scheduler、Controller Manager。

apiserver提供集羣管理的REST API接口，包括認證受權、數據校驗以 及集羣狀態變動等

　　只有API Server才直接操做etcd

　　其餘模塊經過API Server查詢或修改數據

　　提供其餘模塊之間的數據交互和通訊的樞紐

 

scheduler負責分配調度Pod到集羣內的node節點

　   監聽kube-apiserver，查詢還未分配Node的Pod

　　根據調度策略爲這些Pod分配節點

 

controller-manager由一系列的控制器組成，它經過apiserver監控整個 集羣的狀態，並確保集羣處於預期的工做狀態

 

　　　　　

 

　

 

 

1.部署Kubernetes API服務部署

0.準備軟件包

cd /usr/local/src/kubernetes
cp server/bin/kube-apiserver /opt/kubernetes/bin/  \
cp server/bin/kube-controller-manager /opt/kubernetes/bin/ \
cp server/bin/kube-scheduler /opt/kubernetes/bin/

 

1.建立生成CSR的 JSON 配置文件

[root@k8s-master kubernetes]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# vim kubernetes-csr.json
{
  "CN": "kubernetes",
  "hosts": [
    "127.0.0.1",
    "10.0.3.225",     #Master IP地址
    "10.1.0.1",       ？？？
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

 

2.生成 kubernetes 證書和私鑰

cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
   -ca-key=/opt/kubernetes/ssl/ca-key.pem \
   -config=/opt/kubernetes/ssl/ca-config.json \
   -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
   
#拷貝證書到其餘節點
cp kubernetes*.pem /opt/kubernetes/ssl/  
scp kubernetes*.pem 10.0.3.226:/opt/kubernetes/ssl/ 
scp kubernetes*.pem 10.0.3.227:/opt/kubernetes/ssl/

 

3.建立 kube-apiserver 使用的客戶端 token 文件

[root@k8s-master ssl]# head -c 16 /dev/urandom | od -An -t x | tr -d ' '
4c7d89749d1e1a15e5fe55eb5e8446ec
[root@k8s-master ssl]# vim /opt/kubernetes/ssl/bootstrap-token.csv
4c7d89749d1e1a15e5fe55eb5e8446ec,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

 

4.建立基礎用戶名/密碼認證配置

vim /opt/kubernetes/ssl/basic-auth.csv
admin,admin,1
readonly,readonly,2

 

5.部署Kubernetes API Server

vim /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
ExecStart=/opt/kubernetes/bin/kube-apiserver \
  --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \
  --bind-address=10.0.3.225 \
  --insecure-bind-address=127.0.0.1 \
  --authorization-mode=Node,RBAC \
  --runtime-config=rbac.authorization.k8s.io/v1 \
  --kubelet-https=true \
  --anonymous-auth=false \
  --basic-auth-file=/opt/kubernetes/ssl/basic-auth.csv \
  --enable-bootstrap-token-auth \
  --token-auth-file=/opt/kubernetes/ssl/bootstrap-token.csv \
  --service-cluster-ip-range=10.1.0.0/16 \
  --service-node-port-range=20000-40000 \
  --tls-cert-file=/opt/kubernetes/ssl/kubernetes.pem \
  --tls-private-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \
  --client-ca-file=/opt/kubernetes/ssl/ca.pem \
  --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --etcd-cafile=/opt/kubernetes/ssl/ca.pem \
  --etcd-certfile=/opt/kubernetes/ssl/kubernetes.pem \
  --etcd-keyfile=/opt/kubernetes/ssl/kubernetes-key.pem \
  --etcd-servers=https://10.0.3.225:2379,https://10.0.3.226:2379,https://10.0.3.227:2379 \
  --enable-swagger-ui=true \
  --allow-privileged=true \
  --audit-log-maxage=30 \
  --audit-log-maxbackup=3 \
  --audit-log-maxsize=100 \
  --audit-log-path=/opt/kubernetes/log/api-audit.log \
  --event-ttl=1h \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log
Restart=on-failure
RestartSec=5
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

 

6.啓動API Server服務

systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver

查看API Server服務狀態
systemctl status kube-apiserver

　[root@k8s-master ssl]# netstat -lntup|grep kube-apiser
  tcp 0 0 10.0.3.225:6443 0.0.0.0:* LISTEN 27784/kube-apiserve    
  tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 27784/kube-apiserve        

 

 

 

部署ControllerManager服務

剛纔安裝包已經拷貝過去了，直接配置系統服務便可。

[root@k8s-master ssl]# vim /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/opt/kubernetes/bin/kube-controller-manager \
  --address=127.0.0.1 \
  --master=http://127.0.0.1:8080 \
  --allocate-node-cidrs=true \
  --service-cluster-ip-range=10.1.0.0/16 \
  --cluster-cidr=10.2.0.0/16 \
  --cluster-name=kubernetes \
  --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \
  --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \
  --root-ca-file=/opt/kubernetes/ssl/ca.pem \
  --leader-elect=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

 

啓動Controller Manager

systemctl daemon-reload
systemctl enable kube-controller-manager
systemctl start kube-controller-manager

#查看狀態
systemctl status kube-controller-manager
[root@k8s-master ssl]# netstat -lntup|grep kube-controll
tcp        0      0 127.0.0.1:10252         0.0.0.0:*               LISTEN      27899/kube-controll

 

 

 

 

部署Kubernetes Scheduler

[root@k8s-master ssl]#  vim /usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
ExecStart=/opt/kubernetes/bin/kube-scheduler \
  --address=127.0.0.1 \
  --master=http://127.0.0.1:8080 \
  --leader-elect=true \
  --v=2 \
  --logtostderr=false \
  --log-dir=/opt/kubernetes/log

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

 

 

啓動Kubernetes Scheduler

systemctl daemon-reload
systemctl enable kube-scheduler
systemctl start kube-scheduler

#查看服務狀態
systemctl status kube-scheduler
[root@k8s-master ssl]# netstat -lntup|grep kube-schedule
tcp        0      0 127.0.0.1:10251         0.0.0.0:*               LISTEN      27955/kube-schedule 

 

 

 

 

部署kubectl 命令行工具

     kubectl是經過API Server來管理k8s集羣的，kubectl和API Server之間通訊也須要證書認證。kubectl 只在Master管理節點安裝，下面來生成證書。

 

1.準備二進制命令包

[root@k8s-master ssl]# cd /usr/local/src/kubernetes/client/bin
[root@k8s-master bin]# cp kubectl /opt/kubernetes/bin/

 

2.建立 admin 證書籤名請求

[root@k8s-master bin]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# vim admin-csr.json
{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}

 

3.生成 admin 證書和私鑰

[root@k8s-master ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
>    -ca-key=/opt/kubernetes/ssl/ca-key.pem \
>    -config=/opt/kubernetes/ssl/ca-config.json \
>    -profile=kubernetes admin-csr.json | cfssljson -bare admin
2018/11/14 10:11:02 [INFO] generate received request
2018/11/14 10:11:02 [INFO] received CSR
2018/11/14 10:11:02 [INFO] generating key: rsa-2048
2018/11/14 10:11:02 [INFO] encoded CSR
2018/11/14 10:11:03 [INFO] signed certificate with serial number 725437256018406250545228596363344942073012526422
2018/11/14 10:11:03 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

#會生成4個文件
[root@k8s-master ssl]# ls -l admin*
-rw-r--r-- 1 root root 1009 Nov 14 10:11 admin.csr
-rw-r--r-- 1 root root  229 Nov 14 10:10 admin-csr.json
-rw------- 1 root root 1679 Nov 14 10:11 admin-key.pem
-rw-r--r-- 1 root root 1399 Nov 14 10:11 admin.pem

#移動到ssl證書目錄
[root@k8s-master ssl]# mv admin*.pem /opt/kubernetes/ssl/

 

4.設置集羣參數

[root@k8s-master ssl]# kubectl config set-cluster kubernetes \
>    --certificate-authority=/opt/kubernetes/ssl/ca.pem \
>    --embed-certs=true \
>    --server=https://10.0.3.225:6443
Cluster "kubernetes" set.

 

5.設置客戶端認證參數

[root@k8s-master ssl]# kubectl config set-credentials admin \
>    --client-certificate=/opt/kubernetes/ssl/admin.pem \
>    --embed-certs=true \
>    --client-key=/opt/kubernetes/ssl/admin-key.pem
User "admin" set.

 

6.設置上下文參數

[root@k8s-master ssl]#  kubectl config set-context kubernetes \
>    --cluster=kubernetes \
>    --user=admin
Context "kubernetes" created.

 

7.設置默認上下文

[root@k8s-master ssl]# kubectl config use-context kubernetes
Switched to context "kubernetes".

 

#敲了一大堆命令，實際上是在家目錄.kube/ 生成一個config配置文件，kubectl和API Server通訊就要使用到這個文件。 其餘節點想要運行kubectl 就要把這個文件拷貝過去
root@k8s-master ~]# cat  .kube/config

 

8.使用kubectl工具

[root@k8s-master ssl]# kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   
scheduler            Healthy   ok                   
etcd-1               Healthy   {"health": "true"}   
etcd-0               Healthy   {"health": "true"}   
etcd-2               Healthy   {"health": "true"}