Kubernetes1.13.1部署Kuberneted-dashboard v1.10.1

參考文檔

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#deploying-the-dashboard-ui
https://github.com/kubernetes/kubernetes/tree/7f23a743e8c23ac6489340bbb34fa6f1d392db9d/cluster/addons/dashboard
https://github.com/kubernetes/dashboard
https://blog.csdn.net/nklinsirui/article/details/80581286
https://github.com/kubernetes/dashboard/issues/3472

文檔目錄

• kubernetes1.13.1+etcd3.3.10+flanneld0.10集羣部署
• kubernetes1.13.1部署kuberneted-dashboard v1.10.1
• kubernetes1.13.1部署coredns
• kubernetes1.13.1部署ingress-nginx並配置https轉發dashboard
• kubernetes1.13.1部署metrics-server0.3.1
• kubernetes1.13.1集羣使用ceph rbd存儲塊
• kubernetes1.13.1集羣結合ceph rbd部署最新版本jenkins
• kubernetes1.13.1集羣安裝包管理工具helm
• kubernetes1.13.1集羣集成harbor-helm

1、填坑

按照官網文檔一條命令便可，可是國內顯然不是這樣，首先要填許多坑才行

坑一：Docker鏡像

一、註冊阿里雲帳戶構建本身的鏡像

能夠關聯github構建，這樣就能夠把國外鏡像生成爲阿里雲鏡像
https://github.com/minminmsn/k8s1.13/tree/master/kubernetes-dashboard-amd64/Dockerfile

二、下載docker鏡像

docker pull registry.cn-beijing.aliyuncs.com/minminmsn/kubernetes-dashboard:v1.10.1

坑二：SSL證書

證書不對或者用auto建立的證書會報錯，報錯見https://github.com/kubernetes/dashboard/issues/3472

一、若是購買有的證書的話，把證書文件放在certs/目錄下建立secret便可

[root@elasticsearch01 /]# ls certs/
minminmsn.crt  minminmsn.csr  minminmsn.key

[root@elasticsearch01 /]# kubectl create secret generic kubernetes-dashboard-certs --from-file=certs -n kube-system
secret/kubernetes-dashboard-certs created

二、若是沒有購買的話須要自定義生成證書，步驟以下

[root@elasticsearch01 /]# mkdir /certs
[root@elasticsearch01 /]# openssl req -nodes -newkey rsa:2048 -keyout certs/dashboard.key -out certs/dashboard.csr -subj "/C=/ST=/L=/O=/OU=/CN=kubernetes-dashboard"
Generating a 2048 bit RSA private key
................+++
..............................................+++
writing new private key to 'certs/dashboard.key'
-----
No value provided for Subject Attribute C, skipped
No value provided for Subject Attribute ST, skipped
No value provided for Subject Attribute L, skipped
No value provided for Subject Attribute O, skipped
No value provided for Subject Attribute OU, skipped
[root@elasticsearch01 /]# ls /certs
dashboard.csr  dashboard.key

[root@elasticsearch01 /]# openssl x509 -req -sha256 -days 365 -in certs/dashboard.csr -signkey certs/dashboard.key -out certs/dashboard.crt
Signature ok
subject=/CN=kubernetes-dashboard
Getting Private key
[root@elasticsearch01 /]# ls certs/
dashboard.crt  dashboard.csr  dashboard.key

[root@elasticsearch01 /]# kubectl create secret generic kubernetes-dashboard-certs --from-file=certs -n kube-system
secret/kubernetes-dashboard-certs created

坑三：修改service配置，將type: ClusterIP改爲NodePort,便於經過Node端口訪問

[root@elasticsearch01 /]# wget https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
[root@elasticsearch01 /]# vim /k8s/yaml/kubernetes-dashboard.yaml 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard

2、部署Kubernetes-dashboard

修改鏡像地址爲registry.cn-beijing.aliyuncs.com/minminmsn/kubernetes-dashboard:v1.10.1便可部署

[root@elasticsearch01 /]# vim /k8s/yaml/kubernetes-dashboard.yaml 
    spec:
      containers:
      - name: kubernetes-dashboard
        image: registry.cn-beijing.aliyuncs.com/minminmsn/kubernetes-dashboard:v1.10.1

[root@elasticsearch01 /]# kubectl create -f /k8s/yaml/kubernetes-dashboard.yaml 
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard created
Error from server (AlreadyExists): error when creating "/k8s/yaml/kubernetes-dashboard.yaml": secrets "kubernetes-dashboard-certs" already exists

[root@elasticsearch01 /]# kubectl get pods -n kube-system
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-dashboard-cb55bd5bd-4jsh7   1/1     Running   0          21s
[root@elasticsearch01 /]# kubectl get svc -n kube-system
NAME                   TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   NodePort   10.254.140.115   <none>        443:41579/TCP   31s
[root@elasticsearch01 /]# kubectl get pods -n kube-system -o wide
NAME                                   READY   STATUS    RESTARTS   AGE   IP            NODE        NOMINATED NODE   READINESS GATES
kubernetes-dashboard-cb55bd5bd-4jsh7   1/1     Running   0          40s   10.254.73.2   10.2.8.34   <none>           <none>

3、訪問dashboard

一、注意有證書須要域名訪問，若是有DNS能夠配置域名解析，沒有Host綁定便可

二、選擇token訪問，token獲取方法以下

[root@elasticsearch01 ~]# cat /k8s/yaml/admin-token.yaml 
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: admin
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: admin
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile

[root@elasticsearch01 yaml]# kubectl create -f admin-token.yaml 
clusterrolebinding.rbac.authorization.k8s.io/admin created
serviceaccount/admin created

[root@elasticsearch01 yaml]#  kubectl describe secret/$(kubectl get secret -nkube-system |grep admin|awk '{print $1}') -nkube-system
Name:         admin-token-5j2vf
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: admin
              kubernetes.io/service-account.uid: 6b0b0c00-0b45-11e9-85fe-52540089b2b6

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1359 bytes
namespace:  11 bytes
token:      獲取的tocken值

三、效果以下
https://k8s.minminmsn.com
輸入token訪問
上面獲取的tocken值

補充

Apiserver hosts綁定ip錯誤10.0.0.1應該是10.254.0.1，默認pods網端是10.254.0.0/16，其中10.254.0.1會用來kubenetes的clusterip
[root@elasticsearch01 ~]# kubectl get svc --all-namespaces=true
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 6d1h

解決方法
×××文件重啓apiserver服務便可（配置前多檢查，不然後面會增長不少排錯過程）

[root@elasticsearch01 yaml]# kubectl logs kubernetes-dashboard-865b64d96f-g5f9t --namespace=kube-system
2018/12/29 07:49:44 Starting overwatch
2018/12/29 07:49:44 Using in-cluster config to connect to apiserver
2018/12/29 07:49:44 Using service account token for csrf signing
2018/12/29 07:49:44 Error while initializing connection to Kubernetes apiserver. This most likely means that the cluster is misconfigured (e.g., it has invalid apiserver certificates or service account's configuration) or the --apiserver-host param points to a server that does not exist. Reason: Get https://10.254.0.1:443/version: x509: certificate is valid for 10.0.0.1, 127.0.0.1, 10.2.8.44, 10.2.8.65, 10.2.8.34, not 10.254.0.1
Refer to our FAQ and wiki pages for more information: https://github.com/kubernetes/dashboard/wiki/FAQ

修改Hosts裏10.0.0.1爲10.254.0.1

[root@elasticsearch01 ssl]# cat server-csr.json 
{
    "CN": "kubernetes",
    "hosts": [
      "10.254.0.1",
      "127.0.0.1",
      "10.2.8.44",
      "10.2.8.65",
      "10.2.8.34",
      "kubernetes",
      "kubernetes.default",
      "kubernetes.default.svc",
      "kubernetes.default.svc.cluster",
      "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing",
            "O": "k8s",
            "OU": "System"
        }
    ]
}

同步證書並重啓服務

[root@elasticsearch01 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
2018/12/29 15:57:02 [INFO] generate received request
2018/12/29 15:57:02 [INFO] received CSR
2018/12/29 15:57:02 [INFO] generating key: rsa-2048
2018/12/29 15:57:03 [INFO] encoded CSR
2018/12/29 15:57:03 [INFO] signed certificate with serial number 57756035754570455349189088480535470836534926573
2018/12/29 15:57:03 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

[root@elasticsearch01 ssl]# scp server-csr.json server.csr server-key.pem server.pem 10.2.8.65:$PWD
[root@elasticsearch01 ssl]# scp server-csr.json server.csr server-key.pem server.pem 10.2.8.34:$PWD
[root@elasticsearch01 ssl]# systemctl restart kube-apiserver
[root@elasticsearch01 ssl]# systemctl restart kube-scheduler
[root@elasticsearch01 ssl]# systemctl restart kube-controller-manager