利用Helm部署高可用rancher集羣

時間 2020-03-12

原文原文鏈接

1、背景

Rancher HA有多種部署方式：node

Helm HA安裝，將Rancher部署在已有的Kubernetes集羣中，Rancher將使用集羣的etcd存儲數據，並利用Kubernetes調度實現高可用性。
RKE HA安裝，使用RKE工具安裝獨立的Kubernetes集羣，專門用於Rancher HA部署運行，RKE HA安裝僅支持Rancher v2.0.8以及以前的版本，Rancher v2.0.8以後的版本使用helm安裝Rancher。
本方案將基於已有的Kubernetes集羣，利用Helm安裝Rancher HA，並採用四層負載均衡方式。

2、添加Chart倉庫地址

使用helm repo add命令添加Rancher chart倉庫地址
Rancher tag和Chart版本選擇參考：https://www.cnrancher.com/docs/rancher/v2.x/cn/installation/server-tags/安全
```
#替換<CHART_REPO>爲您要使用的Helm倉庫分支(即latest或stable）。
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
```
3、使用自簽名證書安裝Rancher server

Rancher server設計默認須要開啓SSL/TLS配置來保證安全，將ssl證書以Kubernetes Secret卷的形式傳遞給rancher server或Ingress Controller。首先建立證書密文，以便Rancher和Ingress Controller可使用。服務器

一、生成自簽名證書

#腳本
一鍵生成自簽名證書腳本
#執行腳本生成證書
sh create_self-signed-cert.sh --ssl-domain=rancher.sumapay.com --ssl-trusted-ip=172.16.1.21,172.16.1.22 --ssl-size=2048 --ssl-date=3650app

二、使用kubectl建立tls類型的secrets

#建立命名空間負載均衡

[root@k8s-master03 ~]# kubectl create namespace cattle-system
namespace/rancher-system created

#服務證書和私鑰密文dom

[root@k8s-master03 self_CA]# kubectl -n cattle-system create secret tls tls-rancher-ingress --cert=./tls.crt --key=./tls.key 
secret/tls-rancher-ingress created

#ca證書密文tcp

[root@k8s-master03 self_CA]# kubectl -n cattle-system create secret generic tls-ca --from-file=cacerts.pem 
secret/tls-ca created

三、安裝rancher server

#使用helm安裝rancher HAide

[root@k8s-master03 ~]# helm install rancher-stable/rancher --name rancher2 --namespace cattle-system --set hostname=rancher.sumapay.com --set ingress.tls.source=secret --set privateCA=true
NAME:   rancher2
LAST DEPLOYED: Fri Apr 26 14:03:51 2019
NAMESPACE: cattle-system
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRoleBinding
NAME     AGE
rancher2  0s

==> v1/Deployment
NAME     READY  UP-TO-DATE  AVAILABLE  AGE
rancher2  0/3    3           0          0s

==> v1/Pod(related)
NAME                      READY  STATUS             RESTARTS  AGE
rancher-55c884bbf7-2xqpl  0/1    ContainerCreating  0         0s
rancher-55c884bbf7-bqvjh  0/1    ContainerCreating  0         0s
rancher-55c884bbf7-hhlvh  0/1    ContainerCreating  0         0s

==> v1/Service
NAME     TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)  AGE
rancher2  ClusterIP  10.110.148.105  <none>       80/TCP   0s

==> v1/ServiceAccount
NAME     SECRETS  AGE
rancher2  1        0s

==> v1beta1/Ingress
NAME     HOSTS                ADDRESS  PORTS  AGE
rancher2  rancher.sumapay.com  80, 443  0s

NOTES:
Rancher Server has been installed.

NOTE: Rancher may take several minutes to fully initialize. Please standby while Certificates are being issued and Ingress comes up.

Check out our docs at https://rancher.com/docs/rancher/v2.x/en/

Browse to https://rancher.sumapay.com

Happy Containering!

#查看建立工具

[root@k8s-master03 ~]# kubectl get ns
NAME                         STATUS   AGE
cattle-global-data           Active   2d5h
cattle-system                Active   2d5h

[root@k8s-master03 ~]# kubectl get ingress -n cattle-system           
NAME       HOSTS                 ADDRESS   PORTS     AGE
rancher2   rancher.sumapay.com             80, 443   57m

[root@k8s-master03 ~]# kubectl get service -n cattle-system        
NAME       TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
rancher2   ClusterIP   10.111.16.80   <none>        80/TCP    54m

[root@k8s-master03 ~]# kubectl get serviceaccount -n cattle-system           
NAME       TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
rancher2   ClusterIP   10.111.16.80   <none>        80/TCP    51m

[root@k8s-master03 ~]# kubectl get ClusterRoleBinding -n cattle-system -l app=rancher2 -o wide
NAME       AGE   ROLE                        USERS   GROUPS   SERVICEACCOUNTS
rancher2   58m   ClusterRole/cluster-admin                    cattle-system/rancher2

[root@k8s-master03 ~]# kubectl get pods -n cattle-system 
NAME                                    READY   STATUS    RESTARTS   AGE
cattle-cluster-agent-594b8f79bb-pgmdt   1/1     Running   5          2d2h
cattle-node-agent-lg44f                 1/1     Running   0          2d2h
cattle-node-agent-zgdms                 1/1     Running   5          2d2h
rancher2-9774897c-622sc                 1/1     Running   0          50m
rancher2-9774897c-czxxx                 1/1     Running   0          50m
rancher2-9774897c-sm2n5                 1/1     Running   0          50m

[root@k8s-master03 ~]# kubectl get deployment -n cattle-system 
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
cattle-cluster-agent   1/1     1            1           2d4h
rancher2               3/3     3            3           55m

四、爲Agent Pod添加主機別名(/etc/hosts)

若是你沒有內部DNS服務器而是經過添加/etc/hosts主機別名的方式指定的Rancher server域名，那麼無論經過哪一種方式(自定義、導入、Host驅動等)建立K8S集羣，K8S集羣運行起來以後，由於cattle-cluster-agent Pod和cattle-node-agent沒法經過DNS記錄找到Rancher server,最終致使沒法通訊。spa

解決方法

能夠經過給cattle-cluster-agent Pod和cattle-node-agent添加主機別名(/etc/hosts)，讓其能夠正常通訊(前提是IP地址能夠互通)。

#cattle-cluster-agent pod
kubectl -n cattle-system \
patch deployments cattle-cluster-agent --patch '{
    "spec": {
        "template": {
            "spec": {
                "hostAliases": [
                    {
                        "hostnames":
                        [
                            "rancher.sumapay.com"
                        ],
                            "ip": "四層負載均衡地址"
                    }
                ]
            }
        }
    }
}'

#cattle-node-agent pod
kubectl -n cattle-system \
patch  daemonsets cattle-node-agent --patch '{
    "spec": {
        "template": {
            "spec": {
                "hostAliases": [
                    {
                        "hostnames":
                        [
                            "rancher.sumapay.com"
                        ],
                            "ip": "四層負載均衡地址"
                    }
                ]
            }
        }
    }
}'

至此，rancher HA已部署完畢，因爲不是NodePort形式，在沒有部署ingress-controller狀況下，咱們還不能直接去訪問rancher服務。
ingress-controller部署請參考traefik部署與使用。

參考：
https://www.cnrancher.com/docs/rancher/v2.x/cn/installation/ha-install/helm-rancher/tcp-l4/rancher-install/