k8s中負載均衡器【ingress-nginx】部署

時間 2020-05-28

標籤 k8s 負載均衡器 ingress-nginx 部署欄目負載均衡简体版

原文原文鏈接

在Kubernetes中，服務和Pod的IP地址僅能夠在集羣網絡內部使用，對於集羣外的應用是不可見的。爲了使外部的應用可以訪問集羣內的服務，在Kubernetes 目前提供瞭如下幾種方案：html

NodePort前端
LoadBalancerjava
Ingressnode

本節主要就ingress和ingress控制器ingress-nginx-controller的部署做簡單介紹和記錄。python

如下系統組件版本：nginx

雲服務器：centos版本7.6.18十、k8s版本1.15.0、docker版本18.06.1-ce、ingress-nginx-controller版本0.25.0git

Ingressgithub

Ingress 組成？

將Nginx的配置抽象成一個Ingress對象，每添加一個新的服務只需寫一個新的Ingress的yaml文件便可面試
將新加入的Ingress轉化成Nginx的配置文件並使之生效spring
ingress controller
ingress服務

Ingress 工做原理?

ingress controller經過和kubernetes api交互，動態的去感知集羣中ingress規則變化，
而後讀取它，按照自定義的規則，規則就是寫明瞭哪一個域名對應哪一個service，生成一段nginx配置，
再寫到nginx-ingress-controller的pod裏，這個Ingress
controller的pod裏運行着一個Nginx服務，控制器會把生成的nginx配置寫入/etc/nginx.conf文件中，
而後reload一下使配置生效。以此達到域名分配置和動態更新的問題。

Ingress 能夠解決什麼問題？

動態配置服務

若是按照傳統方式, 當新增長一個服務時, 咱們可能須要在流量入口加一個反向代理指向咱們新的服務. 而若是用了Ingress, 只須要配置好這個服務, 當服務啓動時, 會自動註冊到Ingress的中, 不須要而外的操做.

減小沒必要要的端口暴露

配置過k8s的都清楚, 第一步是要關閉防火牆的, 主要緣由是k8s的不少服務會以NodePort方式映射出去, 這樣就至關於給宿主機打了不少孔, 既不安全也不優雅. 而Ingress能夠避免這個問題, 除了Ingress自身服務可能須要映射出去, 其餘服務都不要用NodePort方式

Ingress當前的實現方式？

ingress-nginx-controller

目前最新版本的ingress-nginx-controller，用lua實現了當upstream變化時不用reload，大大減小了生產環境中因爲服務的重啓、升級引發的IP變化致使的nginx reload。

如下就ingress-nginx-controller的部署作簡單記錄：

yaml以下：

kubectl apply -f {以下文件}

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.25.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10

---

在牆內會拉取不到如下鏡像：

quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.25.0

能夠使用國內阿里雲鏡像：

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.25.0

安裝後，在ingress-nginx命名空間下能夠看到pod一直pending，describe pod報以下警告：

查看master節點默認加了污點，通常不容許pod調度到master節點：

若是k8s集羣只有一個節點，能夠在pod的spec下設置容忍該污點：

即：

spec:      
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master

能夠看到ingress-nginx pod被調度到master節點，且變爲Running

看日誌報如下警告：

W0714 13:31:04.883127       6 queue.go:130] requeuing &ObjectMeta{Name:sync status,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,ManagedFields:[],}, err services "ingress-nginx" not found

須要創一個名爲ingress-nginx的service：

kubectl apply -f {以下文件}
kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: https
      port: 443
      targetPort: https

參考：

ingress deploy

https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md

Taint和Toleration（污點和容忍）

https://jimmysong.io/kubernetes-handbook/concepts/taint-and-toleration.html

k8s 1.12部署ingress-nginx

https://www.jianshu.com/p/e30b06906b77

本公衆號免費提供csdn下載服務，海量IT學習資源，若是你準備入IT坑，勵志成爲優秀的程序猿，那麼這些資源很適合你，包括但不限於java、go、python、springcloud、elk、嵌入式、大數據、面試資料、前端等資源。同時咱們組建了一個技術交流羣，裏面有不少大佬，會不定時分享技術文章，若是你想來一塊兒學習提升，能夠公衆號後臺回覆【2】，免費邀請加技術交流羣互相學習提升，會不按期分享編程IT相關資源。

掃碼關注，精彩內容第一時間推給你