kubernetes系列教程(十六)基於nginx ingress實現服務暴露

時間  2020-01-09
標籤 kubernetes 系列 教程 十六 基於 nginx ingress 實現 服務 暴露 欄目 Nginx 简体版
原文   原文鏈接

文章目錄

  • 寫在前面
    1. Ingress簡介
    1. Nginx Ingress
      • 2.1 Nginx ingress介紹
      • 2.2 Nginx ingress安裝
    1. Ingress資源定義
      • 3.1 Ingress定義
      • 3.2 Ingress動態配置
      • 3.3 Ingress路徑轉發
      • 3.4 Ingress虛擬主機
      • 3.5 Ingress TLS加密
    1. Nginx Ingress高級功能
      • 4.1 定製化參數
      • 4.2 虛擬主機和路由
  • 寫在最後
  • 參考文獻

寫在前面

本章介紹kubernetes系列教程的ingress概念，在kubernetes中對外暴露服務的方式有兩種：service（NodePort或者外部LoadBalancer）和ingress，其中service是提供四層的負載均衡，經過iptables DNAT或lvs nat模式實現後端Pod的代理請求。如需實現http，域名，URI，證書等請求方式，service是沒法實現的，須要藉助於ingress來來實現，本文未來介紹ingress相關的內容。html

1. Ingress簡介

An API object that manages external access to the services in a cluster, typically HTTP.
Ingress can provide load balancing, SSL termination and name-based virtual hosting.node

引用官方關於ingress的介紹咱們能夠得知，ingress是一種經過http協議暴露kubernetes內部服務的api對象，即充當Edge Router邊界路由器的角色對外基於七層的負載均衡調度機制，可以提供如下幾個功能：nginx

  • 負載均衡，將請求自動負載均衡到後端的Pod上；
  • SSL加密，客戶端到Ingress Controller爲https加密，到後端Pod爲明文的http；
  • 基於名稱的虛擬主機，提供基於域名或URI更靈活的路由方式

Ingress Controller

實現Ingress包含的組件有：git

  • Ingress，客戶端，負責定義ingress配置，將請求轉發給Ingress Controller；
  • Ingress Controller，Ingress控制器，實現七層轉發的Edge Router，經過調用k8s的api動態感知集羣中Pod的變化而動態更新配置文件並重載， Controller須要部署在k8s集羣中以實現和集羣中的pod通訊，一般以DaemonSets或Deployments的形式部署，並對外暴露80和443端口，對於DaemonSets來講，通常是以hostNetwork或者hostPort的形式暴露，Deployments則以NodePort的方式暴露，控制器的多個節點則藉助外部負載均衡ExternalLB以實現統一接入；
  • Ingress配置規則，Controller控制器經過service服務發現機制動態實現後端Pod路由轉發規則的實現；
  • Service，kuberntes中四層的負載均衡調度機制，Ingress藉助service的服務發現機制實現集羣中Pod資源的動態感知；
  • Pod，後端實際負責響應請求容器，由控制器如Deployment建立，經過標籤Labels和service關聯，服務發現。

簡而言之，ingress控制器藉助service的服務發現機制實現配置的動態更新以實現Pod的負載均衡機制實現，因爲涉及到Ingress Controller的動態更新，目前社區Ingress Controller大致包含兩種類型的控制器：github

  • 傳統的七層負載均衡如Nginx，HAproxy，開發了適應微服務應用的插件，具備成熟，高性能等優勢；
  • 新型微服務負載均衡如Traefik，Envoy，Istio，專門適用於微服務+容器化應用場景，具備動態更新特色；
類型 常見類型 優勢 缺點
傳統負載均衡 nginx，haproxy 成熟，穩定，高性能 動態更新需reload配置文件
微服務負載均衡 Traefik，Envoy，Istio 天生爲微服務而生，動態更新 性能還有待提高

2. Nginx Ingress

2.1 Nginx ingress介紹

By default, pods of Kubernetes services are not accessible from the external network, but only by other pods within the Kubernetes cluster. Kubernetes has a built‑in configuration for HTTP load balancing, called Ingress, that defines rules for external connectivity to Kubernetes services. Users who need to provide external access to their Kubernetes services create an Ingress resource that defines rules, including the URI path, backing service name, and other information. The Ingress controller can then automatically program a frontend load balancer to enable Ingress configuration. The NGINX Ingress Controller for Kubernetes is what enables Kubernetes to configure NGINX and NGINX Plus for load balancing Kubernetes services.web

Nginx ingress

Nginx Ingress Controller是實現ingress的具體實現，包含有兩個版本：Ngnix OSS和Nginx Plus版，後者是商業化加強版，支持更多的功能，詳情參考官方文檔介紹https://www.nginx.com/products/nginx/kubernetes-ingress-controller#compare-versions算法

Nginx ingress版本features對比

2.2 Nginx ingress安裝

首先須要安裝Nginx Ingress Controller控制器，控制器安裝方式包含兩種：DaemonSets和Deployments。apache

  • DaemonSets經過hostPort的方式暴露80和443端口，可經過Node的調度由專門的節點實現部署
  • Deployments則經過NodePort的方式實現控制器端口的暴露，藉助外部負載均衡實現高可用負載均衡

除此以外，還須要部署Namespace，ServiceAccount，RBAC，Secrets，Custom Resource Definitions等資源，以下開始部署。後端

2.2.1 基礎依賴環境準備

一、github中下載源碼包,安裝部署文件在kubernetes-ingress/deployments/目錄下api

[root@node-1 ~]# git clone https://github.com/nginxinc/kubernetes-ingress.git
[root@node-1 ~]# tree kubernetes-ingress/deployments/
kubernetes-ingress/deployments/
├── common
│   ├── custom-resource-definitions.yaml  自定義資源
│   ├── default-server-secret.yaml        Secrets
│   ├── nginx-config.yaml
│   └── ns-and-sa.yaml                    Namspace+ServiceAccount
├── daemon-set
│   ├── nginx-ingress.yaml                DaemonSets控制器
│   └── nginx-plus-ingress.yaml
├── deployment
│   ├── nginx-ingress.yaml                Deployments控制器
│   └── nginx-plus-ingress.yaml
├── helm-chart                            Helm安裝包
│   ├── chart-icon.png
│   ├── Chart.yaml
│   ├── README.md
│   ├── templates
│   │   ├── controller-configmap.yaml
│   │   ├── controller-custom-resources.yaml
│   │   ├── controller-daemonset.yaml
│   │   ├── controller-deployment.yaml
│   │   ├── controller-leader-election-configmap.yaml
│   │   ├── controller-secret.yaml
│   │   ├── controller-serviceaccount.yaml
│   │   ├── controller-service.yaml
│   │   ├── controller-wildcard-secret.yaml
│   │   ├── _helpers.tpl
│   │   ├── NOTES.txt
│   │   └── rbac.yaml
│   ├── values-icp.yaml
│   ├── values-plus.yaml
│   └── values.yaml
├── rbac                                RBAC認證受權
│   └── rbac.yaml
├── README.md
└── service                            Service定義
    ├── loadbalancer-aws-elb.yaml
    ├── loadbalancer.yaml              DaemonSets暴露服務方式
    └── nodeport.yaml                  Deployments暴露服務方式

二、建立Namespace和ServiceAccount, kubectl apply -f common/default-server-secret.yaml 

apiVersion: v1
kind: Namespace
metadata:
  name: nginx-ingress 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress 
  namespace: nginx-ingress

三、建立Secrets自簽名證書，kubectl apply -f common/default-server-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: default-server-secret
  namespace: nginx-ingress
type: Opaque
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=

四、建立ConfigMap自定義配置文件，kubectl apply -f common/nginx-config.yaml 

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:

五、爲虛擬雲主機和虛擬雲主機路由定義自定義資源，支持自定義虛擬主機和虛擬路由，kubectl apply -f common/custom-resource-definitions.yaml

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: virtualservers.k8s.nginx.org
spec:
  group: k8s.nginx.org
  versions:
  - name: v1
    served: true
    storage: true
  scope: Namespaced
  names:
    plural: virtualservers
    singular: virtualserver
    kind: VirtualServer
    shortNames:
    - vs
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: virtualserverroutes.k8s.nginx.org
spec:
  group: k8s.nginx.org
  versions:
  - name: v1
    served: true
    storage: true
  scope: Namespaced
  names:
    plural: virtualserverroutes
    singular: virtualserverroute
    kind: VirtualServerRoute
    shortNames:
    - vsr

六、配置RBAC認證受權，實現ingress控制器訪問集羣中的其餘資源，kubectl apply -f rbac/rbac.yaml

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-ingress
rules:
- apiGroups:
  - ""
  resources:
  - services
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get
  - list
  - watch
  - update
  - create
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs:
  - list
  - watch
  - get
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - k8s.nginx.org
  resources:
  - virtualservers
  - virtualserverroutes
  verbs:
  - list
  - watch
  - get
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-ingress
subjects:
- kind: ServiceAccount
  name: nginx-ingress
  namespace: nginx-ingress
roleRef:
  kind: ClusterRole
  name: nginx-ingress
  apiGroup: rbac.authorization.k8s.io

2.2.2 部署Ingress控制器

一、 部署控制器，控制器能夠DaemonSets和Deployment的形式部署，以下是DaemonSets的配置文件

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
     #annotations:
       #prometheus.io/scrape: "true"
       #prometheus.io/port: "9113"
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:edge
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
          hostPort: 80            #經過hostPort的方式暴露端口
        - name: https
          containerPort: 443
          hostPort: 443
       #- name: prometheus
         #containerPort: 9113
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
         #- -v=3 # Enables extensive logging. Useful for troubleshooting.
         #- -report-ingress-status
         #- -external-service=nginx-ingress
         #- -enable-leader-election
         #- -enable-prometheus-metrics

Deployments的配置文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  replicas: 1                  #副本的個數
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
     #annotations:
       #prometheus.io/scrape: "true"
       #prometheus.io/port: "9113"
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:edge
        imagePullPolicy: Always
        name: nginx-ingress
        ports:                #內部暴露的服務端口，須要經過NodePort的方式暴露給外部
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
       #- name: prometheus
         #containerPort: 9113
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
         #- -v=3 # Enables extensive logging. Useful for troubleshooting.
         #- -report-ingress-status
         #- -external-service=nginx-ingress
         #- -enable-leader-election
         #- -enable-prometheus-metrics

二、咱們以DaemonSets的方式部署，DaemonSet部署集羣中各個節點都是對等，若是有外部LoadBalancer則經過外部負載均衡路由至Ingress中

[root@node-1 deployments]# kubectl apply -f daemon-set/nginx-ingress.yaml 
daemonset.apps/nginx-ingress created
[root@node-1 deployments]# kubectl get daemonsets -n nginx-ingress
NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
nginx-ingress   3         3         3       3            3           <none>          15s

[root@node-1 ~]# kubectl get pods -n nginx-ingress -o wide 
NAME                  READY   STATUS    RESTARTS   AGE     IP             NODE     NOMINATED NODE   READINESS GATES
nginx-ingress-7mpfc   1/1     Running   0          2m44s   10.244.0.50    node-1   <none>           <none>
nginx-ingress-l2rtj   1/1     Running   0          2m44s   10.244.1.144   node-2   <none>           <none>
nginx-ingress-tgf6r   1/1     Running   0          2m44s   10.244.2.160   node-3   <none>           <none>

三、校驗Nginx Ingress安裝狀況，此時三個節點均是對等，即訪問任意一個節點均能實現相同的效果，統一入口則經過外部負載均衡，若是在雲環境下執行kubectl apply -f service/loadbalancer.yaml建立外部負載均衡實現入口調度，自建的能夠經過lvs或nginx等負載均衡實現接入，本文再也不贅述，讀者能夠自行研究。

nginx ingress安裝校驗

備註說明：若是以Deployments的方式部署，則須要執行service/nodeport.yaml建立NodePort類型的Service，實現的效果和DaemonSets相似。

3. Ingress資源定義

上面的章節已安裝了一個Nginx Ingress Controller控制器，有了Ingress控制器後，咱們就能夠定義Ingress資源來實現七層負載轉發了，大致上Ingress支持三種使用方式：1. 基於虛擬主機轉發，2. 基於虛擬機主機URI轉發，3. 支持TLS加密轉發。

3.1 Ingress定義

一、環境準備，先建立一個nginx的Deployment應用，包含2個副本

[root@node-1 ~]# kubectl run ingress-demo --image=nginx:1.7.9 --port=80 --replicas=2
[root@node-1 ~]# kubectl get deployments
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
ingress-demo   2/2     2            2           116s

二、以service方式暴露服務端口

[root@node-1 ~]# kubectl expose deployment ingress-demo --port=80 --protocol=TCP --target-port=80
service/ingress-demo exposed
[root@node-1 ~]# kubectl get services 
NAME           TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
ingress-demo   ClusterIP   10.109.33.91   <none>        80/TCP    2m15s

三、上述兩個步驟已建立了一個service，以下咱們定義一個ingress對象將起轉發至ingress-demo這個service，經過ingress.class指定控制器的類型爲nginx

[root@node-1 nginx-ingress]# cat nginx-ingress-demo.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  labels:
    ingres-controller: nginx
  annotations:
    kubernets.io/ingress.class: nginx
spec:
  rules:
  - host: www.happylau.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: ingress-demo
          servicePort: 80

四、建立ingress對象

[root@node-1 nginx-ingress]# kubectl apply -f nginx-ingress-demo.yaml 
ingress.extensions/nginx-ingress-demo created

查看ingress資源列表
[root@node-1 nginx-ingress]# kubectl get ingresses
NAME                 HOSTS                ADDRESS   PORTS   AGE
nginx-ingress-demo   www.happylau.cn             80      4m4s

五、查看ingress詳情，能夠在Rules規則中看到後端Pod的列表，自動發現和關聯相關Pod

[root@node-1 ~]# kubectl describe ingresses nginx-ingress-demo 
Name:             nginx-ingress-demo
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  www.happylau.cn  
                      /   ingress-demo:80 (10.244.1.146:80,10.244.2.162:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-demo","namespace":"default"},"spec":{"rules":[{"host":"www.happylaulab.cn","http":{"paths":[{"backend":{"serviceName":"ingress-demo","servicePort":80},"path":"/"}]}}]}}

  kubernets.io/ingress.class:  nginx
Events:
  Type    Reason          Age   From                      Message
  ----    ------          ----  ----                      -------
  Normal  AddedOrUpdated  9m7s  nginx-ingress-controller  Configuration for default/nginx-ingress-demo was added or updated
  Normal  AddedOrUpdated  9m7s  nginx-ingress-controller  Configuration for default/nginx-ingress-demo was added or updated
  Normal  AddedOrUpdated  9m7s  nginx-ingress-controller  Configuration for default/nginx-ingress-demo was added or updated

六、測試驗證，ingress規則的配置信息已注入到Ingress Controller中，環境中Ingress Controller是以DaemonSets的方式部署在集羣中，若是有外部的負載均衡，則將www.happylau.cn域名的地址解析爲負載均衡VIP。因爲測試環境沒有搭建負載均衡，將hosts解析執行node-1，node-2或者node-3任意一個IP都能實現相同的功能。

ingress測試

上述測試解析正常，固然也能夠解析爲node-1和node-2的IP，以下：

[root@node-1 ~]# curl -I http://www.happylau.cn --resolve www.happylau.cn:80:10.254.100.101
HTTP/1.1 200 OK
Server: nginx/1.17.6
Date: Tue, 24 Dec 2019 10:32:22 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Tue, 23 Dec 2014 16:25:09 GMT
ETag: "54999765-264"
Accept-Ranges: bytes

[root@node-1 ~]# curl -I http://www.happylau.cn --resolve www.happylau.cn:80:10.254.100.102
HTTP/1.1 200 OK
Server: nginx/1.17.6
Date: Tue, 24 Dec 2019 10:32:24 GMT
Content-Type: text/html
Content-Length: 612
Connection: keep-alive
Last-Modified: Tue, 23 Dec 2014 16:25:09 GMT
ETag: "54999765-264"
Accept-Ranges: bytes

3.2 Ingress動態配置

上面的章節介紹了ingress資源對象的申明配置，在這個章節中咱們探究一下Nginx Ingress Controller的實現機制和動態配置更新機制，以方便了解Ingress控制器的工做機制。

一、 查看Nginx Controller控制器的配置文件，在nginx-ingress pod中存儲着ingress的配置文件

[root@node-1 ~]# kubectl get pods -n nginx-ingress 
NAME                  READY   STATUS    RESTARTS   AGE
nginx-ingress-7mpfc   1/1     Running   0          6h15m
nginx-ingress-l2rtj   1/1     Running   0          6h15m
nginx-ingress-tgf6r   1/1     Running   0          6h15m

#查看配置文件，每一個ingress生成一個配置文件，文件名爲：命名空間-ingres名稱.conf
[root@node-1 ~]# kubectl exec -it nginx-ingress-7mpfc -n nginx-ingress -- ls -l /etc/nginx/conf.d
total 4
-rw-r--r-- 1 nginx nginx 1005 Dec 24 10:06 default-nginx-ingress-demo.conf

#查看配置文件
[root@node-1 ~]# kubectl exec -it nginx-ingress-7mpfc -n nginx-ingress -- cat /etc/nginx/conf.d/default-nginx-ingress-demo.conf
# configuration for default/nginx-ingress-demo

#upstream的配置，會用least_conn算法，經過service服務發現機制動態識別到後端的Pod
upstream default-nginx-ingress-demo-www.happylau.cn-ingress-demo-80 {
    zone default-nginx-ingress-demo-www.happylau.cn-ingress-demo-80 256k;
    random two least_conn;
    server 10.244.1.146:80 max_fails=1 fail_timeout=10s max_conns=0;
    server 10.244.2.162:80 max_fails=1 fail_timeout=10s max_conns=0;
}

server {
    listen 80;
    server_tokens on;
    server_name www.happylau.cn;
    location / {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;
        proxy_pass http://default-nginx-ingress-demo-www.happylau.cn-ingress-demo-80;   #調用upstream實現代理
    }
}

經過上述查看配置文件可得知，Nginx Ingress Controller實際是根據ingress規則生成對應的nginx配置文件，以實現代理轉發的功能，加入Deployments的副本數變動後nginx的配置文件會發生什麼改變呢？

二、更新控制器的副本數，由2個Pod副本擴容至3個

[root@node-1 ~]# kubectl scale --replicas=3 deployment ingress-demo 
deployment.extensions/ingress-demo scaled
[root@node-1 ~]# kubectl get deployments
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
ingress-demo   3/3     3            3           123m

三、再次查看nginx的配置文件，ingress藉助於service的服務發現機制，將加入的Pod自動加入到nginx upstream中

nginx動態更新配置

四、查看nginx pod的日誌（kubectl logs nginx-ingress-7mpfc -n nginx-ingress），有reload優雅重啓的記錄，即經過更新配置文件+reload實現配置動態更新。

nginx控制器重啓日誌記錄

經過上述的配置可知，ingress調用kubernetes api去感知kubernetes集羣中的變化狀況，Pod的增長或減小這些變化，而後動態更新nginx ingress controller的配置文件，並從新載入配置。當集羣規模越大時，會頻繁涉及到配置文件的變更和重載，所以nginx這方面會存在先天的劣勢，專門爲微服務負載均衡應運而生，如Traefik，Envoy，Istio，這些負載均衡工具可以提供大規模，頻繁動態更新的場景，但性能相比Nginx，HAproxy還存在必定的劣勢。日後的章節中，咱們再對其餘的Ingress控制器作介紹。

3.3 Ingress路徑轉發

Ingress支持URI格式的轉發方式，同時支持URL重寫，以下以兩個service爲例演示，service-1安裝nginx，service-2安裝httpd，分別用http://demo.happylau.cn/news和http://demo.happylau.cn/sports轉發到兩個不一樣的service

一、環境準備，建立兩個應用並實現service暴露，建立deployments時指定--explose建立service

[root@node-1 ~]# kubectl run service-1 --image=nginx:1.7.9 --port=80 --replicas=1 --expose=true 
service/service-1 created
deployment.apps/service-1 created

[root@node-1 ~]# kubectl run service-2 --image=httpd --port=80 --replicas=1 --expose=true 
service/service-2 created
deployment.apps/service-2 created

查看deployment狀態
[root@node-1 ~]# kubectl get deployments 
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
ingress-demo   4/4     4            4           4h36m
service-1      1/1     1            1           65s
service-2      1/1     1            1           52s

查看service狀態，服務已經正常
[root@node-1 ~]# kubectl get services 
NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
ingress-demo   ClusterIP   10.109.33.91     <none>        80/TCP    4h36m
kubernetes     ClusterIP   10.96.0.1        <none>        443/TCP   101d
service-1      ClusterIP   10.106.245.71    <none>        80/TCP    68s
service-2      ClusterIP   10.104.204.158   <none>        80/TCP    55s

二、建立ingress對象，經過一個域名將請求轉發至後端兩個service

[root@node-1 nginx-ingress]# cat nginx-ingress-uri-demo.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-uri-demo
  labels:
    ingres-controller: nginx
  annotations:
    kubernets.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: demo.happylau.cn
    http:
      paths:
      - path: /news
        backend:
          serviceName: service-1 
          servicePort: 80
      - path: /sports
        backend:
          serviceName: service-2
          servicePort: 80

三、建立ingress規則，查看詳情

[root@node-1 nginx-ingress]# kubectl apply -f nginx-ingress-uri-demo.yaml 
ingress.extensions/nginx-ingress-uri-demo created

#查看詳情
[root@node-1 nginx-ingress]# kubectl get ingresses.
NAME                     HOSTS              ADDRESS   PORTS   AGE
nginx-ingress-demo       www.happylau.cn              80      4h35m
nginx-ingress-uri-demo   demo.happylau.cn             80      4s
[root@node-1 nginx-ingress]# kubectl describe ingresses nginx-ingress-uri-demo 
Name:             nginx-ingress-uri-demo
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:              #對應的轉發url規則
  Host              Path  Backends
  ----              ----  --------
  demo.happylau.cn  
                    /news     service-1:80 (10.244.2.163:80)
                    /sports   service-2:80 (10.244.1.148:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-uri-demo","namespace":"default"},"spec":{"rules":[{"host":"demo.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-1","servicePort":80},"path":"/news"},{"backend":{"serviceName":"service-2","servicePort":80},"path":"/sports"}]}}]}}

  kubernets.io/ingress.class:                  nginx
  nginx.ingress.kubernetes.io/rewrite-target:  /
Events:
  Type    Reason          Age   From                      Message
  ----    ------          ----  ----                      -------
  Normal  AddedOrUpdated  11s   nginx-ingress-controller  Configuration for default/nginx-ingress-uri-demo was added or updated
  Normal  AddedOrUpdated  11s   nginx-ingress-controller  Configuration for default/nginx-ingress-uri-demo was added or updated
  Normal  AddedOrUpdated  11s   nginx-ingress-controller  Configuration for default/nginx-ingress-uri-demo was added or updated

四、準備測試，站點中建立對應的路徑

[root@node-1 ~]# kubectl exec -it service-1-7b66bf758f-xj9jh /bin/bash
root@service-1-7b66bf758f-xj9jh:/# echo "service-1 website page" >/usr/share/nginx/html/news

[root@node-1 ~]# kubectl exec -it service-2-7c7444684d-w9cv9 /bin/bash
root@service-2-7c7444684d-w9cv9:/usr/local/apache2# echo "service-2 website page" >/usr/local/apache2/htdocs/sports

五、測試驗證

[root@node-1 ~]# curl http://demo.happylau.cn/news --resolve demo.happylau.cn:80:10.254.100.101
service-1 website page
[root@node-1 ~]# curl http://demo.happylau.cn/sports --resolve demo.happylau.cn:80:10.254.100.101
service-2 website page

六、經過上述的驗證測試能夠得知，ingress支持URI的路由方式轉發，其對應在ingress中的配置文件內容是怎樣的呢，咱們看下ingress controller生成對應的nginx配置文件內容，實際是經過ingress的location來實現，將不一樣的localtion轉發至不一樣的upstream以實現service的關聯，配置文件以下：

[root@node-1 ~]# kubectl exec -it nginx-ingress-7mpfc -n nginx-ingress /bin/bash
nginx@nginx-ingress-7mpfc:/$ cat /etc/nginx/conf.d/default-nginx-ingress-uri-demo.conf |grep -v "^$"
# configuration for default/nginx-ingress-uri-demo
#定義兩個upstream和後端的service關聯
upstream default-nginx-ingress-uri-demo-demo.happylau.cn-service-1-80 {
    zone default-nginx-ingress-uri-demo-demo.happylau.cn-service-1-80 256k;
    random two least_conn;
    server 10.244.2.163:80 max_fails=1 fail_timeout=10s max_conns=0;
}

upstream default-nginx-ingress-uri-demo-demo.happylau.cn-service-2-80 {
    zone default-nginx-ingress-uri-demo-demo.happylau.cn-service-2-80 256k;
    random two least_conn;
    server 10.244.1.148:80 max_fails=1 fail_timeout=10s max_conns=0;    
}

server {
    listen 80;
    server_tokens on;
    server_name demo.happylau.cn;

  #定義location實現代理，經過proxy_pass和後端的service關聯
    location /news {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;
        proxy_pass http://default-nginx-ingress-uri-demo-demo.happylau.cn-service-1-80;
    }

    location /sports {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;
        proxy_pass http://default-nginx-ingress-uri-demo-demo.happylau.cn-service-2-80; 
    }   
}

3.4 Ingress虛擬主機

ingress支持基於名稱的虛擬主機，實現單個IP多個域名轉發的需求，經過請求頭部攜帶主機名方式區分開，將上個章節的ingress刪除，使用service-1和service-2兩個service來作演示。

一、建立ingress規則，經過主機名實現轉發規則

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-virtualname-demo
  labels:
    ingres-controller: nginx
  annotations:
    kubernets.io/ingress.class: nginx
spec:
  rules:
  - host: news.happylau.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: service-1 
          servicePort: 80
  - host: sports.happylau.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: service-2 
          servicePort: 80

二、生成ingress規則並查看詳情，一個ingress對應兩個HOSTS

[root@node-1 nginx-ingress]# kubectl apply -f nginx-ingress-virtualname.yaml 
ingress.extensions/nginx-ingress-virtualname-demo created

#查看列表
[root@node-1 nginx-ingress]# kubectl get ingresses nginx-ingress-virtualname-demo 
NAME                             HOSTS                                 ADDRESS   PORTS   AGE
nginx-ingress-virtualname-demo   news.happylau.cn,sports.happylau.cn             80      12s

#查看詳情
[root@node-1 nginx-ingress]# kubectl describe ingresses nginx-ingress-virtualname-demo
Name:             nginx-ingress-virtualname-demo
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  news.happylau.cn    
                      /   service-1:80 (10.244.2.163:80)
  sports.happylau.cn  
                      /   service-2:80 (10.244.1.148:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-virtualname-demo","namespace":"default"},"spec":{"rules":[{"host":"news.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-1","servicePort":80},"path":"/"}]}},{"host":"sports.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-2","servicePort":80},"path":"/"}]}}]}}

  kubernets.io/ingress.class:  nginx
Events:
  Type    Reason          Age   From                      Message
  ----    ------          ----  ----                      -------
  Normal  AddedOrUpdated  28s   nginx-ingress-controller  Configuration for default/nginx-ingress-virtualname-demo was added or updated
  Normal  AddedOrUpdated  28s   nginx-ingress-controller  Configuration for default/nginx-ingress-virtualname-demo was added or updated
  Normal  AddedOrUpdated  28s   nginx-ingress-controller  Configuration for default/nginx-ingress-virtualname-demo was added or updated

三、準備測試數據並測試

[root@node-1 ~]# kubectl exec -it service-1-7b66bf758f-xj9jh /bin/bash
root@service-1-7b66bf758f-xj9jh:/# echo "news demo" >/usr/share/nginx/html/index.html

[root@node-1 ~]# kubectl exec -it service-2-7c7444684d-w9cv9 /bin/bash  
root@service-2-7c7444684d-w9cv9:/usr/local/apache2# echo "sports demo"  >/usr/local/apache2/htdocs/index.html

測試：

[root@node-1 ~]# curl http://news.happylau.cn --resolve news.happylau.cn:80:10.254.100.102
news demo
[root@node-1 ~]# curl http://sports.happylau.cn --resolve sports.happylau.cn:80:10.254.100.102
sports demo

四、查看nginx的配置文件內容，經過在server中定義不一樣的server_name以區分，代理到不一樣的upstream以實現service的代理。

# configuration for default/nginx-ingress-virtualname-demo
upstream default-nginx-ingress-virtualname-demo-news.happylau.cn-service-1-80 {
    zone default-nginx-ingress-virtualname-demo-news.happylau.cn-service-1-80 256k;
    random two least_conn;
    server 10.244.2.163:80 max_fails=1 fail_timeout=10s max_conns=0;
}

upstream default-nginx-ingress-virtualname-demo-sports.happylau.cn-service-2-80 {
    zone default-nginx-ingress-virtualname-demo-sports.happylau.cn-service-2-80 256k;
    random two least_conn;
    server 10.244.1.148:80 max_fails=1 fail_timeout=10s max_conns=0;
}

server {
    listen 80;
    server_tokens on;
    server_name news.happylau.cn;
    location / {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;

        proxy_pass http://default-nginx-ingress-virtualname-demo-news.happylau.cn-service-1-80;

  }
}
server {
    listen 80;  
    server_tokens on;
    server_name sports.happylau.cn;

    location / {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;

        proxy_pass http://default-nginx-ingress-virtualname-demo-sports.happylau.cn-service-2-80;   

    }   
}

3.5 Ingress TLS加密

四層的負載均衡沒法支持https請求，當前大部分業務都要求以https方式接入，Ingress能支持https的方式接入，經過Secrets存儲證書+私鑰，實現https接入，同時還能支持http跳轉功能。對於用戶的請求流量來講，客戶端到ingress controller是https流量，ingress controller到後端service則是http，提升用戶訪問性能，以下介紹ingress TLS功能實現步驟。

一、生成自簽名證書和私鑰

[root@node-1 ~]# openssl req -x509 -newkey rsa:2048 -nodes -days 365 -keyout tls.key -out tls.crt
Generating a 2048 bit RSA private key
....................................................+++
........................................+++
writing new private key to 'tls.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN        #國家
State or Province Name (full name) []:GD    #省份
Locality Name (eg, city) [Default City]:ShenZhen  #城市
Organization Name (eg, company) [Default Company Ltd]:Tencent    #公司 
Organizational Unit Name (eg, section) []:HappyLau  #組織
Common Name (eg, your name or your server's hostname) []:www.happylau.cn  #域名
Email Address []:573302346@qq.com       #郵箱地址

#tls.crt爲證書，tls.key爲私鑰
[root@node-1 ~]# ls tls.* -l
-rw-r--r-- 1 root root 1428 12月 26 13:21 tls.crt
-rw-r--r-- 1 root root 1708 12月 26 13:21 tls.key

二、配置Secrets，將證書和私鑰配置到Secrets中

[root@node-1 ~]# kubectl create secret tls happylau-sslkey --cert=tls.crt --key=tls.key 
secret/happylau-sslkey created

查看Secrets詳情,證書和私要包含在data中，文件名爲兩個不一樣的key：tls.crt和tls.key
[root@node-1 ~]# kubectl describe secrets happylau-sslkey 
Name:         happylau-sslkey
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1428 bytes
tls.key:  1708 bytes

三、配置ingress調用Secrets實現SSL證書加密

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-tls-demo
  labels:
    ingres-controller: nginx
  annotations:
    kubernets.io/ingress.class: nginx
spec:
  tls:
  - hosts:
    - news.happylau.cn
    - sports.happylau.cn
    secretName: happylau-sslkey
  rules:
  - host: news.happylau.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: service-1 
          servicePort: 80
  - host: sports.happylau.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: service-2 
          servicePort: 80

四、建立ingress並查看ingress詳情

[root@node-1 nginx-ingress]# kubectl describe ingresses nginx-ingress-tls-demo 
Name:             nginx-ingress-tls-demo
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
TLS:
  happylau-sslkey terminates news.happylau.cn,sports.happylau.cn
Rules:
  Host                Path  Backends
  ----                ----  --------
  news.happylau.cn    
                      /   service-1:80 (10.244.2.163:80)
  sports.happylau.cn  
                      /   service-2:80 (10.244.1.148:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-tls-demo","namespace":"default"},"spec":{"rules":[{"host":"news.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-1","servicePort":80},"path":"/"}]}},{"host":"sports.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-2","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["news.happylau.cn","sports.happylau.cn"],"secretName":"happylau-sslkey"}]}}

  kubernets.io/ingress.class:  nginx
Events:
  Type    Reason          Age   From                      Message
  ----    ------          ----  ----                      -------
  Normal  AddedOrUpdated  22s   nginx-ingress-controller  Configuration for default/nginx-ingress-tls-demo was added or updated
  Normal  AddedOrUpdated  22s   nginx-ingress-controller  Configuration for default/nginx-ingress-tls-demo was added or updated
  Normal  AddedOrUpdated  22s   nginx-ingress-controller  Configuration for default/nginx-ingress-tls-demo was added or updated

五、 將news.happylau.cn和sports.happylau.cn寫入到hosts文件中，並經過https://news.happylau.cn 的方式訪問，瀏覽器訪問內容提示證書以下，信任證書便可訪問到站點內容。

tls測試頁面

查看證書詳情，正是咱們製做的自簽名證書，生產實際使用時，推薦使用CA機構頒發簽名證書。

證書詳情

六、接下來查看一下tls配置https的nginx配置文件內容，能夠看到在server塊啓用了https並配置證書，同時配置了http跳轉，所以直接訪問http也可以實現自動跳轉到https功能。

# configuration for default/nginx-ingress-tls-demo
upstream default-nginx-ingress-tls-demo-news.happylau.cn-service-1-80 {
    zone default-nginx-ingress-tls-demo-news.happylau.cn-service-1-80 256k;
    random two least_conn;
    server 10.244.2.163:80 max_fails=1 fail_timeout=10s max_conns=0;    
}

upstream default-nginx-ingress-tls-demo-sports.happylau.cn-service-2-80 {
    zone default-nginx-ingress-tls-demo-sports.happylau.cn-service-2-80 256k;
    random two least_conn;
    server 10.244.1.148:80 max_fails=1 fail_timeout=10s max_conns=0;    
}

server {
    listen 80;

    listen 443 ssl;     #https監聽端口，證書和key，實現和Secrets關聯
    ssl_certificate /etc/nginx/secrets/default-happylau-sslkey;
    ssl_certificate_key /etc/nginx/secrets/default-happylau-sslkey;

    server_tokens on;
    server_name news.happylau.cn;

  #http跳轉功能，即訪問http會自動跳轉至https
    if ($scheme = http) {
        return 301 https://$host:443$request_uri;
    }

    location / {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;

        proxy_pass http://default-nginx-ingress-tls-demo-news.happylau.cn-service-1-80; 
  } 
}

server {
    listen 80;
    listen 443 ssl;
    ssl_certificate /etc/nginx/secrets/default-happylau-sslkey;
    ssl_certificate_key /etc/nginx/secrets/default-happylau-sslkey;

    server_tokens on;
    server_name sports.happylau.cn;

    if ($scheme = http) {
        return 301 https://$host:443$request_uri;
    }

    location / {
        proxy_http_version 1.1;
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
        client_max_body_size 1m;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering on;

        proxy_pass http://default-nginx-ingress-tls-demo-sports.happylau.cn-service-2-80;   
    }   
}

4. Nginx Ingress高級功能

4.1 定製化參數

ingress controller提供了基礎反向代理的功能，若是須要定製化nginx的特性或參數，須要經過ConfigMap和Annotations來實現，二者實現的方式有所不一樣，ConfigMap用於指定整個ingress集羣資源的基本參數，修改後會被全部的ingress對象所繼承；Annotations則被某個具體的ingress對象所使用，修改只會影響某個具體的ingress資源，衝突時其優先級高於ConfigMap。

4.1.1 ConfigMap自定義參數

安裝nginx ingress controller時默認會包含一個空的ConfigMap，能夠經過ConfigMap來自定義nginx controller的默認參數，以下以修改一些參數爲例：

一、 定義ConfigMap參數

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  proxy-connect-timeout: "10s"
  proxy-read-timeout: "10s"
  proxy-send-timeout: "10"
  client-max-body-size: "3m"

二、 應用配置並查看ConfigMap配置

[root@node-1 ~]# kubectl get configmaps -n nginx-ingress nginx-config -o yaml
apiVersion: v1
data:
  client-max-body-size: 3m
  proxy-connect-timeout: 10s
  proxy-read-timeout: 10s
  proxy-send-timeout: 10s
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"client-max-body-size":"3m","proxy-connect-timeout":"10s","proxy-read-timeout":"10s","proxy-send-timeout":"10"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"nginx-config","namespace":"nginx-ingress"}}
  creationTimestamp: "2019-12-24T04:39:23Z"
  name: nginx-config
  namespace: nginx-ingress
  resourceVersion: "13845543"
  selfLink: /api/v1/namespaces/nginx-ingress/configmaps/nginx-config
  uid: 9313ae47-a0f0-463e-a25a-1658f1ca0d57

3 、此時，ConfigMap定義的配置參數會被集羣中全部的Ingress資源繼承（除了annotations定義以外）

ConfigMap參數驗證

有不少參數能夠定義，詳情配置可參考方文檔說明：https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/configmap-and-annotations.md#Summary-of-ConfigMap-and-Annotations

4.1.2 Annotations自定義參數

ConfigMap定義的是全局的配置參數，修改後全部的配置都會受影響，若是想針對某個具體的ingress資源自定義參數，則能夠經過Annotations來實現，下面開始以實際的例子演示Annotations的使用。

一、修改ingress資源，添加annotations的定義,經過nginx.org組修改了一些參數，如proxy-connect-timeout，調度算法爲round_robin（默認爲least _conn）

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  labels:
    ingres-controller: nginx
  annotations:
    kubernets.io/ingress.class: nginx
    nginx.org/proxy-connect-timeout: "30s"
    nginx.org/proxy-send-timeout: "20s"
    nginx.org/proxy-read-timeout: "20s"
    nginx.org/client-max-body-size: "2m"
    nginx.org/fail-timeout: "5s"
    nginx.org/lb-method: "round_robin" 
spec:
  rules:
  - host: www.happylau.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: ingress-demo
          servicePort: 80

二、 從新應用ingress對象並查看參數配置狀況

Nginx Annotations演示

由上面的演示可得知，Annotations的優先級高於ConfigMapMap，Annotations修改參數只會影響到某一個具體的ingress資源，其定義的方法和ConfigMap相相近似，但又有差異，部分ConfigMap的參數Annotations沒法支持，反過來Annotations定義的參數ConfigMap也不必定支持，下圖列舉一下常規支持參數狀況：

通用參數

日誌支持

請求頭部

認證和安全

upstream支持

ConfigMap和Annotations詳細支持說明：連接說明

4.2 虛擬主機和路由

安裝nginx ingress時咱們安裝了一個customresourcedefinitions自定義資源，其可以提供除了默認ingress功能以外的一些高級特性如

  • 虛擬主機VirtualServer
  • 虛擬路由VirtualServerRoute
  • 健康檢查Healthcheck
  • 流量切割Split
  • 會話保持SessionCookie
  • 重定向Redirect

這些功能大部分依賴於Nginx Plus高級版本的支持，社區版本僅支持部分，對於企業級開發而言，豐富更多的功能能夠購買企業級Nginx Plus版本。以下以經過VirtualServer和VirtualServerRoute定義upstream配置爲例演示功能使用。

一、定義VirtualServer資源,其配置和ingress資源對象相似，能支持的功能會更豐富一點

apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
  name: cafe
spec:
  host: cafe.example.com
  tls:
    secret: cafe-secret
  upstreams:
  - name: tea
    service: tea-svc
    port: 80
    name: tea
    service: ingress-demo 
    subselector:
    version: canary
    lb-method: round_robin
    fail-timeout: 10s
    max-fails: 1
    max-conns: 32
    keepalive: 32
    connect-timeout: 30s
    read-timeout: 30s
    send-timeout: 30s
    next-upstream: "error timeout non_idempotent"
    next-upstream-timeout: 5s
    next-upstream-tries: 10
    client-max-body-size: 2m
    tls:
      enable: true
  routes:
  - path: /tea
    action:
      pass: tea

二、 應用資源並查看VirtualServer資源列表

[root@node-1 ~]# kubectl apply -f vs.yaml 
virtualserver.k8s.nginx.org/cafe unchanged
[root@node-1 ~]# kubectl get virtualserver
NAME                 AGE
cafe                 2m52s

三、檢查ingress控制器的配置文件狀況,生成的配置和upstream定義一致

nginx@nginx-ingress-7mpfc:/etc/nginx/conf.d$ cat vs_default_cafe.conf 
upstream vs_default_cafe_tea {
    zone vs_default_cafe_tea 256k;
    server 10.244.0.51:80 max_fails=1 fail_timeout=10s max_conns=32;
    server 10.244.1.146:80 max_fails=1 fail_timeout=10s max_conns=32;
    server 10.244.1.147:80 max_fails=1 fail_timeout=10s max_conns=32;
    server 10.244.2.162:80 max_fails=1 fail_timeout=10s max_conns=32;
    keepalive 32;
}

server {
    listen 80;
    server_name cafe.example.com;
    listen 443 ssl;
    ssl_certificate /etc/nginx/secrets/default;
    ssl_certificate_key /etc/nginx/secrets/default;
    ssl_ciphers NULL;
    server_tokens "on";

    location /tea {
        proxy_connect_timeout 30s;
        proxy_read_timeout 30s;
        proxy_send_timeout 30s;
        client_max_body_size 2m;
        proxy_max_temp_file_size 1024m;
        proxy_buffering on;
        proxy_http_version 1.1;
        set $default_connection_header "";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $vs_connection_header;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port $server_port;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_pass https://vs_default_cafe_tea;
        proxy_next_upstream error timeout non_idempotent;
        proxy_next_upstream_timeout 5s;
        proxy_next_upstream_tries 10;   
    }   
}

寫在最後

本文詳細介紹了基於nginx的ingress實現，經過實際的案例演示ingress的安裝部署，基於虛擬主機的配置，基於TLS加密實現https，高級章節中介紹了負載均衡參數定製，自定義資源虛擬主機和虛擬路由的實現，經過該章節相信能加深對ingress服務暴露機制的理解。實現ingress controller的方式有多種，下一個章節咱們將介紹基於HAproxy和TKE Ingress控制器的實現。

參考文獻

Ingress配置：https://kubernetes.io/docs/concepts/services-networking/ingress/

Ingress控制器：https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

Nginx ingress安裝文檔：https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/installation.md

Nginx ingress文檔說明：https://github.com/nginxinc/kubernetes-ingress/tree/master/docs

虛擬主機和路由：https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/virtualserver-and-virtualserverroute.md

當你的才華撐不起你的野心時，你就應該靜下心來學習

返回kubernetes系列教程目錄

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程