本章介紹kubernetes系列教程的ingress概念,在kubernetes中對外暴露服務的方式有兩種:service(NodePort或者外部LoadBalancer)和ingress,其中service是提供四層的負載均衡,經過iptables DNAT或lvs nat模式實現後端Pod的代理請求。如需實現http,域名,URI,證書等請求方式,service是沒法實現的,須要藉助於ingress來來實現,本文未來介紹ingress相關的內容。html
An API object that manages external access to the services in a cluster, typically HTTP.
Ingress can provide load balancing, SSL termination and name-based virtual hosting.node
引用官方關於ingress的介紹咱們能夠得知,ingress是一種經過http協議暴露kubernetes內部服務的api對象,即充當Edge Router邊界路由器的角色對外基於七層的負載均衡調度機制,可以提供如下幾個功能:nginx
實現Ingress包含的組件有:git
簡而言之,ingress控制器藉助service的服務發現機制實現配置的動態更新以實現Pod的負載均衡機制實現,因爲涉及到Ingress Controller的動態更新,目前社區Ingress Controller大致包含兩種類型的控制器:github
類型 | 常見類型 | 優勢 | 缺點 |
---|---|---|---|
傳統負載均衡 | nginx,haproxy | 成熟,穩定,高性能 | 動態更新需reload配置文件 |
微服務負載均衡 | Traefik,Envoy,Istio | 天生爲微服務而生,動態更新 | 性能還有待提高 |
By default, pods of Kubernetes services are not accessible from the external network, but only by other pods within the Kubernetes cluster. Kubernetes has a built‑in configuration for HTTP load balancing, called Ingress, that defines rules for external connectivity to Kubernetes services. Users who need to provide external access to their Kubernetes services create an Ingress resource that defines rules, including the URI path, backing service name, and other information. The Ingress controller can then automatically program a frontend load balancer to enable Ingress configuration. The NGINX Ingress Controller for Kubernetes is what enables Kubernetes to configure NGINX and NGINX Plus for load balancing Kubernetes services.web
Nginx Ingress Controller是實現ingress的具體實現,包含有兩個版本:Ngnix OSS和Nginx Plus版,後者是商業化加強版,支持更多的功能,詳情參考官方文檔介紹https://www.nginx.com/products/nginx/kubernetes-ingress-controller#compare-versions算法
首先須要安裝Nginx Ingress Controller控制器,控制器安裝方式包含兩種:DaemonSets和Deployments。apache
除此以外,還須要部署Namespace,ServiceAccount,RBAC,Secrets,Custom Resource Definitions等資源,以下開始部署。後端
一、github中下載源碼包,安裝部署文件在kubernetes-ingress/deployments/目錄下api
[root@node-1 ~]# git clone https://github.com/nginxinc/kubernetes-ingress.git [root@node-1 ~]# tree kubernetes-ingress/deployments/ kubernetes-ingress/deployments/ ├── common │ ├── custom-resource-definitions.yaml 自定義資源 │ ├── default-server-secret.yaml Secrets │ ├── nginx-config.yaml │ └── ns-and-sa.yaml Namspace+ServiceAccount ├── daemon-set │ ├── nginx-ingress.yaml DaemonSets控制器 │ └── nginx-plus-ingress.yaml ├── deployment │ ├── nginx-ingress.yaml Deployments控制器 │ └── nginx-plus-ingress.yaml ├── helm-chart Helm安裝包 │ ├── chart-icon.png │ ├── Chart.yaml │ ├── README.md │ ├── templates │ │ ├── controller-configmap.yaml │ │ ├── controller-custom-resources.yaml │ │ ├── controller-daemonset.yaml │ │ ├── controller-deployment.yaml │ │ ├── controller-leader-election-configmap.yaml │ │ ├── controller-secret.yaml │ │ ├── controller-serviceaccount.yaml │ │ ├── controller-service.yaml │ │ ├── controller-wildcard-secret.yaml │ │ ├── _helpers.tpl │ │ ├── NOTES.txt │ │ └── rbac.yaml │ ├── values-icp.yaml │ ├── values-plus.yaml │ └── values.yaml ├── rbac RBAC認證受權 │ └── rbac.yaml ├── README.md └── service Service定義 ├── loadbalancer-aws-elb.yaml ├── loadbalancer.yaml DaemonSets暴露服務方式 └── nodeport.yaml Deployments暴露服務方式
二、建立Namespace和ServiceAccount, kubectl apply -f common/default-server-secret.yaml
apiVersion: v1 kind: Namespace metadata: name: nginx-ingress --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress namespace: nginx-ingress
三、建立Secrets自簽名證書,kubectl apply -f common/default-server-secret.yaml
apiVersion: v1 kind: Secret metadata: name: default-server-secret namespace: nginx-ingress type: Opaque data: tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
四、建立ConfigMap自定義配置文件,kubectl apply -f common/nginx-config.yaml
kind: ConfigMap apiVersion: v1 metadata: name: nginx-config namespace: nginx-ingress data:
五、爲虛擬雲主機和虛擬雲主機路由定義自定義資源,支持自定義虛擬主機和虛擬路由,kubectl apply -f common/custom-resource-definitions.yaml
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: virtualservers.k8s.nginx.org spec: group: k8s.nginx.org versions: - name: v1 served: true storage: true scope: Namespaced names: plural: virtualservers singular: virtualserver kind: VirtualServer shortNames: - vs --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: virtualserverroutes.k8s.nginx.org spec: group: k8s.nginx.org versions: - name: v1 served: true storage: true scope: Namespaced names: plural: virtualserverroutes singular: virtualserverroute kind: VirtualServerRoute shortNames: - vsr
六、配置RBAC認證受權,實現ingress控制器訪問集羣中的其餘資源,kubectl apply -f rbac/rbac.yaml
kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: nginx-ingress rules: - apiGroups: - "" resources: - services - endpoints verbs: - get - list - watch - apiGroups: - "" resources: - secrets verbs: - get - list - watch - apiGroups: - "" resources: - configmaps verbs: - get - list - watch - update - create - apiGroups: - "" resources: - pods verbs: - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - extensions resources: - ingresses verbs: - list - watch - get - apiGroups: - "extensions" resources: - ingresses/status verbs: - update - apiGroups: - k8s.nginx.org resources: - virtualservers - virtualserverroutes verbs: - list - watch - get --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: nginx-ingress subjects: - kind: ServiceAccount name: nginx-ingress namespace: nginx-ingress roleRef: kind: ClusterRole name: nginx-ingress apiGroup: rbac.authorization.k8s.io
一、 部署控制器,控制器能夠DaemonSets和Deployment的形式部署,以下是DaemonSets的配置文件
apiVersion: apps/v1 kind: DaemonSet metadata: name: nginx-ingress namespace: nginx-ingress spec: selector: matchLabels: app: nginx-ingress template: metadata: labels: app: nginx-ingress #annotations: #prometheus.io/scrape: "true" #prometheus.io/port: "9113" spec: serviceAccountName: nginx-ingress containers: - image: nginx/nginx-ingress:edge imagePullPolicy: Always name: nginx-ingress ports: - name: http containerPort: 80 hostPort: 80 #經過hostPort的方式暴露端口 - name: https containerPort: 443 hostPort: 443 #- name: prometheus #containerPort: 9113 securityContext: allowPrivilegeEscalation: true runAsUser: 101 #nginx capabilities: drop: - ALL add: - NET_BIND_SERVICE env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name args: - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret #- -v=3 # Enables extensive logging. Useful for troubleshooting. #- -report-ingress-status #- -external-service=nginx-ingress #- -enable-leader-election #- -enable-prometheus-metrics
Deployments的配置文件
apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress namespace: nginx-ingress spec: replicas: 1 #副本的個數 selector: matchLabels: app: nginx-ingress template: metadata: labels: app: nginx-ingress #annotations: #prometheus.io/scrape: "true" #prometheus.io/port: "9113" spec: serviceAccountName: nginx-ingress containers: - image: nginx/nginx-ingress:edge imagePullPolicy: Always name: nginx-ingress ports: #內部暴露的服務端口,須要經過NodePort的方式暴露給外部 - name: http containerPort: 80 - name: https containerPort: 443 #- name: prometheus #containerPort: 9113 securityContext: allowPrivilegeEscalation: true runAsUser: 101 #nginx capabilities: drop: - ALL add: - NET_BIND_SERVICE env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name args: - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret #- -v=3 # Enables extensive logging. Useful for troubleshooting. #- -report-ingress-status #- -external-service=nginx-ingress #- -enable-leader-election #- -enable-prometheus-metrics
二、咱們以DaemonSets的方式部署,DaemonSet部署集羣中各個節點都是對等,若是有外部LoadBalancer則經過外部負載均衡路由至Ingress中
[root@node-1 deployments]# kubectl apply -f daemon-set/nginx-ingress.yaml daemonset.apps/nginx-ingress created [root@node-1 deployments]# kubectl get daemonsets -n nginx-ingress NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE nginx-ingress 3 3 3 3 3 <none> 15s [root@node-1 ~]# kubectl get pods -n nginx-ingress -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-ingress-7mpfc 1/1 Running 0 2m44s 10.244.0.50 node-1 <none> <none> nginx-ingress-l2rtj 1/1 Running 0 2m44s 10.244.1.144 node-2 <none> <none> nginx-ingress-tgf6r 1/1 Running 0 2m44s 10.244.2.160 node-3 <none> <none>
三、校驗Nginx Ingress安裝狀況,此時三個節點均是對等,即訪問任意一個節點均能實現相同的效果,統一入口則經過外部負載均衡,若是在雲環境下執行kubectl apply -f service/loadbalancer.yaml建立外部負載均衡實現入口調度,自建的能夠經過lvs或nginx等負載均衡實現接入,本文再也不贅述,讀者能夠自行研究。
備註說明:若是以Deployments的方式部署,則須要執行service/nodeport.yaml建立NodePort類型的Service,實現的效果和DaemonSets相似。
上面的章節已安裝了一個Nginx Ingress Controller控制器,有了Ingress控制器後,咱們就能夠定義Ingress資源來實現七層負載轉發了,大致上Ingress支持三種使用方式:1. 基於虛擬主機轉發,2. 基於虛擬機主機URI轉發,3. 支持TLS加密轉發。
一、環境準備,先建立一個nginx的Deployment應用,包含2個副本
[root@node-1 ~]# kubectl run ingress-demo --image=nginx:1.7.9 --port=80 --replicas=2 [root@node-1 ~]# kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE ingress-demo 2/2 2 2 116s
二、以service方式暴露服務端口
[root@node-1 ~]# kubectl expose deployment ingress-demo --port=80 --protocol=TCP --target-port=80 service/ingress-demo exposed [root@node-1 ~]# kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-demo ClusterIP 10.109.33.91 <none> 80/TCP 2m15s
三、上述兩個步驟已建立了一個service,以下咱們定義一個ingress對象將起轉發至ingress-demo這個service,經過ingress.class指定控制器的類型爲nginx
[root@node-1 nginx-ingress]# cat nginx-ingress-demo.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-ingress-demo labels: ingres-controller: nginx annotations: kubernets.io/ingress.class: nginx spec: rules: - host: www.happylau.cn http: paths: - path: / backend: serviceName: ingress-demo servicePort: 80
四、建立ingress對象
[root@node-1 nginx-ingress]# kubectl apply -f nginx-ingress-demo.yaml ingress.extensions/nginx-ingress-demo created 查看ingress資源列表 [root@node-1 nginx-ingress]# kubectl get ingresses NAME HOSTS ADDRESS PORTS AGE nginx-ingress-demo www.happylau.cn 80 4m4s
五、查看ingress詳情,能夠在Rules規則中看到後端Pod的列表,自動發現和關聯相關Pod
[root@node-1 ~]# kubectl describe ingresses nginx-ingress-demo Name: nginx-ingress-demo Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- www.happylau.cn / ingress-demo:80 (10.244.1.146:80,10.244.2.162:80) Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-demo","namespace":"default"},"spec":{"rules":[{"host":"www.happylaulab.cn","http":{"paths":[{"backend":{"serviceName":"ingress-demo","servicePort":80},"path":"/"}]}}]}} kubernets.io/ingress.class: nginx Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 9m7s nginx-ingress-controller Configuration for default/nginx-ingress-demo was added or updated Normal AddedOrUpdated 9m7s nginx-ingress-controller Configuration for default/nginx-ingress-demo was added or updated Normal AddedOrUpdated 9m7s nginx-ingress-controller Configuration for default/nginx-ingress-demo was added or updated
六、測試驗證,ingress規則的配置信息已注入到Ingress Controller中,環境中Ingress Controller是以DaemonSets的方式部署在集羣中,若是有外部的負載均衡,則將www.happylau.cn域名的地址解析爲負載均衡VIP。因爲測試環境沒有搭建負載均衡,將hosts解析執行node-1,node-2或者node-3任意一個IP都能實現相同的功能。
上述測試解析正常,固然也能夠解析爲node-1和node-2的IP,以下:
[root@node-1 ~]# curl -I http://www.happylau.cn --resolve www.happylau.cn:80:10.254.100.101 HTTP/1.1 200 OK Server: nginx/1.17.6 Date: Tue, 24 Dec 2019 10:32:22 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Tue, 23 Dec 2014 16:25:09 GMT ETag: "54999765-264" Accept-Ranges: bytes [root@node-1 ~]# curl -I http://www.happylau.cn --resolve www.happylau.cn:80:10.254.100.102 HTTP/1.1 200 OK Server: nginx/1.17.6 Date: Tue, 24 Dec 2019 10:32:24 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Tue, 23 Dec 2014 16:25:09 GMT ETag: "54999765-264" Accept-Ranges: bytes
上面的章節介紹了ingress資源對象的申明配置,在這個章節中咱們探究一下Nginx Ingress Controller的實現機制和動態配置更新機制,以方便了解Ingress控制器的工做機制。
一、 查看Nginx Controller控制器的配置文件,在nginx-ingress pod中存儲着ingress的配置文件
[root@node-1 ~]# kubectl get pods -n nginx-ingress NAME READY STATUS RESTARTS AGE nginx-ingress-7mpfc 1/1 Running 0 6h15m nginx-ingress-l2rtj 1/1 Running 0 6h15m nginx-ingress-tgf6r 1/1 Running 0 6h15m #查看配置文件,每一個ingress生成一個配置文件,文件名爲:命名空間-ingres名稱.conf [root@node-1 ~]# kubectl exec -it nginx-ingress-7mpfc -n nginx-ingress -- ls -l /etc/nginx/conf.d total 4 -rw-r--r-- 1 nginx nginx 1005 Dec 24 10:06 default-nginx-ingress-demo.conf #查看配置文件 [root@node-1 ~]# kubectl exec -it nginx-ingress-7mpfc -n nginx-ingress -- cat /etc/nginx/conf.d/default-nginx-ingress-demo.conf # configuration for default/nginx-ingress-demo #upstream的配置,會用least_conn算法,經過service服務發現機制動態識別到後端的Pod upstream default-nginx-ingress-demo-www.happylau.cn-ingress-demo-80 { zone default-nginx-ingress-demo-www.happylau.cn-ingress-demo-80 256k; random two least_conn; server 10.244.1.146:80 max_fails=1 fail_timeout=10s max_conns=0; server 10.244.2.162:80 max_fails=1 fail_timeout=10s max_conns=0; } server { listen 80; server_tokens on; server_name www.happylau.cn; location / { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-demo-www.happylau.cn-ingress-demo-80; #調用upstream實現代理 } }
經過上述查看配置文件可得知,Nginx Ingress Controller實際是根據ingress規則生成對應的nginx配置文件,以實現代理轉發的功能,加入Deployments的副本數變動後nginx的配置文件會發生什麼改變呢?
二、更新控制器的副本數,由2個Pod副本擴容至3個
[root@node-1 ~]# kubectl scale --replicas=3 deployment ingress-demo deployment.extensions/ingress-demo scaled [root@node-1 ~]# kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE ingress-demo 3/3 3 3 123m
三、再次查看nginx的配置文件,ingress藉助於service的服務發現機制,將加入的Pod自動加入到nginx upstream中
四、查看nginx pod的日誌(kubectl logs nginx-ingress-7mpfc -n nginx-ingress),有reload優雅重啓的記錄,即經過更新配置文件+reload實現配置動態更新。
經過上述的配置可知,ingress調用kubernetes api去感知kubernetes集羣中的變化狀況,Pod的增長或減小這些變化,而後動態更新nginx ingress controller的配置文件,並從新載入配置。當集羣規模越大時,會頻繁涉及到配置文件的變更和重載,所以nginx這方面會存在先天的劣勢,專門爲微服務負載均衡應運而生,如Traefik,Envoy,Istio,這些負載均衡工具可以提供大規模,頻繁動態更新的場景,但性能相比Nginx,HAproxy還存在必定的劣勢。日後的章節中,咱們再對其餘的Ingress控制器作介紹。
Ingress支持URI格式的轉發方式,同時支持URL重寫,以下以兩個service爲例演示,service-1安裝nginx,service-2安裝httpd,分別用http://demo.happylau.cn/news和http://demo.happylau.cn/sports轉發到兩個不一樣的service
一、環境準備,建立兩個應用並實現service暴露,建立deployments時指定--explose建立service
[root@node-1 ~]# kubectl run service-1 --image=nginx:1.7.9 --port=80 --replicas=1 --expose=true service/service-1 created deployment.apps/service-1 created [root@node-1 ~]# kubectl run service-2 --image=httpd --port=80 --replicas=1 --expose=true service/service-2 created deployment.apps/service-2 created 查看deployment狀態 [root@node-1 ~]# kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE ingress-demo 4/4 4 4 4h36m service-1 1/1 1 1 65s service-2 1/1 1 1 52s 查看service狀態,服務已經正常 [root@node-1 ~]# kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-demo ClusterIP 10.109.33.91 <none> 80/TCP 4h36m kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 101d service-1 ClusterIP 10.106.245.71 <none> 80/TCP 68s service-2 ClusterIP 10.104.204.158 <none> 80/TCP 55s
二、建立ingress對象,經過一個域名將請求轉發至後端兩個service
[root@node-1 nginx-ingress]# cat nginx-ingress-uri-demo.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-ingress-uri-demo labels: ingres-controller: nginx annotations: kubernets.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: demo.happylau.cn http: paths: - path: /news backend: serviceName: service-1 servicePort: 80 - path: /sports backend: serviceName: service-2 servicePort: 80
三、建立ingress規則,查看詳情
[root@node-1 nginx-ingress]# kubectl apply -f nginx-ingress-uri-demo.yaml ingress.extensions/nginx-ingress-uri-demo created #查看詳情 [root@node-1 nginx-ingress]# kubectl get ingresses. NAME HOSTS ADDRESS PORTS AGE nginx-ingress-demo www.happylau.cn 80 4h35m nginx-ingress-uri-demo demo.happylau.cn 80 4s [root@node-1 nginx-ingress]# kubectl describe ingresses nginx-ingress-uri-demo Name: nginx-ingress-uri-demo Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: #對應的轉發url規則 Host Path Backends ---- ---- -------- demo.happylau.cn /news service-1:80 (10.244.2.163:80) /sports service-2:80 (10.244.1.148:80) Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-uri-demo","namespace":"default"},"spec":{"rules":[{"host":"demo.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-1","servicePort":80},"path":"/news"},{"backend":{"serviceName":"service-2","servicePort":80},"path":"/sports"}]}}]}} kubernets.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 11s nginx-ingress-controller Configuration for default/nginx-ingress-uri-demo was added or updated Normal AddedOrUpdated 11s nginx-ingress-controller Configuration for default/nginx-ingress-uri-demo was added or updated Normal AddedOrUpdated 11s nginx-ingress-controller Configuration for default/nginx-ingress-uri-demo was added or updated
四、準備測試,站點中建立對應的路徑
[root@node-1 ~]# kubectl exec -it service-1-7b66bf758f-xj9jh /bin/bash root@service-1-7b66bf758f-xj9jh:/# echo "service-1 website page" >/usr/share/nginx/html/news [root@node-1 ~]# kubectl exec -it service-2-7c7444684d-w9cv9 /bin/bash root@service-2-7c7444684d-w9cv9:/usr/local/apache2# echo "service-2 website page" >/usr/local/apache2/htdocs/sports
五、測試驗證
[root@node-1 ~]# curl http://demo.happylau.cn/news --resolve demo.happylau.cn:80:10.254.100.101 service-1 website page [root@node-1 ~]# curl http://demo.happylau.cn/sports --resolve demo.happylau.cn:80:10.254.100.101 service-2 website page
六、經過上述的驗證測試能夠得知,ingress支持URI的路由方式轉發,其對應在ingress中的配置文件內容是怎樣的呢,咱們看下ingress controller生成對應的nginx配置文件內容,實際是經過ingress的location來實現,將不一樣的localtion轉發至不一樣的upstream以實現service的關聯,配置文件以下:
[root@node-1 ~]# kubectl exec -it nginx-ingress-7mpfc -n nginx-ingress /bin/bash nginx@nginx-ingress-7mpfc:/$ cat /etc/nginx/conf.d/default-nginx-ingress-uri-demo.conf |grep -v "^$" # configuration for default/nginx-ingress-uri-demo #定義兩個upstream和後端的service關聯 upstream default-nginx-ingress-uri-demo-demo.happylau.cn-service-1-80 { zone default-nginx-ingress-uri-demo-demo.happylau.cn-service-1-80 256k; random two least_conn; server 10.244.2.163:80 max_fails=1 fail_timeout=10s max_conns=0; } upstream default-nginx-ingress-uri-demo-demo.happylau.cn-service-2-80 { zone default-nginx-ingress-uri-demo-demo.happylau.cn-service-2-80 256k; random two least_conn; server 10.244.1.148:80 max_fails=1 fail_timeout=10s max_conns=0; } server { listen 80; server_tokens on; server_name demo.happylau.cn; #定義location實現代理,經過proxy_pass和後端的service關聯 location /news { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-uri-demo-demo.happylau.cn-service-1-80; } location /sports { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-uri-demo-demo.happylau.cn-service-2-80; } }
ingress支持基於名稱的虛擬主機,實現單個IP多個域名轉發的需求,經過請求頭部攜帶主機名方式區分開,將上個章節的ingress刪除,使用service-1和service-2兩個service來作演示。
一、建立ingress規則,經過主機名實現轉發規則
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-ingress-virtualname-demo labels: ingres-controller: nginx annotations: kubernets.io/ingress.class: nginx spec: rules: - host: news.happylau.cn http: paths: - path: / backend: serviceName: service-1 servicePort: 80 - host: sports.happylau.cn http: paths: - path: / backend: serviceName: service-2 servicePort: 80
二、生成ingress規則並查看詳情,一個ingress對應兩個HOSTS
[root@node-1 nginx-ingress]# kubectl apply -f nginx-ingress-virtualname.yaml ingress.extensions/nginx-ingress-virtualname-demo created #查看列表 [root@node-1 nginx-ingress]# kubectl get ingresses nginx-ingress-virtualname-demo NAME HOSTS ADDRESS PORTS AGE nginx-ingress-virtualname-demo news.happylau.cn,sports.happylau.cn 80 12s #查看詳情 [root@node-1 nginx-ingress]# kubectl describe ingresses nginx-ingress-virtualname-demo Name: nginx-ingress-virtualname-demo Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- news.happylau.cn / service-1:80 (10.244.2.163:80) sports.happylau.cn / service-2:80 (10.244.1.148:80) Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-virtualname-demo","namespace":"default"},"spec":{"rules":[{"host":"news.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-1","servicePort":80},"path":"/"}]}},{"host":"sports.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-2","servicePort":80},"path":"/"}]}}]}} kubernets.io/ingress.class: nginx Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 28s nginx-ingress-controller Configuration for default/nginx-ingress-virtualname-demo was added or updated Normal AddedOrUpdated 28s nginx-ingress-controller Configuration for default/nginx-ingress-virtualname-demo was added or updated Normal AddedOrUpdated 28s nginx-ingress-controller Configuration for default/nginx-ingress-virtualname-demo was added or updated
三、準備測試數據並測試
[root@node-1 ~]# kubectl exec -it service-1-7b66bf758f-xj9jh /bin/bash root@service-1-7b66bf758f-xj9jh:/# echo "news demo" >/usr/share/nginx/html/index.html [root@node-1 ~]# kubectl exec -it service-2-7c7444684d-w9cv9 /bin/bash root@service-2-7c7444684d-w9cv9:/usr/local/apache2# echo "sports demo" >/usr/local/apache2/htdocs/index.html
測試:
[root@node-1 ~]# curl http://news.happylau.cn --resolve news.happylau.cn:80:10.254.100.102 news demo [root@node-1 ~]# curl http://sports.happylau.cn --resolve sports.happylau.cn:80:10.254.100.102 sports demo
四、查看nginx的配置文件內容,經過在server中定義不一樣的server_name以區分,代理到不一樣的upstream以實現service的代理。
# configuration for default/nginx-ingress-virtualname-demo upstream default-nginx-ingress-virtualname-demo-news.happylau.cn-service-1-80 { zone default-nginx-ingress-virtualname-demo-news.happylau.cn-service-1-80 256k; random two least_conn; server 10.244.2.163:80 max_fails=1 fail_timeout=10s max_conns=0; } upstream default-nginx-ingress-virtualname-demo-sports.happylau.cn-service-2-80 { zone default-nginx-ingress-virtualname-demo-sports.happylau.cn-service-2-80 256k; random two least_conn; server 10.244.1.148:80 max_fails=1 fail_timeout=10s max_conns=0; } server { listen 80; server_tokens on; server_name news.happylau.cn; location / { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-virtualname-demo-news.happylau.cn-service-1-80; } } server { listen 80; server_tokens on; server_name sports.happylau.cn; location / { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-virtualname-demo-sports.happylau.cn-service-2-80; } }
四層的負載均衡沒法支持https請求,當前大部分業務都要求以https方式接入,Ingress能支持https的方式接入,經過Secrets存儲證書+私鑰,實現https接入,同時還能支持http跳轉功能。對於用戶的請求流量來講,客戶端到ingress controller是https流量,ingress controller到後端service則是http,提升用戶訪問性能,以下介紹ingress TLS功能實現步驟。
一、生成自簽名證書和私鑰
[root@node-1 ~]# openssl req -x509 -newkey rsa:2048 -nodes -days 365 -keyout tls.key -out tls.crt Generating a 2048 bit RSA private key ....................................................+++ ........................................+++ writing new private key to 'tls.key' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN #國家 State or Province Name (full name) []:GD #省份 Locality Name (eg, city) [Default City]:ShenZhen #城市 Organization Name (eg, company) [Default Company Ltd]:Tencent #公司 Organizational Unit Name (eg, section) []:HappyLau #組織 Common Name (eg, your name or your server's hostname) []:www.happylau.cn #域名 Email Address []:573302346@qq.com #郵箱地址 #tls.crt爲證書,tls.key爲私鑰 [root@node-1 ~]# ls tls.* -l -rw-r--r-- 1 root root 1428 12月 26 13:21 tls.crt -rw-r--r-- 1 root root 1708 12月 26 13:21 tls.key
二、配置Secrets,將證書和私鑰配置到Secrets中
[root@node-1 ~]# kubectl create secret tls happylau-sslkey --cert=tls.crt --key=tls.key secret/happylau-sslkey created 查看Secrets詳情,證書和私要包含在data中,文件名爲兩個不一樣的key:tls.crt和tls.key [root@node-1 ~]# kubectl describe secrets happylau-sslkey Name: happylau-sslkey Namespace: default Labels: <none> Annotations: <none> Type: kubernetes.io/tls Data ==== tls.crt: 1428 bytes tls.key: 1708 bytes
三、配置ingress調用Secrets實現SSL證書加密
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-ingress-tls-demo labels: ingres-controller: nginx annotations: kubernets.io/ingress.class: nginx spec: tls: - hosts: - news.happylau.cn - sports.happylau.cn secretName: happylau-sslkey rules: - host: news.happylau.cn http: paths: - path: / backend: serviceName: service-1 servicePort: 80 - host: sports.happylau.cn http: paths: - path: / backend: serviceName: service-2 servicePort: 80
四、建立ingress並查看ingress詳情
[root@node-1 nginx-ingress]# kubectl describe ingresses nginx-ingress-tls-demo Name: nginx-ingress-tls-demo Namespace: default Address: Default backend: default-http-backend:80 (<none>) TLS: happylau-sslkey terminates news.happylau.cn,sports.happylau.cn Rules: Host Path Backends ---- ---- -------- news.happylau.cn / service-1:80 (10.244.2.163:80) sports.happylau.cn / service-2:80 (10.244.1.148:80) Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernets.io/ingress.class":"nginx"},"labels":{"ingres-controller":"nginx"},"name":"nginx-ingress-tls-demo","namespace":"default"},"spec":{"rules":[{"host":"news.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-1","servicePort":80},"path":"/"}]}},{"host":"sports.happylau.cn","http":{"paths":[{"backend":{"serviceName":"service-2","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["news.happylau.cn","sports.happylau.cn"],"secretName":"happylau-sslkey"}]}} kubernets.io/ingress.class: nginx Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 22s nginx-ingress-controller Configuration for default/nginx-ingress-tls-demo was added or updated Normal AddedOrUpdated 22s nginx-ingress-controller Configuration for default/nginx-ingress-tls-demo was added or updated Normal AddedOrUpdated 22s nginx-ingress-controller Configuration for default/nginx-ingress-tls-demo was added or updated
五、 將news.happylau.cn和sports.happylau.cn寫入到hosts文件中,並經過https://news.happylau.cn 的方式訪問,瀏覽器訪問內容提示證書以下,信任證書便可訪問到站點內容。
查看證書詳情,正是咱們製做的自簽名證書,生產實際使用時,推薦使用CA機構頒發簽名證書。
六、接下來查看一下tls配置https的nginx配置文件內容,能夠看到在server塊啓用了https並配置證書,同時配置了http跳轉,所以直接訪問http也可以實現自動跳轉到https功能。
# configuration for default/nginx-ingress-tls-demo upstream default-nginx-ingress-tls-demo-news.happylau.cn-service-1-80 { zone default-nginx-ingress-tls-demo-news.happylau.cn-service-1-80 256k; random two least_conn; server 10.244.2.163:80 max_fails=1 fail_timeout=10s max_conns=0; } upstream default-nginx-ingress-tls-demo-sports.happylau.cn-service-2-80 { zone default-nginx-ingress-tls-demo-sports.happylau.cn-service-2-80 256k; random two least_conn; server 10.244.1.148:80 max_fails=1 fail_timeout=10s max_conns=0; } server { listen 80; listen 443 ssl; #https監聽端口,證書和key,實現和Secrets關聯 ssl_certificate /etc/nginx/secrets/default-happylau-sslkey; ssl_certificate_key /etc/nginx/secrets/default-happylau-sslkey; server_tokens on; server_name news.happylau.cn; #http跳轉功能,即訪問http會自動跳轉至https if ($scheme = http) { return 301 https://$host:443$request_uri; } location / { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-tls-demo-news.happylau.cn-service-1-80; } } server { listen 80; listen 443 ssl; ssl_certificate /etc/nginx/secrets/default-happylau-sslkey; ssl_certificate_key /etc/nginx/secrets/default-happylau-sslkey; server_tokens on; server_name sports.happylau.cn; if ($scheme = http) { return 301 https://$host:443$request_uri; } location / { proxy_http_version 1.1; proxy_connect_timeout 60s; proxy_read_timeout 60s; proxy_send_timeout 60s; client_max_body_size 1m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_pass http://default-nginx-ingress-tls-demo-sports.happylau.cn-service-2-80; } }
ingress controller提供了基礎反向代理的功能,若是須要定製化nginx的特性或參數,須要經過ConfigMap和Annotations來實現,二者實現的方式有所不一樣,ConfigMap用於指定整個ingress集羣資源的基本參數,修改後會被全部的ingress對象所繼承;Annotations則被某個具體的ingress對象所使用,修改只會影響某個具體的ingress資源,衝突時其優先級高於ConfigMap。
安裝nginx ingress controller時默認會包含一個空的ConfigMap,能夠經過ConfigMap來自定義nginx controller的默認參數,以下以修改一些參數爲例:
一、 定義ConfigMap參數
kind: ConfigMap apiVersion: v1 metadata: name: nginx-config namespace: nginx-ingress data: proxy-connect-timeout: "10s" proxy-read-timeout: "10s" proxy-send-timeout: "10" client-max-body-size: "3m"
二、 應用配置並查看ConfigMap配置
[root@node-1 ~]# kubectl get configmaps -n nginx-ingress nginx-config -o yaml apiVersion: v1 data: client-max-body-size: 3m proxy-connect-timeout: 10s proxy-read-timeout: 10s proxy-send-timeout: 10s kind: ConfigMap metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","data":{"client-max-body-size":"3m","proxy-connect-timeout":"10s","proxy-read-timeout":"10s","proxy-send-timeout":"10"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"nginx-config","namespace":"nginx-ingress"}} creationTimestamp: "2019-12-24T04:39:23Z" name: nginx-config namespace: nginx-ingress resourceVersion: "13845543" selfLink: /api/v1/namespaces/nginx-ingress/configmaps/nginx-config uid: 9313ae47-a0f0-463e-a25a-1658f1ca0d57
3 、此時,ConfigMap定義的配置參數會被集羣中全部的Ingress資源繼承(除了annotations定義以外)
有不少參數能夠定義,詳情配置可參考方文檔說明:https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/configmap-and-annotations.md#Summary-of-ConfigMap-and-Annotations
ConfigMap定義的是全局的配置參數,修改後全部的配置都會受影響,若是想針對某個具體的ingress資源自定義參數,則能夠經過Annotations來實現,下面開始以實際的例子演示Annotations的使用。
一、修改ingress資源,添加annotations的定義,經過nginx.org組修改了一些參數,如proxy-connect-timeout,調度算法爲round_robin(默認爲least _conn)
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-ingress-demo labels: ingres-controller: nginx annotations: kubernets.io/ingress.class: nginx nginx.org/proxy-connect-timeout: "30s" nginx.org/proxy-send-timeout: "20s" nginx.org/proxy-read-timeout: "20s" nginx.org/client-max-body-size: "2m" nginx.org/fail-timeout: "5s" nginx.org/lb-method: "round_robin" spec: rules: - host: www.happylau.cn http: paths: - path: / backend: serviceName: ingress-demo servicePort: 80
二、 從新應用ingress對象並查看參數配置狀況
由上面的演示可得知,Annotations的優先級高於ConfigMapMap,Annotations修改參數只會影響到某一個具體的ingress資源,其定義的方法和ConfigMap相相近似,但又有差異,部分ConfigMap的參數Annotations沒法支持,反過來Annotations定義的參數ConfigMap也不必定支持,下圖列舉一下常規支持參數狀況:
ConfigMap和Annotations詳細支持說明:連接說明
安裝nginx ingress時咱們安裝了一個customresourcedefinitions自定義資源,其可以提供除了默認ingress功能以外的一些高級特性如
這些功能大部分依賴於Nginx Plus高級版本的支持,社區版本僅支持部分,對於企業級開發而言,豐富更多的功能能夠購買企業級Nginx Plus版本。以下以經過VirtualServer和VirtualServerRoute定義upstream配置爲例演示功能使用。
一、定義VirtualServer資源,其配置和ingress資源對象相似,能支持的功能會更豐富一點
apiVersion: k8s.nginx.org/v1 kind: VirtualServer metadata: name: cafe spec: host: cafe.example.com tls: secret: cafe-secret upstreams: - name: tea service: tea-svc port: 80 name: tea service: ingress-demo subselector: version: canary lb-method: round_robin fail-timeout: 10s max-fails: 1 max-conns: 32 keepalive: 32 connect-timeout: 30s read-timeout: 30s send-timeout: 30s next-upstream: "error timeout non_idempotent" next-upstream-timeout: 5s next-upstream-tries: 10 client-max-body-size: 2m tls: enable: true routes: - path: /tea action: pass: tea
二、 應用資源並查看VirtualServer資源列表
[root@node-1 ~]# kubectl apply -f vs.yaml virtualserver.k8s.nginx.org/cafe unchanged [root@node-1 ~]# kubectl get virtualserver NAME AGE cafe 2m52s
三、檢查ingress控制器的配置文件狀況,生成的配置和upstream定義一致
nginx@nginx-ingress-7mpfc:/etc/nginx/conf.d$ cat vs_default_cafe.conf upstream vs_default_cafe_tea { zone vs_default_cafe_tea 256k; server 10.244.0.51:80 max_fails=1 fail_timeout=10s max_conns=32; server 10.244.1.146:80 max_fails=1 fail_timeout=10s max_conns=32; server 10.244.1.147:80 max_fails=1 fail_timeout=10s max_conns=32; server 10.244.2.162:80 max_fails=1 fail_timeout=10s max_conns=32; keepalive 32; } server { listen 80; server_name cafe.example.com; listen 443 ssl; ssl_certificate /etc/nginx/secrets/default; ssl_certificate_key /etc/nginx/secrets/default; ssl_ciphers NULL; server_tokens "on"; location /tea { proxy_connect_timeout 30s; proxy_read_timeout 30s; proxy_send_timeout 30s; client_max_body_size 2m; proxy_max_temp_file_size 1024m; proxy_buffering on; proxy_http_version 1.1; set $default_connection_header ""; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $vs_connection_header; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass https://vs_default_cafe_tea; proxy_next_upstream error timeout non_idempotent; proxy_next_upstream_timeout 5s; proxy_next_upstream_tries 10; } }
本文詳細介紹了基於nginx的ingress實現,經過實際的案例演示ingress的安裝部署,基於虛擬主機的配置,基於TLS加密實現https,高級章節中介紹了負載均衡參數定製,自定義資源虛擬主機和虛擬路由的實現,經過該章節相信能加深對ingress服務暴露機制的理解。實現ingress controller的方式有多種,下一個章節咱們將介紹基於HAproxy和TKE Ingress控制器的實現。
Ingress配置:https://kubernetes.io/docs/concepts/services-networking/ingress/
Ingress控制器:https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
Nginx ingress安裝文檔:https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/installation.md
Nginx ingress文檔說明:https://github.com/nginxinc/kubernetes-ingress/tree/master/docs
虛擬主機和路由:https://github.com/nginxinc/kubernetes-ingress/blob/master/docs/virtualserver-and-virtualserverroute.md
當你的才華撐不起你的野心時,你就應該靜下心來學習