在Kuberbetes中除了使用NodePort,LoadBalancer, Port Proxy(hostPort)等實現外部訪問入口以外,還可使用Kubernetes 自帶的Ingress來實現服務的負載均衡和策略路由的工做,其本質至關於一個Nginx代理服務器,能夠對請求(http)實現精細的控制。php
Ingress 是能夠利用 Nginx、Haproxy 等負載均衡器暴露集羣內服務的工具,html
使用Ingress能夠配置一個用於外部訪問的URL地址,同時實現流量負載,SSL,基於名稱的虛擬主機(例如Nginx的upstream)等。用戶經過發送POST請求到API server,來申請Ingress資源, Ingress控制器負責完成Ingress轉發工做,一般會使用負載均衡,也能夠配置邊緣路由器或其餘前端以幫助以HA方式處理流量。前端
在當前版本中,Ingress處於測試版本,在使用Ingress時,須要先建立Ingress Contronller.node
參考連接nginx
在Kubernetes中,Ingress Controller將以Pod的形式運行,監控apiserver的 ingress接口後端的backend services,若是service發生變化,則Ingress Controller 應自動更新其轉發規則。
Ingress Controller 須要實現基於不一樣HTTP URL向後轉發的負載均衡分發規則,並可靈活設置7層的負載分發策略。若是公有云服務商能提供該類型的HTTP路由LoadBlanacer,也能夠設置其爲Ingress Controller。git
在新的版本中Ingress Controller和具備負載均衡的軟件已經捆綁到一塊兒,目前支持多種類型, 如nginx ,haproxy 等。 github
使用以下的定義的文件,能夠直接從官方下載,須要修改谷歌的鏡像地址(https://kubernetes.github.io/ingress-nginx/deploy/):web
--- apiVersion: v1 kind: Namespace metadata: name: ingress-nginx --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: default-http-backend labels: app: default-http-backend namespace: ingress-nginx spec: replicas: 1 selector: matchLabels: app: default-http-backend template: metadata: labels: app: default-http-backend spec: terminationGracePeriodSeconds: 60 containers: - name: default-http-backend # Any image is permissible as long as: # 1. It serves a 404 page at / # 2. It serves 200 on a /healthz endpoint image: mirrorgooglecontainers/defaultbackend:1.4 livenessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 ports: - containerPort: 8080 resources: limits: cpu: 10m memory: 20Mi requests: cpu: 10m memory: 20Mi --- apiVersion: v1 kind: Service metadata: name: default-http-backend namespace: ingress-nginx labels: app: default-http-backend spec: ports: - port: 80 targetPort: 8080 selector: app: default-http-backend --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "extensions" resources: - ingresses verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx spec: replicas: 1 selector: matchLabels: app: ingress-nginx template: metadata: labels: app: ingress-nginx annotations: prometheus.io/port: '10254' prometheus.io/scrape: 'true' spec: serviceAccountName: nginx-ingress-serviceaccount containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.16.2 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io # - --report-node-internal-ip-address=true securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 33 runAsUser: 33 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 # hostPort: 80 若是不使用NodePort,可使用hostPort的方式直接映射到宿主端口 - name: https containerPort: 443 # hostPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1
使用以下文件建立 Ingress的services:apache
apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx spec: type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP - name: https port: 443 targetPort: 443 protocol: TCP selector: app: ingress-nginx
這裏使用的nodeport的方式,對於其餘雲平臺可使用官方提供的對應yaml文件建立。後端
查看配置是否成功:
# kubectl get pods --all-namespaces -l app=ingress-nginx --watch NAMESPACE NAME READY STATUS RESTARTS AGE ingress-nginx nginx-ingress-controller-6c9fcdf8d9-fvn57 1/1 Running 0 38m # kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE default-http-backend-7fb45cbc-kv24l 1/1 Running 0 39m nginx-ingress-controller-6c9fcdf8d9-fvn57 1/1 Running 0 39m # kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default-http-backend ClusterIP 10.1.74.56 <none> 80/TCP 40m ingress-nginx NodePort 10.1.28.135 <none> 80:34596/TCP,443:28411/TCP 24m
在安裝有kube-proxy的節點上會顯示Nodeport映射的端口:
[root@node-2 ~]# netstat -lntp|egrep "28411|34596" tcp6 0 0 :::28411 :::* LISTEN 1164/kube-proxy tcp6 0 0 :::34596 :::* LISTEN 1164/kube-proxy [root@node-3 ~]# netstat -lntp|egrep "28411|34596" tcp6 0 0 :::28411 :::* LISTEN 1164/kube-proxy tcp6 0 0 :::34596 :::* LISTEN 1164/kube-proxy
訪問任意kube-proxy節點的端口,會返回404(還未配置Ingress)說明Ingress安裝完成:
# curl 10.0.0.3:34596 default backend - 404
固然,在指定NodePort時,也能夠指定端口:
apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx spec: type: NodePort ports: - name: http port: 80 targetPort: 80 nodePort: 20080 protocol: TCP - name: https port: 443 targetPort: 443 nodePort: 20443 protocol: TCP selector: app: ingress-nginx
使用Ingress能夠實現多種訪問策略:
定義以下Ingress配置文件:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test1 spec: rules: - host: test.com http: paths: - path: /demo backend: serviceName: php-apache servicePort: 80 # curl test.com:20080/demo/index.html demo page!
定義以下文件:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test2 spec: rules: - host: test.com http: paths: - path: /tomcat backend: serviceName: tomcat-service servicePort: 8080 - path: /apache backend: serviceName: php-apache servicePort: 80
這裏會將路徑帶入到後端服務中,若是後端的服務沒有此路徑,將會找不到服務,從而返回404。
這裏使用示例定義一個Ingress 用於訪問集羣中的 tomcat和apache,當前的服務以下:
# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE php-apache ClusterIP 10.1.175.166 <none> 80/TCP 4d tomcat-service ClusterIP 10.1.228.9 <none> 8080/TCP 1h # kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE php-apache-56b5765b95-rhvg2 1/1 Running 2 5d 10.2.15.6 10.0.0.2 tomcat-deployment-65799d5fc4-5dx4h 1/1 Running 0 3h 10.2.22.7 10.0.0.3
建立一個Ingress:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test spec: rules: - host: tomcat.com http: paths: - backend: serviceName: tomcat-service servicePort: 8080 - host: apache.com http: paths: - path: /demo/index.html backend: serviceName: php-apache servicePort: 80
建立成功後,會顯示以下信息:
# kubectl get ing -o wide NAME HOSTS ADDRESS PORTS AGE test tomcat.com,apache.com 80 2h
在不使用kubernetes支持的公有云平臺下,默認的 ADDRESS地址會顯示爲空,若是須要顯示節點的IP,能夠添加
--report-node-internal-ip-address=true
的參數,或者修改flags.go z中的源碼,對應的useNodeInternalIP = flags.Bool("report-node-internal-ip-address", false,
中 false 修改成 true。
在本主機host 文件中配置上域名和映射20080端口的主機節點,經過不一樣的域名訪問此節點:
[root@node-1 ingress]# kubectl get nodes NAME STATUS ROLES AGE VERSION 10.0.0.2 Ready <none> 25d v1.10.4 10.0.0.3 Ready <none> 24d v1.10.4 [root@node-1 ingress]# echo "10.0.0.2 tomcat.com apache.com" >> /etc/hosts [root@node-1 ~]# curl apache.com:20080 OK! [root@node-1 ~]# curl tomcat.com:20080 tomcat demo! [root@node-1 ~]# curl 10.0.0.2:20080 default backend - 404 # 當找不到對應的後端Service時,會去訪問指定的默認backend。
查看代理信息:
[root@node-1 ~]# curl --head tomcat.com:20080 HTTP/1.1 200 Server: nginx/1.13.12 Date: Tue, 03 Jul 2018 11:47:31 GMT Content-Type: text/html;charset=UTF-8 Connection: keep-alive Vary: Accept-Encoding [root@node-1 ~]# curl --head apache.com:20080 HTTP/1.1 200 OK Server: nginx/1.13.12 Date: Tue, 03 Jul 2018 11:47:41 GMT Content-Type: text/html; charset=UTF-8 Connection: keep-alive Vary: Accept-Encoding X-Powered-By: PHP/5.6.14
登陸Ingress的Pod,咱們能夠看到已經添加了相應的轉發規則:
# kubectl exec -it nginx-ingress-controller-6c9fcdf8d9-fvn57 -n ingress-nginx -- sh $ cat nginx.conf ... upstream default-php-apache-80 { least_conn; keepalive 32; server 10.2.15.6:80 max_fails=0 fail_timeout=0; } upstream default-tomcat-service-8080 { least_conn; keepalive 32; server 10.2.22.7:8080 max_fails=0 fail_timeout=0; } ... server { server_name apache.com ; listen 80; listen [::]:80; set $proxy_upstream_name "-"; ... ## start server tomcat.com server { server_name tomcat.com ; listen 80; listen [::]:80; set $proxy_upstream_name "-";
對於使用https設置TLS的安全證書方面,Ingress 也能夠支持。
經過如下步驟進行設置:
目前,Ingress僅支持單個TLS端口443,並假定TLS termination。 若是Ingress中的TLS配置部分指定了不一樣的主機,則它們將根據經過SNI TLS擴展指定的主機名(假如Ingress controller支持SNI)在多個相同端口上進行復用。TLS secret中必須包含名爲tls.crt和tls.key的密鑰,這裏麪包含了用於TLS的證書和私鑰,其格式以下:
apiVersion: v1 data: tls.crt: base64 encoded cert # cert 文件內容 tls.key: base64 encoded key # key 文件內容 kind: Secret metadata: name: testsecret namespace: default type: Opaque
在Ingress中引用 Secret:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: no-rules-map spec: tls: - secretName: testsecret backend: serviceName: s1 servicePort: 80
首先生成證書:
# openssl req -x509 -nodes -days 5000 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=myweb.com"
利用生成的key 和 crt文件建立Secret:
# kubectl create secret tls myweb-ingress-secret --key tls.key --cert tls.crt secret "myweb-ingress-secret" created # kubectl get secret NAME TYPE DATA AGE default-token-hmvnc kubernetes.io/service-account-token 3 26d myweb-ingress-secret kubernetes.io/tls 2 13s
咱們也可使用YAML文件的方式建立,其格式和下面的相似:
# kubectl get secret myweb-ingress-secret -o yaml apiVersion: v1 data: tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMr ... Gh0UlJJUkV5bUJpZjZNdmRlOERmUVRiT0x5OUF5Y0xVb2gyL2RlRnhOST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2d0lCQURBTkJna3Foa2lHOXcwQ ... MzekE9PQotLS0tLUVORCBQUklWQVRFIEtFWS0tLS0tCg== kind: Secret metadata: name: myweb-ingress-secret namespace: default type: kubernetes.io/tls
TLS的證書建立到Secret以後,就能夠利用他來建立Ingress對象了,使用以下文件:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myweb-ingress spec: tls: - secretName: myweb-ingress-secret rules: - host: myweb.com http: paths: - path: /demo backend: serviceName: php-apache servicePort: 80
使用默認的443端口訪問(本地使用的NodePort,映射到Ingress的443端口):
# curl -k https://myweb.com:20443/demo/index.html demo test https page!
使用網頁能夠訪問網頁內容: https://myweb.com:20443/demo