Kebernetes 學習總結(5) Ingress

時間 2019-11-06

標籤 kebernetes 學習總結 ingress 简体版

原文原文鏈接

雖然kubernetes集羣內部署的pod、server都有本身的IP，可是卻沒法提供外網訪問，之前能夠經過監聽NodePort的方式暴露服務，可是這種方式並不靈活，生產環境也不建議使用。Ingresss是kubernetes集羣中的一個API資源對象，扮演邊緣路由器(edge router)的角色，也能夠理解爲集羣防火牆、集羣網關，能夠自定義路由規則來轉發、管理、暴露服務(一組pod)，很是靈活，生產環境建議使用這種方式。另外LoadBlancer也能夠暴露服務，不過這種方式須要向雲平臺申請負債均衡器；雖然目前不少雲平臺都支持，可是這種方式深度耦合了雲平臺。
Ingress相關組件：
1) Ingress controller
kubernetes中的controller有不少，好比CronJob、DeamonSet、Deployment、ReplicationSet、StatefulSet等，它們的做用就是監控集羣的變化，使集羣始終保持指望的最終狀態(yml文件)。同理Ingress controller的做用就是實時感知Ingress路由規則集合的變化，再與Api Server交互，獲取Service、Pod在集羣中的IP等信息，而後發送給反向代理web服務器(ingress pods)，刷新其路由配置信息，這就是它的服務發現機制。
2) 反向代理web服務器(ingress pods) 負責轉發外部請求到後端service，好比Nginx、Apache、traefik等等
3) Ingress 定義路由規則集合
4) NodePort類型的service 負責將外部流量引入kubernetes cluster，而後由反向代理web服務器處理。node

本篇總結中的示例大體以下：首先部署ingress controller及 ingress pods；而後利用NodePort類型的service將外部流最引入kubernetes cluster中；而後再定義後端deployment及後端service；而後定義ingress 轉發規則集合；最後訪問測試。
1) 部署ingress podsnginx

[root@docker79 ingress]# mkdir example
[root@docker79 ingress]# cd example/
[root@docker79 example]#  for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml default-backend.yaml udp-services-configmap.yaml ; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file ; done
[root@docker79 example]# kubectl apply -f .
configmap/nginx-configuration created
deployment.extensions/default-http-backend created
service/default-http-backend created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace/ingress-nginx configured
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
configmap/tcp-services created
configmap/udp-services created
deployment.extensions/nginx-ingress-controller created
[root@docker79 example]#
[root@docker79 example]# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-6586bc58b6-8rxtp       1/1       Running   0          59m
nginx-ingress-controller-6bd7c597cb-xdrcm   1/1       Running   0          59m
[root@docker79 example]#

注：較新版本已經再也不支持上述文件，新版本(2019/07)已經把衆多文件合併爲一個mandatory.yaml文件，操做以下所示：git

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
kubectl apply -f mandatory.yaml

2) 在ingress-nginx 名稱空間定義nodePort類型的service 引入外部流量，過程以下所示：github

[root@docker79 ingress]# cat nginx-frontend-ingress-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-frontend-ingress-svc
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30080
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
    nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f nginx-frontend-ingress-svc.yaml
service/nginx-frontend-ingress-svc created
[root@docker79 ingress]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.102.71.36   <none>        80/TCP                       1h
nginx-frontend-ingress-svc     NodePort    10.103.52.44   <none>        80:30080/TCP,443:30443/TCP   11s
[root@docker79 ingress]#

3) 下例中部署兩個後端nginx-backend-deploy、tomcat-backend-deploy。
首先部署nginx-backend-deploy及nginx-backend-svc，過程以下：web

[root@docker79 ingress]# vim backend-nginx.yaml
[root@docker79 ingress]# cat backend-nginx.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-backend-svc
  namespace: default
spec:
  selector:
    app: nginx-backend
    release: canary
  ports:
  - name: http
    targetPort: 80
    port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-backend-deploy
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-backend
      release: canary
  template:
    metadata:
      labels:
        app: nginx-backend
        release: canary
    spec:
      containers:
      - name: nginx-container
        image: nginx:1.15-alpine
        ports:
        - name: http
          containerPort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f backend-nginx.yaml
service/nginx-backend-svc created
deployment.apps/nginx-backend-deploy created
[root@docker79 ingress]# kubectl get pods
NAME                                   READY     STATUS    RESTARTS   AGE
nginx-backend-deploy-9c9b85bf4-6587j   1/1       Running   0          8s
nginx-backend-deploy-9c9b85bf4-k64tv   1/1       Running   0          8s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl get svc
NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes          ClusterIP   10.96.0.1        <none>        443/TCP   3d
nginx-backend-svc   ClusterIP   10.110.150.187   <none>        80/TCP    1m
[root@docker79 ingress]#

而後部署tomcat-backend-deploy及tomcat-backend-svc，過程以下：docker

[root@docker79 ingress]# vim backend-tomcat.yaml
[root@docker79 ingress]# cat backend-tomcat.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat-backend-svc
  namespace: default
spec:
  selector:
    app: tomcat-backend
    release: canary
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: ajp
    port: 8009
    targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat-backend
      release: canary
  template:
    metadata:
      labels:
        app: tomcat-backend
        release: canary
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5.32-jre8-alpine
        ports:
        - name: http
          containerPort: 8080
        - name: ajp
          containerPort: 8009
[root@docker79 ingress]# kubectl apply -f backend-tomcat.yaml
service/tomcat-backend-svc created
deployment.apps/tomcat-deploy created
[root@docker79 ingress]# kubectl get pods
NAME                                   READY     STATUS    RESTARTS   AGE
nginx-backend-deploy-9c9b85bf4-6587j   1/1       Running   0          24m
nginx-backend-deploy-9c9b85bf4-k64tv   1/1       Running   0          24m
tomcat-deploy-7b4c97ff9f-fv8rx         1/1       Running   0          2m
tomcat-deploy-7b4c97ff9f-rj94r         1/1       Running   0          2m
[root@docker79 ingress]# kubectl get svc
NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
kubernetes           ClusterIP   10.96.0.1        <none>        443/TCP             3d
nginx-backend-svc    ClusterIP   10.110.150.187   <none>        80/TCP              24m
tomcat-backend-svc   ClusterIP   10.105.189.191   <none>        8080/TCP,8009/TCP   2m
[root@docker79 ingress]#

4) 部署 ingress 轉發規則集合
首先部署nginx-backend-svc的ingress 轉發規則集合，過程以下所示:shell

[root@docker79 ingress]# vim ingress-rules-ngx.yaml
[root@docker79 ingress]# cat ingress-rules-ngx.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-ngx
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: nginx.inspiry.com
    http:
      paths:
      - path:
        backend:
          serviceName: nginx-backend-svc
          servicePort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f ingress-rules.yaml
ingress.extensions/ingress-rule-ngx created
[root@docker79 ingress]# kubectl get ingress
NAME               HOSTS               ADDRESS   PORTS     AGE
ingress-rule-ngx   nginx.inspiry.com             80        9s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress  ingress-rule-ngx
Name:             ingress-rule-ngx
Namespace:        default
Address:
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  nginx.inspiry.com
                        nginx-backend-svc:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-ngx","namespace":"default"},"spec":{"rules":[{"host":"nginx.inspiry.com","http":{"paths":[{"backend":{"serviceName":"nginx-backend-svc","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
[root@docker79 ingress]#

而後再部署tomcat-backend-svc的ingress 轉發規則集合，過程以下所示：json

[root@docker79 ingress]# vim ingress-rules-tomcat.yaml
[root@docker79 ingress]# cat ingress-rules-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-tomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.inspiry.cn
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat-backend-svc
          servicePort: 8080
[root@docker79 ingress]# kubectl apply -f ingress-rules-tomcat.yaml
ingress.extensions/ingress-rule-tomcat created
[root@docker79 ingress]# kubectl get ingress
NAME                  HOSTS               ADDRESS   PORTS     AGE
ingress-rule-ngx      nginx.inspiry.com             80        11m
ingress-rule-tomcat   tomcat.inspiry.cn             80        6s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress ingress-rule-tomcat
Name:             ingress-rule-tomcat
Namespace:        default
Address:
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  tomcat.inspiry.cn
                        tomcat-backend-svc:8080 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-tomcat","namespace":"default"},"spec":{"rules":[{"host":"tomcat.inspiry.cn","http":{"paths":[{"backend":{"serviceName":"tomcat-backend-svc","servicePort":8080},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
[root@docker79 ingress]#

5) 測試
在kubernetes cluster 外的任何一臺主機上使用 curl http://nginx.inspiry.com:30080/ 或 curl http://tomcat.inspiry.cn:30080/ 便可訪問相應nginx-backend 、tomcat-backend 的pods。
生產環境中還能夠在k8s cluster外再建立一層 LB，將接收到的 nginx.inspiry.com、tomcat.inspiry.cn 域名分別轉發到192.168.20.7九、192.168.20.7八、192.168.20.77爲後端的upstream上。vim

觀察ingress controller的內容以下：後端

[root@docker79 ~]# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-6586bc58b6-8rxtp       1/1       Running   0          1h
nginx-ingress-controller-6bd7c597cb-xdrcm   1/1       Running   0          1h
[root@docker79 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-xdrcm -- /bin/sh
$ ls
fastcgi.conf        geoip    mime.types      nginx.conf         scgi_params      uwsgi_params.default
fastcgi.conf.default    koi-utf  mime.types.default  nginx.conf.default     scgi_params.default  win-utf
fastcgi_params      koi-win  modsecurity         opentracing.json       template
fastcgi_params.default  lua  modules         owasp-modsecurity-crs  uwsgi_params
$ cat nginx.conf
...... (可仔細觀察該文件的內容，發現相應的轉發規則是隨着 ingress rules的變化而變化的)

補充: ingress 的https流量

# kubectl create secret tls domain-name-crt --cert=server.pem --key=server.key
# cat ingress-rule-appapi-svc.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-appapi
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - appapi-devel.domain.cn
    secretName: domain-name-crt
  rules:
  - host: appapi-devel.domain.cn
    http:
      paths:
      - path:
        backend:
          serviceName: appapi-svc
          servicePort: 21900
# kubectl apply  -f   ingress-rule-appapi-svc.yaml

<完>

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。