kubernetes + istio進行流量管理

時間 2019-11-08

標籤 kubernetes istio 進行流量管理简体版

原文原文鏈接

實驗目的：

本文介紹如何經過istio實現域名訪問k8s部署的nginx服務html

前提：

　　已經安裝了kubernetes的服務器node

瞭解 kubernetes 基本命令如何使用 (kubectl create/delete/get/apply 等基本命令)nginx

注意文章紅色加粗字體git

能上網(^_^)web

tip: kubernetes安裝參考：centos7 使用kubeadm 快速部署 kubernetes 國內源 docker

實驗環境：

[root@k8s-master ~]# uname -a
Linux k8s-master 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@k8s-master ~]# kubectl get node,pod,svc -o wide
NAME              STATUS     ROLES    AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
node/k8s-master   Ready      master   2d17h   v1.14.0   10.211.55.6   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.6.1
node/k8s-node     NotReady   <none>   2d14h   v1.14.0   10.211.55.7   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.6.1

NAME                      READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod/qf-test-nginx-45k8x   1/1     Running   0          15h   10.244.0.21   k8s-master   <none>           <none>
pod/qf-test-nginx-k97vc   1/1     Running   0          15h   10.244.10.4   k8s-node     <none>           <none>

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE     SELECTOR
service/kubernetes      ClusterIP   10.96.0.1      <none>        443/TCP        2d17h   <none>
service/qf-test-nginx   NodePort    10.98.49.158   <none>        80:31412/TCP   15h     app=nginx

安裝istio

#下載istio
curl -L https://git.io/getLatestIstio | ISTIO_VERSION=1.1.1 sh -
#進入到istio目錄
cd istio-1.1.1

#修改 ~/.bash_profile 添加下面內容 導入istio
#若是用的是 zsh  修改 ~/.zhsrc

export PATH="$PATH:/root/k8s/istio-1.1.1/bin"

在kubernetes使用istio : istio須要的docker鏡像都須要在docker.io上拉取，因此可能會有一些慢，稍做等待 or 睡一覺以後再看centos

for i in install/kubernetes/helm/istio-init/files/crd*yaml; do kubectl apply -f $i; done

#咱們這裏使用的是寬容模式，要使用嚴禁模式看文章最後的參考文檔
kubectl apply -f install/kubernetes/istio-demo.yaml

#確認下列 Kubernetes 服務已經部署並都具備各自的 CLUSTER-IP
kubectl get svc -n istio-system

#確認必要的 Kubernetes Pod 都已經建立而且其 STATUS 的值是 Running
kubectl get pods -n istio-system

部署完成後咱們看一下 istio 相關Pod 狀態api

[root@k8s-master ~]# kubectl get pods -n istio-system
NAME                                      READY   STATUS      RESTARTS   AGE
grafana-7b9f5d484f-mf28j                  1/1     Running     0          11h
istio-citadel-848f4c8489-s4bm9            1/1     Running     0          11h
istio-cleanup-secrets-1.1.1-4zd5w         0/1     Completed   0          12h
istio-egressgateway-7469db8c68-jlr9b      1/1     Running     0          12h
istio-galley-86bcf86779-858jv             1/1     Running     0          12h
istio-grafana-post-install-1.1.1-t7qqg    0/1     Completed   0          12h
istio-ingressgateway-56bbdd69bf-j7swp     1/1     Running     0          12h
istio-pilot-77b99c499-xxhfk               2/2     Running     1          12h
istio-policy-85f58d8775-wd8wm             2/2     Running     6          12h
istio-security-post-install-1.1.1-nfhb8   0/1     Completed   0          12h
istio-sidecar-injector-5464f674c4-rcvpk   1/1     Running     0          12h
istio-telemetry-9b844886f-h9rzd           2/2     Running     6          12h
istio-tracing-7f5d8c5d98-s72nv            1/1     Running     0          12h
kiali-589d55b4db-vljzq                    1/1     Running     0          12h
prometheus-878999949-qntkc                1/1     Running     0          12h

部署應用添加gateway 和 virtualservice

先來看一下kubernetes的部署文件瀏覽器

[root@k8s-master testnginx]# cat nginx-daemonset.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: qf-test-nginx
spec:
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: qingfenglian/test_nginx
        ports:
        - containerPort: 80


---
apiVersion: v1
kind: Service
metadata:
  name: qf-test-nginx
  namespace: default
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
  - port: 80
    targetPort: 80
    name: http

用上面的yaml文件部署成功後看一下 daemonset, pod,svc,node 信息， bash

tip:: 實驗環境中自己是有兩個node，有一個node是NotReady，緣由是k8s-node這臺機器屬於關機(^_^)狀態，後面我會啓動，不要捉急

[root@k8s-master ~]# kubectl get daemonset,pod,svc,node -o wide
NAME                                 DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE   CONTAINERS   IMAGES                    SELECTOR
daemonset.extensions/qf-test-nginx   1         1         1       1            1           <none>          16h   nginx        qingfenglian/test_nginx   app=nginx

NAME                      READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod/qf-test-nginx-45k8x   1/1     Running   0          16h   10.244.0.21   k8s-master   <none>           <none>
pod/qf-test-nginx-k97vc   1/1     Running   0          16h   10.244.10.4   k8s-node     <none>           <none>

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE     SELECTOR
service/kubernetes      ClusterIP   10.96.0.1      <none>        443/TCP        2d18h   <none>
service/qf-test-nginx   NodePort    10.98.49.158   <none>        80:31412/TCP   16h     app=nginx

NAME              STATUS     ROLES    AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
node/k8s-master   Ready      master   2d18h   v1.14.0   10.211.55.6   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.6.1
node/k8s-node     NotReady   <none>   2d15h   v1.14.0   10.211.55.7   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.6.1

建立gateway

#查看gateway內容
[root@k8s-master testnginx]# cat qingfeng-deve-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: qingfeng-deve
spec:
  selector:
    istio: ingressgateway # 使用 istio 默認的 ingress gateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"

#建立gateway
[root@k8s-master testnginx]# kubectl create -f <(istioctl kube-inject -f qingfeng-deve-gateway.yaml)
gateway.networking.istio.io/qingfeng-deve created

#查看結果
[root@k8s-master testnginx]# kubectl get gateway
NAME            AGE
qingfeng-deve   12s
[root@k8s-master testnginx]#

建立virtualservice

注意：：本文nginx服務命名空間是default ,若是是其餘命名空間的服務須要這樣寫格式 "serviceName.namespaceName.svc.cluster.local" 例 "qf-test-nginx.default.svc.cluster.local"

解釋：： serviceName.namespaceName.svc.cluster.local :: serviceName=k8s的service名稱； namespaceName=service服務所在的命名空間； .svc.cluster.local 這個是固定的不變

[root@k8s-master testnginx]# cat nginx-virutalservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vs-nginx
spec:
  hosts:
  - "nginx.local.com"
  gateways:
  - qingfeng-deve
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: qf-test-nginx
[root@k8s-master testnginx]# kubectl create -f <(istioctl kube-inject -f nginx-virutalservice.yaml)
virtualservice.networking.istio.io/vs-nginx created
[root@k8s-master testnginx]#

修改 istio-ingressgateway

kubectl -n istio-system edit deployment istio-ingressgateway

找到下面內容，並作修改

將 80 端口和 443 端口配置爲 hostPort 模式，

等待幾秒讓 istio-ingressgateway 從新調度

image: docker.io/istio/proxyv2:1.1.1
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 80
          hostPort: 80                ------------######## 這裏增長這一行
          protocol: TCP
        - containerPort: 443
          hostPort: 443　　　　　　　　  ------------######## 這裏增長一行
          protocol: TCP
        - containerPort: 31400
          protocol: TCP
        - containerPort: 15029
          protocol: TCP
        - containerPort: 15030
          protocol: TCP
        - containerPort: 15031
          protocol: TCP
        - containerPort: 15032
          protocol: TCP
        - containerPort: 15443
          protocol: TCP
        - containerPort: 15020
          protocol: TCP
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP

驗證結果

host綁定：：因爲個人域名是 nginx.local.com 沒有域名解析，因此須要在host裏面添加一條記錄

綁定host以後經過瀏覽器訪問 nginx.local.com 查看返回信息，，我是爲了偷懶因此用curl 請求

先 ping 一下看看 host綁定是否生效，，而後用curl 請求

~ » ping nginx.local.com                                                                                                                                          lianqingfeng@bogon
PING nginx.local.com (10.211.55.6): 56 data bytes
64 bytes from 10.211.55.6: icmp_seq=0 ttl=64 time=0.194 ms
64 bytes from 10.211.55.6: icmp_seq=1 ttl=64 time=0.160 ms
^C
--- nginx.local.com ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.160/0.177/0.194/0.017 ms
------------------------------------------------------------
~ » curl nginx.local.com                                                                                                                                          lianqingfeng@bogon
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
<p><em>qingfeng.lian</em></p>
</body>
</html>
------------------------------------------------------------
~ »

讓咱們來看一istio-ingressgateway的日誌, 發現 "10.244.0.21:80" ，回到文章最上面找找這個ip地址會發現這個是nginx的svc服務ip地址，

還有一點上雖然k8s-node節點上的pod仍然是運行狀態，可是注意看返回信息最後面有 "qingfeng.lian" 字樣，這說明服務並無打到 k8s-node節點，由於k8s-node節點上我並無加這樣的輸出

[root@k8s-master testnginx]# kubectl logs -f istio-ingressgateway-64fcc46bb-zx6tx -n istio-system --tail=3
[2019-04-05T03:03:45.837Z] "GET / HTTP/1.1" 304 - "-" 0 0 0 0 "10.211.55.2" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36" "a822ba28-cd5e-9272-a423-7bb051c0dac5" "nginx.local.com" "10.244.0.21:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:57799 -
[2019-04-05T03:03:54.526Z] "GET / HTTP/1.1" 304 - "-" 0 0 0 0 "10.211.55.2" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36" "10ab86c0-8716-977a-965a-007fe11129c0" "nginx.local.com" "10.244.0.21:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:57799 -
[2019-04-05T03:12:37.524Z] "GET / HTTP/1.1" 200 - "-" 0 642 2 0 "10.211.55.2" "curl/7.54.0" "2c51476c-79bf-9969-82a1-588b69e50fa6" "nginx.local.com" "10.244.0.21:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:57877 -

咱們再作一個實驗如今吧k8s-node節點啓動，(把node節點開機便可(^_^)),如今來看一下node節點狀態，發現k8s-node已經就緒，這個基本上是node節點開機成功就會變成Ready ，秒級

這裏爲了看全部信息因此我把 node,svc,pod全列出啦，也能夠只看node

[root@k8s-master testnginx]# kubectl get node,svc,pod -o wide
NAME              STATUS   ROLES    AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
node/k8s-master   Ready    master   2d19h   v1.14.0   10.211.55.6   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.6.1
node/k8s-node     Ready    <none>   2d16h   v1.14.0   10.211.55.7   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://18.6.1

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE     SELECTOR
service/kubernetes      ClusterIP   10.96.0.1      <none>        443/TCP        2d19h   <none>
service/qf-test-nginx   NodePort    10.98.49.158   <none>        80:31412/TCP   18h     app=nginx

NAME                      READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
pod/qf-test-nginx-45k8x   1/1     Running   0          18h   10.244.0.21   k8s-master   <none>           <none>
pod/qf-test-nginx-k97vc   1/1     Running   0          18h   10.244.10.4   k8s-node     <none>           <none>
[root@k8s-master testnginx]#

咱們在來發幾回curl請求驗證一下k8s-node上的pod是否生效， istio-ingressgateway日誌能夠看到請求已經開始打到不一樣的pod上面

[2019-04-10T01:31:52.547Z] "GET / HTTP/1.1" 200 - "-" 0 642 2 0 "10.211.55.2" "curl/7.54.0" "2c3f9423-d05d-94fa-a5cf-9476f16aeb0e" "nginx.local.com" "10.244.0.21:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:58525 -
[2019-04-10T01:31:54.426Z] "GET / HTTP/1.1" 200 - "-" 0 612 2 0 "10.211.55.2" "curl/7.54.0" "5428cb78-4456-910f-bba0-2a76188a3bf3" "nginx.local.com" "10.244.10.4:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:58526 -
[2019-04-10T01:31:55.772Z] "GET / HTTP/1.1" 200 - "-" 0 642 0 0 "10.211.55.2" "curl/7.54.0" "d9213d72-e508-951e-b2ea-1c1a106638b8" "nginx.local.com" "10.244.0.21:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:58527 -
[2019-04-10T01:31:58.441Z] "GET / HTTP/1.1" 200 - "-" 0 612 1 1 "10.211.55.2" "curl/7.54.0" "8f44729b-46c9-9b90-8903-48adbfcb547c" "nginx.local.com" "10.244.10.4:80" outbound|80||qf-test-nginx.default.svc.cluster.local - 10.244.0.39:80 10.211.55.2:58528 -