service mesh istio微服務實驗之監控日誌與可視化

時間  2019-12-02
標籤 service mesh istio 微服 實驗 監控 日誌 可視化 简体版
原文   原文鏈接

簡介

本文是istio微服務實驗的後續文章，實驗前請先參考以前文章。html

分佈式調用鏈追蹤

安裝

# 下載yml文件
mkdir jaeger && cd jaeger
wget https://raw.githubusercontent.com/jaegertracing/jaeger-kubernetes/master/all-in-one/jaeger-all-in-one-template.yml

# 實驗環境不支持 LoadBalancer
# 能夠修改jaeger-all-in-one-template.yml使用nodeport
# 也能夠不修改，這樣的會使用隨機的nodeport

# 啓動
kubectl apply -n istio-system -f jaeger-all-in-one-template.yml

# 查看
kubectl get pods -n istio-system
kubectl get svc -n istio-system

# 屢次訪問以前的vue react界面並點擊發射按鈕

# 訪問
jaegerNodePort=$(kubectl get svc -n istio-system | grep jaeger-query | awk '{print $5}' | cut -d '/' -f 1 | cut -d ':' -f 2)
nodeName=$(kubectl get no | grep '<none>' | head -1 | awk '{print $1}')
nodeIP=$(ping -c 1 $nodeName | grep PING | awk '{print $3}' | tr -d '()')
echo "http://$nodeIP:"$jaegerNodePort

# 選擇 istio-ingress 能夠方便查看整個調用鏈

# 清理
cd jaeger
kubectl delete -n istio-system -f jaeger-all-in-one-template.yml
複製代碼

jaeger的dashboard界面vue

調用鏈node

服務樹展現python

日誌與指標收集

安裝

# 安裝prometheus
cd /usr/local/istio

# 修改支持nodeport
cp install/kubernetes/addons/prometheus.yaml install/kubernetes/addons/prometheus.yaml.ori
vim install/kubernetes/addons/prometheus.yaml
...
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: 'true'
  labels:
    name: prometheus
  name: prometheus
  namespace: istio-system
spec:
  selector:
    app: prometheus
  ports:
  - name: prometheus
    protocol: TCP
    port: 9090
  # 設置使用 nodeport
  type: NodePort
...

# 部署
kubectl apply -f install/kubernetes/addons/prometheus.yaml

# 配置收集
istioctl create -f istio/new_telemetry.yml

# 屢次訪問以前的vue react界面並點擊發射按鈕

# 訪問web測試
prometheusNodePort=$(kubectl get svc -n istio-system | grep prometheus | awk '{print $5}' | cut -d '/' -f 1 | cut -d ':' -f 2)
nodeName=$(kubectl get no | grep '<none>' | head -1 | awk '{print $1}')
nodeIP=$(ping -c 1 $nodeName | grep PING | awk '{print $3}' | tr -d '()')
echo "http://$nodeIP:"$prometheusNodePort

# 使用 istio_double_request_count 關鍵字查詢

# 查看日誌
kubectl -n istio-system logs $(kubectl -n istio-system get pods -l istio=mixer -o jsonpath='{.items[0].metadata.name}') mixer | grep \"instance\":\"newlog.logentry.istio-system\" # 清理 kubectl delete -f install/kubernetes/addons/prometheus.yaml istioctl delete -f istio/new_telemetry.yml 複製代碼

收集TCP服務的指標

安裝

# 安裝prometheus
cd /usr/local/istio

# 修改支持nodeport
cp install/kubernetes/addons/prometheus.yaml install/kubernetes/addons/prometheus.yaml.ori
vim install/kubernetes/addons/prometheus.yaml
...
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: 'true'
  labels:
    name: prometheus
  name: prometheus
  namespace: istio-system
spec:
  selector:
    app: prometheus
  ports:
  - name: prometheus
    protocol: TCP
    port: 9090
  # 設置使用 nodeport
  type: NodePort
...

# 部署
kubectl apply -f install/kubernetes/addons/prometheus.yaml

# 配置收集
istioctl create -f istio/tcp_telemetry.yml

# 部署使用mongodb應用測試

# 訪問web測試
prometheusNodePort=$(kubectl get svc -n istio-system | grep prometheus | awk '{print $5}' | cut -d '/' -f 1 | cut -d ':' -f 2)
nodeName=$(kubectl get no | grep '<none>' | head -1 | awk '{print $1}')
nodeIP=$(ping -c 1 $nodeName | grep PING | awk '{print $3}' | tr -d '()')
echo "http://$nodeIP:"$prometheusNodePort

# 使用 istio_mongo_received_bytes 關鍵字查詢

# 清理
kubectl delete -f install/kubernetes/addons/prometheus.yaml
istioctl delete -f istio/tcp_telemetry.yml
複製代碼

TCP數據流圖react

使用grafana可視化指標

安裝

# 安裝prometheus
cd /usr/local/istio

# 修改支持nodeport
cp install/kubernetes/addons/prometheus.yaml install/kubernetes/addons/prometheus.yaml.ori
vim install/kubernetes/addons/prometheus.yaml
...
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: 'true'
  labels:
    name: prometheus
  name: prometheus
  namespace: istio-system
spec:
  selector:
    app: prometheus
  ports:
  - name: prometheus
    protocol: TCP
    port: 9090
  # 設置使用 nodeport
  type: NodePort
...
cp install/kubernetes/addons/grafana.yaml install/kubernetes/addons/grafana.yaml.ori
vim install/kubernetes/addons/grafana.yaml
kind: Service
metadata:
  name: grafana
  namespace: istio-system
spec:
  # 設置使用 nodeport
  type: NodePort
  ports:
  - port: 3000
    protocol: TCP
    name: http
  selector:
    app: grafana

# 部署
kubectl apply -f install/kubernetes/addons/prometheus.yaml
kubectl apply -f install/kubernetes/addons/grafana.yaml

# 訪問web測試
grafanaNodePort=$(kubectl get svc -n istio-system | grep grafana | awk '{print $5}' | cut -d '/' -f 1 | cut -d ':' -f 2)
nodeName=$(kubectl get no | grep '<none>' | head -1 | awk '{print $1}')
nodeIP=$(ping -c 1 $nodeName | grep PING | awk '{print $3}' | tr -d '()')
echo "http://$nodeIP:"$grafanaNodePort

# 壓力測試查看圖表
# 建立測試用的fortio
kubectl apply -f <(istioctl kube-inject -f istio/fortio-deploy.yaml)

# 正常訪問測試
FORTIO_POD=$(kubectl get pod | grep fortio | awk '{ print $1 }')
kubectl exec -it $FORTIO_POD  -c fortio /usr/local/bin/fortio -- load -curl http://service-python/env

# 加大壓力測試
kubectl exec -it $FORTIO_POD  -c fortio /usr/local/bin/fortio -- load -qps 20 -t 100s -loglevel Warning http://service-python/env
kubectl exec -it $FORTIO_POD  -c fortio /usr/local/bin/fortio -- load -qps 50 -t 100s -loglevel Warning http://service-go/env

# 清理
kubectl delete -f install/kubernetes/addons/prometheus.yaml
kubectl delete -f install/kubernetes/addons/grafana.yaml
kubectl delete -f istio/fortio-deploy.yaml
複製代碼

service mesh 數據監控展現git

pilot數據監控展現github

生成服務樹

安裝

# 修改使用nodeport
cd /usr/local/istio
cp install/kubernetes/addons/servicegraph.yaml install/kubernetes/addons/servicegraph.yaml.ori
vim install/kubernetes/addons/servicegraph.yaml
...
apiVersion: v1
kind: Service
metadata:
  name: servicegraph
  namespace: istio-system
spec:
  # 設置使用 nodeport
  type: NodePort
  ports:
  - name: http
    port: 8088
  selector:
    app: servicegraph
...

# 安裝prometheus
cd /usr/local/istio

# 修改支持nodeport
cp install/kubernetes/addons/prometheus.yaml install/kubernetes/addons/prometheus.yaml.ori
vim install/kubernetes/addons/prometheus.yaml
...
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: 'true'
  labels:
    name: prometheus
  name: prometheus
  namespace: istio-system
spec:
  selector:
    app: prometheus
  ports:
  - name: prometheus
    protocol: TCP
    port: 9090
  # 設置使用 nodeport
  type: NodePort
...

# 部署
kubectl apply -f install/kubernetes/addons/prometheus.yaml
kubectl apply -f install/kubernetes/addons/servicegraph.yaml

# 屢次訪問以前的vue react界面並點擊發射按鈕

# 訪問web測試
servicegraphNodePort=$(kubectl get svc -n istio-system | grep servicegraph | awk '{print $5}' | cut -d '/' -f 1 | cut -d ':' -f 2)
nodeName=$(kubectl get no | grep '<none>' | head -1 | awk '{print $1}')
nodeIP=$(ping -c 1 $nodeName | grep PING | awk '{print $3}' | tr -d '()')
echo "http://$nodeIP:"$servicegraphNodePort/force/forcegraph.html

# 可以使用url
# /force/forcegraph.html
# /dotviz
# /dotgraph
# /d3graph
# /graph

# 清理
kubectl delete -f install/kubernetes/addons/prometheus.yaml
kubectl delete -f install/kubernetes/addons/servicegraph.yaml
複製代碼

服務樹web

使用Fluentd收集日誌

安裝

# 安裝efk
kubectl apply -f istio/logging-stack.yml

# 配置istio使用efk
istioctl create -f istio/fluentd-istio.yml

# 屢次訪問以前的vue react界面並點擊發射按鈕

# 訪問web測試
kibanaNodePort=$(kubectl get svc -n istio-system | grep kibana | awk '{print $5}' | cut -d '/' -f 1 | cut -d ':' -f 2)
nodeName=$(kubectl get no | grep '<none>' | head -1 | awk '{print $1}')
nodeIP=$(ping -c 1 $nodeName | grep PING | awk '{print $3}' | tr -d '()')
echo "http://$nodeIP:"$kibanaNodePort

# 清理
kubectl delete -f istio/logging-stack.yml
istio delete -f istio/fluentd-istio.yml
複製代碼
相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程