微服務監控神器Prometheus的安裝部署

本文涉及：如何在k8s下搭建Prometheus+grafana的監控環境

基本概念

Prometheus提供了容器和雲原生領域數據蒐集、存儲、處理、可視化和告警一套完整的解決方案，最初時是由SoundCloud公司開發的。自2012年開源以來社區成員就不斷遞增。現在的Prometheus已經發展到繼Kubernetes後第2個正式加入CNCF基金會的項目

Prometheus的特色？

• 多維的數據模型（基於時間序列的k/v鍵值對）。
• 靈活的查詢及聚合語句（PromQL）。
• 不依賴分佈式存儲，節點自治。
• 基於HTTP的pull模式採集時間序列數據。
• 能夠使用pushgateway（prometheus的可選中間件）實現push模式。
• 能夠使用動態服務發現或靜態配置採集的目標機器。
• 支持多種圖形及儀表盤。

Prometheus能夠監控什麼？

• k8s、docker、mysql、redis、es、consul、rabbitmq、zabbix等等

Prometheus架構圖

Prometheus安裝部署

Helm 安裝

Helm 是一個命令行下的客戶端工具。主要用於 Kubernetes 應用程序 Chart 的建立、打包、發佈以及建立和管理本地和遠程的 Chart 倉庫。

1 2 3 4 5 6

[root@syj ~]# wget https://storage.googleapis.com/kubernetes-helm/helm-v2.13.1-rc.2-linux-amd64.tar.gz [root@syj ~]# tar -zxvf helm-v2.14.0-rc.2-linux-amd64.tar.gz [root@syj ~]# cp linux-amd64/helm /usr/local/bin/ [root@syj ~]# helm version Client: &version.Version{SemVer:"v2.13.1-rc.2", GitCommit:"05811b84a3f93603dd6c2fcfe57944dfa7ab7fd0", GitTreeState:"clean"} Error: could not find tiller

Tiller 服務器安裝

Tiller 是 Helm 的服務端，部署在 Kubernetes 集羣中。Tiller 用於接收 Helm 的請求，並根據 Chart 生成 Kubernetes 的部署文件（ Helm 稱爲 Release ），而後提交給 Kubernetes 建立應用。Tiller 還提供了 Release 的升級、刪除、回滾等一系列功能。

建立rbac-config.yaml

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

apiVersion: v1 kind: ServiceAccount metadata: name: tiller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tiller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: tiller namespace: kube-system

啓動

1 2 3

[root@syj ~]# kubectl apply -f rbac-config.yaml serviceaccount/tiller created clusterrolebinding.rbac.authorization.k8s.io/tiller created

使用阿里雲鏡像進行安裝

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

[root@syj ~]# helm init --service-account tiller --upgrade -i registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.13.1 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts Creating /root/.helm Creating /root/.helm/repository Creating /root/.helm/repository/cache Creating /root/.helm/repository/local Creating /root/.helm/plugins Creating /root/.helm/starters Creating /root/.helm/cache/archive Creating /root/.helm/repository/repositories.yaml Adding stable repo with URL: https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts Adding local repo with URL: http://127.0.0.1:8879/charts $HELM_HOME has been configured at /root/.helm. Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster. Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy. To prevent this, run `helm init` with the --tiller-tls-verify flag. For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation Happy Helming!

查看結果

1 2 3 4 5 6 7

[root@syj ~]# helm version Client: &version.Version{SemVer:"v2.13.1", GitCommit:"05811b84a3f93603dd6c2fcfe57944dfa7ab7fd0", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.13.1", GitCommit:"05811b84a3f93603dd6c2fcfe57944dfa7ab7fd0", GitTreeState:"clean"} [root@syj ~]# helm repo list NAME URL stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts local http://127.0.0.1:8879/charts

部署 Prometheus Operator

建立命名空間

1	[root@syj ~]# kubectl create namespace monitoring

下載Prometheus Operator

1	[root@syj ~]# wget https://github.com/coreos/prometheus-operator/archive/release-0.29.zip

將下載下來的依賴包解壓並重命名爲prometheus-operator並cd到此目錄
安裝prometheus相關內容

1 2 3

helm install --name prometheus-operator --set rbacEnable=true --namespace=monitoring helm/prometheus-operator helm install --name prometheus --set serviceMonitorsSelector.app=prometheus --set ruleSelector.app=prometheus --namespace=monitoring helm/prometheus helm install --name alertmanager --namespace=monitoring helm/alertmanager

驗證

1 2 3 4 5 6 7 8 9 10 11

[root@syj ~]# kubectl get pod -n monitoring NAME READY STATUS RESTARTS AGE alertmanager-alertmanager-0 2/2 Running 0 58s prometheus-operator-545b59ffc9-6g7dg 1/1 Running 0 6m32s prometheus-prometheus-0 3/3 Running 1 3m31s [root@syj ~]# kubectl get svc -n monitoring NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE alertmanager ClusterIP 10.98.237.7 <none> 9093/TCP 87s alertmanager-operated ClusterIP None <none> 9093/TCP,6783/TCP 87s prometheus ClusterIP 10.104.185.104 <none> 9090/TCP 4m prometheus-operated ClusterIP None <none> 9090/TCP 4m

安裝 kube-prometheus

1 2 3 4

[root@syj ~]# mkdir -p helm/kube-prometheus/charts [root@syj ~]# helm package -d helm/kube-prometheus/charts helm/alertmanager helm/grafana helm/prometheus helm/exporter-kube-dns \ > helm/exporter-kube-scheduler helm/exporter-kubelets helm/exporter-node helm/exporter-kube-controller-manager \ > helm/exporter-kube-etcd helm/exporter-kube-state helm/exporter-coredns helm/exporter-kubernetes

驗證

1 2 3 4 5 6 7 8 9 10 11

[root@syj ~]# kubectl get svc -n monitoring NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE alertmanager ClusterIP 10.98.237.7 <none> 9093/TCP 34m alertmanager-operated ClusterIP None <none> 9093/TCP,6783/TCP 34m kube-prometheus ClusterIP 10.101.249.82 <none> 9090/TCP 29s kube-prometheus-alertmanager ClusterIP 10.100.29.63 <none> 9093/TCP 29s kube-prometheus-exporter-kube-state ClusterIP 10.98.91.146 <none> 80/TCP 29s kube-prometheus-exporter-node ClusterIP 10.98.34.11 <none> 9100/TCP 29s kube-prometheus-grafana ClusterIP 10.108.208.247 <none> 80/TCP 29s prometheus ClusterIP 10.104.185.104 <none> 9090/TCP 36m prometheus-operated ClusterIP None <none> 9090/TCP 36m

將grafana的Service類型改成NodePort

1	kubectl patch svc kube-prometheus-grafana -p '{"spec":{"type":"NodePort"}}' -n monitoring

此時訪問grafana的默認端口31106便可：

1	http://ip:31106

安裝過程參考文章：http://www.javashuo.com/article/p-dqoxwcra-eb.html

grafana的各類模板可參考
https://grafana.com/dashboards