Prometheus node_exporter grafana 持續更新中
什麼是Prometheus
Prometheus是一個最初在SoundCloud上構建的開源系統監視和警報工具包 。自2012年成立以來，許多公司和組織都採用了Prometheus，該項目擁有一個很是活躍的開發人員和用戶社區。它如今是一個獨立的開源項目，能夠獨立於任何公司進行維護。爲了強調這一點，並澄清項目的治理結構，Prometheus 於2016年加入 雲計算本地計算基金會，做爲繼Kubernetes以後的第二個託管項目。html
特色
- 具備由度量名稱和鍵/值對標識的時間序列數據的多維數據模型
- PromQL，一種靈活的查詢語言， 能夠利用這一維度
- 不依賴分佈式存儲; 單個服務器節點是自治的
- 時間序列集合經過HTTP上的拉模型發生
- 推送時間序列經過中間網關支持
- 經過服務發現或靜態配置發現目標
- 多種圖形和儀表板支持模式
架構圖和效果圖
官方網站
安裝配置linux
實驗環境描述：git
筆記本內存不足，只有一臺虛擬機😂，目前所用的組件（node_exporter,prometheus,grafana）均安裝至此github
- 安裝Prometheus
| 官方網站web |
https://prometheus.io/docker |
| 下載地址數據庫 |
|
| 系統環境centos |
Centos 7_1810x64 |
- 運行方式
1，點槓直接運行（不推薦）
2，screen
3，daemonize
4，編寫成系統服務 （本文使用的）
5，nohub &
- 編寫系統服務
在/usr/lib/systemd/system目錄下新建prometheus.service文件。文件內的路徑根據實際修改
1 [root@node1 ~]# cat /usr/lib/systemd/system/prometheus.service 2 [Unit] 3 Description=https://prometheus.io/ 4 [Service] 5 Type=simple 6 ExecStartPre=/mnt/prometheus/promtool check config /mnt/prometheus/prometheus.yml 7 ExecStart=/mnt/prometheus/prometheus --config.file=/mnt/prometheus/prometheus.yml --storage.tsdb.path=/mnt/prometheus/data --web.enable-lifecycle 8 [Install] 9 WantedBy=multi-user.target
經常使用參數解釋
--version 不解釋 --config.file 指定配置文件路徑，絕對路徑 --web.listen-address 監聽地址端口 默認，9090忽略 --web.read-timeout=5m 請求鏈接的最大等待時間， 防止太多的空閒連接，佔用資源 --storage.tsdb.retention 採集監控數據後會存在內存中和硬盤中， 太長的話，硬盤和內存都吃不消，設置15天爲宜 --web.max-connections 最大連接數 --storage.tsdb.path 數據存儲路徑 --query.timeout=2m 查詢超時 --query.max-concurrency=200 最多同時查詢人數 --web.enable-lifecycle 容許經過http請求關閉服務和從新加載
相關命令
1 systemctl daemon-reload #從新加載配置文件 2 systemctl start prometheus #開啓服務 3 systemctl enable prometheus #開機自啓
截圖以下，
訪問webui，默認端口9090 #地址 http://IP:9090
可在此編輯使用PromQL語句進行調試。下圖所示查詢cpu使用率。Prometheus還提供了一個簡單視圖。後期使用grafana用做圖形展現，如今新版本還支持報警功能
安裝node_exporter
白話介紹，linux服務器信息收集工具，可知足平常需求。被動式，等待服務器抓取
官網https://prometheus.io/download/#node_exporter
裏面有各類exporter提供使用（windows，數據庫，各類組件的監控）可知足平常所需，在centos6和7測試無問題。安裝方式參考上面Prometheus。默認端口9100
如需本身編寫exporter監控程序抓取數據，可經過pushgateway來實現。官網有介紹
安裝完成以後，修改Prometheus配置文件（頻率，目標等），最後三行爲新增。靜態文本配置方式（不推薦），動態配置學習中，後續更新
# my global config global: scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. # scrape_timeout is set to the global default (10s). # Alertmanager configuration alerting: alertmanagers: - static_configs: - targets: # - alertmanager:9093 # Load rules once and periodically evaluate them according to the global 'evaluation_interval'. rule_files: # - "first_rules.yml" # - "second_rules.yml" # A scrape configuration containing exactly one endpoint to scrape: # Here it's Prometheus itself. scrape_configs: # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config. - job_name: 'prometheus' # metrics_path defaults to '/metrics' # scheme defaults to 'http'. static_configs: - targets: ['localhost:9090'] - job_name: 'node1' static_configs: - targets: ['127.0.0.1:9100']
以後重啓Prometheus
方式1
#systemctl daemon-reload
#systemctl restart prometheus
方式2
curl -X POST http://127.0.0.1:9090/-/reload
經過Prometheus可查看狀態及詳細信息
http://IP:9090
安裝grafana
Grafana是一個跨平臺的開源的度量分析和可視化工具，能夠經過將採集的數據查詢而後可視化的展現，並及時通知
官網下載地址 https://grafana.com/grafana/download
安裝方式
系統 Ubuntu & Debian wget https://dl.grafana.com/oss/release/grafana_6.3.3_amd64.deb dpkg -i grafana_6.3.3_amd64.deb 系統 Standalone Linux Binaries wget https://dl.grafana.com/oss/release/grafana-6.3.3.linux-amd64.tar.gz 系統 Redhat & Centos wget https://dl.grafana.com/oss/release/grafana-6.3.3-1.x86_64.rpm yum localinstall grafana-6.3.3-1.x86_64.rpm centos 經常使用命令 systemctl daemon-reload systemctl enable grafana-server.service systemctl start grafana-server.service
配置
rpm安裝方式配置文件在 /etc/grafana/grafana.ini 配置先忽略了
官網https://grafana.com/docs/installation/configuration/
訪問界面
訪問地址http://IP:3000 (默認端口3000)
首次登陸後修改密碼（默認帳戶，密碼 admin）。而後首頁以下
點增長數據源，選擇Prometheus
輸入相應信息測試並保存
點dashboards導入系統自帶面板，後續增長用戶和插件。。。。。略
查看剛導入的模板
點擊home選擇剛導入的dashboards,顯示以下，可自定義面板，及監控數據。圖形也可經過插件擴展。
上圖有點醜對吧，哈哈。沒事的，有大神作好了，very beautiful的圖。只需導入就好了，下面貼配置。共2500行我摺疊了
源碼及說明：https://github.com/starsliao/Prometheus
{ "__inputs": [ { "name": "DS_PROMETHEUS_111", "label": "prometheus", "description": "", "type": "datasource", "pluginId": "prometheus", "pluginName": "Prometheus" } ], "__requires": [ { "type": "grafana", "id": "grafana", "name": "Grafana", "version": "5.4.2" }, { "type": "panel", "id": "grafana-piechart-panel", "name": "Pie Chart", "version": "1.3.3" }, { "type": "panel", "id": "graph", "name": "Graph", "version": "5.0.0" }, { "type": "datasource", "id": "prometheus", "name": "Prometheus", "version": "5.0.0" }, { "type": "panel", "id": "singlestat", "name": "Singlestat", "version": "5.0.0" }, { "type": "panel", "id": "table", "name": "Table", "version": "5.0.0" }, { "type": "panel", "id": "text", "name": "Text", "version": "5.0.0" } ], "annotations": { "list": [ { "builtIn": 1, "datasource": "-- Grafana --", "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "type": "dashboard" } ] }, "description": "使用 Node Exporter v0.16+，精簡優化重要指標展現。包含：CPU 內存 磁盤 IO 網絡 溫度等監控指標。https://github.com/starsliao/Prometheus", "editable": true, "gnetId": 8919, "graphTooltip": 0, "id": null, "iteration": 1561954753319, "links": [ { "icon": "external link", "tags": [], "targetBlank": true, "title": "更新node_exporter", "tooltip": "", "type": "link", "url": "https://github.com/prometheus/node_exporter/releases" }, { "icon": "external link", "tags": [], "targetBlank": true, "title": "更新儀表板", "tooltip": "", "type": "link", "url": "https://grafana.com/dashboards/8919" } ], "panels": [ { "content": "", "editable": true, "error": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 }, "id": 11, "links": [], "minSpan": 4, "mode": "html", "repeat": "node", "repeatDirection": "h", "style": {}, "title": "$node", "type": "text" }, { "cacheTimeout": null, "colorBackground": false, "colorPostfix": false, "colorPrefix": false, "colorValue": true, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": 1, "description": "", "format": "s", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 2, "x": 0, "y": 1 }, "hideTimeOverride": true, "id": 15, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": false }, "tableColumn": "", "targets": [ { "expr": "time() - node_boot_time_seconds{instance=~\"$node\"}", "format": "time_series", "hide": false, "instant": true, "intervalFactor": 2, "refId": "A", "step": 40 } ], "thresholds": "1,2", "title": "系統運行時間", "transparent": false, "type": "singlestat", "valueFontSize": "100%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorPostfix": false, "colorValue": true, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS_111}", "description": "", "format": "short", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 2, "w": 2, "x": 2, "y": 1 }, "id": 14, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 4, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": false }, "tableColumn": "", "targets": [ { "expr": "count(count(node_cpu_seconds_total{instance=~\"$node\", mode='system'}) by (cpu))", "format": "time_series", "instant": true, "intervalFactor": 1, "legendFormat": "", "refId": "A", "step": 20 } ], "thresholds": "1,2", "title": "CPU 核數", "type": "singlestat", "valueFontSize": "100%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": true, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": 2, "description": "", "format": "percent", "gauge": { "maxValue": 100, "minValue": 0, "show": true, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 3, "x": 4, "y": 1 }, "id": 167, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 2, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "100 - (avg(irate(node_cpu_seconds_total{instance=~\"$node\",mode=\"idle\"}[5m])) * 100)", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 1, "legendFormat": "", "refId": "A", "step": 20 } ], "thresholds": "50,80", "title": "CPU使用率（5m）", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": true, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": 2, "description": "", "format": "percent", "gauge": { "maxValue": 100, "minValue": 0, "show": true, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 3, "x": 7, "y": 1 }, "id": 20, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 2, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "avg(irate(node_cpu_seconds_total{instance=~\"$node\",mode=\"iowait\"}[5m])) * 100", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 1, "legendFormat": "", "refId": "A", "step": 20 } ], "thresholds": "20,50", "title": "CPU iowait（5m）", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": true, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": 0, "description": "", "format": "percent", "gauge": { "maxValue": 100, "minValue": 0, "show": true, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 3, "x": 10, "y": 1 }, "hideTimeOverride": false, "id": 172, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 4, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "(1 - (node_memory_MemAvailable_bytes{instance=~\"$node\"} / (node_memory_MemTotal_bytes{instance=~\"$node\"})))* 100", "format": "time_series", "hide": false, "interval": "10s", "intervalFactor": 1, "refId": "A", "step": 20 } ], "thresholds": "80,90", "title": "內存使用率", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorPostfix": false, "colorPrefix": false, "colorValue": true, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": 2, "description": "", "format": "short", "gauge": { "maxValue": 10000, "minValue": null, "show": true, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 3, "x": 13, "y": 1 }, "hideTimeOverride": false, "id": 16, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 4, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "node_filefd_allocated{instance=~\"$node\"}", "format": "time_series", "instant": false, "interval": "10s", "intervalFactor": 1, "refId": "B" } ], "thresholds": "7000,9000", "title": "當前打開的文件描述符", "transparent": false, "type": "singlestat", "valueFontSize": "70%", "valueMaps": [], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": true, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": null, "description": "", "format": "percent", "gauge": { "maxValue": 100, "minValue": 0, "show": true, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 4, "x": 16, "y": 1 }, "id": 166, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 4, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "repeatDirection": "h", "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "100 - ((node_filesystem_avail_bytes{instance=~\"$node\",mountpoint=\"/\",fstype=~\"ext4|xfs\"} * 100) / node_filesystem_size_bytes {instance=~\"$node\",mountpoint=\"/\",fstype=~\"ext4|xfs\"})", "format": "time_series", "interval": "10s", "intervalFactor": 1, "refId": "A", "step": 20 } ], "thresholds": "70,90", "title": "根分區使用率", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": true, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": null, "description": "經過變量maxmount獲取最大的分區。", "format": "percent", "gauge": { "maxValue": 100, "minValue": 0, "show": true, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 5, "w": 4, "x": 20, "y": 1 }, "id": 154, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 4, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "repeat": null, "repeatDirection": "h", "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "100 - ((node_filesystem_avail_bytes{instance=~\"$node\",mountpoint=\"$maxmount\",fstype=~\"ext4|xfs\"} * 100) / node_filesystem_size_bytes {instance=~\"$node\",mountpoint=\"$maxmount\",fstype=~\"ext4|xfs\"})", "format": "time_series", "interval": "10s", "intervalFactor": 1, "refId": "A", "step": 20 } ], "thresholds": "70,90", "title": "最大分區($maxmount)使用率", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": true, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS_111}", "decimals": null, "description": "", "format": "bytes", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "gridPos": { "h": 3, "w": 2, "x": 2, "y": 3 }, "id": 75, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "minSpan": 4, "nullPointMode": "null", "nullText": null, "postfix": "", "postfixFontSize": "70%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": false }, "tableColumn": "", "targets": [ { "expr": "node_memory_MemTotal_bytes{instance=~\"$node\"}", "format": "time_series", "instant": true, "intervalFactor": 1, "legendFormat": "{{instance}}", "refId": "A", "step": 20 } ], "thresholds": "2,3", "title": "內存總量", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "aliasColors": { "15分鐘": "#6ED0E0", "1分鐘": "#BF1B00", "5分鐘": "#CCA300" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "editable": true, "error": false, "fill": 1, "grid": {}, "gridPos": { "h": 6, "w": 11, "x": 0, "y": 6 }, "height": "300", "id": 13, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": false, "min": false, "rightSide": true, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 2, "links": [], "minSpan": 4, "nullPointMode": "null as zero", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "repeat": null, "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "node_load1{instance=~\"$node\"}", "format": "time_series", "instant": false, "interval": "10s", "intervalFactor": 2, "legendFormat": "{{instance}}_1m", "metric": "", "refId": "A", "step": 20, "target": "" }, { "expr": "node_load5{instance=~\"$node\"}", "format": "time_series", "instant": false, "interval": "10s", "intervalFactor": 2, "legendFormat": "{{instance}}_5m", "refId": "B", "step": 20 }, { "expr": "node_load15{instance=~\"$node\"}", "format": "time_series", "instant": false, "interval": "10s", "intervalFactor": 2, "legendFormat": "{{instance}}_15m", "refId": "C", "step": 20 } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "系統平均負載", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "cumulative" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "/": "#eab839", "/boot": "#bf1b00", "/data": "#1f78c1" }, "breakPoint": "100%", "cacheTimeout": null, "combine": { "label": "Others", "threshold": "" }, "datasource": "${DS_PROMETHEUS_111}", "decimals": 1, "fontSize": "50%", "format": "bytes", "gridPos": { "h": 6, "w": 5, "x": 11, "y": 6 }, "hideTimeOverride": false, "id": 171, "interval": null, "legend": { "header": "", "percentage": false, "percentageDecimals": 0, "show": true, "sideWidth": 142, "values": true }, "legendType": "Right side", "links": [], "maxDataPoints": 3, "nullPointMode": "connected", "pieType": "pie", "strokeWidth": "2", "targets": [ { "expr": "node_filesystem_size_bytes {instance=~\"$node\",fstype=~\"ext4|xfs\"}", "format": "time_series", "instant": true, "interval": "10s", "intervalFactor": 1, "legendFormat": "{{mountpoint}}", "refId": "A" } ], "title": "磁盤總空間", "type": "grafana-piechart-panel", "valueName": "current" }, { "columns": [], "datasource": "${DS_PROMETHEUS_111}", "fontSize": "120%", "gridPos": { "h": 6, "w": 8, "x": 16, "y": 6 }, "id": 164, "links": [], "pageSize": null, "scroll": true, "showHeader": true, "sort": { "col": 6, "desc": false }, "styles": [ { "alias": "Time", "dateFormat": "YYYY-MM-DD HH:mm:ss", "pattern": "Time", "type": "hidden" }, { "alias": "分區", "colorMode": null, "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "dateFormat": "YYYY-MM-DD HH:mm:ss", "decimals": 2, "mappingType": 1, "pattern": "mountpoint", "thresholds": [ "" ], "type": "string", "unit": "bytes" }, { "alias": "可用空間", "colorMode": "value", "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "dateFormat": "YYYY-MM-DD HH:mm:ss", "decimals": 2, "mappingType": 1, "pattern": "Value #A", "thresholds": [ "10000000000", "20000000000" ], "type": "number", "unit": "bytes" }, { "alias": "使用率", "colorMode": "cell", "colors": [ "rgba(50, 172, 45, 0.97)", "rgba(237, 129, 40, 0.89)", "rgba(245, 54, 54, 0.9)" ], "dateFormat": "YYYY-MM-DD HH:mm:ss", "decimals": 2, "mappingType": 1, "pattern": "Value #B", "thresholds": [ "70", "90" ], "type": "number", "unit": "percentunit" }, { "alias": "總空間", "colorMode": null, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "dateFormat": "YYYY-MM-DD HH:mm:ss", "decimals": 1, "link": false, "mappingType": 1, "pattern": "Value #C", "thresholds": [], "type": "number", "unit": "bytes" }, { "alias": "文件系統", "colorMode": null, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "dateFormat": "YYYY-MM-DD HH:mm:ss", "decimals": 2, "link": false, "mappingType": 1, "pattern": "fstype", "thresholds": [], "type": "number", "unit": "short" }, { "alias": "IP", "colorMode": null, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "dateFormat": "YYYY-MM-DD HH:mm:ss", "decimals": 2, "link": false, "mappingType": 1, "pattern": "instance", "thresholds": [], "type": "number", "unit": "short" }, { "alias": "", "colorMode": null, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "decimals": 2, "pattern": "/.*/", "preserveFormat": true, "sanitize": false, "thresholds": [], "type": "hidden", "unit": "short" } ], "targets": [ { "expr": "node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext4|xfs\"}", "format": "table", "hide": true, "instant": true, "intervalFactor": 1, "legendFormat": "", "refId": "C" }, { "expr": "node_filesystem_avail_bytes {instance=~'$node',fstype=~\"ext4|xfs\"}", "format": "table", "hide": false, "instant": true, "interval": "10s", "intervalFactor": 1, "legendFormat": "", "refId": "A" }, { "expr": "1-(node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext4|xfs\"} / node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext4|xfs\"})", "format": "table", "hide": false, "instant": true, "intervalFactor": 1, "legendFormat": "", "refId": "B" }, { "expr": "", "format": "table", "interval": "10s", "intervalFactor": 1, "legendFormat": "", "refId": "D" } ], "title": "各分區可用空間", "transform": "table", "transparent": false, "type": "table" }, { "aliasColors": { "Idle - Waiting for something to happen": "#052B51", "guest": "#9AC48A", "idle": "#052B51", "iowait": "#EAB839", "irq": "#BF1B00", "nice": "#C15C17", "sdb_每秒I/O操做%": "#d683ce", "softirq": "#E24D42", "steal": "#FCE2DE", "system": "#508642", "user": "#5195CE", "磁盤花費在I/O操做佔比": "#ba43a9" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "decimals": 2, "description": "node_disk_io_time_seconds_total：\n磁盤花費在輸入/輸出操做上的毫秒數。該值爲累加值。（Milliseconds Spent Doing I/Os）\n\nirate(node_disk_io_time_seconds_total[1m])：\n計算每秒的速率：(last值-last前一個值)/時間戳差值，即：1秒鐘內磁盤花費在I/O操做的時間佔比。", "fill": 1, "gridPos": { "h": 7, "w": 16, "x": 0, "y": 12 }, "id": 7, "legend": { "alignAsTable": true, "avg": true, "current": true, "hideEmpty": true, "hideZero": true, "max": true, "min": false, "rightSide": true, "show": true, "sideWidth": null, "sort": null, "sortDesc": null, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "minSpan": 4, "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "repeat": null, "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "avg(irate(node_cpu_seconds_total{instance=~\"$node\",mode=\"system\"}[1m])) by (instance)", "format": "time_series", "hide": false, "instant": false, "interval": "", "intervalFactor": 2, "legendFormat": "{{instance}}_System", "refId": "A", "step": 20 }, { "expr": "avg(irate(node_cpu_seconds_total{instance=~\"$node\",mode=\"user\"}[1m])) by (instance)", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "{{instance}}_User", "refId": "B", "step": 240 }, { "expr": "avg(irate(node_cpu_seconds_total{instance=~\"$node\",mode=\"idle\"}[1m])) by (instance)", "format": "time_series", "hide": true, "intervalFactor": 2, "legendFormat": "{{instance}}_Idle", "refId": "F", "step": 240 }, { "expr": "avg(irate(node_cpu_seconds_total{instance=~\"$node\",mode=\"iowait\"}[1m])) by (instance)", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "{{instance}}_Iowait", "refId": "D", "step": 240 }, { "expr": "irate(node_disk_io_time_seconds_total{instance=~\"$node\"}[1m])", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}_{{device}}_每秒I/O操做%", "refId": "C" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "CPU使用率、磁盤每秒的I/O操做耗費時間（%）", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "transparent": false, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "decimals": null, "format": "percentunit", "label": "", "logBase": 1, "max": "1", "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "內存_Avaliable": "#6ED0E0", "內存_Cached": "#EF843C", "內存_Free": "#629E51", "內存_Total": "#6d1f62", "內存_Used": "#eab839", "可用": "#9ac48a", "總內存": "#bf1b00" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "decimals": 2, "fill": 6, "gridPos": { "h": 7, "w": 8, "x": 16, "y": 12 }, "height": "300", "id": 156, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": false, "min": false, "rightSide": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 3, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "node_memory_MemTotal_bytes{instance=~\"$node\"}", "format": "time_series", "hide": false, "instant": false, "intervalFactor": 2, "legendFormat": "{{instance}}_總內存", "refId": "A", "step": 4 }, { "expr": "node_memory_MemTotal_bytes{instance=~\"$node\"} - node_memory_MemAvailable_bytes{instance=~\"$node\"}", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "{{instance}}_已用", "refId": "B", "step": 4 }, { "expr": "node_memory_MemFree_bytes{instance=~\"$node\"}", "format": "time_series", "hide": true, "intervalFactor": 2, "legendFormat": "內存_Free", "refId": "C", "step": 4 }, { "expr": "node_memory_Buffers_bytes{instance=~\"$node\"}", "format": "time_series", "hide": true, "intervalFactor": 2, "legendFormat": "內存_Buffers", "refId": "D", "step": 4 }, { "expr": "node_memory_Cached_bytes{instance=~\"$node\"}", "format": "time_series", "hide": true, "intervalFactor": 2, "legendFormat": "內存_Cached", "refId": "E", "step": 4 }, { "expr": "node_memory_MemAvailable_bytes{instance=~\"$node\"}", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "{{instance}}_可用", "refId": "F", "step": 4 }, { "expr": "node_memory_MemTotal_bytes{instance=~\"$node\"} - (node_memory_Cached_bytes{instance=~\"$node\"} + node_memory_Buffers_bytes{instance=~\"$node\"} + node_memory_MemFree_bytes{instance=~\"$node\"})", "format": "time_series", "hide": true, "intervalFactor": 1, "refId": "G" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "內存信息", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "bytes", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "vda_write": "#6ED0E0" }, "bars": true, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "description": "Reads completed: 每一個磁盤分區每秒讀完成次數\n\nWrites completed: 每一個磁盤分區每秒寫完成次數\n\nIO now 每一個磁盤分區每秒正在處理的輸入/輸出請求數", "fill": 2, "gridPos": { "h": 8, "w": 6, "x": 0, "y": 19 }, "height": "300", "id": 161, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": true, "hideZero": true, "max": true, "min": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": false, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*_讀取$/", "transform": "negative-Y" } ], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "irate(node_disk_reads_completed_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "{{instance}}_{{device}}_讀取", "refId": "A", "step": 10 }, { "expr": "irate(node_disk_writes_completed_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "{{instance}}_{{device}}_寫入", "refId": "B", "step": 10 }, { "expr": "node_disk_io_now{instance=~\"$node\"}", "format": "time_series", "hide": true, "interval": "", "intervalFactor": 1, "legendFormat": "{{device}}", "refId": "C" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "磁盤讀寫速率（IOPS）", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "decimals": null, "format": "iops", "label": "讀取（-）/寫入（+）I/O ops/sec", "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "vda_write": "#6ED0E0" }, "bars": true, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "description": "Read bytes 每一個磁盤分區每秒讀取的比特數\nWritten bytes 每一個磁盤分區每秒寫入的比特數", "fill": 2, "gridPos": { "h": 8, "w": 6, "x": 6, "y": 19 }, "height": "300", "id": 168, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": true, "hideZero": true, "max": true, "min": false, "show": true, "total": false, "values": true }, "lines": false, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*_讀取$/", "transform": "negative-Y" } ], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "irate(node_disk_read_bytes_total{instance=~\"$node\"}[1m])", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "{{instance}}_{{device}}_讀取", "refId": "A", "step": 10 }, { "expr": "irate(node_disk_written_bytes_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "{{instance}}_{{device}}_寫入", "refId": "B", "step": 10 } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "磁盤讀寫容量大小", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "decimals": null, "format": "Bps", "label": "讀取（-）/寫入（+）", "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "vda": "#6ED0E0" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "description": "Read time ms 每一個磁盤分區讀操做花費的秒數\n\nWrite time ms 每一個磁盤分區寫操做花費的秒數\n\nIO time ms 每一個磁盤分區輸入/輸出操做花費的秒數\n\nIO time weighted 每一個磁盤分區輸入/輸出操做花費的加權秒數", "fill": 3, "gridPos": { "h": 8, "w": 6, "x": 12, "y": 19 }, "height": "300", "id": 160, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": true, "hideZero": true, "max": true, "min": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/,*_讀取$/", "transform": "negative-Y" } ], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "irate(node_disk_io_time_seconds_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": true, "interval": "", "intervalFactor": 2, "legendFormat": "{{device}}", "refId": "A", "step": 10 }, { "expr": "irate(node_disk_io_time_weighted_seconds_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": true, "intervalFactor": 1, "legendFormat": "{{device}}_加權", "refId": "D" }, { "expr": "irate(node_disk_read_time_seconds_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 1, "legendFormat": "{{instance}}_{{device}}_讀取", "refId": "B" }, { "expr": "irate(node_disk_write_time_seconds_total{instance=~\"$node\"}[1m])", "format": "time_series", "hide": false, "intervalFactor": 1, "legendFormat": "{{instance}}_{{device}}_寫入", "refId": "C" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "磁盤IO讀寫時間", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "s", "label": "讀取（-）/寫入（+）", "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "fill": 1, "gridPos": { "h": 8, "w": 6, "x": 18, "y": 19 }, "id": 174, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": false, "hideZero": false, "max": false, "min": false, "rightSide": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "1-(node_filesystem_free_bytes{instance=~'$node',fstype=~\"ext4|xfs\"} / node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext4|xfs\"})", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}：{{mountpoint}}", "refId": "A" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "分時磁盤使用率", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percentunit", "label": null, "logBase": 1, "max": "1", "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "fill": 1, "gridPos": { "h": 8, "w": 12, "x": 0, "y": 27 }, "height": "300", "id": 157, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": true, "hideZero": true, "max": false, "min": false, "rightSide": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 2, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*_out上傳$/", "transform": "negative-Y" } ], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "irate(node_network_receive_bytes_total{instance=~'$node',device!~'tap.*|veth.*|br.*|docker.*|virbr*|lo*'}[5m])*8", "format": "time_series", "intervalFactor": 2, "legendFormat": "{{instance}}_{{device}}_in下載", "refId": "A", "step": 4 }, { "expr": "irate(node_network_transmit_bytes_total{instance=~'$node',device!~'tap.*|veth.*|br.*|docker.*|virbr*|lo*'}[5m])*8", "format": "time_series", "intervalFactor": 2, "legendFormat": "{{instance}}_{{device}}_out上傳", "refId": "B", "step": 4 } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "網絡流量", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "bps", "label": "上傳（-）/下載（+）", "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "TCP": "#6ED0E0" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "description": "CurrEstab - 當前狀態爲 ESTABLISHED 或 CLOSE-WAIT 的 TCP 鏈接數\n\nActiveOpens - 已從 CLOSED 狀態直接轉換到 SYN-SENT 狀態的 TCP 平均鏈接數(1分鐘內)\n\nPassiveOpens - 已從 LISTEN 狀態直接轉換到 SYN-RCVD 狀態的 TCP 平均鏈接數(1分鐘內)\n\nTCP_alloc - 已分配（已創建、已申請到sk_buff）的TCP套接字數量\n\nTCP_inuse - 正在使用（正在偵聽）的TCP套接字數量\n\nTCP_tw - 等待關閉的TCP鏈接數", "fill": 0, "gridPos": { "h": 8, "w": 12, "x": 12, "y": 27 }, "height": "300", "id": 158, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": true, "min": false, "rightSide": true, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "node_netstat_Tcp_CurrEstab{instance=~'$node'}", "format": "time_series", "hide": false, "interval": "10s", "intervalFactor": 1, "legendFormat": "{{instance}}_ESTABLISHED", "refId": "A", "step": 20 }, { "expr": "node_sockstat_TCP_tw{instance=~'$node'}", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}_TCP_tw", "refId": "D" }, { "expr": "irate(node_netstat_Tcp_ActiveOpens{instance=~'$node'}[1m])", "format": "time_series", "hide": false, "intervalFactor": 1, "legendFormat": "{{instance}}_ActiveOpens", "refId": "B" }, { "expr": "irate(node_netstat_Tcp_PassiveOpens{instance=~'$node'}[1m])", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}_PassiveOpens", "refId": "C" }, { "expr": "node_sockstat_TCP_alloc{instance=~'$node'}", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}_TCP_alloc", "refId": "E" }, { "expr": "node_sockstat_TCP_inuse{instance=~'$node'}", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}_TCP_inuse", "refId": "F" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "TCP 鏈接狀況", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "transparent": false, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS_111}", "fill": 0, "gridPos": { "h": 10, "w": 24, "x": 0, "y": 35 }, "id": 169, "legend": { "alignAsTable": true, "avg": true, "current": true, "hideEmpty": true, "hideZero": true, "max": true, "min": false, "rightSide": true, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null as zero", "percentage": false, "pointradius": 0.5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "expr": "node_hwmon_temp_celsius{instance=~'$node'}", "format": "time_series", "intervalFactor": 1, "legendFormat": "{{instance}}_{{chip}}_{{sensor}}", "refId": "A" } ], "thresholds": [], "timeFrom": null, "timeRegions": [], "timeShift": null, "title": "硬件溫度", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "celsius", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } } ], "refresh": false, "schemaVersion": 16, "style": "dark", "tags": [ "Prometheus", "node_exporter", "StarsL.cn" ], "templating": { "list": [ { "auto": true, "auto_count": 10, "auto_min": "10s", "current": { "text": "5s", "value": "5s" }, "hide": 0, "label": "interval", "name": "interval", "options": [ { "selected": false, "text": "auto", "value": "$__auto_interval_interval" }, { "selected": true, "text": "5s", "value": "5s" }, { "selected": false, "text": "10s", "value": "10s" }, { "selected": false, "text": "30s", "value": "30s" }, { "selected": false, "text": "1m", "value": "1m" }, { "selected": false, "text": "10m", "value": "10m" }, { "selected": false, "text": "30m", "value": "30m" }, { "selected": false, "text": "1h", "value": "1h" }, { "selected": false, "text": "6h", "value": "6h" }, { "selected": false, "text": "12h", "value": "12h" }, { "selected": false, "text": "1d", "value": "1d" } ], "query": "5s,10s,30s,1m,10m,30m,1h,6h,12h,1d", "refresh": 2, "skipUrlSync": false, "type": "interval" }, { "allFormat": "glob", "allValue": null, "current": {}, "datasource": "${DS_PROMETHEUS_111}", "definition": "", "hide": 0, "includeAll": true, "label": "環境", "multi": true, "multiFormat": "regex values", "name": "env", "options": [], "query": "label_values(node_exporter_build_info,env)", "refresh": 2, "regex": "", "skipUrlSync": false, "sort": 1, "tagValuesQuery": "", "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { "allFormat": "glob", "allValue": "", "current": {}, "datasource": "${DS_PROMETHEUS_111}", "definition": "label_values(node_exporter_build_info{env=~'$env'},name)", "hide": 0, "includeAll": true, "label": "主機名", "multi": true, "multiFormat": "regex values", "name": "name", "options": [], "query": "label_values(node_exporter_build_info{env=~'$env'},name)", "refresh": 2, "regex": "", "skipUrlSync": false, "sort": 1, "tagValuesQuery": "/.*/", "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { "allFormat": "glob", "allValue": null, "current": {}, "datasource": "${DS_PROMETHEUS_111}", "definition": "label_values(node_exporter_build_info{name=~'$name'},instance)", "hide": 0, "includeAll": true, "label": "IP( 主機名多選時,該項請選ALL)", "multi": true, "multiFormat": "regex values", "name": "node", "options": [], "query": "label_values(node_exporter_build_info{name=~'$name'},instance)", "refresh": 2, "regex": "", "skipUrlSync": false, "sort": 1, "tagValuesQuery": "", "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { "allValue": null, "current": {}, "datasource": "${DS_PROMETHEUS_111}", "definition": "", "hide": 2, "includeAll": false, "label": "", "multi": false, "name": "maxmount", "options": [], "query": "query_result(topk(1,sort_desc (max(node_filesystem_size_bytes{instance=~'$node',fstype=~\"ext4|xfs\"}) by (mountpoint))))", "refresh": 2, "regex": "/.*\\\"(.*)\\\".*/", "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { "datasource": "prometheus_111", "filters": [], "hide": 0, "label": "", "name": "Filters", "skipUrlSync": false, "type": "adhoc" } ] }, "time": { "from": "now-7d", "to": "now" }, "timepicker": { "now": true, "refresh_intervals": [ "5s", "10s", "30s", "1m", "5m", "15m", "30m", "1h", "2h", "1d" ], "time_options": [ "5m", "15m", "1h", "6h", "12h", "24h", "2d", "7d", "30d" ] }, "timezone": "browser", "title": "1 Node Exporter 0.16 + for Prometheus 監控展現看板", "uid": "9CWBz0bik", "version": 65 }
效果圖
自定義監控項
大神的修修改改拿來用確定好，可是咱們也須要會本身配置，後續本身的業務信息抓取配置還需靠本身。
PromQL提供了不少函數供使用，還有linux底層的東西，可否作出」美觀，有價值「的監控須要靠你了。
點擊加號，dashboard，建立一個儀表盤
輸入promql查詢語句,會顯示相應圖形。語句可在promethes進行測試，查看值和圖形，而後再配置在grafana
兩個promql例子：
#cpu使用率 (1 - ((sum(increase(node_cpu_seconds_total{job="node1",mode="idle"} [1m]) ) by (instance)) /(sum(increase(node_cpu_seconds_total[1m])) by (instance)))) * 100 #開機運行時間 ceil(node_time_seconds{instance="127.0.0.1:9100",job="node1"} - node_boot_time_seconds{instance="127.0.0.1:9100",job="node1"})
自定義監控項的詳細信息設置，保存按鈕在右上角
可選擇本身喜歡的圖形（不不不，是甲方爸爸看着舒服的圖形）
圖形插件，dashboards模板等能夠在grafana擴展裏下載
對「自定義監控信息」和」kubernetes監控「持續學習 更新中。。。。
相關文章
- 1. Prometheus+node_exporter+grafana
- 2. prometheus+node_exporter+grafana搭建
- 3. Prometheus Node_exporter
- 4. CentOS7下簡單部署Prometheus+Grafana+node_exporter
- 5. Prometheus+Grafana
- 6. Prometheus+Node_exporter+Grafana+Alertmanager 監控部署
- 7. SpringBoot 2.0+prometheus+Grafana
- 8. prometheus + influxdb + grafana + mysql
- 9. Prometheus+grafana搭建
- 10. Prometheus + Grafana詳解
- 更多相關文章...
- • ADO 更新記錄 - ADO 教程
- • R 繪圖 - 中文支持 - R 語言教程
- • Scala 中文亂碼解決
- • C# 中 foreach 遍歷的用法
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. Prometheus+node_exporter+grafana
- 2. prometheus+node_exporter+grafana搭建
- 3. Prometheus Node_exporter
- 4. CentOS7下簡單部署Prometheus+Grafana+node_exporter
- 5. Prometheus+Grafana
- 6. Prometheus+Node_exporter+Grafana+Alertmanager 監控部署
- 7. SpringBoot 2.0+prometheus+Grafana
- 8. prometheus + influxdb + grafana + mysql
- 9. Prometheus+grafana搭建
- 10. Prometheus + Grafana詳解