使用Prometheus和Grafana監控Mysql服務器性能

時間 2019-12-06

標籤使用 prometheus grafana 監控 mysql 服務器性能欄目 MySQL 简体版

原文原文鏈接

這是一篇快速入門文章，介紹瞭如何使用Prometheus和Grafana對Mysql服務器性能進行監控。內容基於這篇文章，結合了本身的實際實踐並根據最新版本的應用進行了調整。下面是兩張效果圖：node

概述

Prometheus是一個開源的服務監控系統，它經過HTTP協議從遠程的機器收集數據並存儲在本地的時序數據庫上。它提供了一個簡單的網頁界面、一個功能強大的查詢語言以及HTTP接口等等。Prometheus經過安裝在遠程機器上的exporter來收集監控數據，咱們用到了如下兩個exporter：mysql

node_exporter – 用於收集系統數據linux
mysqld_exporter – 用於收集Mysql數據git

Grafana是一個開源的功能豐富的數據可視化平臺，一般用於時序數據的可視化。它內置瞭如下數據源的支持：github

並能夠經過插件擴展支持的數據源。web

架構圖

下面是咱們安裝時用到的架構圖：sql

安裝和運行Prometheus

安裝Prometheus

首先咱們安裝Prometheus：數據庫

$ wget https://github.com/prometheus/prometheus/releases/download/v1.6.3/prometheus-1.6.3.linux-amd64.tar.gz -O prometheus-1.6.3.linux-amd64.tar.gz
$ mkdir /usr/local/services/prometheus
$ tar zxf prometheus-1.6.3.linux-amd64.tar.gz -C /usr/local/services/prometheus --strip-components=1

配置prometheus

而後在安裝目下編輯配置文件 prometheus.yml：json

global:
  scrape_interval:     15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: linux
    static_configs:
      - targets: ['host:9100']
        labels:
          instance: db1

  - job_name: mysql
    static_configs:
      - targets: ['host:9104']
        labels:
          instance: db1

host是咱們數據庫主機的IP，端口則是對應的exporter的監聽端口。bash

運行Prometheus

而後咱們啓動Prometheus：

$ ./prometheus       
INFO[0000] Starting prometheus (version=1.6.3, branch=master, revision=c580b60c67f2c5f6b638c3322161bcdf6d68d7fc)  source=main.go:88
INFO[0000] Build context (go=go1.8.1, user=root@a6410e65f5c7, date=20170522-09:15:06)  source=main.go:89
INFO[0000] Loading configuration file prometheus.yml     source=main.go:251
INFO[0000] Loading series map and head chunks...         source=storage.go:421
INFO[0000] 0 series loaded.                              source=storage.go:432
INFO[0000] Listening on :9090                            source=web.go:259
INFO[0000] Starting target manager...                    source=targetmanager.go:61
INFO[0300] Checkpointing in-memory metrics and chunks...  source=persistence.go:633
INFO[0300] Done checkpointing in-memory metrics and chunks in 75.372924ms.  source=persistence.go:665

Prometheus內置了一個web界面，咱們可經過http://monitor_host:9090進行訪問：

在Status->Targets頁面下，咱們能夠看到咱們配置的兩個Target，它們的State爲DOWN。

部署exporter

下一步咱們須要安裝並運行exporter。下載exporters並解壓：

$ wget https://github.com/prometheus/node_exporter/releases/download/v0.14.0/node_exporter-0.14.0.linux-amd64.tar.gz -O node_exporter-0.14.0.linux-amd64.tar.gz
$ wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.10.0/mysqld_exporter-0.10.0.linux-amd64.tar.gz -O mysqld_exporter-0.10.0.linux-amd64.tar.gz
$ mkdir /usr/local/services/prometheus_exporters
$ tar zxf node_exporter-0.14.0.linux-amd64.tar.gz -C /usr/local/services/prometheus_exporters --strip-components=1
$ tar zxf mysqld_exporter-0.10.0.linux-amd64.tar.gz -C /usr/local/services/prometheus_exporters --strip-components=1

運行node_exporter ：

$ cd /usr/local/services/prometheus_exporters
$ ./node_exporter 
INFO[0000] Starting node_exporter (version=0.14.0, branch=master, revision=840ba5dcc71a084a3bc63cb6063003c1f94435a6)  source="node_exporter.go:140"
INFO[0000] Build context (go=go1.7.5, user=root@bb6d0678e7f3, date=20170321-12:12:54)  source="node_exporter.go:141"
INFO[0000] No directory specified, see --collector.textfile.directory  source="textfile.go:57"
INFO[0000] Enabled collectors:                           source="node_exporter.go:160"
INFO[0000]  - netdev                                     source="node_exporter.go:162"
INFO[0000]  - sockstat                                   source="node_exporter.go:162"
INFO[0000]  - stat                                       source="node_exporter.go:162"
INFO[0000]  - vmstat                                     source="node_exporter.go:162"
INFO[0000]  - filefd                                     source="node_exporter.go:162"
INFO[0000]  - loadavg                                    source="node_exporter.go:162"
INFO[0000]  - mdadm                                      source="node_exporter.go:162"
INFO[0000]  - meminfo                                    source="node_exporter.go:162"
INFO[0000]  - time                                       source="node_exporter.go:162"
INFO[0000]  - uname                                      source="node_exporter.go:162"
INFO[0000]  - edac                                       source="node_exporter.go:162"
INFO[0000]  - hwmon                                      source="node_exporter.go:162"
INFO[0000]  - diskstats                                  source="node_exporter.go:162"
INFO[0000]  - entropy                                    source="node_exporter.go:162"
INFO[0000]  - infiniband                                 source="node_exporter.go:162"
INFO[0000]  - netstat                                    source="node_exporter.go:162"
INFO[0000]  - textfile                                   source="node_exporter.go:162"
INFO[0000]  - wifi                                       source="node_exporter.go:162"
INFO[0000]  - zfs                                        source="node_exporter.go:162"
INFO[0000]  - conntrack                                  source="node_exporter.go:162"
INFO[0000]  - filesystem                                 source="node_exporter.go:162"
INFO[0000] Listening on :9100                            source="node_exporter.go:186"

mysqld_exporter須要鏈接到Mysql，因此須要Mysql的權限，咱們先爲它建立用戶並賦予所需的權限：

mysql> GRANT REPLICATION CLIENT, PROCESS ON *.* TO 'prom'@'localhost' identified by 'abc123';
mysql> GRANT SELECT ON performance_schema.* TO 'prom'@'localhost';

建立.my.cnf文件並運行mysqld_exporter：

$ cd /usr/local/services/prometheus_exporters
$ cat << EOF > .my.cnf
[client]
user=prom
password=abc123
EOF
$ ./mysqld_exporter -config.my-cnf=".my.cnf"  
INFO[0000] Starting mysqld_exporter (version=0.10.0, branch=master, revision=80680068f15474f87847c8ee8f18a2939a26196a)  source="mysqld_exporter.go:460"
INFO[0000] Build context (go=go1.8.1, user=root@3b0154cd9e8e, date=20170425-11:24:12)  source="mysqld_exporter.go:461"
INFO[0000] Listening on :9104                            source="mysqld_exporter.go:479

咱們再次回到Status->Targets頁面，能夠看到兩個Target的狀態已經變成UP了：

安裝和運行Grafana

下載並解壓Grafana：

$ wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.3.1.linux-x64.tar.gz
$ mkdir /usr/local/services/grafana
$ tar zxvf grafana-4.3.1.linux-x64.tar.gz -C /usr/local/services/grafana  --strip-components=1

編輯配置文件/usr/local/services/grafana/conf/defaults.ini，修改dashboards.json段落下兩個參數的值：

[dashboards.json]
enabled = true
path = /var/lib/grafana/dashboards

安裝儀表盤:

$ git clone https://github.com/percona/grafana-dashboards.git
$ cp -r grafana-dashboards/dashboards /var/lib/grafana/

最後咱們運行Grafana服務：

$ cd /usr/local/services/grafana/bin/
$ ./grafana-server
INFO[05-25|15:42:46] Starting Grafana                         logger=main version=4.3.1 commit=befc15c compiled=2017-05-23T21:50:22+0800
INFO[05-25|15:42:46] Config loaded from                       logger=settings file=/usr/local/services/grafana/conf/defaults.ini
INFO[05-25|15:42:46] Path Home                                logger=settings path=/usr/local/services/grafana
INFO[05-25|15:42:46] Path Data                                logger=settings path=/usr/local/services/grafana/data
INFO[05-25|15:42:46] Path Logs                                logger=settings path=/usr/local/services/grafana/data/log
INFO[05-25|15:42:46] Path Plugins                             logger=settings path=/usr/local/services/grafana/data/plugins
INFO[05-25|15:42:46] Initializing DB                          logger=sqlstore dbtype=sqlite3
INFO[05-25|15:42:46] Starting DB migration                    logger=migrator
INFO[05-25|15:42:46] Executing migration                      logger=migrator id="copy data account to org"
INFO[05-25|15:42:46] Skipping migration condition not fulfilled logger=migrator id="copy data account to org"
INFO[05-25|15:42:46] Executing migration                      logger=migrator id="copy data account_user to org_user"
INFO[05-25|15:42:46] Skipping migration condition not fulfilled logger=migrator id="copy data account_user to org_user"
INFO[05-25|15:42:46] Creating json dashboard index for path: /var/lib/grafana/dashboards 
INFO[05-25|15:42:46] Starting plugin search                   logger=plugins
INFO[05-25|15:42:46] Initializing CleanUpService              logger=cleanup
INFO[05-25|15:42:46] Initializing Alerting                    logger=alerting.engine
INFO[05-25|15:42:46] Initializing Stream Manager 
INFO[05-25|15:42:46] Initializing HTTP Server                 logger=http.server address=0.0.0.0:3000 protocol=http subUrl= socket=

咱們可經過http://monitor_host:3000訪問Grafana網頁界面（缺省的賬號/密碼爲admin/admin）：

而後咱們到Data Sources頁面添加數據源：

最後咱們就能夠經過選擇不一樣的儀表盤（左上角）和時間段（右上角）來呈現圖表了：

遇到的問題

Prometheus和Grafana都正常，可是儀表盤不顯示數據

Prometheus和Grafana都運行正常，exporter的狀態也正常，Grafana上的添加的數據源也正常工做，可是儀表盤就是沒有任何數據。惟一的線索是在Prometheus的管理後臺執行查詢時顯示No datapoints found.的錯誤，根據這個錯誤在網上搜索到這個issue：https://github.com/prometheus...，裏面有人提到是系統時間不正確的問題，發現系統的時間確實不對，同步時間後解決問題。