Prometheus + Grafana

Prometheus

ubuntu安裝prometheus很是簡單:html

apt update
apt install prometheus
systemctl enable prometheus
systemctl enable prometheus-node-exporter

apt安裝prometheus和prometheus-node-exporter以後便帶有基本配置,無需修改。node

確保開啓服務開啓:git

systemctl status prometheus
systemctl status prometheus-node-exporter

順便使用它監控mongodb,安裝prometheus-mongodb-exportergithub

apt install prometheus-mongodb-exporter
systemctl enable prometheus-mongodb-exporter

此外因爲mongodb開啓了密碼驗證,須要注意mongodb用戶的權限:mongodb_exporter github連接web

而後須要修改 /etc/default/prometheus-mongodb-exporter 中的 ARGS 以下:mongodb

# ARGS='-mongodb.uri="mongodb://localhost:27017"'
ARGS='-mongodb.uri="mongodb://xxx:xxxxx@localhost:27017"'

mongodb URI格式以下:shell

mongodb://[username:password@]host1[:port1][,...hostN[:portN]][/[database][?options]]

若是 username 或 password 包含 @ : / % 四種符號須要使用 百分號編碼.json

錯誤添加須要刪除時用db.getSiblingDB("admin").dropUser("mongodb_exporter")ubuntu

而後重啓一下服務api

systemctl restart prometheus-mongodb-exporter

Grafana

安裝:

sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
apt update
apt install grafana

配置:

配置文件爲 /etc/grafana/grafana.ini,注意以下內容:

...
[server]
domain = www.xxxx.com
enforce_domain = true
root_url = %(protocol)s://%(domain)s/grafana
...
[security]
admin_password = xxxx

而後訪問 www.xxxx.com/grafana 登陸,用戶名admin,密碼爲上面設置的admin_password。

而後按照 [這裏][https://github.com/percona/grafana-dashboards] 配置數據源使用prometheus,並導入面板。通常導入這些便可:

(注意:json中的pmm-singlestat-panel可能須要替換爲singlestat

使用Viewer角色用戶登陸查看

上述Dashboard配置好以後,不該繼續使用admin登陸系統。

在設置中「邀請」用戶,填寫本身的郵箱而後經過郵箱連接設置密碼,便可以本身的郵箱登陸grafana。


注:

使用prometheus監控兩臺服務器,配置文件 /etc/prometheus/prometheus.yml 內容以下:

# Sample config for Prometheus.

global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s # By default, scrape targets every 15 seconds.
  # scrape_timeout is set to the global default (10s).

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'example'

# Load and evaluate rules in this file every 'evaluation_interval' seconds.
rule_files:
  # - "first.rules"
  # - "second.rules"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    scrape_timeout: 5s

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
        - targets: ['localhost:9090']

  - job_name: "web-server"
    # If prometheus-node-exporter is installed, grab stats about the local
    # machine by default.
    static_configs:
      - targets: ['localhost:9100']

  - job_name: "worker-node1"
    static_configs:
      - targets: ['192.168.0.5:9100']

這個配置是沒問題的,在另外一臺機器 (192.168.0.5) 上安裝並啓用 prometheus-node-exporter 便可。

但若是你僅僅修改了某個job_name(而沒有修改ip),好比把web-server改成node,那麼grafana界面中的singlestat panel將不能正確顯示,顯示「Only queries that return single...」,
這是由於singlestat只能顯示一個結果,而查詢語句查到了兩個結果。解決方式是刪除以前的數據系列:

  • 首先中止prometheus服務,傳入--web.enable-admin-api參數手動運行

  • 而後這樣刪除:

curl -X POST     -g 'http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]={instance="localhost:9100"}'

參考連接:Prometheus: Delete Time Series Metrcs

注2:上述狀況的查詢語句多是這樣的(能夠在grafana中看到):

相關文章
相關標籤/搜索