簡潔的業務指標監控方案

 

1.監控關鍵業務連接的狀態(是否返回200，響應時間等等)

這在prometheus上已經提供了現成的exporter,能夠參考

https://github.com/prometheus/blackbox_exporter

具體使用和效果這裏還有一個博客

https://medium.com/the-telegraph-engineering/how-prometheus-and-the-blackbox-exporter-makes-monitoring-microservice-endpoints-easy-and-free-of-a986078912ee

 

2.監控具體的業務指標

以tomcat爲例，建立一個metrics的應用，而後裏面加入一個index.jsp文件，將須要暴露的指標都寫到這個文件中

好比

[root@master metrics]# cat index.jsp 
# HELP helloworld_ordernumber Number of Order.
# TYPE helloworld_ordernumber gauge
helloworld_ordernumber 10
# HELP helloworld_orderamount Amount of Order.
# TYPE helloworld_orderamount gauge
helloworld_orderamount 100

說明以下：

• 整個是text格式，不須要加html,body什麼的
• 每一個自定義指標前面加上HELP和TYPE, gauge類型意思是可大可小，而不是累加的counter類型。
• 這個具體指標的獲取之後能夠設計成經過調用程序接口或者訪問數據庫的模式，這裏爲了簡化寫死。

 

修改prometheus的配置文件

加入被監控的target

[root@master prometheus-2.7.1.linux-amd64]# cat prometheus.yml 
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9100']
    - targets: ['localhost:8080']

啓動，而後打開http://192.168.56.108:9090/targets

獲取和訪問指標

 

3.在OpenShift容器雲環境下的監控

• index.jsp或者相似的metrics,和業務應用綁定在一塊兒，所以和業務是一個Pod
• 若是須要針對每一個微服務暴露的業務指標進行監控,須要在Openshift容器內部部署Prometheus.
• 若是是在集羣外部署Prometheus,須要把須要監控的服務經過route暴露出來

 

4. 監控類別及方式說明

• 業務監控其實是獲取業務的指標，好比存放在redis或者數據庫，若是存在多個應用實例，只須要走任意一個實例訪問獲取便可。
• 若是是監控每一個實例是否正常工做，能夠經過OpenShift提供的readness Probe和liveness Probe.由Kubernetes來保障