利用prometheus文件服務發現功能,能夠方便自定義主機或虛擬機的監控。python
1. /etc/prometheus/prometheus.yml配置文件docker
文件服務發現配置參考 job_name: 'vm' 段json
# my global config global: scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. # scrape_timeout is set to the global default (10s). # Attach these labels to any time series or alerts when communicating with # external systems (federation, remote storage, Alertmanager). external_labels: monitor: 'codelab-monitor' # Load rules once and periodically evaluate them according to the global 'evaluation_interval'. rule_files: # - "first.rules" # - "second.rules" # A scrape configuration containing exactly one endpoint to scrape: # Here it's Prometheus itself. scrape_configs: # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config. - job_name: 'prometheus' # metrics_path defaults to '/metrics' # scheme defaults to 'http'. static_configs: - targets: ['localhost:9090'] scrape_configs: - job_name: 'vm' file_sd_configs: - files: - /etc/prometheus/files/vm.yml refresh_interval: 5s
2. /etc/prometheus/files/vm.yml配置文件內容api
[ { "targets": [ "192.168.1.6:9273"], "labels": { "hosttype": "vm" } } ]
3. 經過docker啓動prometheusbash
docker run -d -p 9090:9090 \ -v /etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml \ -v /etc/prometheus/files:/etc/prometheus/files \ prom/prometheus:v1.8.2
4. 檢查prometheus的target,是否包含files.yml文件中配置的targetthis
http://192.168.1.6:9090/targetslua
5. 動態生成/etc/prometheus/files.yml url
若是想動態修改target,能夠經過http請求,獲取主機列表,而後動態生成/etc/prometheus/files/vm.yml文件。code
下面是動態生成vm.yml例子。經過http請求,獲取虛擬機的ip地址列表,而後動態生成vm.yml文件。orm
#!/usr/bin/python import json import logging import os import requests import time logging.basicConfig(level=logging.INFO, format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s', datefmt='%Y-%m-%d %H:%M:%S', filename='/var/log/prometheus-vm-http.log', filemode='w') console = logging.StreamHandler() console.setLevel(logging.INFO) formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s') console.setFormatter(formatter) logging.getLogger('').addHandler(console) def refresh(): url = "http://域名/api/vm/ip" resp = requests.get(url, timeout=10) if resp.status_code == 200: logging.info("get " + url + " ok") data = [ { "targets": map(lambda x: x + ":9273", resp.json()), "labels": { "hosttype": "vm" } } ] with open('/etc/prometheus/files/vm.yml.new', 'w') as f: json.dump(data, f, indent=2) f.flush() os.fsync(f.fileno()) os.rename('/etc/prometheus/files/vm.yml.new', '/etc/prometheus/files/vm.yml') else: logging.error("get " + url + " failed. status_code=%d, resp=%s" % (resp.status_code, resp.text)) if __name__ == '__main__': while True: try: refresh() except Exception, e: logging.error(e) time.sleep(10)