prometheus服務發現-consul部署研究

最近在看prometheus-book時，在服務發現這一章節，瞭解到「基於consul的服務發現」，之前一直都是用python代碼生成json推送到服務器上，這種方式很麻煩，以前還想着編寫工具程序部署到prometheus所在節點進行target的自動發現。

最近幾天詳細研究如何利用consul進行prometheus服務發現，首先進行consul部署的研究。

開發模式

使用命令consul agent -dev便可，而後能夠經過 http://127.0.0.1:8500 訪問管理界面

因爲目前處於驗證階段，生產環境確定以多節點集羣的方式部署，要在本機模擬集羣狀況，則能夠經過指定不一樣的端口以及搭建3臺虛擬機的方式來完成。

單節點集羣部署

單節點集羣部署須要給每一個server指定不一樣的端口，涉及的端口類型以下：

1. http-port web服務端口，默認爲8500，須要經過瀏覽器訪問管理端則要加入 -ui 或 ui:true 這個配置項

2. dns-port dns端口，默認爲8600

3. serf-lan-port lan端口，默認8301

4. serf-wan-port wan端口，默認爲8302

5. server-port 服務端口，默認爲8300

命令行傳參

1. server1

consul agent -server -bind 10.106.169.121 -node server1 -data-dir /opt/consul/server1 -ui -client 0.0.0.0 -bootstrap-expect=1 -server-port=18300 -http-port=18500 -dns-port=18600  -serf-lan-port=18301 -serf-wan-port=18302

2. server2

consul agent -server -bind 10.106.169.121 -node server2 -data-dir /opt/consul/server2 -ui -client 0.0.0.0 -server-port=28300 -http-port=28500 -dns-port=28600  -serf-lan-port=28301 -serf-wan-port=28302 -join 10.106.169.121:18301

3. server3

consul agent -server -bind 10.106.169.121 -node server3 -data-dir /opt/consul/server3 -ui -client 0.0.0.0 -server-port=38300 -http-port=38500 -dns-port=38600  -serf-lan-port=38301 -serf-wan-port=38302 -join 10.106.169.121:18301

配置文件傳參

1. server1

consul agent -config-file=/opt/consul/server1.json

{
  "datacenter": "prometheus",
  "bootstrap_expect":1,
  "bind_addr":"10.106.169.121",
  "data_dir": "/opt/consul/server1",
  "log_level": "INFO",
  "node_name": "server1",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "ports": {
     "http": 18500,
     "dns": 18600,
     "serf_lan":18301,
     "serf_wan":18302,
     "server":18300
  }
}

2. server2

consul agent -config-file=/opt/consul/server2.json

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "data_dir": "/opt/consul/server2",
  "log_level": "INFO",
  "node_name": "server2",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 28500,
     "dns": 28600,
     "serf_lan":28301,
     "serf_wan":28302,
     "server":28300
  }
}

3. server3

consul agent -config-file=/opt/consul/server3.json

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "data_dir": "/opt/consul/server3",
  "log_level": "INFO",
  "node_name": "server3",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 38500,
     "dns": 38600,
     "serf_lan":38301,
     "serf_wan":38302,
     "server":38300
  }
}

容器部署

鏡像的entrypoint是：/usr/local/bin/docker-entrypoint.sh，裏面定義了以下兩個環境變量

CONSUL_DATA_DIR=/consul/data
CONSUL_CONFIG_DIR=/consul/config

1. server1

docker run --net=host -v /opt/consul_docker/server1/config:/consul/config -v /opt/consul_docker/server1/data:/consul/data consul:1.4.1

{
  "datacenter": "prometheus",
  "bootstrap_expect":1,
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"09d82408-bc4f-49e0-4208-61ef1d4842f7",
  "node_name": "server1",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "ports": {
     "http": 18500,
     "dns": 18600,
     "serf_lan":18301,
     "serf_wan":18302,
     "server":18300,
     "grpc":-1
  }
}

2. server2

docker run --net=host -v /opt/consul_docker/server2/config:/consul/config -v /opt/consul_docker/server2/data:/consul/data consul:1.4.1

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d",
  "node_name": "server2",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 28500,
     "dns": 28600,
     "serf_lan":28301,
     "serf_wan":28302,
     "server":28300,
     "grpc":-1
  }
}

3. server3

docker run --net=host -v /opt/consul_docker/server3/config:/consul/config -v /opt/consul_docker/server3/data:/consul/data consul:1.4.1

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e",
  "node_name": "server3",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 38500,
     "dns": 38600,
     "serf_lan":38301,
     "serf_wan":38302,
     "server":38300,
     "grpc":-1
  }
}

多節點部署

1. server1

vm : 10.20.0.3

consul agent -server -bind 10.20.0.3 -node server1 -data-dir /root/consul/server1 -ui -client 0.0.0.0 -bootstrap-expect=1

2. server2

vm : 10.20.0.4

consul agent -server -bind 10.20.0.4 -node server2 -data-dir /root/consul/server2 -join 10.20.0.3 -client 0.0.0.0

3. server3

vm : 10.20.0.5

consul agent -server -bind 10.20.0.5 -node server3 -data-dir /root/consul/server3 -join 10.20.0.3 -client 0.0.0.0

部分配置項說明

1. bind_addr，綁定ip，有多IP時須要指定

2. server ，是否開啓server模式

3. client_addr ， 監聽的ip

4. retry_join，非主節點啓動時指定加入的主節點，官方推薦使用retry_join而不是join

5. node_id，可指定節點的uuid，格式：d8a09ffd-7ccb-84bd-7231-8d8b7a01951e，不指定的話爲隨機生成，使用集羣的話最好能指定該id，保證惟一性

6. node_name，節點名稱

7. config_file，指定配置文件