prometheus服務發現-consul生產環境服務部署

模擬生產環境多節點部署，使用linux服務方式部署consul集羣，保證服務高可用即異常掛掉或主機重啓後能自動回覆，同時啓用agent通訊加密和接口認證。

前期準備

1. 安裝

將下載解壓獲得二進制文件consul拷貝到/usr/local/bin目錄

sudo chown root:root /usr/local/bin/consul
consul --version # 驗證

2. 自動補全

consul -autocomplete-install
complete -C /usr/local/bin/consul consul

3. 準備目錄

建立/opt/consul目錄，並在其下準備以下3個server的目錄

/opt/consul$ tree
.
├── server1
│   ├── config
│   └── data
├── server2
│   ├── config
│   └── data
└── server3
    ├── config
    └── data

4. 生成祕鑰

用於集羣間通訊加密，須要保證集羣中全部節點都配置該祕鑰

$ consul keygen
mz8Con27P34D9fiPG1bjHA==

配置

server1

service unit

準備文件：/lib/systemd/system/consul-server1.service，內容以下：

[Unit]
Description="consul server1"
Requires=network-online.target
After=network-online.target
[Service]
ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server1/config
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

配置

準備配置文件：/opt/consul/server1/config/config.json，內容以下：

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"09d82408-bc4f-49e0-4208-61ef1d4842f7",
  "node_name": "server1",
  "data_dir":"/opt/consul/server1/data",
  "server": true,
  "bootstrap_expect":3,
  "encrypt": "mz8Con27P34D9fiPG1bjHA==",
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
  "ports": {
     "http": 18500,
     "dns": 18600,
     "serf_lan":18301,
     "serf_wan":18302,
     "server":18300,
     "grpc":-1
  },
  "acl": {
    "enabled": true,
    "default_policy": "deny",
    "down_policy": "extend-cache",
    "tokens":{
        "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
        "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
    }
  }
}

server2

service unit

準備文件：/lib/systemd/system/consul-server2.service，內容以下：

[Unit]
Description="consul server2"
Requires=network-online.target
After=network-online.target
[Service]
ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server2/config
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

配置

準備配置文件：/opt/consul/server2/config/config.json，內容以下：

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d",
  "node_name": "server2",
  "data_dir":"/opt/consul/server2/data",
  "server": true,
  "bootstrap_expect":3,
  "encrypt": "mz8Con27P34D9fiPG1bjHA==",
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
  "ports": {
     "http": 28500,
     "dns": 28600,
     "serf_lan":28301,
     "serf_wan":28302,
     "server":28300,
     "grpc":-1
  },
  "acl": {
    "enabled": true,
    "default_policy": "deny",
    "down_policy": "extend-cache",
    "tokens":{
        "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
        "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
    }
  }
}

server3

service unit

準備文件：/lib/systemd/system/consul-server3.service，內容以下：

[Unit]
Description="consul server3"
Requires=network-online.target
After=network-online.target
[Service]
ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server3/config
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

配置

準備配置文件：/opt/consul/server3/config/config.json，內容以下：

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e",
  "node_name": "server3",
  "data_dir":"/opt/consul/server3/data",
  "server": true,
  "bootstrap_expect":3,
  "encrypt": "mz8Con27P34D9fiPG1bjHA==",
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
  "ports": {
     "http": 38500,
     "dns": 38600,
     "serf_lan":38301,
     "serf_wan":38302,
     "server":38300,
     "grpc":-1
  },
  "acl": {
    "enabled": true,
    "default_policy": "deny",
    "down_policy": "extend-cache",
    "tokens":{
        "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
        "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
    }
  }
}

啓動

經過命令 ： sudo systemctl enable consul-server1 consul-server2 consul-server3將服務設置爲開機自動啓動，再使用命令啓動服務 ： sudo systemctl restart consul-server1 consul-server2 consul-server3

驗證

UI

在頁面http://127.0.0.1:18500/ui/prometheus/acls/tokens輸入配置中的master token，再刷新界面能夠在services和nodes中查看到信息

API

$ curl http://127.0.0.1:18500/v1/catalog/nodes #未帶token，返回空的節點列表
[]
$ curl http://127.0.0.1:18500/v1/catalog/nodes -H 'x-consul-token: 47eca91b-a5e7-e82d-6424-dba7637e0737' # 經過在header中增長x-consul-token則可返回節點列表
[{"ID":"09d82408-bc4f-49e0-4208-61ef1d4842f7","Node":"server1","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":9,"ModifyIndex":9},{"ID":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d","Node":"server2","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":7,"ModifyIndex":7},{"ID":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e","Node":"server3","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":8,"ModifyIndex":8}]

參考

https://learn.hashicorp.com/consul/advanced/day-1-operations/deployment-guide [官方部署方式]

https://www.consul.io/docs/agent/acl-system.html [acl介紹]

https://learn.hashicorp.com/consul/advanced/day-1-operations/acl-guide [acl配置]

https://learn.hashicorp.com/consul/advanced/day-1-operations/agent-encryption [agent通訊加密]