Azure容器監控部署（上）

時間  2019-11-09
標籤 azure 容器 監控 部署 简体版
原文   原文鏈接

前兩篇簡單的介紹了一下prometheus的，本節本來是寫node_exporter和cAdvisor的搭建，但網上教程不少，因此直接寫整套環境的部署過程html

 1、架構

    咱們原來的系統架構是在AZURE上有兩臺虛機做爲業務機，部署一個Load Balance，用戶訪問LB，LB根據特定規則將流量轉發至內部的虛機VM1/VM2。而且VM1/VM2組成一個局域網，外界是沒法直接訪問的，只能經過LB跳轉到VM1/VM2上node

    prometheus最優的部署方案是將prometheus節點部署到VM1/VM2的局域網內，對外暴露一個端口，或者在LB上設置NAT規則直接鏈接prometheus，這樣作的優勢是：python

        一、VM1/VM2/prometheus組成的局域網，外界沒法訪問；linux

        二、默認狀況下node_exporter和cAdvisor是http協議，避免了VM1/VM2上收集到的數據經過外網傳輸nginx

    但因爲種種緣由，咱們的prometheus只能部署到外部，所以整個系統的架構以下圖 ：VM1/VM2上部署的node_exporter和cAdvisor對外暴露9091和8008端口（可自定義），由LB的NAT端口轉發映射到LB上。而後再由prometheus分別去收集如下四個端口的監控數據。git

　

2、各組件版本

       Prometheus         2.9github

       Grafana                6.1.6web

       cAdvisor               0.17docker

       node_exporter      0.17ubuntu

       stunnel                 5.44

       nginx                    1.14

       certbot                  0.23

3、Load Balance上與prometheus相關的端口（設置的NAT入站規則）

       19101端口鏈接VM1的9101

       18008端口鏈接VM1的8008

       29101端口鏈接VM2的9101

       28008端口鏈接VM2的8008

 

4、部署過程

一、在AZURE上建立prometheus的虛機設置固定IP和域名

二、在prometheus server上安裝docker

安裝腳本以下：

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg|sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -y docker-ce
sudo systemctl restart docker
sudo docker images
sudo groupadd docker
sudo usermod -aG docker $USER

三、在prometheus server上安裝prometheus

下載路徑：

cd /usr/local/share/prometheus/
wget https://github.com/prometheus/prometheus/releases/download/v2.9.1/prometheus-2.9.1.linux-amd64.tar.gz

解壓安裝，並將prometheus加由systemd管理

sudo adduser prometheus
sudo chown -R prometheus:prometheus /usr/local/share/prometheus/


vim /etc/systemd/system/prometheus.service

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network.target
 
[Service]
Restart=on-failure
WorkingDirectory=/usr/local/share/prometheus/
ExecStart=/usr/local/share/prometheus/prometheus \
    --config.file=/usr/local/share/prometheus/prometheus.yml \
    --web.external-url=https://虛機域名 \
    --storage.tsdb.retention.time=30d
 
[Install]
WantedBy=multi-user.target

啓動prometheus

sudo systemctl daemon-reload
sudo systemctl start Prometheus
sudo systemctl enable Prometheus    #設置爲開機自啓
sudo systemctl status prometheus

四、在prometheus server上安裝nginx和certbot

sudo apt -y install nginx

certbot是一款免費生成tls的安全證書，安裝腳本

sudo apt-get update
sudo apt-get install software-properties-common
sudo add-apt-repository universe
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install certbot python-certbot-nginx 
sudo certbot –nginx

根據提示輸入郵箱、主機名等信息【主機名不能亂寫】

生成證書的位置在 /etc/letsencrypt/live/主機名/

ertbot提供的證書有效期是90天，能夠利用官方提供的命令按期從新生成證書，最後將其加入週期性計劃任務中。

certbot renew --dry-run

配置nginx的配置文件

sudo vim /etc/nginx/nginx.conf

#修改server配置
server {
    listen 443 ssl;
    server_name  lkprometheusemu.southeastasia.cloudapp.azure.com;
    ssl_certificate /etc/letsencrypt/live/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/privkey.pem; # managed by Certbot

location / {
   root /var/www/html;
     index index.html;
     proxy_pass http://127.0.0.1:3000;
                }
        }

　　啓動nginx 

       sudo nginx

五、在vm1/VM2上安裝node_exporter

sudo wget https://github.com/prometheus/node_exporter/releases/download/v0.17.0/node_exporter-0.17.0.linux-amd64.tar.gz
sudo tar -xf node_exporter-0.17.0.linux-amd64.tar.gz

加入開機自啓
vim /etc/systemd/system/nodeexporter.service 
[Unit]
Description=Node Exporter
After=network.target

[Service]
ExecStart=/usr/local/share/node_exporter/node_exporter  --web.listen-address=127.0.0.1:9101

[Install]
WantedBy=multi-user.target

啓動並開機自啓
sudo vim /etc/systemd/system/node_exporter.service
sudo systemctl daemon-reload 
sudo systemctl start node_exporter.service
sudo systemctl status node_exporter.service

對外暴露9101端口

六、在VM1/VM2上安裝cAdvisor

切換到root用戶執行

docker run -d  \
--volume=/:/rootfs:ro \
--volume=/var/run:/var/run:rw \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:ro \
-p 8088:8080 \
--restart=always \
--name=cadvisor \
google/cadvisor

　　對外暴露8088端口

七、在VM1/VM2上安裝stunnel

sudo apt install stunnel

啓用stunnel

編輯 sudo vim /etc/default/stunnel4，將ENABLE改爲1

八、在任意一個linux主機上建立自有證書

將生成的證書拷貝的VM1/VM2指定位置，我放在了/etc/stunnel/tls下

sudo mkdir /etc/stunnel/tls 
cd /etc/stunnel/tls
sudo openssl genrsa -out key.pem 2048             #建立一個2048位的祕鑰
sudo openssl req -new -x509 -key key.pem -out cert.pem -days 3650 -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=所在主機的主機名" 
sudo chmod 640 key.pem  cert.pem private.pem

九、修改stunnel的配置

　如下配置爲vm1，在vm2上需將node_exporter一、cAdvisor1改爲node_exporter二、cADVisor2

sudo vim /etc/stunnel/stunnel.conf

pid             = /var/run/stunnel4/stunnel.pid
output          = /var/log/stunnel4/stunnel.log

[node_exporter1]
accept          = 9101
connect         = 127.0.0.1:9100
cert            = /etc/stunnel/tls/cert.pem
key             = /etc/stunnel/tls/key.pem

[cAdvisor1]
accept          = 8008
connect         = 127.0.0.1:8088
cert            = /etc/stunnel/tls/cert.pem
key             = /etc/stunnel/tls/key.pem

重啓stunnel服務

sudo systemctl restart stunnel4

十、在prometheus server上配置prometheus

sudo vim /usr/local/share/prometheus/prometheus.yml

	# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'
    static_configs:
    - targets: ['127.0.0.1:9090']

  - job_name: 'node_exporter1'
    static_configs:
    - targets: ['LBIP:19101']
    scheme: https
tls_config:
      insecure_skip_verify: true

  - job_name: 'node_exporter2'
    static_configs:
    - targets: ['LBIP:29101']
    scheme: https
    tls_config:
      insecure_skip_verify: true

  - job_name: 'cadvisor1'
    static_configs:
    - targets: ['LBIP:18008']
    scheme: https
    tls_config:
      insecure_skip_verify: true

  - job_name: 'cadvisor2'
    static_configs:
    - targets: ['LBIP:28008']
    scheme: https
    tls_config:
      insecure_skip_verify: true

檢查prometheus的配置是否成功，切換到prometheus的安裝目錄下執行

promtool check rules prometheus.yml

十一、在prometheus server上安裝grafana

wget https://dl.grafana.com/oss/release/grafana_6.1.6_amd64.deb 
sudo dpkg -i grafana_6.1.6_amd64.deb 
sudo /bin/systemctl daemon-reload
sudo systemctl start grafana-server.service 
sudo systemctl enable grafana-server.service

配置grafana的郵件功能

sudo vim /etc/grafana/grafana.ini

重啓grafana

sudo systemctl start grafana-server.service

登錄grafana後添加prometheus的數據源

默認的用戶名是 admin

默認密碼是 admin

此時，檢查prometheus是否鏈接正常。

其他步驟見下文

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程