Nginx實現動態負載均衡，首先須要一個服務發現集羣，經過集羣中註冊的信息動態更新nginx的配置，實現動態負載均衡。所以首先準備一個Consul集羣node
Consul集羣準備
此處咱們僅做爲功能架構的測試階段，所以在一臺虛擬機上完成測試，這裏準備一臺Centos 7.4的虛擬機，IP爲192.168.99.12linux
mkdir /data/consul && cd $_
wget https://releases.hashicorp.com/consul/1.9.3/consul_1.9.3_linux_amd64.zip
unzip consul_1.7.3_linux_amd64.zip
mv consul /usr/local/bin/
在完成consul的安裝以後，須要準備一下集羣的基礎環境配置nginx
mkdir -pv /data/consul/node{1..3}
咱們建立三個consul節點使用的配置文件，分別對應各自的文件夾內git
-
/data/consul/node1/consul_config1.jsongithub{
"datacenter": "dev",
"data_dir": "/data/consul/node1",
"log_file": "/data/consul/node1/consul.log",
"log_level": "INFO",
"server": true,
"node_name": "node1",
"ui": true,
"bind_addr": "192.168.99.12",
"client_addr": "192.168.99.12",
"advertise_addr": "192.168.99.12",
"bootstrap_expect": 3,
"ports":{
"http": 8510,
"dns": 8610,
"server": 8310,
"serf_lan": 8311,
"serf_wan": 8312
}
} -
/data/consul/node2/consul_config2.jsonweb{
"datacenter": "dev",
"data_dir": "/data/consul/node2",
"log_file": "/data/consul/node2/consul.log",
"log_level": "INFO",
"server": true,
"node_name": "node2",
"ui": true,
"bind_addr": "192.168.99.12",
"client_addr": "192.168.99.12",
"advertise_addr": "192.168.99.12",
"bootstrap_expect": 3,
"ports":{
"http": 8520,
"dns": 8620,
"server": 8320,
"serf_lan": 8321,
"serf_wan": 8322
}
} -
/data/consul/node3/consul_config3.jsondocker{
"datacenter": "dev",
"data_dir": "/data/consul/node3",
"log_file": "/data/consul/node3/consul.log",
"log_level": "INFO",
"server": true,
"node_name": "node3",
"ui": true,
"bind_addr": "192.168.99.12",
"client_addr": "192.168.99.12",
"advertise_addr": "192.168.99.12",
"bootstrap_expect": 3,
"ports":{
"http": 8530,
"dns": 8630,
"server": 8330,
"serf_lan": 8331,
"serf_wan": 8332
}
}
而後即可以啓動consul集羣了json
nohup consul agent -config-file=/data/consul/node1/consul_config1.json > /dev/null 2>&1 &
nohup consul agent -config-file=/data/consul/node2/consul_config2.json -retry-join=192.168.99.12:8311 > /dev/null 2>&1 &
nohup consul agent -config-file=/data/consul/node3/consul_config3.json -retry-join=192.168.99.12:8311 > /dev/null 2>&1 &
啓動以後，即可以經過地址http://192.168.99.12:8510地址訪問，此處192.168.99.12:8510便是Leader角色bootstrap
編譯Nginx
作動態負載均衡的時候須要添加nginx-upsync-module和nginx_upstream_check_module兩個模塊，所以此處咱們須要將這兩個模塊給編譯到nginx中去。由於此前咱們在基礎鏡像的時候已經寫好了nginx自動化編譯的Dockerfile，因此咱們這裏直接使用便可vim
-
Dockerfile ( Nginx 1.14.2)
FROM debian:stretch-slim
RUN useradd www && \
mkdir -p /logs/nginx/ /webserver/nginx /webserver/nginx/conf/upsync && \
chown -R www:www /logs/nginx/ /webserver/nginx && \
echo 'deb http://mirrors.163.com/debian/ stretch main non-free contrib' > /etc/apt/sources.list && \
echo 'deb http://mirrors.163.com/debian/ stretch-updates main non-free contrib' >> /etc/apt/sources.list && \
echo 'deb-src http://mirrors.163.com/debian/ stretch main non-free contrib' >> /etc/apt/sources.list && \
echo 'deb-src http://mirrors.163.com/debian/ stretch-updates main non-free contrib' >> /etc/apt/sources.list && \
echo 'deb-src http://mirrors.163.com/debian/ stretch-backports main non-free contrib' >> /etc/apt/sources.list && \
echo 'deb-src http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib' >> /etc/apt/sources.list && \
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
apt-get update && \
apt-get install -y wget vim net-tools unzip libjemalloc-dev && \
apt-get build-dep -y nginx
RUN \
cd /usr/local/src/ && \
wget -c http://nginx.org/download/nginx-1.14.2.tar.gz && \
wget -c https://www.openssl.org/source/old/1.0.2/openssl-1.0.2m.tar.gz && \
wget -c https://github.com/simplresty/ngx_devel_kit/archive/v0.3.1rc1.tar.gz && \
wget -c https://github.com/openresty/lua-nginx-module/archive/v0.10.11.tar.gz && \
wget -c https://github.com/xiaokai-wang/nginx_upstream_check_module/archive/master.zip -O nginx_upstream_check_module.zip && \
wget -c https://github.com/weibocom/nginx-upsync-module/archive/master.zip -O nginx-upsync-module.zip && \
tar zxf ./nginx-1.14.2.tar.gz && rm nginx-1.14.2.tar.gz && \
tar zxf ./openssl-1.0.2m.tar.gz && rm openssl-1.0.2m.tar.gz && \
tar zxf ./v0.3.1rc1.tar.gz && rm v0.3.1rc1.tar.gz && \
tar zxf ./v0.10.11.tar.gz && rm v0.10.11.tar.gz && \
unzip ./nginx_upstream_check_module.zip && rm nginx_upstream_check_module.zip && \
unzip ./nginx-upsync-module.zip && rm nginx-upsync-module.zip
RUN \
cd /usr/local/src/nginx-1.14.2 &&\
patch -p1 < /usr/local/src/nginx_upstream_check_module-master/check_1.12.1+.patch &&\
./configure \
--prefix=/webserver/nginx \
--user=www --group=www --with-pcre \
--with-stream \
--with-http_v2_module \
--with-http_ssl_module \
--with-ld-opt=-ljemalloc \
--with-http_realip_module \
--with-http_gzip_static_module \
--with-http_stub_status_module \
--http-log-path=/logs/nginx/access.log \
--error-log-path=/logs/nginx/error.log \
--with-openssl=/usr/local/src/openssl-1.0.2m \
--add-module=/usr/local/src/ngx_devel_kit-0.3.1rc1 \
--add-module=/usr/local/src/lua-nginx-module-0.10.11 \
--add-module=/usr/local/src/nginx_upstream_check_module-master \
--add-module=/usr/local/src/nginx-upsync-module-master && \
make && \
make install
upstream測試服務
此處咱們能夠寫個http的demo服務也行，或者更簡單點，也能夠直接使用Docker運行兩個服務.此處咱們向consul集羣中註冊兩個已有的服務
curl -X PUT -d '{"weight":1, "max_fails":2, "fail_timeout":10, "down":0}' http://192.168.99.12:8510/v1/kv/upstreams/app/192.168.99.12:9000
curl -X PUT -d '{"weight":1, "max_fails":2, "fail_timeout":10, "down":0}' http://192.168.99.12:8510/v1/kv/upstreams/app/192.168.99.12:9001
執行完以後在Dashboard中的k/v處便可看到
準備Nginx動態更新的配置文件
咱們這裏做爲功能測試實驗，因此準備一個簡潔的nignx.conf配置文件
-
nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream app {
upsync 192.168.99.12:8510/v1/kv/upstreams/app/ upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
upsync_dump_path /webserver/nginx/conf/app.conf; # 當consul故障時候，就能夠把此做爲備份配置文件
include /webserver/nginx/conf/app.conf; # 準備一個兼容的nginx測試文件，若是沒有第一次啓動可能起不來
check interval=1000 rise=2 fall=2 timeout=3000 type=http default_down=false;
check_http_send "HEAD / HTTP/1.0\r\n\r\n";
check_http_expect_alive http_2xx http_3xx;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://app;
}
location /upstream_list {
upstream_show;
}
location /upstream_status {
check_status;
access_log off;
}
}
}
-
app.conf
server 127.0.0.1:80 weight=1 max_fails=2 fail_timeout=10s down;
server 127.0.0.1:81 weight=1 max_fails=2 fail_timeout=10s down;
準備完成以後，咱們就能夠啓動nginx服務了
docker run -d --name nginx-consul -v /Users/marionxue/Downloads/webserver/nginx.conf:/webserver/nginx/conf/nginx.conf -v /Users/marionxue/Downloads/webserver/app.conf:/webserver/nginx/conf/app.conf -p 80:80 -p 443:443 nginx:upsync
容器運行正常後，就能夠經過http://127.0.0.1/upstream_list查看upstream主機了，此時app.conf的內容也會被動態的更新，這裏能夠做爲consul故障後的備份配置文件
容器啓動後的app.conf的內容
root@be1b245c1c47:/webserver/nginx/sbin# cat ../conf/app.conf
server 192.168.99.12:9001 weight=1 max_fails=2 fail_timeout=10s;
server 192.168.99.12:9000 weight=1 max_fails=2 fail_timeout=10s;
此處經過curl增長一個新的主機節點觀察是否會自動的更新
> curl -X PUT -d '{"weight":1, "max_fails":2, "fail_timeout":10, "down":1}' http://192.168.99.12:8510/v1/kv/upstreams/app/192.168.99.12:8510
true
這和咱們預期的同樣，不須要干預nginx服務，便可自動完成nginx配置的更新。此處即基本上完成nginx實現動態負載均衡的基礎實驗。
本文分享自微信公衆號 - 雲原生生態圈（CloudNativeEcoSystem）。
若有侵權，請聯繫 support@oschina.cn 刪除。
本文參與「OSC源創計劃」，歡迎正在閱讀的你也加入，一塊兒分享。