Docker+Nginx+Keepalived實現架構高可用

1、背景

經過keepalived實現nginx高可用，因爲在家不想弄多臺主機來搞，因此將運行環境用docker封裝來模擬跨主機

docker基礎鏡像：centos

 

說以前，簡單介紹一下：

Keepalived是基於vrrp協議的一款高可用軟件。Keepailived有一臺主服務器和多臺備份服務器，在主服務器和備份服務器上面部署相同的服務配置，使用一個虛擬IP地址對外提供服務，當主服務器出現故障時，虛擬IP地址會自動漂移到備份服務器。

 

雙機高可用方法目前分爲兩種：

• 雙機主從（也叫雙機熱備）
• 雙機主主 （也叫雙機互備）

下述介紹，高可用中的雙機主從模式，雙機主主模式，主要是keepalived.conf配置會有所不一樣，方法都是同樣。

2、具體操做

一、安裝centos 鏡像

docker pull centos

說明：經過用centos鏡像來安裝高可用所須要的全部環境，再啓兩個容器，再真實模擬跨主機的場景

 

二、在centos上安裝所需環境（nginx和其它工具）

運行centos容器

docker run -it centos /bash/bin

 

安裝依賴和所須要的包　　

#使用yum安裝nginx須要包括Nginx的庫，安裝Nginx的庫

rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

# 使用下面命令安裝nginx
#yum install nginx

#安裝網絡包(須要使用ifconfig和ping命令)
yum install net-tools

#安裝vim
yum install vim

 

三、在centos安裝keepalvied

#安裝keepalived環境依賴

yum install -y gcc openssl-devel popt-devel

#安裝keepalived

經過yum install keepalived



#或者經過源碼安裝

wget http://124.205.69.132/files/90630000053A2BB4/www.keepalived.org/software/keepalived-1.3.4.tar.gz

tar zxvf keepalived-1.3.4.tar.gz 

cd keepalived-1.3.4

./configure --prefix=/usr/local/keepalived

make && make install



拷貝幾個文件到CentOS7環境中：

cp keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

 

四、修改/etc/keepalived/keepalived.conf文件

! Configuration File for keepalived
global_defs {
    notification_email {
    	762357658@qq.com
    }
    notification_email_from itsection@example.com
    smtp_server mail.example.com
    smtp_connect_timeout 30
    router_id LVS_DEVEL  
}


vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh"  
    interval 2
    weight -5
    fall 3
    rise 2
}


vrrp_instance VI_1 {
    state MASTER   
    interface eth0   
    virtual_router_id 2  
    priority 101  
    advert_int 2
    authentication {
        auth_type PASS  
        auth_pass 1111
    }
    virtual_ipaddress {
        172.17.0.210   
    }
    track_script {
       chk_nginx 
    }

}

　　

/etc/keepalived/check_nginx.sh文件

A=`ps -ef | grep nginx | grep -v grep | wc -l`
if [ $A -eq 0 ];then
    nginx
    sleep 2
    if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then
        #killall keepalived
        ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9 
    fi

fi　　

 

再對check_nginx.sh賦於執行權限：

chmod +x check_nginx.sh

 

注：keepalived是經過檢測keepalived進程是否存在判斷服務器是否宕機，若是keepalived進程在可是nginx進程不在了那麼keepalived是不會作主備切換，因此咱們須要寫個腳原本監控nginx進程是否存在，若是nginx不存在就將keepalived進程殺掉。

在主nginx上須要編寫nginx進程檢測腳本（check_nginx.sh），判斷nginx進程是否存在，若是nginx不存在就將keepalived進程殺掉，並將vip漂移到備份機器上

 

五、設置開機啓動

chkconfig keepalived on

或者

systemctl enable keepalived.service  設置開機自動啓動

啓動keepalived服務：

systemctl start keepalived.service 啓動

 

六、安裝全部須要的依賴和環境後，將容器新增的內容從新提交

docker commit 5d112 centos_keepalived_nginx:v1

注：5d112爲，上述安裝軟件所對應的容器id

 

六、啓動含有(keepalived+nginx)的容器

docker run --privileged  -tid --name  keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init

 

 

進入keepalived_master容器：

docker exec -it keepalived_master bash

進入/usr/share/nginx/html，修改index.html文件

修改標題爲:

Welcome to nginx Master!

七、啓動keepalived_salve容器

#啓動一個容器

docker run --privileged  -tid --name  keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init

#進入容器

docker exec -it keepalived_slave bash

 

八、修改keepalived_salve容器中nginx index.html文件

vim /usr/share/nginx/html/index.html

 

 

修改標題爲:

Welcome to nginx Slave!

 

九、修改keepalived_salve容器中keepalived.conf文件 (master容器中，保持和鏡像中設置同樣便可，不須要更改)

! Configuration File for keepalived
global_defs {
notification_email {
762357658@qq.com
}
notification_email_from itsection@example.com
smtp_server mail.example.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}

vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval 2
weight -5
fall 3
rise 2
}


vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 2
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.17.0.210
}
track_script {
chk_nginx
}

}

其實，由配置中能夠看出，主要是state和priority兩個參數的調整，其中master節點的priority值必定要比backup大才行！

 

原理說明：
一、 經過vrrp協議廣播，每一個keepalived vrrp都去爭取master
二、 以virtual_router_id爲組隊標識。  同爲一個vip服務的keepalived的virtual_router_id要保持相同

三、 以priority 爲權值，同一個virtual_router_id下那個priority大那個就是master,其它爲backup

 

 

改完以後，從新加載

systemctl daemon-reload 
systemctl restart keepalived.service

 

十、驗證

查看兩個容器中keepalived服務狀態

systemctl status keepalived.service 

keepalived_master服務狀態效果：

keepalived_slave服務狀態效果圖：

 

能夠看到，keepalived服務運行正常

 

啓動nginx: nginx

 

在master容器中 curl 172.17.0.210

 

在slave容器中 curl 172.17.0.210:

能夠看現，此時master和slave容器兩邊經過虛擬vip : 172.17.0.210 訪問nginx數據，請求返回的數據都是master容器中nginx配置的數據： welcome to nginx master

 

繼續驗證，關掉master容器的keepalived服務：

 

 驗證獲得的結果是當master容器中的keepalived服務關掉後，curl 172.17.0.210請求返回的數據來自slave，welcome to nginx slave

 

再繼續驗證，把關掉master容器的keepalived服務再開啓：

 

能夠看到，當master容器中的keepalived服務開啓後，請求返回的數據會再次轉到master中。

 

到此，全部的驗證和預期的一致，也達到咱們藉助docker爲基礎來實現了整套基於Nginx+Keepalived高可用的方案了。

 

3、Keepalived服務命令

• systemctl daemon-reload  從新加載
• systemctl enable keepalived.service  設置開機自動啓動
• systemctl disable keepalived.service 取消開機自動啓動
• systemctl start keepalived.service 啓動
• systemctl stop keepalived.service中止
• systemctl status keepalived.service  查看服務狀態