Nginx+Keepalived搭建高可用負載均衡集羣

Nginx+Keepalived搭建高可用負載均衡集羣

 

一。 環境說明

前端雙Nginx+keepalived,nginx反向代理到後端的tomcat集羣實現負載均衡，Keepalived實現集羣高可用.

• 操做系統： Centos 6.6_X64

• Nginx版本： nginx-1.9.5

• Keepalived版本：keepalived-1.2.13  

結構:

  Keepalived+nginx-MASTER：10.6.1.210
         Keepalived+nginx-BACKUP:  10.6.1.211
         VIP: 10.6.1.1.214
         Realsever:10.6.1.212,10.6.1.213  

---

2、安裝

    前端兩臺主機分別安裝nginx和keepalived。

1.    安裝keepalived

Yum install keepalived –y

默認安裝目錄: /etc/keepalived/

配置文件:/etc/keepalived/keepalived.conf

 

2.    安裝Nginx

1) 安裝nginx所依賴的包.

yum install gcc gcc-c++ autoconf automake zlib zlib-developenssl openssl-devel pcre pcre-deve –y

2）Nginx官網下載安裝包 (http://nginx.org/en/download.html)

此處版本選擇nginx-1.9.5.tar.gz

 

3）編譯安裝

tar –zxvf nginx-1.9.5.tar.gz

cd nginx-1.9.5

./configure

Make && make install

完成

 

3、配置

1）前端兩臺主機nginx的配置徹底同樣

Vim /usr/local/nginx/conf/nginx.conf

upstream tomcat_8080 {

        server10.6.1.212:8080;

        server10.6.1.213:8080;

        ip_hash;

        }

 

 

    server {

        listen       80;

       server_name  test;

 

        #charsetkoi8-r;

 

       #access_log logs/host.access.log  main;

 

        location /{

        proxy_passhttp://tomcat_8080;

 

        }

 

 

2）nginx_master的keepalived配置

Vim /etc/keepalived/keepalived.conf

global_defs {

   notification_email {

     sxl_youcun@qq.com

   }

   notification_email_from sxl_youcun@qq.com

   smtp_server 127.0.0.1

#   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

 

vrrp_scriptcheck_nginx {

 #檢查nginx狀態的腳本,後面會介紹.同理mysql也能夠這樣作。

    script"/etc/keepalived/check_nginx.sh"

    # 執行間隔2秒

    interval 2

}

 

#VIP1

vrrp_instance VI_1{

    state BACKUP      #兩臺主機都是BACKUP

    interface eth0

#同一keepalived集羣的virtual_router_id必須相同，默認51   

virtual_router_id 51

    priority 100  #主的優先級高

    advert_int 1

#不搶佔：若是集羣裏已存在MASTER狀態的主機，即便優先級高於MASTER也不搶佔爲MASTER。只在優先級高的主機上設置便可。

    nopreempt

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

#虛擬IP

        10.6.1.214

    }

    track_script {

        check_nginx

    }

}

3）nginx_backup的keepalived配置

global_defs {

   notification_email {

     sxl_youcun@qq.com

   }

   notification_email_from sxl_youcun@qq.com

   smtp_server 127.0.0.1

#   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

 

vrrp_scriptcheck_nginx {

    script"/etc/keepalived/check_nginx.sh"

    # 執行間隔2秒

    interval 2

}

 

#VIP1

vrrp_instance VI_1{

    state BACKUP

    interface eth0

    virtual_router_id 51

    priority 80

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        10.6.1.214

    }

    track_script {

        check_nginx

    }

}

4）防火牆設置

Iptables -I INPUT -d 224.0.0.18 -j ACCEPT

或者直接關閉防火牆service iptables stop

    VRRP報文是經過IP多播形式發送的，組播地址224.0.0.18是VRRP報文的目的地址。
    兩個主機都是BACKUP，若是同時啓動keepalived，VRRP協議經過競選使優先級高的主機作爲MASTER。若是防火牆沒有容許VRRP報文經過的話，兩個BACKUP都會成爲MASTER，你會發現兩個主機都啓動了虛擬IP。
    5）nginx狀態檢查腳本check_nginx.sh

 Vim /etc/keepalived/check_nginx.sh

#!/bin/bash

#This script is used to check_nginx status for keepalived

KK=`ps -ef |grep "nginx: master process" | grep -v grep | wc -l`

if [ $KK -eq 0 ]

   then

echo "`date`nginx is dead,prepare to startnginx">>/usr/local/nginx/logs/nginx_status.log 2>&1

  /usr/local/nginx/sbin/nginx_start.sh

   sleep 5

   else

   echo "`date` nginx isOK">>/usr/local/nginx/logs/nginx_status.log 2>&1

fi

    6）開啓keepalived的日誌

    編輯/etc/sysconfig/keepalived：

1	KEEPALIVED_OPTIONS="-D -d -S 0"

編輯/etc/rsyslog.conf：

# 配置文件最後面加上下面一行

local0.*                /var/log/keepalived.log

 

重啓rsyslog：

service rsyslog restart

按上面配置後，keepalived會把日誌記錄到/var/log/keepalived.log。

 

7）啓動服務

設置nginx環境變量。

+++++++++++++++++++++++++++++++++++++++++++++

NGINX=/usr/local/nginx

PATH=$PATH:$HOME/bin:$NGINX/sbin

export NGINX PATH

++++++++++++++++++++++++++++++++++++++++++++++

 

啓動nginx, 用腳本啓動, 腳本以下：

+++++++++++++++++++++++++++++++++++++++++++

#!/bin/bash

. $HOME/.bash_profile

nohup /usr/local/nginx/sbin/nginx -c/usr/local/nginx/conf/nginx.conf >/dev/null 2>&1 &

+++++++++++++++++++++++++++++++++++++++++++++++++

啓動keepalived

service keepalived start

 

    4、驗證

1. nginx_master和nginx_slave同時啓動keepalived，觀察日誌/var/log/keepalived.log，你會發現nginx_master搶佔爲MASTER，綁定了虛擬IP。

    nginx_master：

 

    nginx_backup：

    

2. 把nginx_master的keepalived服務停掉或者重啓系統，同時不斷的ping虛擬IP。會出現短暫的中斷或者時延較大，虛擬IP會漂移到nginx_backup上面：

    nginx_backup：

注意觀察在主備切換時nginx_backup日誌keepalived.log的變化，以下：

VRRP_Instance(VI_1) Transition to MASTER STATE

VRRP_Instance(VI_1) Entering MASTER STATE

VRRP_Instance(VI_1) setting protocol VIPs.

VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.6.1.214

Netlink reflector reports IP 10.6.1.214 added

VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.6.1.214   

 

3. 最後把nginx_master的keepalived服務開啓，虛擬IP並無漂移回到nginx_master，這是由於nginx_master開啓了不搶佔模式，即便優先級高，也不會搶佔MASTER。