Nginx+Keepalived雙主輪詢負載均衡

將兩臺LBServer（Nginx）分別添加VIP1與VIP2（Keepalived心跳監測），兩臺LBServer互爲主從，共同提供服務，提升資源利用率。同時使用DNS管理系統（如Dnspod）作多條A記錄解析至對應的VIP1與VIP2，實現對兩臺Nginx的輪詢(負載均衡)，當一臺LBServer故障時，VIP會轉移到另外一臺繼續提供服務，這裏配置Nginx+keepalived雙主高可用。

系統

兩臺Nginx

CentOS6.7 x86_64

兩臺Web

Ubuntu15.04 desktop

IP地址

Nginx1:192.168.15.132

Nginx2:192.168.15.133

VIP1:192.168.15.135

VIP2:192.168.15.136

Real1:192.168.15.128

Real2:192.168.15.130

部署整個環境用到的軟件爲:

nginx-1.6.3.tar.gz  
prce-8.38.tar.gz 

zlib-1.2.8.tar.gz

①2臺Web主機（Ubuntu）上部署環境，安裝Nginx+PHP-FPM+MySQL

②分別在二臺Nginx負載均衡器上安裝Nginx，配置

安裝GCC編譯器等工具：

yum install -y gcc gcc-c++ autoconf automake libtool make openssl openssl-devel

安裝Nginx:

wget http://exim.mirror.fr/pcre/pcre-8.38.tar.gz

tar -zxvf pcre-8.38.tar.gz

cd pcre-8.38

./configure

make && make install

wget http://zlib.net/zlib-1.2.8.tar.gz

tar -zxvf zlib-1.2.8.tar.gz

cd zlib-1.2.8

./configure

make && make install

wget http://nginx.org/download/nginx-1.6.3.tar.gz

tar -zxvf nginx-1.6.3.tar.gz 
cd nginx-1.6.3/  

./configure --prefix=/usr/local/nginx

--sbin-path=/usr/local/nginx/sbin/nginx

--conf-path=/usr/local/nginx/conf/nginx.conf

--pid-path=/usr/local/nginx/logs/nginx.pid \

--with-http_ssl_module \

--with-http_stub_status_module \

--with-http_gzip_static_module \ 

make && make install 

注：查詢"./configure --help"相關模塊，按需求指定啓用

Nginx.conf配置文件，二個nginx負載均衡器的文件同樣

user  www-data www-data;

worker_processes  1;

 

error_log  /usr/local/nginx/logs/error.log notice;

pid        /usr/local/nginx/logs/nginx.pid;

 

worker_rlimit_nofile 51200;

 

events {

    use epoll;

    worker_connections  51200;

}

 

http {

    include       mime.types;

    default_type  application/octet-stream;

 

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                      '$status $body_bytes_sent "$http_referer" '

                      '"$http_user_agent" "$http_x_forwarded_for"';

 

    access_log  logs/access.log  main;

 

    server_names_hash_bucket_size 128;

    client_header_buffer_size 32k;

    large_client_header_buffers 4 32k;

    client_max_body_size 8m;

    sendfile        on;

    tcp_nopush     on;

    server_tokens off;

    keepalive_timeout  60;

    fastcgi_connect_timeout 300;

    fastcgi_send_timeout 300;

    fastcgi_read_timeout 300;

    fastcgi_buffer_size 64k;

    fastcgi_buffers 4 64k;

    fastcgi_busy_buffers_size 128k;

    fastcgi_temp_file_write_size 128k;

    gzip  on;

 

    upstream backend

    {

    server 192.168.15.128;

    server 192.168.15.130;

    }

 

    server {

        listen       80;

        server_name  192.168.15.135;  #Nginx2改成192.168.15.136

 

        location / {

            root   html;

            index  index.php index.html index.htm;

            proxy_redirect off;

            proxy_set_header Host $host;

            proxy_set_header X-Real-IP $remote_addr;

            #後端的Web服務器能夠經過X-Forwarded-For獲取用戶真實IP

            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_pass http://backend;

        }

 

        error_page   500 502 503 504  /50x.html;

        location = /50x.html {

            root   html;

        }

        

        location /nginx_status {

            stub_status on; 

            auth_basic "NginxStatus";

            auth_basic_user_file /usr/local/nginx/htpasswd;

            #allow 127.0.0.1;

            #deny all;

        }

        location ~* \.(ini|docx|txt|doc|pdf)$ {

        #禁止訪問文檔性文件

        root /usr/share/nginx/html;

        deny all;

        }

        location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|html|htm|css)$ {

        root /home/image;

        proxy_store on;

        proxy_store_access user:rw group:rw all:rw;

        proxy_temp_path /home/image;

        if ( !-e $request_filename) {

           proxy_pass  http://backend;

        }

    }

}

}

③在二臺Nginx上安裝及配置keepalived：

wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz 

tar -zxvf keepalived-1.2.15.tar.gz

cd keepalived-1.2.15

./configure --sysconf=/etc/  --with-kernel-dir=/usr/src/kernels/2.6.32-573.8.1.el6.x86_64

make && make install

ln -s /usr/local/sbin/keepalived  /sbin/  

#這一步很重要，不執行ln -s會報錯「Starting keepalived: /bin/bash: keepalived: command not found」

service keepalived start

二臺Nginx上keepalived.conf配置以下，配置完成後分別service keepalived start啓動，檢測keepalived配置是否成功

Nginx1：

global_defs {

   notification_email {

     test@163.com

   }

   notification_email_from keepalived@localhost

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id NGINX_VIP1

}

 

vrrp_script chk_http_port {

script "/usr/local/src/check_nginx_pid.sh"

interval 2                           #（檢測腳本執行的間隔）

weight 2

}

 

vrrp_instance VI_1 {

    state MASTER

    interface bond0

    virtual_router_id 51

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

}

 

track_script {

    chk_http_port            #（調用檢測腳本）

}

    virtual_ipaddress {

 

    192.168.15.135/24 broadcast 192.168.15.255 dev bond0 label bond0:1

}

}

 

 

vrrp_instance VI_2 {

    state BACKUP

    interface bond0

    virtual_router_id 52

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

}

 

track_script {

     chk_http_port            #（調用檢測腳本）

}

 

    virtual_ipaddress {

    192.168.15.136/24 broadcast 192.168.15.255 dev bond0 label bond0:2

    }

} 

 

Nginx2： 

global_defs {

   notification_email {

     test@163.com

   }

   notification_email_from keepalived@localhost

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id NGINX_VIP2

}

 

vrrp_script chk_http_port {

script "/usr/local/src/check_nginx_pid.sh"

interval 2                           #（檢測腳本執行的間隔）

weight 2

}

 

vrrp_instance VI_1 {

    state BACKUP

    interface bond0

    virtual_router_id 51

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

}

track_script {

        chk_http_port            #（調用檢測腳本）

}

 

    virtual_ipaddress {

 

 

    192.168.15.135/24 broadcast 192.168.15.255 dev bond0 label bond0:1

}

}

 

 

vrrp_instance VI_2 {

    state MASTER

    interface bond0

    virtual_router_id 52

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

}

 

track_script {

       chk_http_port            #（調用檢測腳本）

}

 

    virtual_ipaddress {

 

    192.168.15.136/24 broadcast 192.168.15.255 dev bond0 label bond0:2

    }

} 

如下是針對nginx狀態進行檢測的腳本，第一次nginx服務死掉時會從新啓動，若是Nginx沒法正常啓動，則殺掉keepalived進程

vim  /usr/local/src/check_nginx_pid.sh

#!/bin/bash

A=`ps -C nginx --no-header |wc -l`        

if [ $A -eq 0 ];then                            

      /usr/local/nginx/sbin/nginx        

      if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then  

              killall keepalived                    

      fi

fi

配置完畢，測試停掉其中一臺的任何服務，不影響整個系統的運做。經過DNS管理系統輪詢VIP1與VIP2，實現Nginx的負載均衡｡