centos7中keepalived+nginx作雙機熱備和反向代理

時間 2019-11-17

標籤 centos7 centos keepalived+nginx keepalived nginx 反向代理欄目 Tomcat 简体版

原文原文鏈接

我虛擬了三臺centos7，要將one（192.168.56.31）、two（192.168.56.32）配置成nginx+keepalived雙機熱備（思路是當一臺服務器的nginx掛掉時，能夠自動用另外一臺的nginx,就是當主服務器nginx的端口不通時，自動關閉本身的keepalived服務，這樣虛擬ip就會指到備用ip上），three（192.168.56.33）作tomcat服務器。實驗環境下先把防火牆和selinux關閉：setenforce 0 && systemctl stop firewalld
首先進行nginx的安裝，我用的是最簡單的yum安裝方式，自動的yum源中沒有nginx，咱們須要安裝一下yum源而後再來安裝nginx：yum -y install epel-release && yum -y install nginx。而後再來安裝一下keepalived：yum -y install keepalived。安裝完成後可使用keepalived --help 來進行keepalived的命令幫助。二者的配置文件都在etc目錄下。再將jdk和tomcat上傳到three中並安裝，我這設置了三個tomcat實例，端口分別爲：8180、8280、8380。
而後先來配置nginx，配置文件在/etc/nginx/中：html

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;                          #運行用戶，避免權限問題，就用root吧
worker_processes auto;               #工做進程數，表示自動選擇，cpu多的話，能夠手動設置通常爲cpu的倍數。
error_log /var/log/nginx/error.log;  #錯誤日誌路徑
pid /run/nginx.pid;                  #PID文件路徑

# Load dynamic modules. See /usr/share/nginx/README.dynamic.  
include /usr/share/nginx/modules/*.conf;   #包含這個文件的信息

events {       
    worker_connections 1024;         #工做鏈接數，指一個進程能夠產生多少個鏈接，用 ulimit -n 來查看，可是這是理論上，不必設置這麼大
}

http {     
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '   #定義訪問日誌的顯示格式
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;                                #訪問日誌的路徑

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;                                 #將指定的配置文件引入到這個文件中來
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

    server {                                          #服務器設置，能夠設置多個服務器
        listen       80 default_server;               #設置監聽的端口
        listen       [::]:80 default_server;            
        server_name  _;                               #服務器名稱
        root         /usr/share/nginx/html;           #服務器默認網站的目錄

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {                                  #設置請求轉向
        proxy_pass http://cs;
        proxy_set_header HOST $host;                   #不添加下面三行時，有次轉發後點擊鏈接前面域名會變爲cs
        proxy_set_header X-Real-IP $remote_addr; 
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
        
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
upstream cs {                      #這是一個反向鏈接池,指向了三個tomcat實例
 server 192.168.56.33:8180;
 server 192.168.56.33:8280;
 server 192.168.56.33:8380;
}
# Settings for a TLS enabled server.
#
#    server {
#        listen       443 ssl http2 default_server;
#        listen       [::]:443 ssl http2 default_server;
#        server_name  _;
#        root         /usr/share/nginx/html;
#
#        ssl_certificate "/etc/pki/nginx/server.crt";
#        ssl_certificate_key "/etc/pki/nginx/private/server.key";
#        ssl_session_cache shared:SSL:1m;
#        ssl_session_timeout  10m;
#        ssl_ciphers HIGH:!aNULL:!MD5;
#        ssl_prefer_server_ciphers on;
#
#        # Load configuration files for the default server block.
#        include /etc/nginx/default.d/*.conf;
#
#        location / {
#        }
#
#        error_page 404 /404.html;
#            location = /40x.html {
#        }
#
#        error_page 500 502 503 504 /50x.html;
#            location = /50x.html {
#        }
#    }

}

配置完成後把，nginx和三個tomcat服務起來後，就能成功訪問了，接下來咱們再來配置一下keepalived，配置文件keeplived.conf在安裝好後的/etc/keeplived中，下面是配置文件的解釋：node

! Configuration File for keepalived

global_defs {                                                       #全局定義塊
   notification_email {                                             #指定keepalived在發生切換時須要發送email到的對象，一行一個。
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc            #發件人是誰
   smtp_server 192.168.200.1                                        #指定smtp服務器地址                              
   smtp_connect_timeout 30                                          #指定smtp鏈接超時時間
   router_id LVS_DEVEL                                              #Lvs負載均衡器標識（lvs_id）。在一個網絡內，它應該是惟一的。
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {                                               #虛擬vrrp設置模塊                                        
    state MASTER                                                   #指定那個爲master，那個爲backup，若是設置了nopreempt這個值不起做用，主備靠priority決定。
    interface eth0                                                 #設置實例綁定的網卡
    virtual_router_id 51                                           #虛擬路由的id
    priority 100                                                   #優先級，高優先級競選爲master
    advert_int 1                                                   #檢查間隔，默認1秒
    authentication {                                               #認證設置
        auth_type PASS                                             #認證方式
        auth_pass 1111                                             #認證密碼
    }
    virtual_ipaddress {                                            #虛擬ip地址設置
        192.168.200.16
        192.168.200.17
        192.168.200.18
    }
}

#虛擬服務器virtual_server定義塊 ，該部分是用來管理LVS的，是實現keepalive和LVS相結合的模塊。ipvsadm命令能夠實現的管理在這裏均可以經過參數配置實現。
virtual_server 192.168.200.100 443 {                             #虛擬IP地址，要和vrrp_instance模塊中的virtual_ipaddress地址一致
    delay_loop 6                                                 #健康檢查時間間隔          
    lb_algo rr                                                   #lvs調度算法有rr|wrr|lc|wlc|lblc|sh|dh 
    lb_kind NAT                                                  #負載均衡轉發規則NAT|DR|RUN                                                       
    persistence_timeout 50                                       #會話保持時間
    protocol TCP                                                 #使用的協議

    real_server 192.168.201.100 443 {                            #真實ip地址
        weight 1                                                 #默認爲1,0爲失效
        SSL_GET {
            url {
              path /
              digest ff20ad2481f97b1754ef3e12ecd3a9cc
            }
            url {
              path /mrtg/
              digest 9b3a0c85a887a256d6939da88aabd8cd
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

virtual_server 10.10.10.2 1358 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP

    sorry_server 192.168.200.200 1358

    real_server 192.168.200.2 1358 {
        weight 1
        HTTP_GET {
            url {
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url {
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url {
              path /testurl3/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.200.3 1358 {
        weight 1
        HTTP_GET {
            url {
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334c
            }
            url {
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334c
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

virtual_server 10.10.10.3 1358 {
    delay_loop 3
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP

    real_server 192.168.200.4 1358 {
        weight 1
        HTTP_GET {
            url {
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url {
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url {
              path /testurl3/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.200.5 1358 {
        weight 1
        HTTP_GET {
            url {
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url {
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url {
              path /testurl3/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

配置文件有點多，咱們用不到的功能能夠不往上配，這樣能使性能更好！個人腳本以下：linux

global_defs {
   notification_email {
     652179279@qq.com
   }
   notification_email_from
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id one
}
vrrp_script chk_nginx {                  #由於要檢測nginx服務狀態，因此建立一個檢查腳本
    script "/usr/local/check_ng.sh"
    interval 3
}
vrrp_instance VI_1 {
    state MASTER
    interface enp0s8
    virtual_router_id 60
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

    virtual_ipaddress {
        192.168.56.100
    }
    track_script {
        chk_nginx
    }
}

接下來就是去上面配置的路徑建立腳本了vi /usr/local/check_ng.sh ：nginx

#!/bin/bash
d=`date --date today +%Y%m%d_%H:%M:%S`
n=`ps -C nginx --no-heading|wc -l`
if [ $n -eq "0" ]; then
        systemctl start nginx
        n2=`ps -C nginx --no-heading|wc -l`
        if [ $n2 -eq "0"  ]; then
                echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
                systemctl stop keepalived
        fi
fi

而後增長執行權限。再配置備用機：算法

global_defs {
   notification_email {
     652179279@qq.com
   }
   notification_email_from
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id two
}
vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"
    interval 3
}
vrrp_instance VI_1 {
    state BACKUP
    interface enp0s8
    virtual_router_id 60
    priority 90
    advert_int 1
    authentication {                                               
        auth_type PASS              
        auth_pass 1111                                             
    }
    
    virtual_ipaddress {
        192.168.56.100
    }
    track_script {
        chk_nginx
    }
}

配置檢測腳本和配置主機同樣！生產環境下須要開放112端口，是keepalived的！centos