LVS+DR模式+keepalived

時間 2019-11-30

原文原文鏈接

LVS+DR模式+keepalived

上一篇博客咱們說了三種模式，NAT,TUN,DR此次是DR+Keepalived

先了解什麼是Keepalived

keepalived採用VRRP熱備份協議實現Linux服務器的多機熱備功能
VRRP,虛擬路由冗雜協議，是針對路由器的一種備份解決方案html

keepalived能夠實現多機熱備，每一個熱備組可有多臺服務器，最經常使用的就是雙機熱備
雙擊熱備的故障切換是由虛擬IP地址的飄逸來實現，適用於各類應用服務器前端

LVS-DR的ARP問題

在LVS-DR的負載均衡羣集中，負載均衡器與節點服務器都要配置相同的VIP地址
在局域網中具備相同的IP地址，勢必會形成服務器APR通訊的紊亂
當一個ARP廣播發送到LVS-DR集羣時，由於負載均衡器和節點服務器都是鏈接到相同的網絡上，它們都會接收到ARP廣播
此時只有前端的負載均衡器進行響應。其餘節點服務器不該該響應ARP廣播

LVS-DR的ARP問題解決方案

對節點服務器進行處理，使其不響應針對VIP的ARP請求
使用虛接口lo:0承載VIP地址
設置內核參數arp_ignore=1:系統只響應目的IP爲本地IP的ARP請求

下面咱們來作實驗

實驗規劃

咱們須要五臺虛擬機
DR1 主服務器 192.168.100.201
DR4 備份服務器 192.168.100.202
web 5 192.168.100.221
web 7 192.168.100.222
vip 192.168.100.10
clent 192.168.100.50web

把須要的LVS模塊，keepalived,http分別裝好

#1，4
[root@localhost ~]# yum install ipvsadm keepalived -y
#5，7
[root@localhost ~]# yum install httpd -y

第一臺主負載均衡調度器選擇僅主機模式，配置網卡

[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 
BOOTPROTO=static #靜態
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=849aa04e-1874-490f-8cb0-b2fde4b9a6f8
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.201 #地址
NETMASK=255.255.255.0 #子網 掩碼
GATEWAY=192.168.100.1 #網關
[root@localhost ~]# systemctl restart network #ch重啓網絡服務
[root@localhost ~]# ifconfig #查看固定地址是否生效

第二臺4備份負載均衡調度器，綁定僅主機模式，配置網卡

[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 

BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=c3f0a196-6819-4702-9b54-7cad18402591
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.202
NETMASK=255.255.255.0
GATEWAY=192.168.100.1
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig

第三臺5web服務器綁定僅主機模式，配置網卡

[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 

BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=a6cf69fe-eb42-4a99-9239-0da4cdeae0c7
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.221
NETMASK=255.255.255.0
GATEWAY=192.168.100.1
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig

第四臺7web服務器綁定僅主機模式，配置網卡

[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=447e510f-fea3-4b6c-8f47-d0c6319ead28
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.222
NETMASK=255.255.255.0
GATEWAY=192.168.100.1
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig

配置第一臺1主服務器

[root@localhost ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward=1
#proc響應關閉重定向功能
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
[root@localhost ~]# sysctl -p #生效
net.ipv4.ip_forward = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0

建立虛擬網卡

[root@localhost ~]# cd /etc/sysconfig/network-scripts/

[root@localhost network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33:0
[root@localhost network-scripts]# vim ifcfg-ens33:0
100dd

DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.100.10
NETMASK=255.255.255.0

[root@localhost network-scripts]# ifup ens33:0 #開啓網卡，這個網關是用來作虛擬IP的，至關於一給訪問入口

[root@localhost network-scripts]# ifconfig
[root@localhost network-scripts]# cd /etc/init.d/
[root@localhost init.d]# vim dr.sh
#!/bin/bash
GW=192.168.100.1
VIP=192.168.100.10
RIP1=192.168.100.221
RIP2=192.168.100.222
case "$1" in
start)
        /sbin/ipvsadm --save > /etc/sysconfig/ipvsadm
        systemctl start ipvsadm
        /sbin/ifconfig ens33:0 $VIP broadcast $VIP netmask 255.255.255.255 broadcast $VIP up
        /sbin/route add -host $VIP dev ens33:0
        /sbin/ipvsadm -A -t $VIP:80 -s rr
        /sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
        /sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
        echo "ipvsadm starting --------------------[ok]"
        ;;
        stop)
        /sbin/ipvsadm -C
        systemctl stop ipvsadm
        ifconfig ens33:0 down
        route del $VIP
        echo "ipvsadm stoped---------------------[ok]"
        ;;
        status)
        if [ ! -e /var/lock/subsys/ipvsadm ];then
        echo "ipvsadm stoped---------------"
        exit 1
                else
                echo "ipvsamd Runing ---------[ok]"
        fi
        ;;
        *)
        echo "Usage: $0 {start|stop|status}"
    exit 1
        esac
        exit 0
   [root@localhost init.d]# service dr.sh start
ipvsadm starting --------------------[ok]
[root@localhost init.d]# systemctl status ipvsadm

配置第一臺5web服務器

[root@localhost ~]# systemctl start httpd.service 
[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0
[root@localhost ~]# cd /var/www/html/
[root@localhost html]# echo "this is accp web" > index.html
[root@localhost html]# cd /etc/sysconfig/network-scripts/
[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:0
[root@localhost network-scripts]# vim ifcfg-lo:0
DEVICE=lo:0
IPADDR=192.168.100.10
NETMASK=255.255.255.0
ONBOOT=yes

[root@localhost network-scripts]# cd /etc/init.d/
[root@localhost init.d]# vim web.sh
#寫一個腳本讓它能響應調度服務器的控制，服務的啓動和關閉
#!/bin/bash
VIP=192.168.100.10
        case "$1" in
        start)
                ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
                /sbin/route add -host $VIP dev lo:0
                echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
                echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
                echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
                echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
                sysctl -p >/dev/null 2>&1
                echo "RealServer Start OK "
                ;;
        stop)
                ifconfig lo:0 down
                route del $VIP /dev/null 2>&1
                echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
                echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
                echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
                echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
                echo "RealServer Stopd"
                ;;
        *)
                echo "Usage: $0 {start|stop}"
                exit 1
        esac
        exit 0

[root@localhost init.d]# chmod +x web.sh
[root@localhost init.d]# service web.sh start
RealServer Start OK 
[root@localhost init.d]# ifup lo:0 #開啓虛擬網卡
[root@localhost network-scripts]# ifconfig
lo:0: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 192.168.100.10  netmask 255.255.255.0
        loop  txqueuelen 1  (Local Loopback)

[root@localhost network-scripts]# firefox "http://127.0.0.1/" & #用火狐瀏覽器去訪問回訪地址，放在後臺運行
[2] 17973

配置第二臺7web服務器

[root@localhost ~]# systemctl start httpd

[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0

[root@localhost ~]# cd /var/www/html/
[root@localhost html]# echo "this is kgv web" > index.html
[root@localhost html]# cd /etc/sysconfig/network-scripts/
[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:0
[root@localhost network-scripts]# vim ifcfg-lo:0
DEVICE=lo:0
IPADDR=192.168.100.10
NETMASK=255.255.255.0
ONBOOT=yes
[root@localhost network-scripts]# cd /etc/init.d/
[root@localhost init.d]# vim web.sh

#!/bin/bash
VIP=192.168.100.10
        case "$1" in
        start)
                ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
                /sbin/route add -host $VIP dev lo:0
                echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
                echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
                echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
                echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
                sysctl -p >/dev/null 2>&1
                echo "RealServer Start OK "
                ;;
        stop)
                ifconfig lo:0 down
                route del $VIP /dev/null 2>&1
                echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
                echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
                echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
                echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
                echo "RealServer Stopd"
                ;;
        *)
                echo "Usage: $0 {start|stop}"
                exit 1
        esac
        exit 0
[root@localhost init.d]# chmod +x web.sh 
[root@localhost init.d]# ifup lo:0
[root@localhost init.d]# service web.sh start
RealServer Start OK 
[root@localhost init.d]# firefox "http://127.0.0.1/" &
[1] 17916

去客戶端去測試一下，先設置同一網段的地址

去1主服務器測試能不能Ping通兩個web服務器

[root@localhost init.d]# ping 192.168.100.221
PING 192.168.100.221 (192.168.100.221) 56(84) bytes of data.
64 bytes from 192.168.100.221: icmp_seq=1 ttl=64 time=0.416 ms
64 bytes from 192.168.100.221: icmp_seq=2 ttl=64 time=0.420 ms
[root@localhost init.d]# ping 192.168.100.222
PING 192.168.100.222 (192.168.100.222) 56(84) bytes of data.
64 bytes from 192.168.100.222: icmp_seq=1 ttl=64 time=0.348 ms
64 bytes from 192.168.100.222: icmp_seq=2 ttl=64 time=0.490 ms

再去客戶端測試一下，咱們的LVS沒問題

（咱們刷新一下就跳出第二個網頁了）

去主服務器1配置keepalive

[root@localhost keepalived]# vim keepalived.conf

 10    smtp_server 127.0.0.1
 12    router_id LVS_01
22     virtual_router_id 10
 27         auth_pass abc123
30         192.168.100.10  #把原來的刪掉咱們只須要一個

36 virtual_server 192.168.100.10 80 {
 39     lb_kind DR
45         TCP_CHECK {

43     real_server 192.168.100.221 80 {
 44         weight 1
 45         TCP_CHECK {
 46             connect_port 80
 47             connect_timeout 3
 48             nb_get_retry 3
 49             delay_before_retry 3
 50         }   
 51     }         
 52     real_server 192.168.100.222 80 {
 53         weight 1
 54         TCP_CHECK {
 55             connect_port 80
 56             connect_timeout 3

[root@localhost keepalived]# systemctl start keepalived.service 
[root@localhost keepalived]# systemctl status keepalived.service #查看狀態，若是沒起來在開啓一次
[root@localhost keepalived]# ip addr show dev ens33:0 #查看網卡有沒有被啓動起來
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c9:dd:05 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.201/24 brd 192.168.100.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.100.10/24 brd 192.168.100.255 scope global secondary ens33:0
       valid_lft forever preferred_lft forever
    inet6 fe80::e3c7:14af:6e4d:7216/64 scope link 
       valid_lft forever preferred_lft forever

一樣的配置第二臺備份的4服務器

[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0
[root@localhost ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward=1
#proc響應關閉重定向功能
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
[root@localhost ~]# sysctl -p #生效
net.ipv4.ip_forward = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0

建立虛擬網卡

[root@localhost ~]# cd /etc/sysconfig/network-scripts/

[root@localhost network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33:0
[root@localhost network-scripts]# vim ifcfg-ens33:0
100dd

DEVICE=ens33:0
ONBOOT=yes
IPADDR=192.168.100.10
NETMASK=255.255.255.0
[root@localhost network-scripts]# service network restart
Restarting network (via systemctl):                        [  肯定  ]

[root@localhost network-scripts]# ifup ens33:0 #開啓網卡，這個網關是用來作虛擬IP的，至關於一給訪問入口

[root@localhost network-scripts]# ifconfig
[root@localhost network-scripts]# cd /etc/init.d/
[root@localhost init.d]# vim dr.sh
#!/bin/bash
GW=192.168.100.1
VIP=192.168.100.10
RIP1=192.168.100.221
RIP2=192.168.100.222
case "$1" in
start)
        /sbin/ipvsadm --save > /etc/sysconfig/ipvsadm
        systemctl start ipvsadm
        /sbin/ifconfig ens33:0 $VIP broadcast $VIP netmask 255.255.255.255 broadcast $VIP up
        /sbin/route add -host $VIP dev ens33:0
        /sbin/ipvsadm -A -t $VIP:80 -s rr
        /sbin/ipvsadm -a -t $VIP:80 -r $RIP1:80 -g
        /sbin/ipvsadm -a -t $VIP:80 -r $RIP2:80 -g
        echo "ipvsadm starting --------------------[ok]"
        ;;
        stop)
        /sbin/ipvsadm -C
        systemctl stop ipvsadm
        ifconfig ens33:0 down
        route del $VIP
        echo "ipvsadm stoped---------------------[ok]"
        ;;
        status)
        if [ ! -e /var/lock/subsys/ipvsadm ];then
        echo "ipvsadm stoped---------------"
        exit 1
                else
                echo "ipvsamd Runing ---------[ok]"
        fi
        ;;
        *)
        echo "Usage: $0 {start|stop|status}"
    exit 1
        esac
        exit 0

[root@localhost init.d]# chmod +x dr.sh
[root@localhost init.d]# service dr.sh start
ipvsadm starting --------------------[ok]
[root@localhost init.d]# cd /etc/keepalived/
[root@localhost keepalived]# vim keepalived.conf 
 10    smtp_server 127.0.0.1
 12    router_id LVS_02
 20     state BACKUP
 22     virtual_router_id 10
 23     priority 99
27         auth_pass abc123
30         192.168.100.10

[root@localhost keepalived]# systemctl start keepalived.service 
[root@localhost keepalived]# systemctl status keepalived.service #查看狀態，若是沒起來在開啓一次

咱們把主服務器1關掉，去客戶端去測試一下是否是還能Ping通

[root@localhost keepalived]# ifdown ens33:0

C:\Users\CHEN>ping 192.168.100.10

正在 Ping 192.168.100.10 具備 32 字節的數據:
來自 192.168.100.10 的回覆: 字節=32 時間<1ms TTL=64
來自 192.168.100.10 的回覆: 字節=32 時間<1ms TTL=64
來自 192.168.100.10 的回覆: 字節=32 時間<1ms TTL=64
來自 192.168.100.10 的回覆: 字節=32 時間<1ms TTL=6