實現lvs調度及lvs+keeplivead高可用

時間  2020-05-26
標籤 實現 lvs 調度 lvs+keeplivead keeplivead 可用 欄目 負載均衡 简体版
原文   原文鏈接

1.簡述下lvs四種集羣特色及使用場景html

LVS 有三種負載均衡的模式，分別是VS/NAT（nat 模式）,VS/DR(路由模式),VS/TUN（隧道模式）,VS/FULLNAT
1、NAT模式（VS-NAT）
原理：就是把客戶端發來的數據包的IP頭的目的地址，在負載均衡器上換成其中一臺RS的IP地址併發至此RS來處理,RS處理完後把數據交給負載均衡器,負載均衡器再把數據包原IP地址改成本身的IP將目的地址改成客戶端IP地址便可期間,不管是進來的流量,仍是出去的流量,都必須通過負載均衡器
優勢：集羣中的物理服務器可使用任何支持TCP/IP操做系統，只有負載均衡器須要一個合法的IP地址
缺點：擴展性有限。當服務器節點（普通PC服務器）增加過多時,負載均衡器將成爲整個系統的瓶頸
由於全部的請求包和應答包的流向都通過負載均衡器。當服務器節點過多時
大量的數據包都交匯在負載均衡器那，速度就會變慢！
2、IP隧道模式（VS-TUN）
原理：首先要知道，互聯網上的大多Internet服務的請求包很短小，而應答包一般很大那麼隧道模式就是，把客戶端發來的數據包，封裝一個新的IP頭標記(僅目的IP)發給RSRS收到後,先把數據包的頭解開,還原數據包,處理後,直接返回給客戶端,不須要再通過負載均衡器。注意,因爲RS須要對負載均衡器發過來的數據包進行還原,因此說必須支持IPTUNNEL協議，因此,在RS的內核中,必須編譯支持IPTUNNEL這個選項
優勢：負載均衡器只負責將請求包分發給後端節點服務器，而RS將應答包直接發給用戶因此，減小了負載均衡器的大量數據流動，負載均衡器再也不是系統的瓶頸，就能處理很巨大的請求量這種方式，一臺負載均衡器可以爲不少RS進行分發。並且跑在公網上就能進行不一樣地域的分發。
缺點：隧道模式的RS節點須要合法IP，這種方式須要全部的服務器支持」IP Tunneling」(IP Encapsulation)協議，服務器可能只侷限在部分Linux系統上
3、直接路由模式（VS-DR）
原理：負載均衡器和RS都使用同一個IP對外服務但只有DR對ARP請求進行響應全部RS對自己這個IP的ARP請求保持靜默也就是說,網關會把對這個服務IP的請求所有定向給DR而DR收到數據包後根據調度算法,找出對應的RS,把目的MAC地址改成RS的MAC（由於IP一致）並將請求分發給這臺RS這時RS收到這個數據包,處理完成以後，因爲IP一致，能夠直接將數據返給客戶則等於直接從客戶端收到這個數據包無異,處理後直接返回給客戶端
因爲負載均衡器要對二層包頭進行改換,因此負載均衡器和RS之間必須在一個廣播域也能夠簡單的理解爲在同一臺交換機上
優勢：和TUN（隧道模式）同樣，負載均衡器也只是分發請求，應答包經過單獨的路由方法返回給客戶端與VSTUN相比，VS-DR這種實現方式不須要隧道結構，所以可使用大多數操做系統作爲物理服務器。
缺點：（不能說缺點，只能說是不足）要求負載均衡器的網卡必須與物理網卡在一個物理段上。
4、fullnat模式
lvs-fullnat：經過同時修改請求報文的源IP地址和目標IP地址進行轉發
(1) VIP是公網地址，RIP和DIP是私網地址，且一般不在同一IP網絡；所以，RIP的網關通常不會指向DIP
(2) RS收到的請求報文源地址是DIP，所以，只需響應給DIP；但Director還要將其發往Client
(3) 請求和響應報文都經由Director
(4) 支持端口映射
注意：此類型kernel默認不支持

2.描述LVS-DR工做原理，並配置實現node

原理：負載均衡器和RS都使用同一個IP對外服務但只有DR對ARP請求進行響應全部RS對自己這個IP的ARP請求保持靜默也就是說,網關會把對這個服務IP的請求所有定向給DR而DR收到數據包後根據調度算法,找出對應的RS,把目的MAC地址改成RS的MAC（由於IP一致）並將請求分發給這臺RS這時RS收到這個數據包,處理完成以後，因爲IP一致，能夠直接將數據返給客戶則等於直接從客戶端收到這個數據包無異,處理後直接返回給客戶端
因爲負載均衡器要對二層包頭進行改換,因此負載均衡器和RS之間必須在一個廣播域也能夠簡單的理解爲在同一臺交換機上
規劃：c1,c2,c3,c4都是centos7.6
c1  客戶端
c2  vs
c3  web1
c4  web2
2.1 安裝web服務器
[root@c3 ~]# yum install httpd -y
[root@c3 ~]# echo rs1 > /var/www/html/index.html
[root@c3 ~]# systemctl start httpd
[root@c3 ~]# curl c3
rs1

[root@c4 ~]# yum install httpd -y
[root@c4 ~]# echo rs2 > /var/www/html/index.html
[root@c4 ~]# systemctl start httpd
[root@c4 ~]# curl c4
rs2
2.2 配置rs服務器
[root@c3 ~]# yum install net-tools -y   ###使用ifconfig命令須要安裝net-tools包
[root@c3 ~]# cat rs.sh 
#!/bin/bash
vip=10.0.0.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac
[root@c3 ~]# sh rs.sh start
[root@c3 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.0.0.100/32 scope global lo:1
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:f1:37:a8 brd ff:ff:ff:ff:ff:ff
    inet 10.1.1.244/24 brd 10.1.1.255 scope global noprefixroute dynamic eth0
       valid_lft 14582sec preferred_lft 14582sec
    inet6 fe80::5025:c937:77d0:2b28/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@c4 ~]# yum install net-tools -y
[root@c4 ~]# sh rs.sh start
[root@c4 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.0.0.100/32 scope global lo:1
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:05:32:f0 brd ff:ff:ff:ff:ff:ff
    inet 10.1.1.245/24 brd 10.1.1.255 scope global noprefixroute dynamic eth0
       valid_lft 16671sec preferred_lft 16671sec
    inet6 fe80::96c3:3cc3:b39e:dee3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
2.3 配置vs服務器
[root@c2 ~]# yum install -y ipvsadm  ###安裝lvs包
[root@c2 ~]# cat vs.sh 
#!/bin/bash
vip='10.0.0.100'
iface='lo:1'
mask='255.255.255.255'
port='80'
rs1='10.1.1.244'
rs2='10.1.1.245'
scheduler='rr'      ###爲了測試容易出效果，採用rr輪詢算法
type='-g'
case $1 in
start)
    ifconfig $iface $vip netmask $mask #broadcast $vip up
    iptables -F
    ipvsadm -A -t ${vip}:${port} -s $scheduler
    ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w 1
    ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w 1
    ;;
stop)
    ipvsadm -C
    ifconfig $iface down
    ;;
*)
    echo "Usage $(basename $0) start|stop"
    exit 1
esa
[root@c2 ~]# sh vs.sh start
[root@c2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.0.0.100/32 scope global lo:1
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
[root@c2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.100:80 rr
  -> 10.1.1.244:80                Route   1      0          0         
  -> 10.1.1.245:80                Route   1      0          0
2.4 測試：
[root@c1 ~]# route -n     ###無去往10.0.0.0網段的路由
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.1.1.254      0.0.0.0         UG    100    0        0 eth0
10.1.1.0        0.0.0.0         255.255.255.0   U     100    0        0 eth0
[root@c1 ~]# curl 10.0.0.100
^C
[root@c1 ~]# route add -host 10.0.0.100 dev eth0  ###增長去往10.0.0.0網段的路由
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2

3.實現LVS+Keepalived高可用web

keepalive使用vrrp（虛擬路由協議）實現負載冗餘功能。路由器經過週期的組播通告宣稱本身是主路由器，而後與網絡中的路由器對比優先級，以選出主、備路由器。主路由器提供相對應的路由功能，備路由器在主路由器故障的時候再次對比優先級選出新的主路由器提供服務，其他的成爲備份路由器。此場景在第2小節的基礎上完成，c2是主路由器，c5是備路由器
3.1 實現c2與c5互爲免密登陸
[root@c2 ~]# ssh-keygen -t rsa -P "" ###生成密鑰
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:C1wDPspULjsjJQs/hSUjac50V4BLQXCQFkwxbViT/DM root@c2
The key's randomart image is:
+---[RSA 2048]----+
|+O#*=.=.         |
|.Oo&.= .         |
|B * O + o        |
| = O E o .       |
|  = * = S        |
|   o o . .       |
|        .        |
|                 |
|                 |
+----[SHA256]-----+
[root@c2 ~]# ssh-copy-id c5 ###傳輸密鑰
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'c5 (10.1.1.246)' can't be established.
ECDSA key fingerprint is SHA256:ilZ46J85JC8Xhr2dVvYsUxMGyj17SDhD6/JrhmNy6GY.
ECDSA key fingerprint is MD5:2f:c5:a9:d6:d7:5f:5e:4e:c3:94:7c:92:3a:d2:55:63.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@c5's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'c5'"
and check to make sure that only the key(s) you wanted were added.

[root@c2 ~]# ssh c5   ###測試免密登陸
Last login: Mon May 25 21:40:03 2020 from 192.168.10.45
[root@c5 ~]# 

[root@c5 ~]# ssh-keygen -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:abVEGoN7+mbpGU0aZY4VssjdndMC+cjLZYK5Icy+S/U root@c5
The key's randomart image is:
+---[RSA 2048]----+
|       .+ +.     |
|     ..o O.+ o   |
|     oo.++Bo= .  |
|      = =X+.+o   |
|     . +S++=     |
|      oo.*o      |
|      .oo.E      |
|     .. =o       |
|      .=o        |
+----[SHA256]-----+
[root@c5 ~]# ssh-copy-id c2
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'c2 (10.1.1.243)' can't be established.
ECDSA key fingerprint is SHA256:dldJTKtxApZyQT/FT6WKQsqKgtf4cPuAxBTiLMFdxSk.
ECDSA key fingerprint is MD5:1a:07:07:69:3f:0e:94:b3:f3:c5:04:dc:73:6b:ba:3e.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@c2's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'c2'"
and check to make sure that only the key(s) you wanted were added.

[root@c5 ~]# ssh c2
Last login: Mon May 25 21:40:01 2020 from 192.168.10.45
[root@c2 ~]# 
3.2 安裝與配置keepalived
3.2.1 先清除c2上的ipvsadm策略
[root@c2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.100:80 rr
  -> 10.1.1.244:80                Route   1      0          0         
  -> 10.1.1.245:80                Route   1      0          0         
[root@c2 ~]# ls
anaconda-ks.cfg  original-ks.cfg  vs.sh
[root@c2 ~]# sh vs.sh stop
[root@c2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

3.2.2 安裝keepalived服務，ipvsadm工具
[root@c2 ~]# yum install keepalived.x86_64 -y
[root@c5 ~]# yum install keepalived -y
[root@c5 ~]# yum install ipvsadm -y

[root@c2 keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node1
   vrrp_mcast_group4 224.0.100.100
}

vrrp_instance VI_1 {
    state MASTER
    interface bond0
    virtual_router_id 5
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        10.0.0.100/24 dev bond0 label bond0:0
    }
}

virtual_server 10.0.0.100 80 {
    delay_loop 1
    lb_algo rr
    lb_kind DR
    protocol TCP

    real_server 10.1.1.244 80 {
        weight 1   ###後端服務檢測
        HTTP_GET {          
        url {
            path /
            status_code 200
        }
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    real_server 10.1.1.245 80 {
        weight 1
        HTTP_GET {
                url {
                        path /
                        status_code 200
                }       
            } 
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }

    }
}
[root@c2 keepalived]# systemctl start keepalived
[root@c2 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
    link/ether 00:0c:29:ba:03:94 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
    link/ether 00:0c:29:ba:03:9e brd ff:ff:ff:ff:ff:ff
7: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:0c:29:ba:03:94 brd ff:ff:ff:ff:ff:ff
    inet 10.1.1.243/24 brd 10.1.1.255 scope global noprefixroute bond0
       valid_lft forever preferred_lft forever
    inet 10.0.0.100/24 scope global bond0:0
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:feba:394/64 scope link 
       valid_lft forever preferred_lft forever
[root@c5 keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id node2
   vrrp_mcast_group4 224.0.100.100
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 5
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        10.0.0.100/24 dev eth0 label eth0:0
    }
}

virtual_server 10.0.0.100 80 {
    delay_loop 1
    lb_algo rr
    lb_kind DR
    protocol TCP

    real_server 10.1.1.244 80 {
        weight 1
        HTTP_GET {
        url {
            path /
            status_code 200
        }
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    real_server 10.1.1.245 80 {
        weight 1
        HTTP_GET {
                url {
                        path /
                        status_code 200
                }       
            } 
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }

    }
}
[root@c5 keepalived]# systemctl start keepalived.service

3.3 測試
3.3.1 先測試lvs是否正常工做
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# 
3.3.2 停掉c2上的keepalived服務，再測試是否正常調度
[root@c2 keepalived]# systemctl stop keepalived

[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
3.3.3 測試後端服務器的健康性檢查
[root@c3 ~]# systemctl stop httpd
[root@c1 ~]# while true;do curl 10.0.0.100;sleep 1;done
rs2
rs1
rs2
rs1
rs2
curl: (7) Failed connect to 10.0.0.100:80; Connection refused
rs2
curl: (7) Failed connect to 10.0.0.100:80; Connection refused
rs2
rs2
rs2
rs2
rs2
rs2
rs2

[root@c3 ~]# systemctl start httpd
[root@c1 ~]# while true;do curl 10.0.0.100;sleep 1;done
rs2
rs2
rs2
rs2
rs1
rs2
rs1
rs2
rs1
rs2
rs1
rs2
rs1
相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程