【均衡負載之LVS 系列四】 - OSPF(ECMP)-LVS 集羣
Keepalived-LVS 可以提升集羣的高可用性並增長後端檢測功能、簡化配置，知足常規需求。但Keepalived-LVS集羣中，同一個VIP只能由一臺設備進行宣告，爲一主多備的架構，不能橫向拓展集羣的性能，爲此咱們引入OSPF來解決該問題。
OSPF(ECMP)
ECMP（Equal-CostMultipathRouting）等價多路徑，存在多條不一樣鏈路到達同一目的地址的網絡環境中，若是使用傳統的路由技術，發往該目的地址的數據包只能利用其中的一條鏈路，其它鏈路處於備份狀態或無效狀態，而且在動態路由環境下相互的切換須要必定時間，而等值多路徑路由協議能夠在該網絡環境下同時使用多條鏈路，不只增長了傳輸帶寬，而且能夠無時延無丟包地備份失效鏈路的數據傳輸。後端
特色：centos
- 基於流的均衡負載
- 最大鏈路數受設備限制（最高16）
- 全部鏈路都active，故障鏈路自動剔除
LVS+OSPF(ECMP)
利用ECMP以上特性，能夠將LVS集羣進行橫向拓展，利用quagga啓ospf
爲模擬集羣環境，咱們準備了六臺虛擬機分別爲Client、LVS-一、LVS-二、RealServer一、RealServer二、Router，VIP設爲192.168.0.100網絡
- Router ：192.168.0.1 192.168.1.1
- Client ：192.168.1.2
- LVS-1 ：192.168.0.2
- LVS-2 ：192.168.0.3
- RealServer1 ：192.168.0.4
- RealServer2 ：192.168.0.5
Router 配置
LVS-一、LVS-2與Router需處於同一ospf域中，經過 IP 192.168.0.1 與LVS1/LVS2/RealServer1/RealServer2 通信，IP 192.168.1.1 與 Client通信架構
[root@router ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
// 開啓轉發
[root@router ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.1 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fe6e:d10e prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:6e:d1:0e txqueuelen 1000 (Ethernet)
RX packets 63921 bytes 5978914 (5.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 24354 bytes 2334494 (2.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens38: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::20c:29ff:fe6e:d122 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:6e:d1:22 txqueuelen 1000 (Ethernet)
RX packets 60501 bytes 5206254 (4.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 18358 bytes 1432690 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 1119 bytes 88568 (86.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1119 bytes 88568 (86.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
// ens33：192.168.0.1 與LVS1/LVS2/RealServer1/RealServer同一子網 ， ens38：192.168.1.1 與Client同一子網
[root@LVS1 ~]# yum install quagga.x86_64 -y
[root@router ~]# vi /etc/quagga/zebra.conf
hostname Router
password test
enable password test
log file /var/log/quagga/zebra.log
service password-encryption
interface ens33
interface ens38
access-list 1 permit 127.0.0.1
ip prefix-list ANY seq 5 permit 0.0.0.0/0 le 32
route-map ANY deny 10
match ip address prefix-list ANY
ip protocol ospf route-map ANY
line vty
access-class 1
[root@router ~]# vi /etc/quagga/ospfd.conf
hostname Router
password test
log file /var/log/quagga/ospfd.log
log stdout
log syslog
service password-encryption
interface ens33
ip ospf hello-interval 1
ip ospf dead-interval 4
ip ospf priority 1
ip ospf cost 1
router ospf
ospf router-id 192.168.0.7
log-adjacency-changes
network 192.168.0.0/24 area 0.0.0.0
network 192.168.1.0/24 area 0.0.0.0
access-list 1 permit 127.0.0.1
line vty
access-class 1
RealServer 配置
realserver.sh 參考上一章節curl
[root@RealServer1 ~]# ./realserver.sh start
[root@RealServer1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.4 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:febd:38da prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:bd:38:da txqueuelen 1000 (Ethernet)
RX packets 48635 bytes 4087456 (3.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 46101 bytes 5700308 (5.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 30 bytes 2613 (2.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 30 bytes 2613 (2.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo:0: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 192.168.0.100 netmask 255.255.255.255
loop txqueuelen 1000 (Local Loopback)
LVS 配置
因爲咱們使用ospf來實現高可用，不開啓keepalived的vrrp功能（LVS2也一樣配置），只是用其後端檢測功能。
在keepalived-LVS集羣的搶佔模式下，Master在網卡上掛VIP並進行ARP廣播，此時VIP對應的設備是惟一的。
但在OSPF-LVS集羣中，Router根據ospf信息經過修改報文的目的mac地址轉發到對應的LVS來實現均衡負載，並不根據VIP對應的ARP信息，因此對應的每臺LVS將VIP掛在在lo上。tcp
[root@LVS1 ~]# vi /etc/keepalived/keepalived.conf
global_defs {
router_id LVS1 #路由器標識
script_user root
enable_script_security
}
virtual_server 192.168.0.100 80 {
delay_loop 5
lb_algo wrr
lb_kind DR
persistence_timeout 60
persistence_granularity 255.255.255.255
protocol tcp
inhibit_on_failure on
ha_suspend
sorry_server 127.0.0.1 80
real_server 192.168.0.4 80 {
weight 10
HTTP_GET{
url{
path /
status_code 200
}
connect_port 80
connect_timeout 2
retry 1
delay_before_retry 1
}
}
real_server 192.168.0.5 80 {
weight 10
HTTP_GET{
url{
path /
status_code 200
}
connect_port 80
connect_timeout 2
retry 1
delay_before_retry 1
}
}
}
[root@LVS1 ~]# systemctl reload keepalived
[root@LVS1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.100:80 wrr persistent 60
-> 192.168.0.4:80 Route 10 0 0
-> 192.168.0.5:80 Route 10 0 0
[root@LVS1 ~]# ifconfig lo:0 192.168.0.100 netmask 255.255.255.255 up
//添加VIP到lo
[root@LVS1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.0.100/32 scope global lo:0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:af:6b:f7 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.2/24 brd 192.168.0.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:feaf:6bf7/64 scope link
valid_lft forever preferred_lft forever
[root@LVS1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 100 0 0 ens33
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
安裝quagga並進行配置oop
[root@LVS1 ~]# yum install quagga.x86_64 -y [root@LVS1 ~]# vi /etc/quagga/zebra.conf hostname LVS1 # HOSTNAME改成IP也能夠 password test enable password test log file /var/log/quagga/zebra.log #log syslog service password-encryption interface ens33 access-list 1 permit 127.0.0.1 ip prefix-list ANY seq 5 permit 0.0.0.0/0 le 32 route-map ANY deny 10 match ip address prefix-list ANY ip protocol ospf route-map ANY line vty access-class 1 [root@LVS1 ~]# vi /etc/quagga/ospfd.conf hostname LVS1 password test log file /var/log/quagga/ospfd.log log stdout log syslog service password-encryption interface ens33 ip ospf hello-interval 1 ip ospf dead-interval 4 ip ospf priority 0 ip ospf cost 1 router ospf ospf router-id 192.168.0.2 log-adjacency-changes network 192.168.0.2/24 area 0.0.0.0 access-list 1 permit 127.0.0.1 line vty access-class 1 [root@LVS1 ~]# systemctl start zebra [root@LVS1 ~]# systemctl start ospfd
同理咱們配置完LVS2後進行檢測ospf狀態性能
[root@LVS1 ~]# vtysh
Hello, this is Quagga (version 0.99.22.4).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
LVS1# show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
192.168.0.7 1 Full/DR 3.817s 192.168.0.1 ens33:192.168.0.2 0 0 0
192.168.0.3 0 2-Way/DROther 3.518s 192.168.0.3 ens33:192.168.0.2 0 0 0
LVS1# show ip ospf route
============ OSPF network routing table ============
N 192.168.0.0/24 [1] area: 0.0.0.0
directly attached to ens33
N 192.168.0.100/32 [1] area: 0.0.0.0
directly attached to lo
N 192.168.1.0/24 [2] area: 0.0.0.0
via 192.168.0.1, ens33
============ OSPF router routing table =============
============ OSPF external routing table ===========
在Route上查看路由測試
router# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, A - Babel,
> - selected route, * - FIB route
K>* 0.0.0.0/0 via 192.168.0.10, ens33
C>* 127.0.0.0/8 is directly connected, lo
O 192.168.0.0/24 [110/1] is directly connected, ens33, 00:11:04
C>* 192.168.0.0/24 is directly connected, ens33
O 192.168.0.100/32 [110/2] via 192.168.0.2, ens33 inactive, 00:00:11
via 192.168.0.3, ens33 inactive, 00:00:11
O 192.168.1.0/24 [110/1] is directly connected, ens38, 00:12:19
C>* 192.168.1.0/24 is directly connected, ens38
//能夠看到 到 192.168.0.100/32 的下一跳分配到 192.168.0.二、192.168.0.3
Client 測試
[root@Client ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 ens33 192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33 [root@Client ~]# traceroute 192.168.0.100 traceroute to 192.168.0.100 (192.168.0.100), 30 hops max, 60 byte packets 1 192.168.1.1 (192.168.1.1) 0.575 ms 0.258 ms 0.478 ms 2 192.168.0.100 (192.168.0.100) 1.901 ms 1.746 ms 1.370 ms [root@Client ~]# curl 192.168.0.100 RealServer1 192.168.0.4
自動化
以上是最基礎的ospf-Lvs集羣配置，在實際生產環境中會有自動化部署、監控、告警等需求，例如quagga的初始化配置this
quagga 配置
HOSTNAME=`hostname`
PASSWORD=shenyangchangkuan
#獲取默認路由對應網卡
NIC=`/sbin/route -n|awk '$1=="default"||$1=="0.0.0.0"{print $NF}'|head -n 1`
if [ -z $NIC ];then
echo "get NIC err,NIC is null!"
fi
#獲取該網卡IP
#centos7改成下面這行
#IP=`/sbin/ifconfig $NIC |grep "inet "|sed 's/:/ /'|awk '{print $2}'`
IP=`/sbin/ifconfig $NIC|grep "inet addr"|sed 's/:/ /'|awk '{print $3}'`
if [ -z $IP ];then
echo "get IP err,IP is null!"
exit 1
fi
#將該網卡及子網卡IP信息寫入tmp/lvs_network.tmp
echo "">/tmp/lvs_network.tmp
for cfg in ` ls -l /etc/sysconfig/network-scripts/ifcfg-${NIC}*|awk '{print $NF}'`
do
cat $cfg |grep IPADDR=|awk -F "=" '{print $2}' |sed 's/"//g'>>/tmp/lvs_network.tmp
done
#將keepalived中配置的vip寫入/tmp/lvs_network.tmp
if [ -f /etc/keepalived/keepalived.conf ];then
cat /etc/keepalived/keepalived.conf | grep virtual_server |awk '{print $2}'>>/tmp/lvs_network.tmp
fi
[ -d /usr/local/lvs ] || mkdir /usr/local/lvs
[ -d /usr/local/lvs/etc ] || mkdir /usr/local/lvs/etc
cat /tmp/lvs_network.tmp|egrep -v "^$" |awk -F "." '{print $1"."$2"."$3"."0}'|sort|uniq|sort >/usr/local/lvs/etc/networks
#將上述ip聚類爲網段，例如10.0.1.0/24與10.0.2.0/24，主要針對vip與lvs ip不一樣網段狀況
i=0
for n in `cat /usr/local/lvs/etc/networks|egrep "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+"`
do
echo $n;
NETWORKS[$i]=$n;
let i++;
done
TIME=`date +%Y/%m/%d" "%H":"%M":"%S`
ZEBRA="/etc/quagga/zebra.conf"
OSPFD="/etc/quagga/ospfd.conf"
[ -d /etc/quagga/bak ] || mkdir /etc/quagga/bak
[ -f /etc/quagga/zebra.conf ] && /bin/cp /etc/quagga/zebra.conf /etc/quagga/bak/zebra.conf_`date +%Y%m%d%H%M%S`
[ -f /etc/quagga/ospfd.conf ] && /bin/cp /etc/quagga/ospfd.conf /etc/quagga/bak/ospfd.conf_`date +%Y%m%d%H%M%S`
echo -e "!$TIME" >$ZEBRA
echo -e "hostname $HOSTNAME" >>$ZEBRA
# HOSTNAME改成IP也能夠
echo -e "password shenyangchangkuan" >>$ZEBRA
echo -e "enable password shenyangchangkuan" >>$ZEBRA
echo -e "log file /var/log/quagga/zebra.log" >>$ZEBRA
#echo -e "log syslog" >>$ZEBRA
echo -e "service password-encryption" >>$ZEBRA
echo -e "interface $NIC" >>$ZEBRA
#echo -e " ip address ${IP}/24" >>$ZEBRA
echo -e "access-list 1 permit 127.0.0.1" >>$ZEBRA
echo -e "ip prefix-list ANY seq 5 permit 0.0.0.0/0 le 32" >> $ZEBRA
echo -e "route-map ANY deny 10" >>$ZEBRA
echo -e " match ip address prefix-list ANY" >>$ZEBRA
echo -e "ip protocol ospf route-map ANY" >>$ZEBRA
echo -e "line vty" >>$ZEBRA
echo -e " access-class 1">>$ZEBRA
echo -e "!$TIME" > $OSPFD
echo -e "hostname $HOSTNAME" >>$OSPFD
echo -e "password $PASSWORD ">>$OSPFD
echo -e "log file /var/log/quagga/ospfd.log">>$OSPFD
echo -e "log stdout">>$OSPFD
echo -e "log syslog">>$OSPFD
echo -e "service password-encryption">>$OSPFD
echo -e "interface $NIC" >>$OSPFD
echo -e " ip ospf hello-interval 1">>$OSPFD
echo -e " ip ospf dead-interval 4">>$OSPFD
echo -e " ip ospf priority 0">>$OSPFD
echo -e " ip ospf cost 1">>$OSPFD
echo -e "interface lo">>$OSPFD
echo -e " ip ospf cost 1">>$OSPFD
echo -e "router ospf" >>$OSPFD
echo -e " ospf router-id ${IP}">>$OSPFD
echo -e " log-adjacency-changes">>$OSPFD
i=0
for NETWORK in ${NETWORKS[@]}
do
echo -e " network $NETWORK/24 area 0.0.0.0" >>$OSPFD
let i++
done
echo -e "access-list 1 permit 127.0.0.1" >>$OSPFD
echo -e "line vty">>$OSPFD
echo -e " access-class 1" >>$OSPFD
chmod 640 $OSPFD
chmod 640 $ZEBRA
/bin/chown quagga:quagga $OSPFD
/bin/chown quagga:quagga $ZEBRA
/sbin/service zebra reload
/sbin/service ospfd reload
相關文章
- 1. Lvs負載均衡羣集
- 2. 負載均衡集羣LVS
- 3. LVS負載均衡集羣
- 4. LVS負載均衡羣集
- 5. lvs負載均衡集羣
- 6. LVS-負載均衡集羣
- 7. 【LVS】負載均衡集羣
- 8. 【均衡負載之LVS 系列三】 - 高可用 LVS+KeepAlived 集羣
- 9. 負載均衡之LVS集羣
- 10. Centos 7之LVS負載均衡羣集
- 更多相關文章...
- • Swarm 集羣管理 - Docker教程
- • Scala List(列表) - Scala教程
- • RxJava操作符（四）Combining
- • ☆技術問答集錦（13）Java Instrument原理
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. Lvs負載均衡羣集
- 2. 負載均衡集羣LVS
- 3. LVS負載均衡集羣
- 4. LVS負載均衡羣集
- 5. lvs負載均衡集羣
- 6. LVS-負載均衡集羣
- 7. 【LVS】負載均衡集羣
- 8. 【均衡負載之LVS 系列三】 - 高可用 LVS+KeepAlived 集羣
- 9. 負載均衡之LVS集羣
- 10. Centos 7之LVS負載均衡羣集