大綱html
1、前言前端
2、Keepalived 詳解node
3、環境準備vim
4、LVS+Keepalived 實現高可用的前端負載均衡器後端
1、前言centos
這篇文章是前幾篇文章的總結,咱們先簡單的總結一下咱們前面講解的內容,前面咱們講解了,LVS(負載均衡器)、Heartbeat、Corosync、Pacemaker、Web高可用集羣、MySQL高可用集羣、DRDB、iscsi、gfs二、cLVM等,惟一沒有講解的就是LVS可用,也就是前端高可用,咱們這一篇博文主要講解內容。在說這個以前咱們得和你們討論一個問題,也是好多博友問的問題。Heartbeat、Corosync、Keepalived這三個集羣組件咱們到底選哪一個好,首先我想說明的是,Heartbeat、Corosync是屬於同一類型,Keepalived與Heartbeat、Corosync,根本不是同一類型的。Keepalived使用的vrrp協議方式,虛擬路由冗餘協議 (Virtual Router Redundancy Protocol,簡稱VRRP);Heartbeat或Corosync是基於主機或網絡服務的高可用方式;簡單的說就是,Keepalived的目的是模擬路由器的高可用,Heartbeat或Corosync的目的是實現Service的高可用。因此通常Keepalived是實現前端高可用,經常使用的前端高可用的組合有,就是咱們常見的LVS+Keepalived、Nginx+Keepalived、HAproxy+Keepalived。而Heartbeat或Corosync是實現服務的高可用,常見的組合有Heartbeat v3(Corosync)+Pacemaker+NFS+Httpd 實現Web服務器的高可用、Heartbeat v3(Corosync)+Pacemaker+NFS+MySQL 實現MySQL服務器的高可用。總結一下,Keepalived中實現輕量級的高可用,通常用於前端高可用,且不須要共享存儲,通常經常使用於兩個節點的高可用。而Heartbeat(或Corosync)通常用於服務的高可用,且須要共享存儲,通常用於多節點的高可用。這個問題咱們說明白了,又有博友會問了,那heartbaet與corosync咱們又應該選擇哪一個好啊,我想說咱們通常用corosync,由於corosync的運行機制更優於heartbeat,就連從heartbeat分離出來的pacemaker都說在之後的開發當中更傾向於corosync,因此如今corosync+pacemaker是最佳組合。但說實話我對於軟件沒有任何傾向性,因此我把全部的集羣軟件都和你們說了一下,我認爲無論什麼軟件,只要它能存活下來都有它的特色和應用領域,只有把特定的軟件放在特定的位置才能發揮最大的做用,那首先咱們得對這個軟件有全部瞭解。學習一種軟件的最好方法,就是去查官方文檔。好了說了那麼多但願你們有所收穫,下面咱們來講一說keepalived。安全
2、Keepalived 詳解bash
1.Keepalived 定義 服務器
Keepalived 是一個基於VRRP協議來實現的LVS服務高可用方案,能夠利用其來避免單點故障。一個LVS服務會有2臺服務器運行Keepalived,一臺爲主服務器(MASTER),一臺爲備份服務器(BACKUP),可是對外表現爲一個虛擬IP,主服務器會發送特定的消息給備份服務器,當備份服務器收不到這個消息的時候,即主服務器宕機的時候, 備份服務器就會接管虛擬IP,繼續提供服務,從而保證了高可用性。Keepalived是VRRP的完美實現,所以在介紹keepalived以前,先介紹一下VRRP的原理。網絡
2.VRRP 協議簡介
在現實的網絡環境中,兩臺須要通訊的主機大多數狀況下並無直接的物理鏈接。對於這樣的狀況,它們之間路由怎樣選擇?主機如何選定到達目的主機的下一跳路由,這個問題一般的解決方法有二種:
在主機上使用動態路由協議(RIP、OSPF等)
在主機上配置靜態路由
很明顯,在主機上配置動態路由是很是不切實際的,由於管理、維護成本以及是否支持等諸多問題。配置靜態路由就變得十分流行,但路由器(或者說默認網關default gateway)卻常常成爲單點故障。VRRP的目的就是爲了解決靜態路由單點故障問題,VRRP經過一競選(election)協議來動態的將路由任務交給LAN中虛擬路由器中的某臺VRRP路由器。
3.VRRP 工做機制
在一個VRRP虛擬路由器中,有多臺物理的VRRP路由器,可是這多臺的物理的機器並不能同時工做,而是由一臺稱爲MASTER的負責路由工做,其它的都是BACKUP,MASTER並不是一成不變,VRRP讓每一個VRRP路由器參與競選,最終獲勝的就是MASTER。MASTER擁有一些特權,好比,擁有虛擬路由器的IP地址,咱們的主機就是用這個IP地址做爲靜態路由的。擁有特權的MASTER要負責轉發發送給網關地址的包和響應ARP請求。
VRRP經過競選協議來實現虛擬路由器的功能,全部的協議報文都是經過IP多播(multicast)包(多播地址224.0.0.18)形式發送的。虛擬路由器由VRID(範圍0-255)和一組IP地址組成,對外表現爲一個周知的MAC地址。因此,在一個虛擬路由 器中,無論誰是MASTER,對外都是相同的MAC和IP(稱之爲VIP)。客戶端主機並不須要由於MASTER的改變而修改本身的路由配置,對客戶端來講,這種主從的切換是透明的。
在一個虛擬路由器中,只有做爲MASTER的VRRP路由器會一直髮送VRRP通告信息(VRRPAdvertisement message),BACKUP不會搶佔MASTER,除非它的優先級(priority)更高。當MASTER不可用時(BACKUP收不到通告信息), 多臺BACKUP中優先級最高的這臺會被搶佔爲MASTER。這種搶佔是很是快速的(<1s),以保證服務的連續性。因爲安全性考慮,VRRP包使用了加密協議進行加密。
4.VRRP 工做流程
(1).初始化:
路由器啓動時,若是路由器的優先級是255(最高優先級,路由器擁有路由器地址),要發送VRRP通告信息,併發送廣播ARP信息通告路由器IP地址對應的MAC地址爲路由虛擬MAC,設置通告信息定時器準備定時發送VRRP通告信息,轉爲MASTER狀態;不然進入BACKUP狀態,設置定時器檢查定時檢查是否收到MASTER的通告信息。
(2).Master
設置定時通告定時器;
用VRRP虛擬MAC地址響應路由器IP地址的ARP請求;
轉發目的MAC是VRRP虛擬MAC的數據包;
若是是虛擬路由器IP的擁有者,將接受目的地址是虛擬路由器IP的數據包,不然丟棄;
當收到shutdown的事件時刪除定時通告定時器,發送優先權級爲0的通告包,轉初始化狀態;
若是定時通告定時器超時時,發送VRRP通告信息;
收到VRRP通告信息時,若是優先權爲0,發送VRRP通告信息;不然判斷數據的優先級是否高於本機,或相等並且實際IP地址大於本地實際IP,設置定時通告定時器,復位主機超時定時器,轉BACKUP狀態;不然的話,丟棄該通告包;
(3).Backup
設置主機超時定時器;
不能響應針對虛擬路由器IP的ARP請求信息;
丟棄全部目的MAC地址是虛擬路由器MAC地址的數據包;
不接受目的是虛擬路由器IP的全部數據包;
當收到shutdown的事件時刪除主機超時定時器,轉初始化狀態;
主機超時定時器超時的時候,發送VRRP通告信息,廣播ARP地址信息,轉MASTER狀態;
收到VRRP通告信息時,若是優先權爲0,表示進入MASTER選舉;不然判斷數據的優先級是否高於本機,若是高的話認可MASTER有效,復位主機超時定時器;不然的話,丟棄該通告包;
5.ARP查詢處理
當內部主機經過ARP查詢虛擬路由器IP地址對應的MAC地址時,MASTER路由器回覆的MAC地址爲虛擬的VRRP的MAC地址,而不是實際網卡的 MAC地址,這樣在路由器切換時讓內網機器覺察不到;而在路由器從新啓動時,不能主動發送本機網卡的實際MAC地址。若是虛擬路由器開啓的ARP代理 (proxy_arp)功能,代理的ARP迴應也迴應VRRP虛擬MAC地址;好了VRRP的簡單講解就到這裏,咱們下來說解一下Keepalived的案例。
3、環境準備
1.操做系統
CentOS 6.4 X86_64
2.軟件版本
ipvsadm.x86_64 0:1.25-10.el6
keepalived.x86_64 0:1.2.7-3.el6
httpd-2.2.15-29.el6.centos.x86_64
3.實驗拓撲
4.時間同步
node1:
[root@node1 ~]# ntpdate 202.120.2.101
node2:
[root@node2 ~]# ntpdate 202.120.2.101
master:
[root@master ~]# ntpdate 202.120.2.101
slave:
[root@slave ~]# ntpdate 202.120.2.101
5.主機名互相解析
node1:
[root@node1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.18.201 node1.test.com node1 192.168.18.202 node2.test.com node2
node2:
[root@node2 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.18.201 node1.test.com node1 192.168.18.202 node2.test.com node2
6.安裝yum源
node1:
[root@node1 ~]# rpm -ivh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm [root@node1 ~]# rpm -ivh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
node2:
[root@node2 ~]# rpm -ivh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm [root@node2 ~]# rpm -ivh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
master:
[root@master ~]# rpm -ivh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm [root@master ~]# rpm -ivh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
slave:
[root@slave ~]# rpm -ivh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm [root@slave ~]# rpm -ivh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
4、LVS+Keepalived 實現高可用的前端負載均衡器
node1:
1.安裝httpd
[root@node1 ~]# yum install -y httpd
2.配置httpd
[root@node1 ~]# vim /var/www/html/index.html <h1>RS1.test.com</h1>
3.啓動httpd
[root@node1 ~]# service httpd start
4.測試
5.設置開機自啓動
[root@node1 ~]# chkconfig httpd on [root@node1 ~]# chkconfig httpd --list httpd 0:關閉 1:關閉 2:啓用 3:啓用 4:啓用 5:啓用 6:關閉
6.配置node1
[root@node1 ~]# mkdir src [root@node1 ~]# cd src/ [root@node1 src]# vim realserver.sh #!/bin/bash # # Script to start LVS DR real server. # description: LVS DR real server # . /etc/rc.d/init.d/functions VIP=192.168.18.200 #修改你的VIP host=`/bin/hostname` case "$1" in start) # Start LVS-DR real server on this machine. /sbin/ifconfig lo down /sbin/ifconfig lo up echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up /sbin/route add -host $VIP dev lo:0 ;; stop) # Stop LVS-DR real server loopback device(s). /sbin/ifconfig lo:0 down echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce ;; status) # Status of LVS-DR real server. islothere=`/sbin/ifconfig lo:0 | grep $VIP` isrothere=`netstat -rn | grep "lo:0" | grep $VIP` if [ ! "$islothere" -o ! "isrothere" ];then # Either the route or the lo:0 device # not found. echo "LVS-DR real server Stopped." else echo "LVS-DR real server Running." fi ;; *) # Invalid entry. echo "$0: Usage: $0 {start|status|stop}" exit 1 ;; esac [root@node1 src]# chmod +x realserver.sh [root@node1 src]# ll 總用量 4 -rwxr-xr-x 1 root root 1485 8月 22 10:18 realserver.sh [root@node1 src]# ./realserver.sh start
7.查看配置
[root@node1 src]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:11:92:E4 inet addr:192.168.18.201 Bcast:192.168.18.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe11:92e4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:115061 errors:0 dropped:0 overruns:0 frame:0 TX packets:14979 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:43448483 (41.4 MiB) TX bytes:1224926 (1.1 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:168 (168.0 b) TX bytes:168 (168.0 b) lo:0 Link encap:Local Loopback inet addr:192.168.18.200 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1 [root@node1 src]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.18.200 0.0.0.0 255.255.255.255 UH 0 0 0 lo 192.168.18.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0 0.0.0.0 192.168.18.254 0.0.0.0 UG 0 0 0 eth0 [root@node1 src]# cat /proc/sys/net/ipv4/conf/lo/arp_ignore 1 [root@node1 src]# cat /proc/sys/net/ipv4/conf/lo/arp_announce 2 [root@node1 src]# cat /proc/sys/net/ipv4/conf/all/arp_ignore 1 [root@node1 src]# cat /proc/sys/net/ipv4/conf/all/arp_announce 2
好了,node1到這裏基本配置完成,下面咱們來配置node2。
node2:
1.安裝httpd
[root@node2 ~]# yum install -y httpd
2.配置httpd
[root@node2 ~]# vim /var/www/html/index.html <h1>RS2.test.com</h1>
3.啓動httpd
[root@node2 ~]# service httpd start
4.測試
5.設置開機自啓動
[root@node2 ~]# chkconfig httpd on [root@node2 ~]# chkconfig httpd --list httpd 0:關閉 1:關閉 2:啓用 3:啓用 4:啓用 5:啓用 6:關閉
6.配置node2
[root@node2 ~]# mkdir src [root@node2 ~]# cd src/ [root@node2 src]# vim realserver.sh #!/bin/bash # # Script to start LVS DR real server. # description: LVS DR real server # . /etc/rc.d/init.d/functions VIP=192.168.18.200 host=`/bin/hostname` case "$1" in start) # Start LVS-DR real server on this machine. /sbin/ifconfig lo down /sbin/ifconfig lo up echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up /sbin/route add -host $VIP dev lo:0 ;; stop) # Stop LVS-DR real server loopback device(s). /sbin/ifconfig lo:0 down echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce ;; status) # Status of LVS-DR real server. islothere=`/sbin/ifconfig lo:0 | grep $VIP` isrothere=`netstat -rn | grep "lo:0" | grep $VIP` if [ ! "$islothere" -o ! "isrothere" ];then # Either the route or the lo:0 device # not found. echo "LVS-DR real server Stopped." else echo "LVS-DR real server Running." fi ;; *) # Invalid entry. echo "$0: Usage: $0 {start|status|stop}" exit 1 ;; esac [root@node2 src]# chmod +x realserver.sh [root@node2 src]# ./realserver.sh start
7.查看配置
[root@node2 src]# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:B8:DF:6A inet addr:192.168.18.202 Bcast:192.168.18.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:feb8:df6a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:110545 errors:0 dropped:0 overruns:0 frame:0 TX packets:10461 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:32853789 (31.3 MiB) TX bytes:889109 (868.2 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:616 (616.0 b) TX bytes:616 (616.0 b) lo:0 Link encap:Local Loopback inet addr:192.168.18.200 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1 [root@node2 src]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.18.200 0.0.0.0 255.255.255.255 UH 0 0 0 lo 192.168.18.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0 0.0.0.0 192.168.18.254 0.0.0.0 UG 0 0 0 eth0 [root@node2 src]# cat /proc/sys/net/ipv4/conf/lo/arp_ignore 1 [root@node2 src]# cat /proc/sys/net/ipv4/conf/lo/arp_announce 2 [root@node2 src]# cat /proc/sys/net/ipv4/conf/all/arp_ignore 1 [root@node2 src]# cat /proc/sys/net/ipv4/conf/all/arp_announce 2
好了,到這裏node2也基本配置完成。下面咱們來配置master與slave。
masterg與slave:
1.安裝keepalived與ipvsadm
[root@master ~]# yum install -y keepalived ipvsadm [root@slave ~]# yum install -y keepalived ipvsadm
2.修改配置文件
[root@master ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com #配置管理員郵箱 } notification_email_from root #配置發件人 smtp_server 127.0.0.1 #配置郵件服務器 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state MASTER #配置模式 interface eth0 virtual_router_id 51 priority 101 #配置優先級 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 #配置虛擬IP地址 } } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 #persistence_timeout 50 protocol TCP real_server 192.168.18.201 80 { #配置realaserver weight 1 HTTP_GET { #監控配置 url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } }
3.將配置文件同步到slave
[root@master ~]# scp /etc/keepalived/keepalived.conf root@192.168.18.207:/etc/keepalived/
4.簡單修改一下slave配置文件
[root@slave keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state BACKUP #修改成BACKUP interface eth0 virtual_router_id 51 priority 100 #修改優先級 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 #persistence_timeout 50 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } }
5.啓動master與slave的keepalived服務
[root@master ~]# service keepalived start 正在啓動 keepalived: [肯定] [root@slave ~]# service keepalived start 正在啓動 keepalived: [肯定]
6.查看一下LVS狀態
[root@master ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.18.200:80 rr -> 192.168.18.201:80 Route 1 0 0 -> 192.168.18.202:80 Route 1 0 0
7.測試
8.模擬故障
(1).中止一下node1
[root@node1 src]# service httpd stop 中止 httpd: [肯定]
(2).查看一下的lvs
[root@master ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.18.200:80 rr -> 192.168.18.202:80 Route 1 0 0
(3).測試一下
(4).查看一下郵件
(5).從新啓動一下node1
[root@node1 src]# service httpd start 正在啓動 httpd: [肯定]
(6).再查看一下lvs狀態
[root@master ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.18.200:80 rr -> 192.168.18.201:80 Route 1 0 0 -> 192.168.18.202:80 Route 1 0 0
(7).再查看一下郵件
(8).關閉master上keepalived
[root@master ~]# service keepalived stop 中止 keepalived: [肯定] [root@master ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn
(9).查看一下slave狀態
[root@slave ~]# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:f9:e6:26 brd ff:ff:ff:ff:ff:ff inet 192.168.18.207/24 brd 192.168.18.255 scope global eth0 inet 192.168.18.200/32 scope global eth0 inet6 fe80::20c:29ff:fef9:e626/64 scope link valid_lft forever preferred_lft forever [root@slave ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.18.200:80 rr -> 192.168.18.201:80 Route 1 0 0 -> 192.168.18.202:80 Route 1 0 0
(10).再次測試一下
注,你們能夠看到,通過上面的演示咱們如今LVS的高可用即前端負載均衡的高可用,同時實現對後端realserver監控,也實現後端realserver宕機時會給管理員發送郵件。但還有幾個問題咱們尚未解決,問題以下:
全部realserver都down機,怎麼處理?是否是用戶就無法打開,仍是提供一下維護頁面。
怎麼完成維護模式keepalived切換?
如何在keepalived故障時,發送警告郵件給指定的管理員?
9.全部realserver都down機,怎麼處理?
問題:在集羣中若是全部real server所有宕機了,客戶端訪問時就會出現錯誤頁面,這樣是很不友好的,咱們得提供一個維護頁面來提醒用戶,服務器正在維護,什麼時間能夠訪問等,下面咱們就來解決一下這個問題。解決方案有兩種,一種是提供一臺備用的real server當全部的服務器宕機時,提供維護頁面,但這樣作有點浪費服務器。另外一種就是在負載均衡器上提供維護頁面,這樣是比較靠譜的,也比較經常使用。下面咱們就來具體操做一下。
(1).master與slave安裝上httpd
[root@master ~]# yum install -y httpd [root@slave ~]# yum install -y httpd
(2).配置維護頁面
[root@master ~]# vim /var/www/html/index.html Website is currently under maintenance, please come back later! [root@slave ~]# vim /var/www/html/index.html Website is currently under maintenance, please come back later!
(3).啓動httpd服務並測試
[root@master ~]# service httpd start 正在啓動 httpd: [肯定] [root@slave ~]# service httpd start 正在啓動 httpd: [肯定]
(4).修改配置文件
master:
[root@master ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 101 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } sorry_server 127.0.0.1 80 #增長一行sorry_server }
slave:
[root@slave ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } sorry_server 127.0.0.1 80 #增長一行sorry_server }
(5).關閉全部的real server並從新啓動一下master與slave的keepalived
[root@node1 ~]# service httpd stop 中止 httpd: [肯定] [root@node2 ~]# service httpd stop 中止 httpd: [肯定] [root@master ~]# service keepalived restart 中止 keepalived: [肯定] 正在啓動 keepalived: [肯定] [root@slave ~]# service keepalived restart 中止 keepalived: [肯定] 正在啓動 keepalived: [肯定]
(6).查看一下lvs
[root@master ~]# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.18.200:80 rr -> 127.0.0.1:80 Local 1 0 0
(7).測試
注,sorry_server測試成功,下面咱們繼續。
10.怎麼完成維護模式keepalived切換?
問題:咱們通常進行主從切換測試時都是關閉keepalived或關閉網卡接口,有沒有一種方法能實如今不關閉keepalived下或網卡接口來實現維護呢?方法確定是有的,在keepalived新版本中,支持腳本vrrp_srcipt,具體如何使用你們能夠man keepalived.conf查看。下面咱們來演示一下具體怎麼實現。
(1).定義腳本
vrrp_srcipt chk_schedown { script "[ -e /etc/keepalived/down ] && exit 1 || exit 0" interval 1 #監控間隔 weight -5 #減少優先級 fall 2 #監控失敗次數 rise 1 #監控成功次數 }
(2).執行腳本
track_script { chk_schedown #執行chk_schedown腳本 }
(3).修改配置文件
master:
[root@master ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_schedown { #定義vrrp執行腳本 script "[ -e /etc/keepalived/down ] && exit 1 || exit 0" #查看是否有down文件,有就進入維護模式 interval 1 #監控間隔時間 weight -5 #下降優先級 fall 2 #失敗次數 rise 1 #成功數次 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 101 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } track_script { #執行腳本 chk_schedown } } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } sorry_server 127.0.0.1 80 }
slave:
[root@slave ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_schedown { script "[ -e /etc/keepalived/down ] && exit 1 || exit 0" interval 1 weight -5 fall 2 rise 1 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } track_script { chk_schedown } } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } sorry_server 127.0.0.1 80
(4).測試
master:
[root@master keepalived]# touch down #新建一下down文件 [root@master keepalived]# ll 總用量 4 -rw-r--r-- 1 root root 0 8月 22 13:39 down -rw-r--r-- 1 root root 1317 8月 22 13:35 keepalived.conf [root@master keepalived]# tail -f /var/log/messages #查看一下日誌 Aug 22 13:43:52 master Keepalived_vrrp[12003]: VRRP_Instance(VI_1) Entering MASTER STATE Aug 22 13:43:52 master Keepalived_vrrp[12003]: VRRP_Instance(VI_1) setting protocol VIPs. Aug 22 13:43:52 master Keepalived_vrrp[12003]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.18.200 Aug 22 13:43:52 master Keepalived_vrrp[12003]: VRRP_Instance(VI_1) Received higher prio advert Aug 22 13:43:52 master Keepalived_vrrp[12003]: VRRP_Instance(VI_1) Entering BACKUP STATE Aug 22 13:43:52 master Keepalived_vrrp[12003]: VRRP_Instance(VI_1) removing protocol VIPs. Aug 22 13:43:52 master Keepalived_healthcheckers[12002]: Netlink reflector reports IP 192.168.18.200 added Aug 22 13:43:52 master Keepalived_healthcheckers[12002]: Netlink reflector reports IP 192.168.18.200 removed Aug 22 13:43:52 master Keepalived_healthcheckers[12002]: SMTP alert successfully sent. Aug 22 13:43:52 master Keepalived_healthcheckers[12002]: SMTP alert successfully sent. ^C [root@master keepalived]# ip add show #查看VIP 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:4b:a1:85 brd ff:ff:ff:ff:ff:ff inet 192.168.18.208/24 brd 192.168.18.255 scope global eth0 inet6 fe80::20c:29ff:fe4b:a185/64 scope link valid_lft forever preferred_lft forever
slave:
[root@slave ~]# ip addr show #查看一下VIP已轉移到slave上 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:f9:e6:26 brd ff:ff:ff:ff:ff:ff inet 192.168.18.207/24 brd 192.168.18.255 scope global eth0 inet 192.168.18.200/32 scope global eth0 inet6 fe80::20c:29ff:fef9:e626/64 scope link valid_lft forever preferred_lft forever
好了,自寫監測腳本,完成維護模式切換,到這裏就演示成功,下面咱們來解決最後一個問題,就是keepalived主從切換的郵件通告。
11.如何在keepalived故障時(或主備切換時),發送警告郵件給指定的管理員?
(1).keepalived通知腳本進階示例
下面的腳本能夠接受選項,其中
-s, --service SERVICE,...:指定服務腳本名稱,當狀態切換時可自動啓動、重啓或關閉此服務;
-a, --address VIP: 指定相關虛擬路由器的VIP地址;
-m, --mode {mm|mb}:指定虛擬路由的模型,mm表示主主,mb表示主備;它們表示相對於同一種服務而方,其VIP的工做類型;
-n, --notify {master|backup|fault}:指定通知的類型,即vrrp角色切換的目標角色;
-h, --help:獲取腳本的使用幫助;
#!/bin/bash # Author: freeloda # description: An example of notify script # Usage: notify.sh -m|--mode {mm|mb} -s|--service SERVICE1,... -a|--address VIP -n|--notify {master|backup|falut} -h|--help contact='1521076067@163.com' helpflag=0 serviceflag=0 modeflag=0 addressflag=0 notifyflag=0 Usage() { echo "Usage: notify.sh [-m|--mode {mm|mb}] [-s|--service SERVICE1,...] <-a|--address VIP> <-n|--notify {master|backup|falut}>" echo "Usage: notify.sh -h|--help" } ParseOptions() { local I=1; if [ $# -gt 0 ]; then while [ $I -le $# ]; do case $1 in -s|--service) [ $# -lt 2 ] && return 3 serviceflag=1 services=(`echo $2|awk -F"," '{for(i=1;i<=NF;i++) print $i}'`) shift 2 ;; -h|--help) helpflag=1 return 0 shift ;; -a|--address) [ $# -lt 2 ] && return 3 addressflag=1 vip=$2 shift 2 ;; -m|--mode) [ $# -lt 2 ] && return 3 mode=$2 shift 2 ;; -n|--notify) [ $# -lt 2 ] && return 3 notifyflag=1 notify=$2 shift 2 ;; *) echo "Wrong options..." Usage return 7 ;; esac done return 0 fi } #workspace=$(dirname $0) RestartService() { if [ ${#@} -gt 0 ]; then for I in $@; do if [ -x /etc/rc.d/init.d/$I ]; then /etc/rc.d/init.d/$I restart else echo "$I is not a valid service..." fi done fi } StopService() { if [ ${#@} -gt 0 ]; then for I in $@; do if [ -x /etc/rc.d/init.d/$I ]; then /etc/rc.d/init.d/$I stop else echo "$I is not a valid service..." fi done fi } Notify() { mailsubject="`hostname` to be $1: $vip floating" mailbody="`date '+%F %H:%M:%S'`, vrrp transition, `hostname` changed to be $1." echo $mailbody | mail -s "$mailsubject" $contact } # Main Function ParseOptions $@ [ $? -ne 0 ] && Usage && exit 5 [ $helpflag -eq 1 ] && Usage && exit 0 if [ $addressflag -ne 1 -o $notifyflag -ne 1 ]; then Usage exit 2 fi mode=${mode:-mb} case $notify in 'master') if [ $serviceflag -eq 1 ]; then RestartService ${services[*]} fi Notify master ;; 'backup') if [ $serviceflag -eq 1 ]; then if [ "$mode" == 'mb' ]; then StopService ${services[*]} else RestartService ${services[*]} fi fi Notify backup ;; 'fault') Notify fault ;; *) Usage exit 4 ;; esac
(2).在keepalived.conf配置文件中,其調用方法以下所示:
notify_master "/etc/keepalived/notify.sh -n master -a 192.168.18.200"
notify_backup "/etc/keepalived/notify.sh -n backup -a 192.168.18.200"
notify_fault "/etc/keepalived/notify.sh -n fault -a 192.168.18.200"
(3).修改配置文件
master:
[root@master keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_schedown { script "[ -e /etc/keepalived/down ] && exit 1 || exit 0" interval 1 weight -5 fall 2 rise 1 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 101 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } track_script { chk_schedown } #增長如下三行 notify_master "/etc/keepalived/notify.sh -n master -a 192.168.18.200" notify_backup "/etc/keepalived/notify.sh -n backup -a 192.168.18.200" notify_fault "/etc/keepalived/notify.sh -n fault -a 192.168.18.200" } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } sorry_server 127.0.0.1 80 }
slave:
[root@slave keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { 15251076067@163.com } notification_email_from root smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_schedown { script "[ -e /etc/keepalived/down ] && exit 1 || exit 0" interval 1 weight -5 fall 2 rise 1 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.18.200 } track_script { chk_schedown } #增長如下三行 notify_master "/etc/keepalived/notify.sh -n master -a 192.168.18.200" notify_backup "/etc/keepalived/notify.sh -n backup -a 192.168.18.200" notify_fault "/etc/keepalived/notify.sh -n fault -a 192.168.18.200" } virtual_server 192.168.18.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 protocol TCP real_server 192.168.18.201 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.18.202 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } sorry_server 127.0.0.1 80 }
(4).增長腳本
[root@slave keepalived]# pwd /etc/keepalived [root@slave keepalived]# vim notify.sh [root@slave keepalived]# cat notify.sh #!/bin/bash # Author: freeloda # description: An example of notify script # Usage: notify.sh -m|--mode {mm|mb} -s|--service SERVICE1,... -a|--address VIP -n|--notify {master|backup|falut} -h|--help contact='15251076067@163.com' helpflag=0 serviceflag=0 modeflag=0 addressflag=0 notifyflag=0 Usage() { echo "Usage: notify.sh [-m|--mode {mm|mb}] [-s|--service SERVICE1,...] <-a|--address VIP> <-n|--notify {master|backup|falut}>" echo "Usage: notify.sh -h|--help" } ParseOptions() { local I=1; if [ $# -gt 0 ]; then while [ $I -le $# ]; do case $1 in -s|--service) [ $# -lt 2 ] && return 3 serviceflag=1 services=(`echo $2|awk -F"," '{for(i=1;i<=NF;i++) print $i}'`) shift 2 ;; -h|--help) helpflag=1 return 0 shift ;; -a|--address) [ $# -lt 2 ] && return 3 addressflag=1 vip=$2 shift 2 ;; -m|--mode) [ $# -lt 2 ] && return 3 mode=$2 shift 2 ;; -n|--notify) [ $# -lt 2 ] && return 3 notifyflag=1 notify=$2 shift 2 ;; *) echo "Wrong options..." Usage return 7 ;; esac done return 0 fi } #workspace=$(dirname $0) RestartService() { if [ ${#@} -gt 0 ]; then for I in $@; do if [ -x /etc/rc.d/init.d/$I ]; then /etc/rc.d/init.d/$I restart else echo "$I is not a valid service..." fi done fi } StopService() { if [ ${#@} -gt 0 ]; then for I in $@; do if [ -x /etc/rc.d/init.d/$I ]; then /etc/rc.d/init.d/$I stop else echo "$I is not a valid service..." fi done fi } Notify() { mailsubject="`hostname` to be $1: $vip floating" mailbody="`date '+%F %H:%M:%S'`, vrrp transition, `hostname` changed to be $1." echo $mailbody | mail -s "$mailsubject" $contact } # Main Function ParseOptions $@ [ $? -ne 0 ] && Usage && exit 5 [ $helpflag -eq 1 ] && Usage && exit 0 if [ $addressflag -ne 1 -o $notifyflag -ne 1 ]; then Usage exit 2 fi mode=${mode:-mb} case $notify in 'master') if [ $serviceflag -eq 1 ]; then RestartService ${services[*]} fi Notify master ;; 'backup') if [ $serviceflag -eq 1 ]; then if [ "$mode" == 'mb' ]; then StopService ${services[*]} else RestartService ${services[*]} fi fi Notify backup ;; 'fault') Notify fault ;; *) Usage exit 4 ;; esac
(5).給腳本增長執行權限
[root@slave keepalived]# chmod +x notify.sh
(6).將master上腳本複製到slave上
[root@slave keepalived]# scp -p notify.sh root@192.168.18.207:/etc/keepalived/
(7).測試一下腳本
[root@slave keepalived]# ./notify.sh -h Usage: notify.sh [-m|--mode {mm|mb}] [-s|--service SERVICE1,...] <-a|--address VIP> <-n|--notify {master|backup|falut}> Usage: notify.sh -h|--help [root@slave keepalived]# ./notify.sh --help Usage: notify.sh [-m|--mode {mm|mb}] [-s|--service SERVICE1,...] <-a|--address VIP> <-n|--notify {master|backup|falut}> Usage: notify.sh -h|--help [root@slave keepalived]# ./notify.sh -m mb -a 1.1.1.1 -n master
(8).查看一下郵件
注,你們能夠看到成功收到郵件,測試成功。在模擬故障時先重啓一下keepalived服務。
(9).模擬故障
[root@master keepalived]# ip addr show #查看一下VIP 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:4b:a1:85 brd ff:ff:ff:ff:ff:ff inet 192.168.18.208/24 brd 192.168.18.255 scope global eth0 inet 192.168.18.200/32 scope global eth0 inet6 fe80::20c:29ff:fe4b:a185/64 scope link valid_lft forever preferred_lft forever [root@master keepalived]# touch down #進入維護模式 [root@master keepalived]# ll 總用量 8 -rw-r--r-- 1 root root 0 8月 22 14:39 down -rw-r--r-- 1 root root 1543 8月 22 14:04 keepalived.conf -rwxr-xr-x 1 root root 2516 8月 22 14:15 notify.sh [root@master keepalived]# ip addr show #再次查看VIP 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:4b:a1:85 brd ff:ff:ff:ff:ff:ff inet 192.168.18.208/24 brd 192.168.18.255 scope global eth0 inet6 fe80::20c:29ff:fe4b:a185/64 scope link valid_lft forever preferred_lft forever [root@slave keepalived]# ip addr show #你們能夠看到VIP成功移動到slave上 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:f9:e6:26 brd ff:ff:ff:ff:ff:ff inet 192.168.18.207/24 brd 192.168.18.255 scope global eth0 inet 192.168.18.200/32 scope global eth0 inet6 fe80::20c:29ff:fef9:e626/64 scope link valid_lft forever preferred_lft forever
(10).查看一下郵件
注,你們能夠看到,主備切換時,會發送郵件報警,好了到這裏全部演示所有完成。但願你們有所收穫^_^……