Keepalived是基於vrrp協議的一款高可用軟件。Keepailived有一臺主服務器和多臺備份服務器,在主服務器和備份服務器上面部署相同的服務配置,使用一個虛擬IP地址對外提供服務,當主服務器出現故障時,虛擬IP地址會自動漂移到備份服務器。php
VRRP(Virtual Router Redundancy Protocol,虛擬路由器冗餘協議),VRRP是爲了解決靜態路由的高可用。VRRP的基本架構
虛擬路由器由多個路由器組成,每一個路由器都有各自的IP和共同的VRID(0-255),其中一個VRRP路由器經過競選成爲MASTER,佔有VIP,對外提供路由服務,其餘成爲BACKUP,MASTER以IP組播(組播地址:224.0.0.18)形式發送VRRP協議包,與BACKUP保持心跳鏈接,若MASTER不可用(或BACKUP接收不到VRRP協議包),則BACKUP經過競選產生新的MASTER並繼續對外提供路由服務,從而實現高可用。css
vrrp協議的相關術語:html
虛擬路由器:Virtual Router 虛擬路由器標識:VRID(0-255) 物理路由器: master :主設備 backup :備用設備 priority:優先級 VIP:Virtual IP VMAC:Virutal MAC (00-00-5e-00-01-VRID) GraciousARP
安全認證:java
簡單字符認證、HMAC機制,只對信息作認證 MD5(leepalived不支持)
工做模式:node
主/備:單虛擬路徑器; 主/主:主/備(虛擬路徑器),備/主(虛擬路徑器)
工做類型:nginx
搶佔式:當出現比現有主服務器優先級高的服務器時,會發送通告搶佔角色成爲主服務器 非搶佔式:
vrrp stack:vrrp協議的實現 ipvs wrapper:爲集羣內的全部節點生成IPVS規則 checkers:對IPVS集羣的各RS作健康狀態檢測 控制組件:配置文件分析器,用來實現配置文件的分析和加載 IO複用器 內存管理組件,用來管理keepalived高可用是的內存管理
注意:web
從CentOS 6.4開始keepalived隨系統base倉庫提供,能夠使用yun -y install keepalived
安裝。算法
主配置文件:/etc/keepalived/keepalived.conf 主程序文件:/usr/sbin/keepalived 提供校驗碼:/usr/bin/genhash Unit File:keepalived.service Unit File的環境配置文件:/etc/sysconfig/keepalived
! Configuration File for keepalived global_defs { notification_email { #發送報警郵件收件地址 acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from Alexandre.Cassen@firewall.loc #指明報警郵件的發送地址 smtp_server 192.168.200.1 #郵件服務器地址 smtp_connect_timeout 30 #smtp的超時時間 router_id LVS_DEVEL #物理服務器的主機名 vrrp_mcast_group4 #定義一個組播地址 static_ipaddress { 192.168.1.1/24 dev eth0 scope global } static_routes { 192.168.2.0/24 via 192.168.1.100 dev eth0 } } vrrp_sync_group VG_1 { #定義一個故障組,組內有一個虛擬路由出現故障另外一個也會一塊兒跟着轉移,適用於LVS的NAT模型。 group { VI1 # name of vrrp_instance (below) VI2 # One for each moveable IP } } vrrp_instance VI_1 { #定義一個虛擬路由 state MASTER|BACKUP #當前節點在此虛擬路由器上的初始狀態;只能有一個是MASTER,餘下的都應該爲BACKUP; interface eth0 #綁定爲當前虛擬路由器使用的物理接口; virtual_router_id 51 #當前虛擬路由器的唯一標識,範圍是0-255; priority 100 #當前主機在此虛擬路徑器中的優先級;範圍1-254; advert_int 1 #通告發送間隔,包含主機優先級、心跳等。 authentication { #認證配置 auth_type PASS #認證類型,PASS表示簡單字符串認證 auth_pass 1111 #密碼,PASS密碼最長爲8位 virtual_ipaddress { 192.168.200.16 #虛擬路由IP地址,以輔助地址方式設置 192.168.200.18/24 dev eth2 label eth2:1 #以別名的方式設置 } track_interface { eth0 eth1 } #配置要監控的網絡接口,一旦接口出現故障,則轉爲FAULT狀態; nopreempt #定義工做模式爲非搶佔模式; preempt_delay 300 #搶佔式模式下,節點上線後觸發新選舉操做的延遲時長; virtual_routes { #配置路由信息,可選項 # src <IPADDR> [to] <IPADDR>/<MASK> via|gw <IPADDR> [or <IPADDR>] dev <STRING> scope <SCOPE> tab src 192.168.100.1 to 192.168.109.0/24 via 192.168.200.254 dev eth1 192.168.112.0/24 via 192.168.100.254 192.168.113.0/24 via 192.168.200.254 or 192.168.100.254 dev eth1 blackhole 192.168.114.0/24 } notify_master <STRING>|<QUOTED-STRING> #當前節點成爲主節點時觸發的腳本。 notify_backup <STRING>|<QUOTED-STRING> #當前節點轉爲備節點時觸發的腳本。 notify_fault <STRING>|<QUOTED-STRING> #當前節點轉爲「失敗」狀態時觸發的腳本。 notify <STRING>|<QUOTED-STRING> #通用格式的通知觸發機制,一個腳本可完成以上三種狀態的轉換時的通知。 smtp_alert #若是加入這個選項,將調用前面設置的郵件設置,並自動根據狀態發送信息 } virtual_server 192.168.200.100 443 { #LVS配置段 ,設置LVS的VIP地址和端口 delay_loop #服務輪詢的時間間隔;檢測RS服務器的狀態。 lb_algo rr #調度算法,可選rr|wrr|lc|wlc|lblc|sh|dh。 lb_kind NAT #集羣類型。 nat_mask 255.255.255.0 #子網掩碼,可選項。 persistence_timeout 50 #是否啓用持久鏈接,鏈接保存時長 protocol TCP #協議,只支持TCP sorry_server <IPADDR> <PORT> #備用服務器地址,可選項。 real_server 192.168.201.100 443 { #配置RS服務器的地址和端口 weight 1 #權重 SSL_GET { #檢測RS服務器的狀態,發送請求報文 url { path / #請求的URL digest ff20ad2481f97b1754ef3e12ecd3a9cc #對請求的頁面進行hash運算,而後和這個hash碼進行比對,若是hash碼同樣就表示狀態正常 status_code <INT> #判斷上述檢測機制爲健康狀態的響應碼,和digest二選一便可。 } #這個hash碼能夠使用genhash命令請求這個頁面生成 connect_timeout 3 #鏈接超時時間 nb_get_retry 3 #超時重試次數 delay_before_retry 3 #每次超時事後多久再進行鏈接 connect_ip <IP ADDRESS> #向當前RS的哪一個IP地址發起健康狀態檢測請求 connect_port <PORT> #向當前RS的哪一個PORT發起健康狀態檢測請求 bindto <IP ADDRESS> #發出健康狀態檢測請求時使用的源地址; bind_port <PORT> #發出健康狀態檢測請求時使用的源端口; } } }
TCP_CHECK {
connect_ip <IP ADDRESS> #向當前RS的哪一個IP地址發起健康狀態檢測請求; connect_port <PORT> #向當前RS的哪一個PORT發起健康狀態檢測請求; bindto <IP ADDRESS> #發出健康狀態檢測請求時使用的源地址; bind_port <PORT> #發出健康狀態檢測請求時使用的源端口; connect_timeout <INTEGER> #鏈接請求的超時時長; }
實驗主機
虛擬IP:192.168.166.100shell
2臺CentOS 7.3vim
CentOS 7.3 主服務器, IP:192.168.166.130 CentOS 7.3-1 備份服務器,IP:192.168.166.132
2臺CentOS 6.9
CentOS 6.9 IP:192.168.166.129 CentOS6.9-1 IP:192.168.166.131
主調度器配置
[root@CentOS7.3 ~]#yum -y install keepalived ipvsadm #安裝keepalived和LVS管理軟件ipvsadm [root@CentOS7.3 ~]#vim /etc/keepalived/keepalived.conf #配置keepalived ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 #郵件服務器的地址 smtp_connect_timeout 30 router_id CentOS7.3 #主調度器的主機名 vrrp_mcast_group4 224.26.1.1 #發送心跳信息的組播地址 } vrrp_instance VI_1 { state MASTER #主調度器的初始角色 interface eth0 #虛擬IP工做的網卡接口 virtual_router_id 66 #虛擬路由的ID priority 100 #主調度器的選舉優先級 advert_int 1 authentication { auth_type PASS #集羣主機的認證方式 auth_pass 123456 #密鑰,最長8位 } virtual_ipaddress { 192.168.166.100 #虛擬IP } } virtual_server 192.168.166.100 80 { #LVS配置段,VIP delay_loop 6 lb_algo rr #調度算法輪詢 lb_kind DR #工做模式DR nat_mask 255.255.255.0 # persistence_timeout 50 #持久鏈接,在測試時須要註釋,不然會在設置的時間內把請求都調度到一臺RS服務器上面 protocol TCP sorry_server 127.0.0.1 80 #Sorry server的服務器地址及端口 #Sorry server就是在後端的服務器所有宕機的狀況下緊急提供服務。 real_server 192.168.166.129 80 { #RS服務器地址和端口 weight 1 #RS的權重 HTTP_GET { #健康狀態檢測方法 url { path / status_code 200 #狀態斷定規則 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.166.131 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } [root@CentOS7.3 keepalived]#systemctl start keepalived #啓動keepalived [root@CentOS7.3 keepalived]#ip a l eth0 #查看虛擬路由綁定的網卡 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:b9:7d:cb brd ff:ff:ff:ff:ff:ff inet 192.168.166.130/24 brd 192.168.166.255 scope global eth0 valid_lft forever preferred_lft forever inet 192.168.166.100/32 scope global eth0 #虛擬IP已經綁定在了eth網卡上 valid_lft forever preferred_lft forever inet6 fe80::50fe:a3f3:83a0:d38a/64 scope link valid_lft forever preferred_lft forever
備份調度器的配置
[root@centos7.3-1 ~]#yum -y install keepalived ipvsadm ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id CentOS7.3-1 #備份調度器的主機名 vrrp_mcast_group4 224.26.1.1 #這個組播地址需與集羣內的其餘主機相同 } vrrp_instance VI_1 { state BACKUP #初始角色,備份服務器需設置爲BACKUP interface eth0 virtual_router_id 66 #虛擬路由的ID必定要和集羣內的其餘主機相同 priority 90 #選舉優先級,要比主調度器地一些 advert_int 1 authentication { auth_type PASS auth_pass 123456 #密鑰須要和集羣內的主服務器相同 } virtual_ipaddress { 192.168.166.100 } } #餘下配置和主服務器相同 virtual_server 192.168.166.100 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 # persistence_timeout 50 protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.166.129 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.166.131 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } [root@centos7.3-1 ~]#systemctl start keepalived #啓動備份keepalived [root@centos7.3-1 ~]#ip a l eth0 #查看虛擬路由綁定的網卡接口 [root@centos7.3-1 ~]#ip a l eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:7e:ec:ef brd ff:ff:ff:ff:ff:ff inet 192.168.166.132/24 brd 192.168.166.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::9aab:52b3:cc1e:fbef/64 scope link valid_lft forever preferred_lft forever
測試虛擬IP地址漂移
關閉主服務器的keepalived,並查看eth0接口
查看備份服務器的eth0接口,地址已經漂移到了備份服務器上面
能夠看到上圖提示有新郵件。使用mail命令查看郵件列表,都是後端服務器狀態檢測的郵件,說明配置的報警郵件生效了。應爲後端服務器尚未配置因此檢測的狀態全是down。
啓動主服務器,地址又漂移回了主服務器
配置RS服務器
RS1配置
[root@CentOS6.9 ~]#yum -y install httpd #安裝httpd服務 [root@CentOS6.9 ~]#vim lvs.sh #建立一個配置腳本 #!/bin/bash vip=192.168.166.100 #VIP地址 mask=255.255.255.255 dev=eth0:1 case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $dev $vip netmask $mask broadcast $vip up route add -host $vip dev $dev ;; stop) ifconfig $dev down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esac [root@CentOS6.9 ~]#bash lvs.sh start [root@CentOS6.9 ~]#ip a l eth0:0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:23:38:c9 brd ff:ff:ff:ff:ff:ff inet 192.168.166.129/24 brd 192.168.166.255 scope global eth0 inet 192.168.166.100/32 brd 192.168.166.100 scope global eth0:1 inet6 fe80::20c:29ff:fe23:38c9/64 scope link valid_lft forever preferred_lft forever [root@CentOS6.9 ~]#echo WebServer1 > /var/www/html/index.html [root@CentOS6.9 ~]#cat /var/www/html/index.html WebServer1 [root@CentOS6.9 ~]#service httpd start
使用ipvsadm 命令查看lvs配置信息,RS1服務器已經調度器添加進集羣。
RS2配置
[root@CentOS6.9-1 ~]#yum -y install httpd [root@CentOS6.9-1 ~]#vim lvs.sh #和上面RS1的lvx.sh內容相同 [root@CentOS6.9-1 ~]#echo WebServer2 > /var/www/html/index.html [root@CentOS6.9-1 ~]#cat /var/www/html/index.html WebServer2 [root@CentOS6.9-1 ~]#service httpd start
第二臺RS服務器上線
客戶端測試
由於使用的是輪詢算法,因此會在Web1和Web2之間來回調度。
關閉主調度器
客戶端訪問
第二套LVS信息
VIP:192.168.166.200
第一臺調度器爲備份服務器
第二臺調度器爲主服務器
第一臺調度器配置
! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id CentOS7.3 vrrp_mcast_group4 224.26.1.1 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 66 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.166.100 } } virtual_server 192.168.166.100 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 # persistence_timeout 50 protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.166.129 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.166.131 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } #第二套虛擬路由 vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 88 #ID不要和第一套虛擬路由相同 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 12345678 } virtual_ipaddress { 192.168.166.200 } } virtual_server 192.168.166.200 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 # persistence_timeout 50 protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.166.129 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.166.131 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } }
第一臺服務器
第二臺服務器
RS配置
#!/bin/bash vip=192.168.166.100 mask=255.255.255.255 dev=eth0:1 vip2=192.168.166.200 #添加一個VIP2 mask2=255.255.255.255 dev2=eth0:2 #再添加一個eth0:2別名 case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $dev $vip netmask $mask broadcast $vip up ifconfig $dev2 $vip2 netmask $mask2 broadcast $vip2 up #設置地址 route add -host $vip dev $dev route add -host $vip2 dev $dev2 ;; stop) ifconfig $dev down ifconfig $dev2 down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esac
RS
RS2
調度器LVS規則
測試
配置Nginx主機
[root@centos7.3 ~]#yum -y install nginx #安裝nginx,nginx在epel源。 [root@centos7.3 ~]#vim /etc/nginx/nginx.conf #修改nginx主配置文件 user nginx; worker_processes auto; error_log /var/log/nginx/error.log; pid /run/nginx.pid; # Load dynamic modules. See /usr/share/nginx/README.dynamic. include /usr/share/nginx/modules/*.conf; events { worker_connections 1024; } http { log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; include /etc/nginx/mime.types; default_type application/octet-stream; upstream web { #在http上下文中添加一個服務器組,web是組名。 server 192.168.166.129:80; #後端服務器的地址和端口 server 192.168.166.132:80; } # Load modular configuration files from the /etc/nginx/conf.d directory. # See http://nginx.org/en/docs/ngx_core_module.html#include # for more information. include /etc/nginx/conf.d/*.conf; server { listen [::]:80 default_server; server_name _; root /usr/share/nginx/html; # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; location / { } error_page 404 /404.html; location = /40x.html { } include /etc/nginx/conf.d/*.conf; server { listen [::]:80 default_server; server_name _; root /usr/share/nginx/html; # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; location / { } error_page 404 /404.html; location = /40x.html { } error_page 500 502 503 504 /50x.html; location = /50x.html { } }
定義兩個Nginx虛擬主機
[root@centos7.3 nginx]#vim /etc/nginx/conf.d/host.conf server { server_name www.test.com; listen 80; index index.html; root /app/web; location / { proxy_pass http://web; } }
配置keepalived
第一臺
! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id CentOS7.3 vrrp_mcast_group4 224.24.1.1 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 66 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 111111 } virtual_ipaddress { 192.168.166.100/24 dev eth0 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 88 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 11111111 } virtual_ipaddress { 192.168.166.200/24 dev eth0 } }
第二臺Nginx
! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id CentOS7.3 vrrp_mcast_group4 224.24.1.1 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 66 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 111111 } virtual_ipaddress { 192.168.166.100/24 dev eth0 } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 88 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 11111111 } virtual_ipaddress { 192.168.166.200/24 dev eth0 } }
! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id CentOS7.3-1 vrrp_mcast_group4 224.24.1.1 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 66 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 111111 } virtual_ipaddress { 192.168.166.100/24 dev eth0 }
測試
keepalived能夠調用外部的輔助腳本進行資源監控,並根據監控的結果狀態能實現優先動態調整;
先定義一個腳本
vrrp_script <SCRIPT_NAME> { #定義腳本 script "killall -0 sshd" #能夠在引號內調用命令或者腳本路徑,若是腳本執行成功則不變,若是失敗則執行下面的命令 interval INT #檢測間隔時間 weight -INT #減掉權重 fall 2 #檢測幾回斷定爲失敗 rise 2 #檢測幾回斷定爲成功 }
killall -0 只是測試,並不執行操做,用來測試進程是否運行正常
調用此腳本
track_script {
SCRIPT_NAME_1 #調用腳本 SCRIPT_NAME_2 weight -2 #若是腳本健康狀態檢測失敗優先級減2 }
配置文件
! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id CentOS7.3 vrrp_mcast_group4 224.24.1.1 } vrrp_script nginx { script "killall -0 nginx && exit 0 || exit 1" interval 1 weight -15 fall 2 rise 1 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 66 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 111111 } virtual_ipaddress { 192.168.166.100/24 dev eth0 } track_script { nginx } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 88 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 11111111 } virtual_ipaddress { 192.168.166.200/24 dev eth0 } }
測試