keepalived 安裝及簡單示例

時間 2020-04-12

原文原文鏈接

keepalived做用

在大型網站建設中，咱們一般會用一些負載均衡技術（LVS、Nginx、Haproxy），將請求分發到後端的服務集羣中。web

這時，負載均衡均衡節點就成爲單點故障的節點，爲了保證系統的高可用，能夠引入keepalived，將多個負載均衡節點聯合起來做爲一個總體對外服務，從而防止單點故障。spring

keepalived工做原理

keepalived是以VRRP協議爲基礎實現的，VRRP全稱Virtual Router Redundancy Protocol，是一種路由容錯協議，也能夠叫作備份路由協議。一組VRRP路由器協同工做，共同構成一臺虛擬路由器。該虛擬路由器對外表現爲一個具備MAC地址和惟一固定IP地址的邏輯路由器。實際工做時，這一組路由器中的一個做爲master節點，其他做爲backup節點。工做期間，master會定時發送VRRP協議包，當master節點出現故障，backup節點收不到VRRP包，其中一個backup節點（priority大的）會自動成爲master節點，繼續提供服務，此過程徹底自動，不須要人工干預。ubuntu

keepalived利用這個原理，將不一樣的服務器節點（負載均衡節點、web服務器）組件成一個服務器組，對外提供統一的（VIP）地址，從而保證系統高可用。後端

keepalived安裝

$ mkdir -p /opt/k8s/keepalive-haproxy
$ cd /opt/k8s/keepalive-haproxy
$ wget https://www.keepalived.org/software/keepalived-2.0.20.tar.gz    
$ tar -xzvf keepalived-2.0.20.tar.gz    
$ cd /opt/k8s/keepalive-haproxy/keepalived-2.0.20
# 執行configure，prefix指定安裝目錄
$ mkdir -p /opt/k8s/keepalived/
$ ./configure --prefix=/opt/k8s/keepalived/

$ make & make install

# 查看安裝目錄
$ ls /opt/k8s/keepalived/
bin  etc  sbin  share

簡單例子

下面來以keepalived結合tomcat來實現一個web服務器的雙機熱備。網絡topology以下：tomcat

整個系統兩個節點，節點web1 192.168.0.114 (主節點), 節點web2 192.168.0.107(備用節點)，虛擬IP(對外提供服務的IP 192.168.0.100)
兩個節點上面都運行着一個一樣的web服務，暴露在8086端口，在兩個節點上都安裝keepalived，外部經過虛擬IP訪問系統
在實際工做過程當中虛擬IP在某時刻只能屬於某一個節點，另外一個節點做爲備用節點存在
當主節點正常的時候:節點web1上的keepalived廣播出去的信息中，192.168.0.100 這個IP對應的MAC地址爲節點web1網卡的MAC地址
同一網段內的機器會更新本身的ARP表，對應192.168.0.100的MAC地址=節點web1網卡的MAC地址
當節點web1發生故障的時候，節點web2上的keepalived會檢測到，而且將下面的信息廣播出去:
192.168.8.100 這個IP對應的MAC地址爲節點web2網卡的MAC地址
同一網段內的其它電腦如客戶端會更新本身的ARP表，對應192.168.8.100的MAC地址=節點web2網卡的MAC地址

配置主節點的keepalivedbash

$ mkdir /opt/k8s/keepalived/conf
$ cd /opt/k8s/keepalived/conf
$ cat > keepalived.conf<<EOF
global_defs {  
	router_id web1  
}  
vrrp_instance VI_1 {  
    state MASTER    #設置爲主服務器  
    interface wlo1  #監測網絡接口  
    virtual_router_id 55  #主、備必須同樣  
    priority 200   #(主、備機取不一樣的優先級，主機值較大，備份機值較小,值越大優先級越高)  
    advert_int 1   #VRRP Multicast廣播週期秒數  
    authentication {  
	    auth_type PASS  #VRRP認證方式，主備必須一致  
	    auth_pass 1111   #(密碼)  
    }
	virtual_ipaddress {  
	    192.168.0.100/24  #VRRP HA虛擬地址  
	}
}
EOF

配置備用節點的keepalived服務器

$ mkdir /opt/k8s/keepalived/conf
$ cd /opt/k8s/keepalived/conf
$ cat > keepalived.conf<<EOF
global_defs {  
	router_id web2  
}  
vrrp_instance VI_1 {  
    state BACKUP    #設置爲主服務器  
    interface wlp3s0  #監測網絡接口  
    virtual_router_id 55  #主、備必須同樣  
    priority 100   #(主、備機取不一樣的優先級，主機值較大，備份機值較小,值越大優先級越高)  
    advert_int 1   #VRRP Multicast廣播週期秒數  
    authentication {  
	    auth_type PASS  #VRRP認證方式，主備必須一致  
	    auth_pass 1111   #(密碼)  
    }
	virtual_ipaddress {  
	    192.168.0.100/24  #VRRP HA虛擬地址  
	}
}
EOF

在主備節點上啓動keepalived服務restful

$ /opt/k8s/keepalived/sbin/keepalived -f /opt/k8s/keepalived/conf/keepalived.conf

查看日誌網絡

主節點(ubuntu系統，在syslog中）app

$ tail -f /var/log/syslog | grep Keepalived
Mar 13 17:21:42 slave Keepalived[11843]: Starting Keepalived v2.0.20 (01/22,2020)
Mar 13 17:21:42 slave Keepalived[11843]: Running on Linux 5.3.0-28-generic #30~18.04.1-Ubuntu SMP Fri Jan 17 06:14:09 UTC 2020 (built for Linux 4.15.18)
Mar 13 17:21:42 slave Keepalived[11843]: Command line: '/opt/k8s/keepalived/sbin/keepalived' '-f' '/opt/k8s/keepalived/conf/keepalived.conf'
Mar 13 17:21:42 slave Keepalived[11843]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:21:42 slave Keepalived[11843]: Remove a zombie pid file /run/keepalived.pid
Mar 13 17:21:42 slave Keepalived[11844]: Starting VRRP child process, pid=11845
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Registering Kernel netlink reflector
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Registering Kernel netlink command channel
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: Registering gratuitous ARP shared channel
Mar 13 17:21:42 slave Keepalived_vrrp[11845]: (VI_1) Entering BACKUP STATE (init)
Mar 13 17:21:45 slave Keepalived_vrrp[11845]: (VI_1) Entering MASTER STATE

從節點

$ tail -f /var/log/syslog | grep Keepalived
Mar 13 17:24:30 master Keepalived[3118]: Starting Keepalived v2.0.20 (01/22,2020)
Mar 13 17:24:30 master Keepalived[3118]: Running on Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020 (built for Linux 4.15.18)
Mar 13 17:24:30 master Keepalived[3118]: Command line: '/opt/k8s/keepalived/sbin/keepalived' '-f' '/opt/k8s/keepalived/conf/keepalived.conf'
Mar 13 17:24:30 master Keepalived[3118]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:24:30 master Keepalived[3118]: Remove a zombie pid file /run/keepalived.pid
Mar 13 17:24:30 master Keepalived[3120]: Starting VRRP child process, pid=3122
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Registering Kernel netlink reflector
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Registering Kernel netlink command channel
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Opening file '/opt/k8s/keepalived/conf/keepalived.conf'.
Mar 13 17:24:30 master Keepalived_vrrp[3122]: Registering gratuitous ARP shared channel
Mar 13 17:24:30 master Keepalived_vrrp[3122]: (VI_1) Entering BACKUP STATE (init)

查看主節點網路設備

$ ip addr show wlo1
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether c4:8e:8f:d0:33:89 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.114/24 brd 192.168.0.255 scope global dynamic noprefixroute wlo1
       valid_lft 4948sec preferred_lft 4948sec
    inet 192.168.0.100/24 scope global secondary wlo1
       valid_lft forever preferred_lft forever
    inet6 fe80::db1e:449b:5431:7392/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

能夠看到在網卡wlo1下面的多了一個地址192.168.0.100/24

部署web服務

基於spring boot構建了一個應用，利用embeded-tomcat，裏面提供了一個restful接口,對應暴露的端口設置成8086
```
@RequestMapping("/health")
public String health(HttpServletRequest request) {

	return "OK";
}
```
在主備節點上都啓動spring boot應用

經過虛擬IP訪問

$ curl http://192.168.0.100:8086/health
OK

模擬故障以及恢復

停掉主節點的keepalived進程

$ ps -elf | grep -e keepa -e PID
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD
1 S root     11844     1  0  80   0 -  4188 ep_pol 17:21 ?        00:00:00 /opt/k8s/keepalived/sbin/keepalived -f /opt/k8s/keepalived/conf/keepalived.conf
5 S root     11845 11844  0  80   0 -  4188 ep_pol 17:21 ?        00:00:00 /opt/k8s/keepalived/sbin/keepalived -f /opt/k8s/keepalived/conf/keepalived.conf
0 S root     19389 16663  0  80   0 -  5383 pipe_w 18:06 pts/0    00:00:00 grep --color=auto -e keepa -e PID
$ kill -9 11844

查看主從節點網路設備

主節點

$ ip addr show wlo1
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether c4:8e:8f:d0:33:89 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.114/24 brd 192.168.0.255 scope global dynamic noprefixroute wlo1
       valid_lft 5707sec preferred_lft 5707sec
    inet6 fe80::db1e:449b:5431:7392/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

從節點

$ ip addr show wlp3s0
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether d0:c5:d3:57:73:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.107/24 brd 192.168.0.255 scope global dynamic noprefixroute wlp3s0
       valid_lft 4597sec preferred_lft 4597sec
    inet 192.168.0.100/24 scope global secondary wlp3s0
       valid_lft forever preferred_lft forever
    inet6 fe80::1fda:e90a:207a:67e4/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

能夠看到虛擬IP漂移到了從節點的網卡上了

經過虛擬IP訪問

$ curl http://192.168.0.100:8086/health
OK

服務正常響應

從新啓動web1上的keepalived進程，再次查看主節點上的wlo1網卡，會發現虛擬IP又自動漂移回來

腳本檢測

上一小節咱們經過模擬keepalived進程down掉，驗證了VIP自動遷移。但是當主節點上的keepalived功能正常，而只是web服務出現故障，按照以前的簡單配置，VIP是不會自動遷移的，這時服務就變的不可訪問了。解決方法是在keepalived的配置文件中追加vrrp_script

示例以下：

編寫檢測腳本

$ mkdir /opt/k8s/keepalived/script
$ cd /opt/k8s/keepalived/script
$ cat > checkproxy.sh<<EOF
#!/bin/bash
count = `ps aux | grep -v grep | grep haproxy | wc -l`
if [ $count > 0 ]; then
    exit 0
else
    exit 1
fi

在配置文件中追加檢測

...
vrrp_script checkhaproxy
{
    script "/home/checkproxy.sh"
    interval 3
    weight -150
}

vrrp_instance test
{
...

track_script
{
    checkhaproxy
}

...
}

關於check中的weight
- vrrp_script 裏的script返回值爲0時認爲檢測成功，其它值都會當成檢測失敗；
- weight 爲正時，腳本檢測成功時此weight會加到priority上，檢測失敗時不加；
  - 主失敗: 主 priority < 從 priority + weight 時會切換。
  - 主成功：主 priority + weight > 從 priority + weight 時，主依然爲主
- weight 爲負時，腳本檢測成功時此weight不影響priority，檢測失敗時priority – abs(weight)
  - 主失敗: 主 priority – abs(weight) < 從priority 時會切換主從
  - 主成功: 主 priority > 從priority 主依然爲主

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。