【原創】運維基礎之keepalived

時間 2019-11-21

原文原文鏈接

keepalived 2.0.12html

官方：http://www.keepalived.org/oracle

一簡介

Keepalived is a routing software written in C. The main goal of this project is to provide simple and robust facilities for loadbalancing and high-availability to Linux system and Linux based infrastructures. Loadbalancing framework relies on well-known and widely used Linux Virtual Server (IPVS) kernel module providing Layer4 loadbalancing. Keepalived implements a set of checkers to dynamically and adaptively maintain and manage loadbalanced server pool according their health. On the other hand high-availability is achieved by VRRP protocol. VRRP is a fundamental brick for router failover. In addition, Keepalived implements a set of hooks to the VRRP finite state machine providing low-level and high-speed protocol interactions. In order to offer fastest network failure detection, Keepalived implements BFD protocol. VRRP state transition can take into account BFD hint to drive fast state transition. Keepalived frameworks can be used independently or all together to provide resilient infrastructures.負載均衡

keepalived是用c寫的路由軟件，使用vrrp協議（Virtual Router Redundancy Protocol）和arp協議 (Address Resolution Protocol)實現簡單和健壯的負載均衡和高可用；tcp

VRRP 將局域網的一組路由器（包括一個Master 即活動路由器和若干個Backup 即備份路由器）組織成一個虛擬路由器，稱之爲一個備份組。這個虛擬的路由器擁有本身的IP 地址10.100.10.1（這個IP 地址能夠和備份組內的某個路由器的接口地址相同，相同的則稱爲ip擁有者），備份組內的路由器也有本身的IP 地址（如Master的IP 地址爲10.100.10.2，Backup 的IP 地址爲10.100.10.3）。局域網內的主機僅僅知道這個虛擬路由器的IP 地址10.100.10.1，而並不知道具體的Master 路由器的IP 地址10.100.10.2 以及Backup 路由器的IP 地址10.100.10.3。它們將本身的缺省路由下一跳地址設置爲該虛擬路由器的IP 地址10.100.10.1。ide

原理

主從節點之間經過廣播或組播的方式發送vrrp包，而後根據priority來選舉出masteroop

14:20:21.521870 IP 192.168.0.1 > 192.168.0.2: VRRPv2, Advertisement, vrid 99, prio 100, authtype simple, intvl 1s, length 20ui

一旦master必定時間內沒有及時發出vrrp包出來，則其餘standby會發vrrp包再根據priority選舉出master；this

master會發送arp包，日誌

Jan 28 19:04:26 cdp-test-server-05 Keepalived_vrrp[27675]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 192.168.0.3
Jan 28 19:04:26 cdp-test-server-05 Keepalived_vrrp[27675]: Sending gratuitous ARP on eth0 for 192.168.0.3router

注意這裏是虛擬ip（vip）的gratuitous ARP，

先看ARP (Address Resolution Protocol, 地址解析協議)，將IP地址轉換爲MAC地址

ARP的過程：在Host A上發送ARP請求，內容爲who has [IP_B], tell [IP_A], 包裏攜帶了主機B的IP地址，以及主機A的IP和MAC。收到廣播包的全部主機會檢查請求的IP 地址是不是本身的，若是是，就會發送一個ARP應答（單播，從B到A），內容爲 [IP_B] is at [MAC_B]，包裏攜帶了主機A和B的MAC及IP地址。

# arping 192.168.0.1

剛纔的場景中若是Host A發請求的時候，內容爲who has [IP_A], tell [IP_A]，則這是一個gratuitous ARP，爲何會請求本身的IP，由於：正常的ARP是向其餘主機請求信息，而免費ARP是主動向其餘主機廣播本身的信息，因此免費ARP不期待響應；

發送gratuitous ARP後收到廣播包的全部主機或者交換機均可以經過命令查看vip和mac（master mac）映射：

# arp -a

這樣其餘主機就能夠經過vip訪問到master，也能夠經過arp手工綁定

# arp -s 192.168.0.3 00-02-b3-3c-16-95

另外能夠經過設置vrrp_garp_master_refresh來讓master按期發送gratuitous ARP包；

若是是在雲主機環境，就不用考慮搭keepalived了，由於雲上一般會禁止vrrp協議的組播以及arp，能夠考慮直接用雲平臺的虛擬ip服務；

二安裝

# yum install keepalived

主節點配置

# vi /etc/keepalived/keepalived.conf

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    unicast_src_ip 192.168.0.1

    unicast_peer {

        192.168.0.2

    }

    virtual_router_id 51

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.0.3

    }

}

virtual_server 192.168.0.3 81 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

    nat_mask 255.255.255.0

    persistence_timeout 50

    protocol TCP

    #sorry_server 127.0.0.1 80

    real_server 192.168.0.1 80 {

        weight 1

    }

    real_server 192.168.0.2 80 {

        weight 1

    }

}

這裏使用的是單播（unicast_src_ip、unicast_peer ）的方式，由於不少環境下組播不能用，若是想用組播，把單播參數去掉便可；

若是real_server和keealived部署在一臺機器上，不須要配置virtual_server；

註釋掉

#vrrp_strict

不然會在iptables裏生成一條drop規則；

從節點修改配置

state BACKUP
unicast_src_ip 對調
unicast_peer 對調
priority 50

若是開啓iptables須要增長規則

# iptables -I INPUT -d 224.0.0.0/8 -j ACCEPT
# iptables -A INPUT -p vrrp -j ACCEPT

啓動

# service keepalived start

查看vip

# ip a

查看tcp包

# tcpdump -p vrrp -n

參考：https://docs.oracle.com/cd/E37670_01/E41138/html/section_ksr_psb_nr.html

日誌位於/var/log/messages，若是報錯：

Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: Registering Kernel netlink reflector
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: Registering Kernel netlink command channel
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: Registering gratuitous ARP shared channel
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: (VI_1): Cannot start in MASTER state if not address owner
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: Unable to load ipset library - libipset.so.11: cannot open shared object file: No such file or directory
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: VRRP_Instance(VI_1) removing protocol VIPs.
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: VRRP_Instance(VI_1) removing protocol iptable drop rule
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: Using LinkWatch kernel netlink reflector...
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 27 17:05:05 cdp-test-server-05 Keepalived_vrrp[26508]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]

須要安裝ipset

# yum install ipset

而後正常

Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: Registering Kernel netlink reflector
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: Registering Kernel netlink command channel
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: Registering gratuitous ARP shared channel
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: (VI_1): Cannot start in MASTER state if not address owner
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: iptc_commit returned 0: No chain/target/match by that name
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) removing protocol VIPs.
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) removing protocol iptable drop rule
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: Using LinkWatch kernel netlink reflector...
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 27 21:40:35 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Jan 27 21:40:38 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) setting protocol iptable drop rule
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 192.168.0.3
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:39 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:44 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:44 cdp-test-server-05 Keepalived_vrrp[7001]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 192.168.0.3
Jan 27 21:40:44 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:44 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:44 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3
Jan 27 21:40:44 cdp-test-server-05 Keepalived_vrrp[7001]: Sending gratuitous ARP on eth0 for 192.168.0.3