keepalived實現haproxy高可用詳解

時間  2019-12-19
標籤 keepalived 實現 haproxy 可用 詳解 欄目 負載均衡 简体版
原文   原文鏈接

一，keepalived介紹bash


keepalived是一個能夠實現某些資源高可用的開源軟件，其主要的組件包括core，check，vrrp，libipfwc，libipvs，這裏說下各個組件的功能。
服務器

core：keepalived的核心組件，負責主進程的啓動和維護以及加載解析配置文件等。
frontend

check：負責healthchecker,負責各類健康檢查方式，和對應的配置解析以及LVS的配置解析。
dom

vrrp：vrrpd的子進程。
tcp

libipfwc：結合iptables的ipchains庫來使用。
ide

libipvs：結合LVS使用。
oop

keepalived啓動後會生成3個進程，master主進程，VRRP子進程，healthchecker子進程。
ui

VRRP協議是實現keepalived高可用的一個基礎，下面說一下VRRP的實現原理：
spa

VRRP虛擬路由(VRRP router)，VRRP是一個「選舉」協議，它可以動態地將一個虛擬路由器的責任指定至同一個VRRP組中的其它路由器上，VRRP的優點：
router

冗餘：可使用多個路由器設備做爲LAN客戶端的默認網關，大大下降了默認網關成爲單點故障的可能性；

負載共享：容許來自LAN客戶端的流量由多個路由器設備所共享；

多VRRP組：在一個路由器物理接口上可配置多達255個VRRP組；

多IP地址：基於接口別名在同一個物理接口上配置多個IP地址，從而支持在同一個物理接口上接入多個子網；

搶佔：在master故障時容許優先級更高的backup成爲master；

通告協議：使用IANA所指定的組播地址224.0.0.18進行VRRP通告；

VRRP追蹤：基於接口狀態來改變其VRRP優先級來肯定最佳的VRRP路由器成爲master；


二，實驗環境：

192.168.30.116 OS：Centos 6.4 x86_64   master.luojianlong.com

192.168.30.117 OS：Centos 6.4 x86_64   backup.luojianlong.com

keepalived版本：keepalived-1.2.7

wKioL1M7yieRtbFrAAD5jvAENKQ351.jpg


首先，分別在2臺服務器上，安裝keepalived,haproxy，因爲系統版本是Centos 6.4，這2個軟件已經被整合在內部了，因此使用yum來安裝

[root@master ~]# yum -y install keepalived haproxy
[root@backup ~]# yum -y install keepalived haproxy


接下來，先配置master與backup服務器的優先級，virtual_router_id（同一個實例2臺服務器必須相同），編輯配置文件:

[root@master ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@backup ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@master ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.30.230
    }
}
[root@backup ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.30.230
    }
}


global_defs：全局配置標識破；

notification_email：表示告警時發送的郵件地址；

notification_email_from：表示發送郵件的源地址；

smtp_server：發送郵件的smtp服務器地址；

router_id：機器標識；

vrrp_instance：定義一個vrrp實例；

state：state指定instance(Initial)的初始狀態，就是說在配置好後，這臺服務器的初始狀態就是這裏指定的，但這裏指定的不算，仍是得要經過競選經過優先級來肯定，裏若是這裏設置爲master，但如若他的優先級不及另一臺，那麼這臺在發送通告時，會發送本身的優先級，另一臺發現優先級不如本身的高，那麼他會就回搶佔爲master；

interface：實例綁定的網卡，由於在配置虛擬IP的時候必須是在已有的網卡上添加的；

virtual router id：這裏設置VRID，這裏很是重要，相同的VRID爲一個組，他將決定多播的MAC地址；

priority 100：設置本節點的優先級，優先級高的爲master；

advert int：檢查間隔，默認爲1秒；

virtual ipaddress：這裏設置的就是VIP，也就是虛擬IP地址，他隨着state的變化而增長刪除，當state爲master的時候就添加，當state爲backup的時候刪除，這裏主要是有優先級來決定的，和state設置的值沒有多大關係，這裏能夠設置多個IP地址；

authentication：這裏設置認證；

auth type：認證方式，能夠是PASS或AH兩種認證方式；

auth pass：認證密碼；


啓動倆臺服務器的keepalived

[root@master ~]# service keepalived start
Starting keepalived:                                       [  OK  ]
[root@backup ~]# service keepalived start
Starting keepalived:                                       [  OK  ]
[root@master ~]# tail -f /var/log/messages
Jan 10 11:40:56 localhost Keepalived_healthcheckers[19368]: Using LinkWatch kernel netlink reflector...
Jan 10 11:40:56 localhost Keepalived_vrrp[19369]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 10 11:40:56 localhost Keepalived_vrrp[19369]: Configuration is using : 63019 Bytes
Jan 10 11:40:56 localhost Keepalived_vrrp[19369]: Using LinkWatch kernel netlink reflector...
Jan 10 11:40:56 localhost Keepalived_vrrp[19369]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jan 10 11:40:57 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 10 11:40:58 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 10 11:40:58 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 10 11:40:58 localhost Keepalived_healthcheckers[19368]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 11:40:58 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 11:41:03 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 11:41:14 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Jan 10 11:41:14 localhost Keepalived_vrrp[19369]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
[root@master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:f3:fc:ba brd ff:ff:ff:ff:ff:ff
    inet 192.168.30.116/24 brd 192.168.30.255 scope global eth0
    inet 192.168.30.230/32 scope global eth0
    inet6 fe80::20c:29ff:fef3:fcba/64 scope link
       valid_lft forever preferred_lft forever

發現剛纔定義的virtual ipaddress在master服務器上，由於優先級較高


中止master服務器的keepalived服務器，看IP會不會轉移到backup

[root@master ~]# service keepalived stop
Stopping keepalived:                                       [  OK  ]
[root@backup ~]# tail -f /var/log/messages
Jan 10 12:12:46 localhost Keepalived_vrrp[18581]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 10 12:12:46 localhost Keepalived_vrrp[18581]: Configuration is using : 63017 Bytes
Jan 10 12:12:46 localhost Keepalived_vrrp[18581]: Using LinkWatch kernel netlink reflector...
Jan 10 12:12:46 localhost Keepalived_healthcheckers[18580]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 10 12:12:46 localhost Keepalived_healthcheckers[18580]: Configuration is using : 7324 Bytes
Jan 10 12:12:46 localhost Keepalived_healthcheckers[18580]: Using LinkWatch kernel netlink reflector...
Jan 10 12:12:46 localhost Keepalived_vrrp[18581]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jan 10 12:12:47 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 10 12:12:47 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Received higher prio advert
Jan 10 12:12:47 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 10 12:16:27 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 10 12:16:28 localhost Keepalived_healthcheckers[18580]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 12:16:28 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:16:33 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
[root@backup ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:5b:50:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.30.117/24 brd 192.168.30.255 scope global eth0
    inet 192.168.30.230/32 scope global eth0
    inet6 fe80::20c:29ff:fe5b:50f9/64 scope link
       valid_lft forever preferred_lft forever

發現IP已經轉移到backup服務器

下面從新啓動master的keepalived

[root@master ~]# service keepalived start
Starting keepalived:                                       [  OK  ]
[root@backup ~]# tail -f /var/log/messages
Jan 10 12:12:46 localhost Keepalived_vrrp[18581]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jan 10 12:12:47 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 10 12:12:47 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Received higher prio advert
Jan 10 12:12:47 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 10 12:16:27 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 10 12:16:28 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 10 12:16:28 localhost Keepalived_healthcheckers[18580]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 12:16:28 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:16:33 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:18:20 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Received higher prio advert
Jan 10 12:18:20 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 10 12:18:20 localhost Keepalived_vrrp[18581]: VRRP_Instance(VI_1) removing protocol VIPs.
Jan 10 12:18:20 localhost Keepalived_healthcheckers[18580]: Netlink reflector reports IP 192.168.30.230 removed
[root@master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:f3:fc:ba brd ff:ff:ff:ff:ff:ff
    inet 192.168.30.116/24 brd 192.168.30.255 scope global eth0
    inet 192.168.30.230/32 scope global eth0
    inet6 fe80::20c:29ff:fef3:fcba/64 scope link
       valid_lft forever preferred_lft forever

發現IP已經從新轉移到master服務器


如今編寫haproxy狀態檢測腳本，來實現haproxy的健康檢測：

[root@master ~]# cat haproxy_pid.sh
#!/bin/bash
while :
do
haproxypid=`ps -C haproxy --no-header | wc -l`
if [ $haproxypid -eq 0 ];then
   service haproxy start
   sleep 5
   haproxypid=`ps -C haproxy --no-header | wc -l`
   echo $haproxypid
     if [ $haproxypid -eq 0 ];then
   /etc/init.d/keepalived stop
     fi
fi
sleep 5
done
# 啓動backup的haproxy
[root@backup ~]# service haproxy start
Starting haproxy:                                          [  OK  ]

模擬故障，先讓httpd進程開啓，修改haproxy監聽端口爲80，使得haproxy進程沒法啓動，看資源會不會轉移到backup服務器

[root@master ~]# vi /etc/haproxy/haproxy.cfg
frontend  main *:5000 改成frontend  main *:80
# 啓動httpd進程
[root@master ~]# scp -pr /etc/haproxy/haproxy.cfg root@192.168.30.117:/etc/haproxy/
[root@master ~]# service httpd start
Starting httpd: httpd: apr_sockaddr_info_get() failed for master.luojianlong.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
                                                           [  OK  ]
[root@master ~]# netstat -antpl | grep :80
tcp        0      0 :::80                       :::*                        LISTEN      19965/httpd
#運行狀態檢測腳本
[root@master ~]# nohup /root/haproxy_pid.sh &
[root@master ~]# scp -pr haproxy_pid.sh root@192.168.30.117:/root/
[root@bakcup ~]# nohup /root/haproxy_pid.sh &
[root@master ~]# tail -f /var/log/messages
Jan 10 12:02:29 localhost Keepalived_vrrp[19849]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jan 10 12:02:29 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 10 12:02:30 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 10 12:02:30 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 10 12:02:30 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:02:30 localhost Keepalived_healthcheckers[19848]: Netlink reflector reports IP 192.168.30.230 added
Jan 10 12:02:35 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.30.230
Jan 10 12:14:49 localhost Keepalived[19847]: Stopping Keepalived v1.2.7 (02/21,2013)
Jan 10 12:14:49 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) sending 0 priority
Jan 10 12:14:49 localhost Keepalived_vrrp[19849]: VRRP_Instance(VI_1) removing protocol VIPs.
[root@master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:f3:fc:ba brd ff:ff:ff:ff:ff:ff
    inet 192.168.30.116/24 brd 192.168.30.255 scope global eth0
    inet6 fe80::20c:29ff:fef3:fcba/64 scope link
       valid_lft forever preferred_lft forever
[root@backup ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:5b:50:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.30.117/24 brd 192.168.30.255 scope global eth0
    inet 192.168.30.230/32 scope global eth0
    inet6 fe80::20c:29ff:fe5b:50f9/64 scope link
       valid_lft forever preferred_lft forever
[root@backup ~]# ps aux | grep haproxy
haproxy  19054  0.0  0.0  18688  1280 ?        Ss   12:47   0:00 /usr/sbin/haproxy -D -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid
root     19097  0.0  0.0 103248   828 pts/0    S+   12:58   0:00 grep haproxy

發現IP已經轉移到backup服務器，實現了keepalived對於haproxy故障的高可用。


keepalived常見的啓動報錯：

5913 May 16 15:26:04 localhost Keepalived_vrrp: ip address associated with VRID not present in received packet : 192.168.57.75
5914 May 16 15:26:04 localhost Keepalived_vrrp: one or more VIP associated with VRID mismatch actual MASTER advert
5915 May 16 15:26:04 localhost Keepalived_vrrp: bogus VRRP packet received on eth0 !!!
5916 May 16 15:26:04 localhost Keepalived_vrrp: VRRP_Instance(VI_1) ignoring received advertisment...
5917 May 16 15:26:05 localhost Keepalived_vrrp: ip address associated with VRID not present in received packet : 192.168.57.75
5918 May 16 15:26:05 localhost Keepalived_vrrp: one or more VIP associated with VRID mismatch actual MASTER advert
5919 May 16 15:26:05 localhost Keepalived_vrrp: bogus VRRP packet received on eth0 !!!
5920 May 16 15:26:05 localhost Keepalived_vrrp: VRRP_Instance(VI_1) ignoring received advertisment.

解決方法：

在同一網段內virtual_router_id 值不能相同，若是相同會在messages中收到VRRP錯誤包 ，因此須要更改 virual_router_id。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程