經過Keepalived實現Redis Failover自動故障切換功能

時間 2019-12-23

標籤經過 keepalived 實現 redis failover 自動故障切換功能欄目負載均衡简体版

原文原文鏈接

參考資料： html

http://patrick-tang.blogspot.com/2012/06/redis-keepalived-failover-system.html redis

http://deidara.blog.51cto.com/400447/302402 vim

目前，Redis尚未一個相似於MySQL Proxy或Oracle RAC的官方HA方案。
Redis做者有一個名爲Redis Sentinel的計劃(http://redis.io/topics/sentinel)，據稱將會有監控，報警和自動故障轉移三大功能，很是不錯。
但惋惜的是短時間內恐怕還不能開發完成。 bash

所以，如何在出現故障時自動轉移是一個須要解決的問題。負載均衡

經過對網上一些資料的搜索，有建議採用HAProxy或Keepalived來實現的，事實上若是是作Failover而非負載均衡的話，Keepalived的效率確定是超過HAProxy的，因此我決定採用Keepalived的方案。測試

環境介紹：
Master: 10.6.1.143
Slave: 10.6.1.144
Virtural IP Address (VIP): 10.6.1.200 加密

設計思路：
當 Master 與 Slave 均運做正常時, Master負責服務，Slave負責Standby；
當 Master 掛掉，Slave 正常時, Slave接管服務，同時關閉主從複製功能；
當 Master 恢復正常，則從Slave同步數據，同步數據以後關閉主從複製功能，恢復Master身份，於此同時Slave等待Master同步數據完成以後，恢復Slave身份。
而後依次循環。 spa

須要注意的是，這樣作須要在Master與Slave上都開啓本地化策略，不然在互相自動切換的過程當中，未開啓本地化的一方會將另外一方的數據清空，形成數據徹底丟失。設計

下面，是具體的實施步驟：日誌

在Master和Slave上安裝Keepalived
$ sudo apt-get install keepalived

修改Master和Slave的/etc/hosts文件
$ sudo vim /etc/hosts

1	127.0.0.1 localhost

2	10.6.1.143 redis

3	10.6.1.144 redis-slave

默認安裝完成keepalived以後是沒有配置文件的，所以咱們須要手動建立：

首先，在Master上建立以下配置文件：
$ sudo vim /etc/keepalived/keepalived.conf

01	vrrp_script chk_redis {

02	script "/etc/keepalived/scripts/redis_check.sh" ###監控腳本

03	interval 2 ###監控時間

04

}

05	vrrp_instance VI_1 {

06	state MASTER ###設置爲MASTER

07	interface eth0 ###監控網卡

08	virtual_router_id 51

09	priority 101 ###權重值

10	authentication {

11	auth_type PASS ###加密

12	auth_pass redis ###密碼

13

}

14	track_script {

15	chk_redis ###執行上面定義的chk_redis

16

}

17	virtual_ipaddress {

18	10.6.1.200 ###VIP

19

}

20	notify_master /etc/keepalived/scripts/redis_master.sh

21	notify_backup /etc/keepalived/scripts/redis_backup.sh

22	notify_fault /etc/keepalived/scripts/redis_fault.sh

23	notify_stop /etc/keepalived/scripts/redis_stop.sh

24

}

而後，在Slave上建立以下配置文件：
$ sudo vim /etc/keepalived/keepalived.conf

01	vrrp_script chk_redis {

02	script "/etc/keepalived/scripts/redis_check.sh" ###監控腳本

03	interval 2 ###監控時間

04

}

05	vrrp_instance VI_1 {

06	state BACKUP ###設置爲BACKUP

07	interface eth0 ###監控網卡

08	virtual_router_id 51

09	priority 100 ###比MASTRE權重值低

10	authentication {

11	auth_type PASS

12	auth_pass redis ###密碼與MASTRE相同

13

}

14	track_script {

15	chk_redis ###執行上面定義的chk_redis

16

}

17	virtual_ipaddress {

18	10.6.1.200 ###VIP

19

}

20	notify_master /etc/keepalived/scripts/redis_master.sh

21	notify_backup /etc/keepalived/scripts/redis_backup.sh

22	notify_fault /etc/keepalived/scripts/redis_fault.sh

23	notify_stop /etc/keepalived/scripts/redis_stop.sh

24

}

在Master和Slave上建立監控Redis的腳本
$ sudo mkdir /etc/keepalived/scripts
$ sudo vim /etc/keepalived/scripts/redis_check.sh

01	#!/bin/bash

02

03	ALIVE=`/opt/redis/bin/redis-cli PING`

04	if [ "$ALIVE" == "PONG" ]; then

05	echo $ALIVE

06

   exit 0 

07

 else 

08	echo $ALIVE

09

   exit 1 

10

fi

編寫如下負責運做的關鍵腳本：
notify_master /etc/keepalived/scripts/redis_master.sh
notify_backup /etc/keepalived/scripts/redis_backup.sh
notify_fault /etc/keepalived/scripts/redis_fault.sh
notify_stop /etc/keepalived/scripts/redis_stop.sh

由於Keepalived在轉換狀態時會依照狀態來呼叫：
當進入Master狀態時會呼叫notify_master
當進入Backup狀態時會呼叫notify_backup
當發現異常狀況時進入Fault狀態呼叫notify_fault
當Keepalived程序終止時則呼叫notify_stop

首先，在Redis Master上建立notity_master與notify_backup腳本：
$ sudo vim /etc/keepalived/scripts/redis_master.sh

01	#!/bin/bash

02

03	REDISCLI="/opt/redis/bin/redis-cli"

04	LOGFILE="/var/log/keepalived-redis-state.log"

05

06	echo "[master]" >> $LOGFILE

07	date >> $LOGFILE

08	echo "Being master...." >> $LOGFILE 2>&1

09

10	echo "Run SLAVEOF cmd ..." >> $LOGFILE

11	$REDISCLI SLAVEOF 10.6.1.144 6379 >> $LOGFILE 2>&1

12	sleep 10 #延遲10秒之後待數據同步完成後再取消同步狀態

13

14	echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE

15	$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

$ sudo vim /etc/keepalived/scripts/redis_backup.sh

01	#!/bin/bash

02

03	REDISCLI="/opt/redis/bin/redis-cli"

04	LOGFILE="/var/log/keepalived-redis-state.log"

05

06	echo "[backup]" >> $LOGFILE

07	date >> $LOGFILE

08	echo "Being slave...." >> $LOGFILE 2>&1

09

10	sleep 15 #延遲15秒待數據被對方同步完成以後再切換主從角色

11	echo "Run SLAVEOF cmd ..." >> $LOGFILE

12	$REDISCLI SLAVEOF 10.6.1.144 6379 >> $LOGFILE 2>&1

接着，在Redis Slave上建立notity_master與notify_backup腳本：

$ sudo vim /etc/keepalived/scripts/redis_master.sh

01	#!/bin/bash

02

03	REDISCLI="/opt/redis/bin/redis-cli"

04	LOGFILE="/var/log/keepalived-redis-state.log"

05

06	echo "[master]" >> $LOGFILE

07	date >> $LOGFILE

08	echo "Being master...." >> $LOGFILE 2>&1

09

10	echo "Run SLAVEOF cmd ..." >> $LOGFILE

11	$REDISCLI SLAVEOF 10.6.1.143 6379 >> $LOGFILE 2>&1

12	sleep 10 #延遲10秒之後待數據同步完成後再取消同步狀態

13

14	echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE

15	$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

$ sudo vim /etc/keepalived/scripts/redis_backup.sh

01	#!/bin/bash

02

03	REDISCLI="/opt/redis/bin/redis-cli"

04	LOGFILE="/var/log/keepalived-redis-state.log"

05

06	echo "[backup]" >> $LOGFILE

07	date >> $LOGFILE

08	echo "Being slave...." >> $LOGFILE 2>&1

09

10	sleep 15 #延遲15秒待數據被對方同步完成以後再切換主從角色

11	echo "Run SLAVEOF cmd ..." >> $LOGFILE

12	$REDISCLI SLAVEOF 10.6.1.143 6379 >> $LOGFILE 2>&1

而後在Master與Slave建立以下相同的腳本：
$ sudo vim /etc/keepalived/scripts/redis_fault.sh

1	#!/bin/bash

2

3	LOGFILE=/var/log/keepalived-redis-state.log

4

5	echo "[fault]" >> $LOGFILE

6	date >> $LOGFILE

$ sudo vim /etc/keepalived/scripts/redis_stop.sh

1	#!/bin/bash

2

3	LOGFILE=/var/log/keepalived-redis-state.log

4

5	echo "[stop]" >> $LOGFILE

6	date >> $LOGFILE

給腳本都加上可執行權限：
$ sudo chmod +x /etc/keepalived/scripts/*.sh

腳本建立完成之後，咱們開始按照以下流程進行測試：
1.啓動Master上的Redis
$ sudo /etc/init.d/redis start

2.啓動Slave上的Redis
$ sudo /etc/init.d/redis start

3.啓動Master上的Keepalived
$ sudo /etc/init.d/keepalived start

4.啓動Slave上的Keepalived
$ sudo /etc/init.d/keepalived start

5.嘗試經過VIP鏈接Redis:
$ redis-cli -h 10.6.1.200 INFO

鏈接成功，Slave也鏈接上來了。
role:master
slave0:10.6.1.144,6379,online

6.嘗試插入一些數據：
$ redis-cli -h 10.6.1.200 SET Hello Redis
OK

從VIP讀取數據
$ redis-cli -h 10.6.1.200 GET Hello
"Redis"

從Master讀取數據
$ redis-cli -h 10.6.1.143 GET Hello
"Redis"

從Slave讀取數據
$ redis-cli -h 10.6.1.144 GET Hello
"Redis"

下面，模擬故障產生：
將Master上的Redis進程殺死：
$ sudo killall -9 redis-server

查看Master上的Keepalived日誌
$ tailf /var/log/keepalived-redis-state.log
[fault]
Thu Sep 27 08:29:01 CST 2012

同時Slave上的日誌顯示：
$ tailf /var/log/keepalived-redis-state.log
[master]
Fri Sep 28 14:14:09 CST 2012
Being master....
Run SLAVEOF cmd ...
OK
Run SLAVEOF NO ONE cmd ...
OK

而後咱們能夠發現，Slave已經接管服務，而且擔任Master的角色了。
$ redis-cli -h 10.6.1.200 INFO
$ redis-cli -h 10.6.1.144 INFO
role:master

而後咱們恢復Master的Redis進程
$ sudo /etc/init.d/redis start

查看Master上的Keepalived日誌
$ tailf /var/log/keepalived-redis-state.log
[master]
Thu Sep 27 08:31:33 CST 2012
Being master....
Run SLAVEOF cmd ...
OK
Run SLAVEOF NO ONE cmd ...
OK

同時Slave上的日誌顯示：
$ tailf /var/log/keepalived-redis-state.log
[backup]
Fri Sep 28 14:16:37 CST 2012
Being slave....
Run SLAVEOF cmd ...
OK

能夠發現目前的Master已經再次恢復了Master的角色，故障切換以及自動恢復都成功了。