Redis 哨兵（sentinel）模式集羣配置（5.0.3版本）

時間 2020-04-22

標籤 redis 哨兵 sentinel 模式集羣配置 5.0.3 版本欄目 Redis 简体版

原文原文鏈接

1、準備工做

1.系統環境：centos6.4redis

2.服務器六臺（1主5從）：mongodb

192.168.1.161（master）centos

192.168.1.162（slave）服務器

192.168.1.163（slave）app

192.168.1.141（slave）工具

192.168.1.142（slave）測試

192.168.1.143（slave）ui

2.redis版本：5.0.3spa

3.安裝：3d

進入到目錄：cd /usr/local

下載redis：wget http://download.redis.io/releases/redis-5.0.3.tar.gz

下載完成後解壓：tar zxvf redis-5.0.3.tar.gz

重命名爲redis文件夾（這一步純屬我的喜愛）：mv redis-5.0.3 redis

進入到redis文件夾：cd redis

編譯及安裝：make && make install

特別說明：官方文檔只給出了make（編譯），沒有給出make install（安裝）

2、配置

1.Master配置，把master的redis.conf拷貝了一份命名爲redis_master.conf

設置：

### NETWORK 設置：
# bind 127.0.0.1 //綁定監控的網卡IP，註釋掉bind，任何ip都可訪問，
protected-mode no //保護模式關閉，這裏使用密碼訪問
port 17000 //設置端口，建議測試時可使用默認端口，我這裏改掉了，建議生產環境均使用自定義端口
timeout 30 //Client 端空閒斷開鏈接的時間

### GENERAL 設置：
daemonize yes //後臺模式運行
pidfile /var/run/redis_17000.pid //pid進程文件名
logfile /usr/local/redis/logs/redis.log //日誌文件的位置

### SNAPSHOTTING 設置：
dir /usr/local/redis/datas //快照文件的路徑

### APPEND ONLY MODE 設置：
appendonly yes //默認值是No，意思是不使用AOF增量持久化的方式，使用RDB全量持久化的方式。把No值改爲Yes，使用AOF增量持久化的方式
appendfsync always

###SECURITY 設置密碼:，生產環境必定要使用複雜密碼
requirepass 123456

2.Slave配置，我把master的redis.conf拷貝了一份命名爲redis_slave.conf

### NETWORK 設置：
# bind 127.0.0.1 //註釋掉bind，任何ip都可訪問
port 17000 //設置端口
protected-mode no //保護模式關閉，使用密碼訪問
timeout 30 //Client 端空閒斷開鏈接的時間

### GENERAL 設置：
daemonize yes //後臺模式運行
pidfile /var/run/redis_17000.pid
logfile /usr/local/redis/logs/redis.log //日誌文件的位置

### SNAPSHOTTING 設置：
dir /usr/local/redis/datas //SNAPSHOTTING文件的路徑

### REPLICATION 設置：
replicaof 192.168.1.161 17000 //主服務器的Ip地址和Port端口號
replica-serve-stale-data no //若是slave 沒法與master 同步，設置成slave不可讀，方便監控腳本發現問題。
masterauth 123456 //master的密碼

###SECURITY 設置密碼:，生產環境必定要使用複雜密碼
requirepass 123456

3.Sentinel（哨兵）配置，配置文件爲sentinel.conf

port 16000 //哨兵端口號
protected-mode no //關閉保護模式
daemonize yes //守護進程
dir /usr/local/redis/sentinel/ //哨兵程序的工做路徑
sentinel auth-pass mymaster 123456 //master的訪問密碼

//Sentinel去監視一個名爲mymaster的主redis實例，這個主實例的IP地址爲本機地址192.168.1.161，端口號爲17000，而將這個主實例判斷爲失效至少須要1個 Sentinel進程的贊成，只要贊成Sentinel的數量不達標，自動failover就不會執行

sentinel monitor mymaster 192.168.1.161 17000 1
sentinel down-after-milliseconds mymaster 5000 //哨兵程序每5秒檢測一次Master是否正常

//指定了在執行故障轉移時，最多能夠有多少個從Redis實例在同步新的主實例，在從Redis實例較多的狀況下這個數字越小，同步的時間越長，完成故障轉移所需的時間就越長
sentinel parallel-syncs mymaster 2

sentinel failover-timeout mymaster 300000 //若是在該時間（ms）內未能完成failover操做，則認爲該failover失敗，生產環境須要根據數據量設置該值

3、啓動Redis服務

1.在各服務器上先創建配置文件中須要的文件夾

mkdir -p /usr/local/redis/datas
mkdir -p /usr/local/redis/logs
mkdir -p /usr/local/redis/sentinel

2.啓動服務

先開啓master服務，也就是192.168.1.161服務器

進入到redis文件夾中：cd /usr/local/redis

### 啓動master:
src/redis-server redis_master.conf

注意：這裏指定了配置文件redis_master.conf

使用redis-cli訪問服務端，查看狀態

src/redis-cli -p 17000 //因爲修改了端口，因此在使用redis-cli工具時，須要指定端口

看到127.0.0.1:17000> 表示鏈接到服務端

輸入info replication 查看集羣狀態，這裏報錯：NOAUTH Authentication required.

緣由是：master和slave均設置了密碼，若是要查看replication，則須要輸入密碼受權：

AUTH 123456 --密碼就是123456（配置文件裏配置的），看到OK表示受權經過，再次輸入info replication即可以顯示集羣狀態：

[root@mongodb-161 redis]# src/redis-server redis_master.conf
[root@mongodb-161 redis]# src/redis-cli -p 17000
127.0.0.1:17000> info replication
NOAUTH Authentication required.
127.0.0.1:17000> AUTH 123456
OK
127.0.0.1:17000> info replication
# Replication
role:master
connected_slaves:0
master_replid:96942e0268bff14dba3b6d1e22a9b55e9e77343b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:17000>

這裏只是剛啓動了master一個服務，因此connected_slaves:0

再啓動192.168.1.162服務器上的redis

src/redis-server redis_slave.conf

注意：這裏指定了配置文件redis_slave.conf

啓動完成後使用redis-cli -p 17000 查看狀態

[root@mongodb-162 redis]# src/redis-server redis_slave.conf 
[root@mongodb-162 redis]# src/redis-cli -p 17000
127.0.0.1:17000> AUTH 123456
OK
127.0.0.1:17000> info replication
# Replication
role:slave
master_host:192.168.1.161
master_port:17000
master_link_status:up
master_last_io_seconds_ago:9
master_sync_in_progress:0
slave_repl_offset:28
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:b3f15ec093bb8ab2b1e908b41740f4bdabbfeba4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:28
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:28
127.0.0.1:17000>

能夠看到當前的狀態是slave，鏈接的master是192.168.1.161，這裏再來查看192.168.1.161的master上的狀態

127.0.0.1:17000> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.1.162,port=17000,state=online,offset=14,lag=1
master_replid:b3f15ec093bb8ab2b1e908b41740f4bdabbfeba4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:14
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:14
127.0.0.1:17000>

能夠看到鏈接到master有一個slave了，繼續把剩餘服務器上的redis都啓動起來，而後再查看master狀態

127.0.0.1:17000> info replication
# Replication
role:master
connected_slaves:5
slave0:ip=192.168.1.162,port=17000,state=online,offset=308,lag=1
slave1:ip=192.168.1.163,port=17000,state=online,offset=308,lag=0
slave2:ip=192.168.1.141,port=17000,state=online,offset=308,lag=1
slave3:ip=192.168.1.142,port=17000,state=online,offset=308,lag=1
slave4:ip=192.168.1.143,port=17000,state=online,offset=308,lag=0
master_replid:b3f15ec093bb8ab2b1e908b41740f4bdabbfeba4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:308
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:308
127.0.0.1:17000>

這裏5個slave均已成功鏈接到master上了。

4、啓動Sentinel（哨兵）進程

哨兵進程不必定與redis數量一致，也不必定要放在redis服務器上，sentinel的做用是監控全部服務及與其它哨兵通訊，若sentinel單獨放其它服務器上，則也須要安裝redis，sentinel只是redis軟件包中的一個服務

每臺服務器上都放了一個sentinel進程

啓動命令：src/redis-sentinel sentinel.conf

啓動後查看日誌：

[root@redis redis]# src/redis-sentinel sentinel.conf
[root@redis redis]# cat sentinel/sentinel.log 
2747:X 15 Mar 2019 16:18:52.042 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2747:X 15 Mar 2019 16:18:52.042 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2747, just started
2747:X 15 Mar 2019 16:18:52.042 # Configuration loaded
2748:X 15 Mar 2019 16:18:52.059 * Increased maximum number of open files to 10032 (it was originally set to 1024).
2748:X 15 Mar 2019 16:18:52.061 * Running mode=sentinel, port=16000.
2748:X 15 Mar 2019 16:18:52.061 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2748:X 15 Mar 2019 16:18:52.071 # Sentinel ID is 53e37f3442336b7b229fa42c655bdca9eaa6578c
2748:X 15 Mar 2019 16:18:52.072 # +monitor master mymaster 192.168.1.161 17000 quorum 1
2748:X 15 Mar 2019 16:18:52.073 * +slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.075 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.076 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.078 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.079 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.515 * +sentinel sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:54.057 * +sentinel sentinel 93da0f13c96f22e0263e4288c68390cce5e11cea 192.168.1.161 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:54.078 * +sentinel sentinel 605bf637a218de833308aa590cb3346de0eaacaf 192.168.1.162 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:55.520 * +sentinel sentinel 3607a3f65b224ee03e49a825604b40d1cc1fbeaf 192.168.1.142 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:58.040 * +sentinel sentinel d8e333f5daec7809605b8f276b1f2b44c1711459 192.168.1.143 16000 @ mymaster 192.168.1.161 17000

這裏有個警告是因爲TCP backlog 設置了511，而系統目前僅支持128，能夠先忽略。

目前全部master-slave服務均正常。

5、故障轉移

模擬master宕機，這裏直接把master進程殺掉

查看redis進程：

[root@mongodb-161 redis]# ps -e | grep redis
 2730 ?        00:00:01 redis-server
 2748 ?        00:00:02 redis-sentinel

redis-server進程號爲2730

按進程號殺掉：kill -9 2730

[root@mongodb-161 redis]# kill -9 2730
[root@mongodb-161 redis]# ps -e | grep redis
 2748 ?        00:00:02 redis-sentinel
[root@mongodb-161 redis]#

殺掉後再查看進程，已經沒有了redis-server進程了。

而後查看192.168.1.161上的sentinel日誌

[root@mongodb-161 redis]# cat sentinel/sentinel.log 
3129:X 15 Mar 2019 16:18:42.113 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3129:X 15 Mar 2019 16:18:42.113 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=3129, just started
3129:X 15 Mar 2019 16:18:42.113 # Configuration loaded
3130:X 15 Mar 2019 16:18:42.127 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3130:X 15 Mar 2019 16:18:42.129 * Running mode=sentinel, port=16000.
3130:X 15 Mar 2019 16:18:42.129 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
3130:X 15 Mar 2019 16:18:42.139 # Sentinel ID is 93da0f13c96f22e0263e4288c68390cce5e11cea
3130:X 15 Mar 2019 16:18:42.139 # +monitor master mymaster 192.168.1.161 17000 quorum 1
3130:X 15 Mar 2019 16:18:42.141 * +slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:42.143 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:42.144 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:42.146 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:42.147 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:50.335 * +sentinel sentinel 605bf637a218de833308aa590cb3346de0eaacaf 192.168.1.162 16000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:52.836 * +sentinel sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:54.374 * +sentinel sentinel 53e37f3442336b7b229fa42c655bdca9eaa6578c 192.168.1.141 16000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:55.841 * +sentinel sentinel 3607a3f65b224ee03e49a825604b40d1cc1fbeaf 192.168.1.142 16000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:18:58.361 * +sentinel sentinel d8e333f5daec7809605b8f276b1f2b44c1711459 192.168.1.143 16000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:25:46.300 # +new-epoch 1
3130:X 15 Mar 2019 16:25:46.301 # +vote-for-leader 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3130:X 15 Mar 2019 16:25:46.336 # +sdown master mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:25:46.336 # +odown master mymaster 192.168.1.161 17000 #quorum 1/1
3130:X 15 Mar 2019 16:25:46.336 # Next failover delay: I will not start a failover before Fri Mar 15 16:35:47 2019
3130:X 15 Mar 2019 16:25:46.635 # +config-update-from sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
3130:X 15 Mar 2019 16:25:46.635 # +switch-master mymaster 192.168.1.161 17000 192.168.1.162 17000
3130:X 15 Mar 2019 16:25:46.636 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.162 17000
3130:X 15 Mar 2019 16:25:46.636 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.162 17000
3130:X 15 Mar 2019 16:25:46.636 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.162 17000
3130:X 15 Mar 2019 16:25:46.636 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.162 17000
3130:X 15 Mar 2019 16:25:46.636 * +slave slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000
3130:X 15 Mar 2019 16:25:51.638 # +sdown slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000

能夠看到時間在16:25:46時master宕機後，已經進行了故障轉移，新的master是192.168.1.162，查看192.168.1.162的sentinel日誌：

[root@mongodb-162 redis]# cat sentinel/sentinel.log 
3103:X 15 Mar 2019 16:18:48.245 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3103:X 15 Mar 2019 16:18:48.245 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=3103, just started
3103:X 15 Mar 2019 16:18:48.245 # Configuration loaded
3104:X 15 Mar 2019 16:18:48.250 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3104:X 15 Mar 2019 16:18:48.251 * Running mode=sentinel, port=16000.
3104:X 15 Mar 2019 16:18:48.252 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
3104:X 15 Mar 2019 16:18:48.254 # Sentinel ID is 605bf637a218de833308aa590cb3346de0eaacaf
3104:X 15 Mar 2019 16:18:48.254 # +monitor master mymaster 192.168.1.161 17000 quorum 1
3104:X 15 Mar 2019 16:18:48.256 * +slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:48.257 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:48.259 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:48.261 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:48.262 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:48.270 * +sentinel sentinel 93da0f13c96f22e0263e4288c68390cce5e11cea 192.168.1.161 16000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:52.815 * +sentinel sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:54.355 * +sentinel sentinel 53e37f3442336b7b229fa42c655bdca9eaa6578c 192.168.1.141 16000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:55.820 * +sentinel sentinel 3607a3f65b224ee03e49a825604b40d1cc1fbeaf 192.168.1.142 16000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:18:58.341 * +sentinel sentinel d8e333f5daec7809605b8f276b1f2b44c1711459 192.168.1.143 16000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:25:46.277 # +new-epoch 1
3104:X 15 Mar 2019 16:25:46.279 # +vote-for-leader 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3104:X 15 Mar 2019 16:25:46.309 # +sdown master mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:25:46.309 # +odown master mymaster 192.168.1.161 17000 #quorum 1/1
3104:X 15 Mar 2019 16:25:46.309 # Next failover delay: I will not start a failover before Fri Mar 15 16:35:46 2019
3104:X 15 Mar 2019 16:25:46.612 # +config-update-from sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
3104:X 15 Mar 2019 16:25:46.612 # +switch-master mymaster 192.168.1.161 17000 192.168.1.162 17000
3104:X 15 Mar 2019 16:25:46.614 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.162 17000
3104:X 15 Mar 2019 16:25:46.614 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.162 17000
3104:X 15 Mar 2019 16:25:46.614 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.162 17000
3104:X 15 Mar 2019 16:25:46.614 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.162 17000
3104:X 15 Mar 2019 16:25:46.614 * +slave slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000
3104:X 15 Mar 2019 16:25:51.622 # +sdown slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000

跟161上日誌幾乎是同樣的，那麼162是怎麼被選舉出做爲master的呢，繼續查看163日誌：

[root@mongodb-163 redis]# cat sentinel/sentinel.log 
3067:X 15 Mar 2019 16:18:50.731 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3067:X 15 Mar 2019 16:18:50.731 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=3067, just started
3067:X 15 Mar 2019 16:18:50.731 # Configuration loaded
3068:X 15 Mar 2019 16:18:50.747 * Increased maximum number of open files to 10032 (it was originally set to 1024).
3068:X 15 Mar 2019 16:18:50.749 * Running mode=sentinel, port=16000.
3068:X 15 Mar 2019 16:18:50.749 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
3068:X 15 Mar 2019 16:18:50.751 # Sentinel ID is 4a32c88aa8bff64295e45f8bb9c25154f4533d2a
3068:X 15 Mar 2019 16:18:50.751 # +monitor master mymaster 192.168.1.161 17000 quorum 1
3068:X 15 Mar 2019 16:18:50.754 * +slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:50.756 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:50.757 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:50.759 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:50.760 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:52.329 * +sentinel sentinel 93da0f13c96f22e0263e4288c68390cce5e11cea 192.168.1.161 16000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:52.331 * +sentinel sentinel 605bf637a218de833308aa590cb3346de0eaacaf 192.168.1.162 16000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:54.330 * +sentinel sentinel 53e37f3442336b7b229fa42c655bdca9eaa6578c 192.168.1.141 16000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:55.796 * +sentinel sentinel 3607a3f65b224ee03e49a825604b40d1cc1fbeaf 192.168.1.142 16000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:18:58.316 * +sentinel sentinel d8e333f5daec7809605b8f276b1f2b44c1711459 192.168.1.143 16000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.247 # +sdown master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.248 # +odown master mymaster 192.168.1.161 17000 #quorum 1/1
3068:X 15 Mar 2019 16:25:46.248 # +new-epoch 1
3068:X 15 Mar 2019 16:25:46.248 # +try-failover master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.250 # +vote-for-leader 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.253 # 53e37f3442336b7b229fa42c655bdca9eaa6578c voted for 53e37f3442336b7b229fa42c655bdca9eaa6578c 1
3068:X 15 Mar 2019 16:25:46.254 # 93da0f13c96f22e0263e4288c68390cce5e11cea voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.255 # 605bf637a218de833308aa590cb3346de0eaacaf voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.255 # 3607a3f65b224ee03e49a825604b40d1cc1fbeaf voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.255 # d8e333f5daec7809605b8f276b1f2b44c1711459 voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.316 # +elected-leader master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.316 # +failover-state-select-slave master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.372 # +selected-slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.372 * +failover-state-send-slaveof-noone slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.424 * +failover-state-wait-promotion slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.528 # +promoted-slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.528 # +failover-state-reconf-slaves master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.584 * +slave-reconf-sent slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.584 * +slave-reconf-sent slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-inprog slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-done slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-inprog slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-done slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.602 * +slave-reconf-sent slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.602 * +slave-reconf-sent slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:48.588 * +slave-reconf-inprog slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:48.588 * +slave-reconf-done slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:48.588 * +slave-reconf-inprog slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:49.661 * +slave-reconf-done slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:49.732 # +failover-end master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:49.732 # +switch-master mymaster 192.168.1.161 17000 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:54.743 # +sdown slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000

在16:25:46.247時sentinel已將161進行了主觀下線（sdown，標記爲不可用），而後緊接着進行了客觀下線（odown，sentinel認爲確實不可用），這裏說明一下，爲何主觀下線後緊跟着客觀下線，緣由是配置了：

sentinel monitor mymaster 192.168.1.161 17000 1

最後這個1的意思是隻要大於等於1個sentinel認爲sdown，則認爲能夠進行odown了，因此日誌中有一句：odown master mymaster 192.168.1.161 17000 #quorum 1/1，因此odown緊跟在sdown後面。

在163的sentinel的日誌中：Sentinel ID is 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 表示163上的sentinel的ID，後面會用到這個。

當sentinel設置161爲odown（客觀下線）後，這些sentinel開始進行投票選舉，這裏特別注意，選舉的不是redis_server服務，而是sentinel服務，意思就是先選舉一個sentinel，而後由它決定將哪一個slave提高爲master。

查看141服務器上的sentinel日誌：

[root@redis redis]# cat sentinel/sentinel.log 
2747:X 15 Mar 2019 16:18:52.042 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2747:X 15 Mar 2019 16:18:52.042 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=2747, just started
2747:X 15 Mar 2019 16:18:52.042 # Configuration loaded
2748:X 15 Mar 2019 16:18:52.059 * Increased maximum number of open files to 10032 (it was originally set to 1024).
2748:X 15 Mar 2019 16:18:52.061 * Running mode=sentinel, port=16000.
2748:X 15 Mar 2019 16:18:52.061 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
2748:X 15 Mar 2019 16:18:52.071 # Sentinel ID is 53e37f3442336b7b229fa42c655bdca9eaa6578c
2748:X 15 Mar 2019 16:18:52.072 # +monitor master mymaster 192.168.1.161 17000 quorum 1
2748:X 15 Mar 2019 16:18:52.073 * +slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.075 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.076 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.078 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.079 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:52.515 * +sentinel sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:54.057 * +sentinel sentinel 93da0f13c96f22e0263e4288c68390cce5e11cea 192.168.1.161 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:54.078 * +sentinel sentinel 605bf637a218de833308aa590cb3346de0eaacaf 192.168.1.162 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:55.520 * +sentinel sentinel 3607a3f65b224ee03e49a825604b40d1cc1fbeaf 192.168.1.142 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:18:58.040 * +sentinel sentinel d8e333f5daec7809605b8f276b1f2b44c1711459 192.168.1.143 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:25:45.962 # +sdown master mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:25:45.963 # +odown master mymaster 192.168.1.161 17000 #quorum 1/1
2748:X 15 Mar 2019 16:25:45.963 # +new-epoch 1
2748:X 15 Mar 2019 16:25:45.963 # +try-failover master mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:25:45.966 # +vote-for-leader 53e37f3442336b7b229fa42c655bdca9eaa6578c 1
2748:X 15 Mar 2019 16:25:45.966 # 4a32c88aa8bff64295e45f8bb9c25154f4533d2a voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
2748:X 15 Mar 2019 16:25:45.967 # 93da0f13c96f22e0263e4288c68390cce5e11cea voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
2748:X 15 Mar 2019 16:25:45.968 # 605bf637a218de833308aa590cb3346de0eaacaf voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
2748:X 15 Mar 2019 16:25:45.968 # 3607a3f65b224ee03e49a825604b40d1cc1fbeaf voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
2748:X 15 Mar 2019 16:25:45.968 # d8e333f5daec7809605b8f276b1f2b44c1711459 voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
2748:X 15 Mar 2019 16:25:46.301 # +config-update-from sentinel 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 192.168.1.163 16000 @ mymaster 192.168.1.161 17000
2748:X 15 Mar 2019 16:25:46.301 # +switch-master mymaster 192.168.1.161 17000 192.168.1.162 17000
2748:X 15 Mar 2019 16:25:46.302 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.162 17000
2748:X 15 Mar 2019 16:25:46.302 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.162 17000
2748:X 15 Mar 2019 16:25:46.303 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.162 17000
2748:X 15 Mar 2019 16:25:46.303 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.162 17000
2748:X 15 Mar 2019 16:25:46.303 * +slave slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000
2748:X 15 Mar 2019 16:25:51.320 # +sdown slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000

由141上日誌看出：選舉後，141上的sentinel並無進行操做failover，因此能夠證明選舉是sentinel。

日誌說明：

這段日誌就是在進行選舉：

3068:X 15 Mar 2019 16:25:46.248 # +new-epoch 1
3068:X 15 Mar 2019 16:25:46.248 # +try-failover master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.250 # +vote-for-leader 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.253 # 53e37f3442336b7b229fa42c655bdca9eaa6578c voted for 53e37f3442336b7b229fa42c655bdca9eaa6578c 1
3068:X 15 Mar 2019 16:25:46.254 # 93da0f13c96f22e0263e4288c68390cce5e11cea voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.255 # 605bf637a218de833308aa590cb3346de0eaacaf voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.255 # 3607a3f65b224ee03e49a825604b40d1cc1fbeaf voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1
3068:X 15 Mar 2019 16:25:46.255 # d8e333f5daec7809605b8f276b1f2b44c1711459 voted for 4a32c88aa8bff64295e45f8bb9c25154f4533d2a 1

能夠看出id爲4a32c88aa8bff64295e45f8bb9c25154f4533d2a的sentinel有4票（共6個sentinel），它決定哪一個slave升爲master：

3068:X 15 Mar 2019 16:25:46.316 # +elected-leader master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.316 # +failover-state-select-slave master mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.372 # +selected-slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.372 * +failover-state-send-slaveof-noone slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.424 * +failover-state-wait-promotion slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.528 # +promoted-slave slave 192.168.1.162:17000 192.168.1.162 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.528 # +failover-state-reconf-slaves master mymaster 192.168.1.161 17000

最終163上的sentinel選擇了162爲master，並進行了failover

其它各服務器則進行更新配置：

3068:X 15 Mar 2019 16:25:46.584 * +slave-reconf-sent slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:46.584 * +slave-reconf-sent slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-inprog slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-done slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-inprog slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.546 * +slave-reconf-done slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.602 * +slave-reconf-sent slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:47.602 * +slave-reconf-sent slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:48.588 * +slave-reconf-inprog slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:48.588 * +slave-reconf-done slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:48.588 * +slave-reconf-inprog slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000
3068:X 15 Mar 2019 16:25:49.661 * +slave-reconf-done slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.161 17000

最後，造成新的master-slave集羣：

3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.163:17000 192.168.1.163 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.142:17000 192.168.1.142 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.143:17000 192.168.1.143 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.141:17000 192.168.1.141 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:49.733 * +slave slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000
3068:X 15 Mar 2019 16:25:54.743 # +sdown slave 192.168.1.161:17000 192.168.1.161 17000 @ mymaster 192.168.1.162 17000

查看162上的replication信息：

[root@mongodb-162 redis]# src/redis-cli -p 17000
127.0.0.1:17000> AUTH 123456
OK
127.0.0.1:17000> info replication
# Replication
role:master
connected_slaves:4
slave0:ip=192.168.1.163,port=17000,state=online,offset=947470,lag=1
slave1:ip=192.168.1.142,port=17000,state=online,offset=947470,lag=1
slave2:ip=192.168.1.141,port=17000,state=online,offset=947754,lag=1
slave3:ip=192.168.1.143,port=17000,state=online,offset=947470,lag=1
master_replid:b6c12c4d48aa953cb473480198359da669f171cc
master_replid2:b3f15ec093bb8ab2b1e908b41740f4bdabbfeba4
master_repl_offset:948180
second_repl_offset:172534
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:948180
127.0.0.1:17000>

再查看163上集羣配置：

[root@mongodb-163 redis]# src/redis-cli -p 17000
127.0.0.1:17000> AUTH 123456
OK
127.0.0.1:17000> info replication
# Replication
role:slave
master_host:192.168.1.162
master_port:17000
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:955166
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:b6c12c4d48aa953cb473480198359da669f171cc
master_replid2:b3f15ec093bb8ab2b1e908b41740f4bdabbfeba4
master_repl_offset:955166
second_repl_offset:172534
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:253
repl_backlog_histlen:954914
127.0.0.1:17000>

最後，master宕機後，master-slave自動切換成功。

6、遇到的坑

1.slave不能鏈接到master，緣由是防火牆開着的，解決方案是關閉防火牆，或把相應端口添加到防火牆中

2.master宕機後，不能自動選舉新的master，查看日誌報以下錯誤：

2230:X 15 Mar 2019 09:11:16.006 # -failover-abort-not-elected master mymaster 192.168.1.161 17000
2230:X 15 Mar 2019 09:11:16.007 # Next failover delay: I will not start a failover before Fri Mar 15 09:17:05 2019

緣由是：原先開啓了保護模式：protected-mode yes，而bind參數沒有設置正確，實際上redis_master.conf、redis_slave.conf、sentinel.conf均須要設置正確，綁定上正確的網卡IP，我這裏嫌麻煩改每臺服務器配置（配置文件是作好後上傳到服務器的），就把保護模式關閉了：protected-mode no。

官方配置中是默認開啓的，必定要注意這點。不過文檔中也建議，若是設置了訪問密碼，也能夠關閉。