Redis 搭建一主二從三哨兵高可用集羣

一、單個redis服務搭建請參考：redis服務搭建html

二、在/usr/local下建立目錄redis-cluster,並在redis-cluster下建立 637九、6380、6381目錄以及data、temp目錄java

# cd /usr/local # mkdir redis-cluster

--其餘文件建立相似，此處不一一寫出

三、複製安裝後的redis的配置文件（個人在/etc/目錄下）的redis.conf 和 sentinel.conf文件到 637九、6380、6381目錄中.redis

# cp /etc/redis.conf /usr/local/redis-cluster/6379 # cp /etc/redis-sentinel.conf /usr/local/redis-cluster/6379

--6380和6379相似，此處不一一寫出

四、主從配置，修改 redis.conf文件 centos

主redis 6379目錄:服務器

protected-mode yes port 6379 daemonize yes pidfile /var/run/redis_6379.pid
logfile /var/log/redis/redis_6379.log
dir /usr/local/redis-cluster/6379/data
slave-read-only yes
requirepass foo(設置訪問登陸密碼)

#從服務設置了密碼須要加上 
masterauth foo

從slave1 6380:測試

protected-mode yes port 6380 daemonize yes pidfile /var/run/redis_6380.pid logfile /var/log/redis/redis_6380.log dir /usr/local/redis-cluster/6380/data slaveof 127.0.0.1 6379 
#若主服務設置了密碼須要加上 
masterauth foo #從服務密碼設置 
requirepass foo

從slave2 6381:ui

protected-mode yes port 6381 daemonize yes pidfile /var/run/redis_6381.pid logfile /var/log/redis/redis_6381.log dir /usr/local/redis-cluster/6381/data slaveof 127.0.0.1 6379 #若主服務設置了密碼須要加上，在設置哨兵時主從之間鏈接須要 masterauth foo #從服務密碼設置 requirepass foo

master既能夠讀,也能夠寫,而從服務器是隻能夠讀,不可寫的.spa

五、哨兵配置.日誌

主redis 6379 sentinel.confcode

protected-mode no port 26379
dir "/usr/local/redis-cluster/6379/temp" sentinel monitor redis1 127.0.0.1 6379 2 sentinel down-after-milliseconds redis1 10000 sentinel failover-timeout redis1 60000

其中redis1可自定義

從slave1 6380 centinel.conf

protected-mode no port 26380
dir "/usr/local/redis-cluster/6380/temp" sentinel monitor redis1 127.0.0.1 6379 2 sentinel down-after-milliseconds redis1 10000 sentinel failover-timeout redis1 60000

從slave2 7003 sentinel.conf

protected-mode no port 26381
dir "/home/redis/redis-cluster/6381/temp" sentinel monitor redis1 127.0.0.1 6381 2 sentinel down-after-milliseconds redis1 10000 sentinel failover-timeout redis1 60000

哨兵配置完成.

六、啓動

啓動redis
分別到637九、6380、6381的目錄下執行啓動命令:

redis-server ./redis.conf

啓動哨兵

redis-server ./sentinel.conf

七、查看主從信息:

127.0.0.1:6379> info replication # Replication role:master connected_slaves:2 slave0:ip=127.0.0.1,port=6380,state=online,offset=420,lag=1 slave1:ip=127.0.0.1,port=6381,state=online,offset=420,lag=1 master_repl_offset:434 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:433

127.0.0.1:6380> info replication # Replication role:slave master_host:127.0.0.1 master_port:6379 master_link_status:up master_last_io_seconds_ago:1 master_sync_in_progress:0 slave_repl_offset:168 slave_priority:100 slave_read_only:1 connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

127.0.0.1:6381> info replication # Replication role:slave master_host:127.0.0.1 master_port:6379 master_link_status:up master_last_io_seconds_ago:9 master_sync_in_progress:0 slave_repl_offset:406 slave_priority:100 slave_read_only:1 connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

八、驗證

127.0.0.1:6379> set test 123 OK

127.0.0.1:6380> get test "123"
127.0.0.1:6380> set hh 6380 (error) READONLY You can't write against a read only slave.

127.0.0.1:6381> get test "123"
127.0.0.1:6381> set xx 6381 (error) READONLY You can't write against a read only slave.

此時,咱們能夠把主服務或者從服務停掉進行測試來看看效果

停掉主服務6379:

[root@VM_0_15_centos 6379]# ps -ef |grep redis root 10690 30720  0 18:24 pts/4    00:00:00 tailf -n 200 /var/log/redis/sentinel.log root 15650     1  0 19:01 ?        00:00:01 redis-sentinel *:26379 [sentinel] root 16404     1  0 19:06 ?        00:00:00 redis-sentinel *:26381 [sentinel] root 16565     1  0 19:07 ?        00:00:00 redis-sentinel *:26380 [sentinel] root 17248 28933  0 19:12 pts/0    00:00:00 grep --color=auto redis root 31060     1  0 16:55 ?        00:00:06 redis-server *:6379 root 31488     1  0 16:58 ?        00:00:06 redis-server *:6380 root 31563 29043  0 16:58 pts/1    00:00:00 redis-cli -c -p 6380 -a foo root 31952     1  0 17:01 ?        00:00:06 redis-server *:6381 root 32057 29125  0 17:02 pts/2    00:00:00 redis-cli -c -p 6381 -a foo [root@VM_0_15_centos 6379]# kill 31060

sentinel日誌：

15650:X 18 Dec 19:13:44.834 # +sdown master mymaster 127.0.0.1 6379
16565:X 18 Dec 19:13:44.852 # +sdown master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:44.861 # +sdown master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:44.920 # +odown master mymaster 127.0.0.1 6379 #quorum 3/2
16404:X 18 Dec 19:13:44.920 # +new-epoch 1
16404:X 18 Dec 19:13:44.920 # +try-failover master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:44.927 # +vote-for-leader 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 1
15650:X 18 Dec 19:13:44.934 # +new-epoch 1
16565:X 18 Dec 19:13:44.934 # +new-epoch 1
16565:X 18 Dec 19:13:44.940 # +vote-for-leader 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 1
15650:X 18 Dec 19:13:44.940 # +vote-for-leader 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 1
16404:X 18 Dec 19:13:44.940 # 69c00be66f0461192b2db901ece6282e00b6462c voted for 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 1
16404:X 18 Dec 19:13:44.940 # f51e8307952eba4264cc9089adf3c716e658609f voted for 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 1
16565:X 18 Dec 19:13:44.942 # +odown master mymaster 127.0.0.1 6379 #quorum 2/2
16565:X 18 Dec 19:13:44.942 # Next failover delay: I will not start a failover before Tue Dec 18 19:14:05 2018
16404:X 18 Dec 19:13:45.003 # +elected-leader master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:45.003 # +failover-state-select-slave master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:45.075 # +selected-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:45.075 * +failover-state-send-slaveof-noone slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:45.166 * +failover-state-wait-promotion slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
15650:X 18 Dec 19:13:45.900 # +odown master mymaster 127.0.0.1 6379 #quorum 3/2
15650:X 18 Dec 19:13:45.900 # Next failover delay: I will not start a failover before Tue Dec 18 19:14:05 2018
16404:X 18 Dec 19:13:45.960 # +promoted-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:45.960 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:46.011 * +slave-reconf-sent slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
16565:X 18 Dec 19:13:46.012 # +config-update-from sentinel 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 127.0.0.1 26381 @ mymaster 127.0.0.1 6379
16565:X 18 Dec 19:13:46.012 # +switch-master mymaster 127.0.0.1 6379 127.0.0.1 6381
16565:X 18 Dec 19:13:46.012 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381
16565:X 18 Dec 19:13:46.012 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
15650:X 18 Dec 19:13:46.013 # +config-update-from sentinel 6d5a34396cd5912cbfe1134a70cd3e14338ebf24 127.0.0.1 26381 @ mymaster 127.0.0.1 6379
15650:X 18 Dec 19:13:46.013 # +switch-master mymaster 127.0.0.1 6379 127.0.0.1 6381
15650:X 18 Dec 19:13:46.013 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381
15650:X 18 Dec 19:13:46.013 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
16404:X 18 Dec 19:13:47.002 * +slave-reconf-inprog slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:47.002 * +slave-reconf-done slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:47.067 # -odown master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:47.067 # +failover-end master mymaster 127.0.0.1 6379
16404:X 18 Dec 19:13:47.067 # +switch-master mymaster 127.0.0.1 6379 127.0.0.1 6381
16404:X 18 Dec 19:13:47.067 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381
16404:X 18 Dec 19:13:47.067 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
16565:X 18 Dec 19:14:16.015 # +sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
15650:X 18 Dec 19:14:16.017 # +sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
16404:X 18 Dec 19:14:17.154 # +sdown slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381

查看replication:

127.0.0.1:6380> info replication # Replication role:slave master_host:127.0.0.1 master_port:6381 master_link_status:up master_last_io_seconds_ago:1 master_sync_in_progress:0 slave_repl_offset:72129 slave_priority:100 slave_read_only:1 connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

127.0.0.1:6381> info replication # Replication role:master connected_slaves:1 slave0:ip=127.0.0.1,port=6380,state=online,offset=78422,lag=0 master_repl_offset:78422 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:78421

此時從服務6381成爲主服務

重啓6379服務：

[root@VM_0_15_centos 6379]# redis-server ./redis.conf

sentinel日誌：

16404:X 18 Dec 19:22:33.181 * +convert-to-slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381

查看replication:

127.0.0.1:6379> info replication # Replication role:slave master_host:127.0.0.1 master_port:6381 master_link_status:up master_last_io_seconds_ago:-1 master_sync_in_progress:0 slave_repl_offset:1 master_link_down_since_seconds:1545132401 slave_priority:100 slave_read_only:1 connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

再次停掉從服務器6379能夠測試從服務器宕機時的效果

九、java集成

引入依賴：

<!-- https://mvnrepository.com/artifact/redis.clients/jedis -->
        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.9.0</version>
        </dependency>

具體測試見：java集成測試參考

十、注意事項

只有Sentinel 集羣中大多數服務器認定master主觀下線時master纔會被認定爲客觀下線，才能夠進行故障遷移，也就是說，即便無論咱們在sentinel monitor中設置的數是多少，就算是知足了該值，只要達不到大多數，就不會發生故障遷移。
官方建議sentinel至少部署三臺，且分佈在不一樣機器。這裏主要考慮到sentinel的可用性，假如咱們只部署了兩臺sentinel，且quorum設置爲1，也能夠實現自動故障遷移，但假如其中一臺sentinel掛了，就永遠不會觸發自動故障遷移，由於永遠達不到大多數sentinel認定master主觀下線了。
sentinel monitor配置中的master IP儘可能不要寫127.0.0.1或localhost，由於客戶端，如jedis獲取master是根據這個獲取的，若這樣配置，jedis獲取的ip則是127.0.0.1，這樣就可能致使程序鏈接不上master
當sentinel 啓動後會自動的修改sentinel.conf文件，如已發現的master的slave信息，和集羣中其它sentinel 的信息等,這樣即便重啓sentinel也能保持原來的狀態。注意，當集羣服務器調整時，如更換sentinel的機器，或者新配置一個sentinel，請不要直接複製原來運行過得sentinel配置文件，由於其裏面自動生成了以上說的那些信息，應該複製一個新的配置文件或者把自動生成的信息給刪掉。
當發生故障遷移的時候，master的變動記錄與slave更換master的修改會自動同步到redis的配置文件，這樣即便重啓redis也能保持變動後的狀態。