深刻理解Redis系列之集羣環境搭建

時間 2019-12-05

標籤深刻理解 redis 系列集羣環境搭建欄目 Redis 简体版

原文原文鏈接

前面分別寫了關於單機版Redis搭建以及使用SpringBoot來訪問Redis服務, 後面也就順着寫一些關於分佈式環境的搭建以及高可用的大概原理;

分佈式環境準備

由於在前面一篇深刻理解Redis系列之單機Redis環境搭建中已經安裝好了Redis的介質, 後面只須要使用配置文件指定不一樣端口來作Master以及Slave節點便可;由於是在一臺機器安裝, 因此只能經過端口來啓動不一樣角色的Redis進程; 規劃以下:node

機器IP	端口	角色	配置文件目錄
127.0.0.1	6379	Master1	/usr/local/redis/conf/6379/redis.conf
127.0.0.1	6380	Master2	/usr/local/redis/conf/6380/redis.conf
127.0.0.1	6381	Master3	/usr/local/redis/conf/6381/redis.conf
127.0.0.1	26379	Slave1	/usr/local/redis/conf/26379/redis.conf
127.0.0.1	26380	Slave2	/usr/local/redis/conf/26380/redis.conf
127.0.0.1	26381	Slave3	/usr/local/redis/conf/26381/redis.conf

注: 原本使用的1637九、16380、16381做爲slave節點的端口的, 可是發現每次啓動6379節點的時候老是把16379端口也同時佔用, 經查證發現每個redis集羣的節點須要開通兩個TCP端口。一個是用於客戶端的Redis TCP，如6379。另外一個由客戶端加10000所得，如16379，用於Redis集羣總線鏈接；故改成2637九、26380、26381redis

生成配置文件

使用以下配置能夠批量修改與端口相關內容segmentfault

vi redis.conf
esc 命令以後
:%s/6379/{replace_port}/g

分別修改以下配置:緩存

port 6379 //根據上面的規劃調整
daemonize yes // 從no => yes
pidfile /var/run/redis_6379.pid //根據規劃調整 
cluster-enabled yes //將該行註釋去掉, 若是沒有這一行就添加到配置文件
bind 127.0.0.1 //若是是外部訪問，改成本機的IP地址，而不是本地迴路127.0.0.1
dbfilename dump_6379.rdb //根據端口設置
logfile "6379.log" //啓動日誌也能夠打開, 便於看出錯日誌

啓動master與slave進程

cd /usr/local/redis/conf/6379
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/6380
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/6381
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/26379
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/26380
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/26381
../../bin/redis-server redis.conf

咱們能夠經過ps -ef | grep redis 驗證進程是否起來:分佈式

501 82074     1   0 12:08上午 ??         0:00.11 ../../bin/redis-server 127.0.0.1:6379 [cluster] 
  501 82077     1   0 12:08上午 ??         0:00.09 ../../bin/redis-server 127.0.0.1:6380 [cluster] 
  501 82079     1   0 12:08上午 ??         0:00.07 ../../bin/redis-server 127.0.0.1:6381 [cluster] 
  501 82082     1   0 12:08上午 ??         0:00.05 ../../bin/redis-server 127.0.0.1:26379 [cluster] 
  501 82085     1   0 12:08上午 ??         0:00.04 ../../bin/redis-server 127.0.0.1:26380 [cluster] 
  501 82087     1   0 12:08上午 ??         0:00.02 ../../bin/redis-server 127.0.0.1:26381 [cluster] 
  501 82090 81954   0 12:08上午 ttys003    0:00.00 grep redis

將Redis各進程加入到Redis集羣

一、使用以下命令啓動集羣:3d

redis-cli --cluster create 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:26379 127.0.0.1:26380 127.0.0.1:26381 --cluster-replicas 1

啓動日誌輸出以下:日誌

terrydeMacBook-Air:bin terrylmay$ ./redis-cli --cluster create 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:26379 127.0.0.1:26380 127.0.0.1:26381 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:26379 to 127.0.0.1:6379
Adding replica 127.0.0.1:26380 to 127.0.0.1:6380
Adding replica 127.0.0.1:26381 to 127.0.0.1:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 3dbb14f3561c413a29c79eb75b39c670692bb1c1 127.0.0.1:6379
   slots:[0-5460] (5461 slots) master
M: d0fc2996f417ac47d4ca174817d7485148534f97 127.0.0.1:6380
   slots:[5461-10922] (5462 slots) master
M: 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 127.0.0.1:6381
   slots:[10923-16383] (5461 slots) master
S: 5594d0d4e6114533c4b598307a94eeb56605970f 127.0.0.1:26379
   replicates d0fc2996f417ac47d4ca174817d7485148534f97
S: 8d6edc0b4d95b989d94e12d9d7915b45db7a6a0b 127.0.0.1:26380
   replicates 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa
S: 493d5dfe6d6aec9ce629dc55de58d4072ad1be93 127.0.0.1:26381
   replicates 3dbb14f3561c413a29c79eb75b39c670692bb1c1
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 3dbb14f3561c413a29c79eb75b39c670692bb1c1 127.0.0.1:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 8d6edc0b4d95b989d94e12d9d7915b45db7a6a0b 127.0.0.1:26380
   slots: (0 slots) slave
   replicates 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa
S: 5594d0d4e6114533c4b598307a94eeb56605970f 127.0.0.1:26379
   slots: (0 slots) slave
   replicates d0fc2996f417ac47d4ca174817d7485148534f97
S: 493d5dfe6d6aec9ce629dc55de58d4072ad1be93 127.0.0.1:26381
   slots: (0 slots) slave
   replicates 3dbb14f3561c413a29c79eb75b39c670692bb1c1
M: d0fc2996f417ac47d4ca174817d7485148534f97 127.0.0.1:6380
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 127.0.0.1:6381
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
terrydeMacBook-Air:bin terrylmay$

到此, Master-Slave集羣已經搭建成功了; code

注:剛開始沒發覺, 後面驗證集羣master切換的時候才發現, 原來建立集羣的時候並非按順序一對一的關係, 因此咱們能夠看到 127.0.0.1:26381 這個進程是 127.0.0.1:6379節點的Slave節點, 這也就是後面再啓動6379進程的時候日誌是鏈接到26381 master節點了;orm

那麼集羣master節點與slave節點都有主從複製的功能, 下面咱們來驗證一下, 首先登陸redis任意主節點，放入內容以後, 登陸從節點server

./redis-cli -c -h 127.0.0.1 -p 6380

terrydeMacBook-Air:bin terrylmay$ ./redis-cli -c -h 127.0.0.1 -p 6380
127.0.0.1:6380> set name 123456
OK
127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=26379,state=online,offset=110378,lag=0

./redis-cli -c -h 127.0.0.1 -p 26379登陸從節點, 發現經過name也能夠拿到值

terrydeMacBook-Air:bin terrylmay$ ./redis-cli -c -h 127.0.0.1 -p 26379
127.0.0.1:26379> get name
-> Redirected to slot [5798] located at 127.0.0.1:6380
"123456"
127.0.0.1:6380>

經過上面日誌，咱們看到該信息redirected到了master節點, 那麼問題來了? 從slave裏面讀取數據，會再走到master中讀數據麼? 答案應該不會, 那樣就沒人用slave來當本地緩存了. 可是爲何不會呢? 仍是想要了解一下! 放在後面探索吧;

這時候，咱們刪除其中的一個master 6379節點，看一下會發生什麼狀況?

ps -ef | grep :6379 | awk '{print $2}' |xargs  kill -9

咱們在master6379的從節點26379節點上能夠看到日誌輸出以下:

82082:S 05 Nov 2018 22:56:08.343 * FAIL message received from 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa about 3dbb14f3561c413a29c79eb75b39c670692bb1c1
82082:S 05 Nov 2018 22:56:08.345 # Cluster state changed: fail
82082:S 05 Nov 2018 22:56:09.153 # Cluster state changed: ok

這時候, 咱們還看不出master節點切換了, 僅僅看着像是切換了;

那麼, 咱們使用redis-cli命令進入到redis控制檯

./redis-cli -c -h 127.0.0.1 -p 6380

使用命令cluster nodes 查看節點狀態，能夠看到

127.0.0.1:6380> cluster nodes
8d6edc0b4d95b989d94e12d9d7915b45db7a6a0b 127.0.0.1:26380@36380 slave 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 0 1541431115317 5 connected
493d5dfe6d6aec9ce629dc55de58d4072ad1be93 127.0.0.1:26381@36381 master - 0 1541431115000 7 connected 0-5460
3dbb14f3561c413a29c79eb75b39c670692bb1c1 127.0.0.1:6379@16379 slave 493d5dfe6d6aec9ce629dc55de58d4072ad1be93 1541431110864 1541431109000 7 disconnected
5594d0d4e6114533c4b598307a94eeb56605970f 127.0.0.1:26379@36379 slave d0fc2996f417ac47d4ca174817d7485148534f97 0 1541431116330 4 connected
2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 127.0.0.1:6381@16381 master - 0 1541431115000 3 connected 10923-16383
d0fc2996f417ac47d4ca174817d7485148534f97 127.0.0.1:6380@16380 myself,master - 0 1541431115000 2 connected 5461-10922

6379節點已經處於disconnected狀態了

使用命令cluster info 查看集羣狀態, 發現cluster 狀態正常

cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384

說明master節點已經切換到了26381節點;

而後, 咱們試着重啓剛纔刪掉的6379節點，能夠看到6379日誌打印以下:

90819:C 05 Nov 2018 23:05:56.067 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
90819:C 05 Nov 2018 23:05:56.067 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=90819, just started
90819:C 05 Nov 2018 23:05:56.067 # Configuration loaded
90820:M 05 Nov 2018 23:05:56.070 * Increased maximum number of open files to 10032 (it was originally set to 256).
90820:M 05 Nov 2018 23:05:56.073 * Node configuration loaded, I'm 3dbb14f3561c413a29c79eb75b39c670692bb1c1
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in cluster mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 90820
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

90820:M 05 Nov 2018 23:05:56.077 # Server initialized
90820:M 05 Nov 2018 23:05:56.077 * DB loaded from disk: 0.000 seconds
90820:M 05 Nov 2018 23:05:56.077 * Ready to accept connections
90820:M 05 Nov 2018 23:05:56.079 # Configuration change detected. Reconfiguring myself as a replica of 493d5dfe6d6aec9ce629dc55de58d4072ad1be93
90820:S 05 Nov 2018 23:05:56.079 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
90820:S 05 Nov 2018 23:05:56.080 # Cluster state changed: ok
90820:S 05 Nov 2018 23:05:57.088 * Connecting to MASTER 127.0.0.1:26381
90820:S 05 Nov 2018 23:05:57.096 * MASTER <-> REPLICA sync started
90820:S 05 Nov 2018 23:05:57.097 * Non blocking connect for SYNC fired the event.
90820:S 05 Nov 2018 23:05:57.097 * Master replied to PING, replication can continue...
90820:S 05 Nov 2018 23:05:57.098 * Trying a partial resynchronization (request 385e7338e98550e9dc9d1cd5a3ea77d4e39e5f51:1).
90820:S 05 Nov 2018 23:05:57.099 * Full resync from master: 306b56738fb37bd1d1fcb9e0f709584c56f2df52:107352
90820:S 05 Nov 2018 23:05:57.099 * Discarding previously cached master state.
90820:S 05 Nov 2018 23:05:57.188 * MASTER <-> REPLICA sync: receiving 178 bytes from master
90820:S 05 Nov 2018 23:05:57.189 * MASTER <-> REPLICA sync: Flushing old data
90820:S 05 Nov 2018 23:05:57.189 * MASTER <-> REPLICA sync: Loading DB in memory
90820:S 05 Nov 2018 23:05:57.189 * MASTER <-> REPLICA sync: Finished with success

Connecting to MASTER 127.0.0.1:26381 說明先前master節點切換到26381節點上; 集羣信息都有了更新; 固然咱們能夠經過./redis-server -c -h 127.0.0.1 -p 26381 而後使用 info replication查看slave節點信息

terrydeMacBook-Air:bin terrylmay$ ./redis-cli -c -h 127.0.0.1 -p 26381
127.0.0.1:26381> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6379,state=online,offset=108430,lag=0
master_replid:306b56738fb37bd1d1fcb9e0f709584c56f2df52
master_replid2:37b3866dc3956911c64eca1ea3df03aa786d4813
master_repl_offset:108430
second_repl_offset:107353
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:108430

到這裏, 基本上主從切換就OK了; 下一篇文章寫一下如何使用SpringBoot訪問Redis集羣;