Redis多機多節點集羣實驗

第一步：環境準備

咱們搞兩臺虛擬機

 

局域網IP 分別是 192.168.1.109和192.168.1.110

咱們約定把192.168.1.109做爲集羣控制端，須要安裝redis-trib.rb

 

第二步：安裝Redis

前面已經安裝過了 不解釋，

192.168.1.109虛擬機裏，Reids安裝包裏有個集羣工具，要複製到/usr/local/bin裏去

cp redis-3.2.9/src/redis-trib.rb /usr/local/bin

 

第三步：修改配置，建立節點

首先咱們在192.168.1.109虛擬機裏建立三個節點，端口分別是7001，7002，7003

咱們先在root目錄下新建一個redis_cluster目錄，而後該目錄下再建立3個目錄，

分別是7001，7002，7003，用來存redis配置文件；

這裏咱們要使用redis集羣，要先修改redis的配置文件redis.conf

mkdir redis_cluster 新建目錄

[root@localhost ~]#  cd redis_cluster/

[root@localhost redis_cluster]# mkdir 7001 7002 7003

[root@localhost redis_cluster]# ll

總用量 0

drwxr-xr-x. 2 root root 6 7月  27 19:49 7001

drwxr-xr-x. 2 root root 6 7月  27 19:49 7002

drwxr-xr-x. 2 root root 6 7月  27 19:49 7003

[root@localhost redis_cluster]# 

 

先複製一份配置文件到7001目錄下

[root@localhost redis_cluster]# cd

[root@localhost ~]# cp redis-3.2.9/redis.conf redis_cluster/7001/

 

咱們修改下這個配置文件

vi redis_cluster/7001/redis.conf

修改一下幾個

port 7001  //六個節點配置文件分別是7001-7003

bind 192.168.1.109    //默認ip爲127.0.0.1 須要改成其餘節點機器可訪問的ip 不然建立集羣時沒法訪，和單機集羣有區別

daemonize yes        //redis後臺運行

pidfile /var/run/redis_7001.pid   //pidfile文件對應7001-7003

cluster-enabled yes   //開啓集羣

cluster-config-file nodes_7001.conf  //保存節點配置，自動建立，自動更新對應7001-7003

cluster-node-timeout 5000    //集羣超時時間，節點超過這個時間沒反應就判定是宕機

appendonly yes   //存儲方式，aof，將寫操做記錄保存到日誌中

 

7001下的修改完後，咱們把7001下的配置分別複製到7002-7003 而後對應的再修改下配置便可；

[root@localhost ~]# cp redis_cluster/7001/redis.conf redis_cluster/7002/

[root@localhost ~]# cp redis_cluster/7001/redis.conf redis_cluster/7003/

 

[root@localhost ~]# vi redis_cluster/7002/redis.conf 

[root@localhost ~]# vi redis_cluster/7003/redis.conf 

[root@localhost ~]# vi redis_cluster/7004/redis.conf 

編輯後面5個配置文件，把 port ，pidfile，cluster-config-file 分別修改下便可；

 

 

同理 192.168.1.110機器，也搞一個redis_cluster目錄，而後再新建7004，7005，7006目錄，

複製三份配置文件 其餘同樣，就端口和bind地址改下便可；這裏不囉嗦了；

 

第四步：啓動兩臺機器的六個節點

192.168.1.109機器

[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7001/redis.conf 

[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7002/redis.conf 

[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7003/redis.conf 

[root@localhost ~]# ps -ef | grep redis  

root       2242      1  0 19:55 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]

root       2252      1  0 19:59 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7002 [cluster]

root       2256      1  0 19:59 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]

root       2260   2214  0 19:59 pts/0    00:00:00 grep --color=auto redis

 

192.168.1.110機器

[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7004/redis.conf 

[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7005/redis.conf 

[root@localhost ~]# /usr/local/redis/bin/redis-server redis_cluster/7006/redis.conf 

[root@localhost ~]# ps -ef | grep redis  

root       2347      1  0 20:31 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]

root       2351      1  0 20:31 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]

root       2355      1  0 20:31 ?        00:00:00 /usr/local/redis/bin/redis-server 192.168.1.110:7006 [cluster]

root       2363   2270  0 20:32 pts/0    00:00:00 grep --color=auto redis

 

說明都啓動OK

 

第五步：設置防火牆，開放集羣端口

兩臺機器的防火牆咱們直接關掉

systemctl stop firewalld.service

 

第六步：建立集羣

192.168.1.109機器做爲集羣控制端

redis官方提供了redis-trib.rb工具，第一步裏已經房到裏bin下 ；

可是在使用以前 須要安裝ruby，以及redis和ruby鏈接

yum -y install ruby ruby-devel rubygems rpm-build

gem install redis

 

redis-trib.rb create --replicas 1  192.168.1.109:7001 192.168.1.109:7002 192.168.1.109:7003 192.168.1.110:7004 192.168.1.110:7005 192.168.1.110:7006

建立集羣

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

192.168.1.109:7001

192.168.1.110:7004

192.168.1.109:7002

Adding replica 192.168.1.110:7005 to 192.168.1.109:7001

Adding replica 192.168.1.109:7003 to 192.168.1.110:7004

Adding replica 192.168.1.110:7006 to 192.168.1.109:7002

M: 08d5f59fa053d79e3cc71fc5bb8759bec191ff41 192.168.1.109:7001

   slots:0-5460 (5461 slots) master

M: 4bda83bae0aeb9988aeb7d34eff2cc75226a0edd 192.168.1.109:7002

   slots:10923-16383 (5461 slots) master

S: b782f527ead6f63c8278c1b0713b92a9fd4fe995 192.168.1.109:7003

   replicates 86c8cb7975f85a859147fef9672d44b24c5a718b

M: 86c8cb7975f85a859147fef9672d44b24c5a718b 192.168.1.110:7004

   slots:3671,5461-10922,11797,15924 (5465 slots) master

S: 2d44accacb1bb4cd1eef5f98f7af64a43f01544d 192.168.1.110:7005

   replicates 08d5f59fa053d79e3cc71fc5bb8759bec191ff41

S: 2c42433953bcd3ad47015d9f8ac72e7a7cf447c5 192.168.1.110:7006

   replicates 4bda83bae0aeb9988aeb7d34eff2cc75226a0edd

Can I set the above configuration? (type 'yes' to accept):

 

從運行結果看 主節點就是7001 7004 7002 從節點分別是7005 7003 7006 

7001分配到的哈希槽是 0-5460

7004分配到的哈希槽是 5461-10922

7002分配到的哈希槽是 10923-16383

最後問咱們是否接受上面的設置，輸入yes 就表示接受，咱們輸入yes

而後顯示：

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 192.168.1.109:7001)

M: d06aebf92e99169864893decc371e34cd321640f 192.168.1.109:7001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 189dc189dd743f9fc61c439d44cc3379d8cbe535 192.168.1.110:7004

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: fe2b0038304c02bc807c38fe006338bd5dd85a70 192.168.1.110:7006

   slots: (0 slots) slave

   replicates 25a6be3b62fa675a3835e6c62641c3784ec62257

M: 25a6be3b62fa675a3835e6c62641c3784ec62257 192.168.1.109:7002

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: df28512d63af4a3e6dc1739e1c267ba916f3a579 192.168.1.110:7005

   slots: (0 slots) slave

   replicates d06aebf92e99169864893decc371e34cd321640f

S: aaa3310d3c30cc9a7f235db27e3c395a5e180a52 192.168.1.109:7003

   slots: (0 slots) slave

   replicates 189dc189dd743f9fc61c439d44cc3379d8cbe535

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

顯示配置哈希槽，以及集羣建立成功，能夠用了；

 

第七步：集羣數據測試

咱們先鏈接任意一個節點，而後添加一個key：

redis-cli是redis默認的客戶端工具，啓動時加上｀-c｀參數，`-p`指定端口，就能夠鏈接到集羣。

這裏還得加-h 指定機器IP 

鏈接任意一個節點端口：

[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.1.109 -c -p 7002

192.168.1.109:7002> 

鏈接到7002節點

192.168.1.109:7002> set xxx 'fadfa'

-> Redirected to slot [4038] located at 192.168.1.109:7001

OK

前面說過Redis Cluster值分配規則，因此分配key的時候，它會使用CRC16(‘my_name’)%16384算法，來計算，將這個key 放到哪一個節點，這裏分配到了4038slot 就分配到了7001(0-5460)這個節點上。因此有：

Redirected to slot [4038] located at 192.168.1.109:7001

咱們從其餘集羣節點 ，均可以獲取到數據

[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.1.110 -c -p 7005

192.168.1.110:7005> 

192.168.1.110:7005> get xxx

-> Redirected to slot [4038] located at 192.168.1.109:7001

"fadfa"

192.168.1.109:7001> 

 

第八步：集羣宕機測試

假如咱們幹掉一個節點，好比7002 這個主節點

[root@localhost ~]# ps -ef | grep redis  

root       9799      1  0 21:33 ?        00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]

root       9803      1  0 21:33 ?        00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7002 [cluster]

root       9807      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]

root       9830   2214  0 21:53 pts/0    00:00:00 grep --color=auto redis

[root@localhost ~]# kill -9 9803

[root@localhost ~]# ps -ef | grep redis  

root       9799      1  0 21:33 ?        00:00:02 /usr/local/redis/bin/redis-server 192.168.1.109:7001 [cluster]

root       9807      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.109:7003 [cluster]

root       9832   2214  0 21:54 pts/0    00:00:00 grep --color=auto redis

[root@localhost ~]# 

而後咱們查看下集羣狀況

[root@localhost ~]# redis-trib.rb check 192.168.1.109:7001

>>> Performing Cluster Check (using node 192.168.1.109:7001)

M: d06aebf92e99169864893decc371e34cd321640f 192.168.1.109:7001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 189dc189dd743f9fc61c439d44cc3379d8cbe535 192.168.1.110:7004

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: fe2b0038304c02bc807c38fe006338bd5dd85a70 192.168.1.110:7006

   slots:10923-16383 (5461 slots) master

   0 additional replica(s)

S: df28512d63af4a3e6dc1739e1c267ba916f3a579 192.168.1.110:7005

   slots: (0 slots) slave

   replicates d06aebf92e99169864893decc371e34cd321640f

S: aaa3310d3c30cc9a7f235db27e3c395a5e180a52 192.168.1.109:7003

   slots: (0 slots) slave

   replicates 189dc189dd743f9fc61c439d44cc3379d8cbe535

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

咱們發現 7006原本是從節點，因爲他對應的主節點掛了，就自動變成主節點master，全部會有最後一個說明

 

 All 16384 slots covered. 全部哈希槽均可覆蓋了； 集羣能夠正常使用；

 

假如咱們把7006也幹掉，試試看

[root@localhost ~]# ps -ef | grep redis  

root       2578      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]

root       2582      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]

root       2586      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7006 [cluster]

root       2623   2270  0 21:56 pts/0    00:00:00 grep --color=auto redis

[root@localhost ~]# kill -9 2586

[root@localhost ~]# ps -ef | grep redis  

root       2578      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7004 [cluster]

root       2582      1  0 21:33 ?        00:00:03 /usr/local/redis/bin/redis-server 192.168.1.110:7005 [cluster]

root       2625   2270  0 21:56 pts/0    00:00:00 grep --color=auto redis

[root@localhost ~]# 

 

[root@localhost ~]# redis-trib.rb check 192.168.1.109:7001

這裏咱們發現 出事了，由於主從節點都掛了 因此有一部分哈希槽沒得分配，最後一句

[ERR] Not all 16384 slots are covered by nodes.  沒有安全覆蓋；

因此不能正常使用集羣