redis 消息隊列（發佈訂閱）、持久化(RDB、AOF)、集羣(cluster)

一：訂閱：

192.168.10.205:6379> SUBSCRIBE test
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "test"
3) (integer) 1


在另外一個終端向訂閱test發佈消息：
127.0.0.1:6379> PUBLISH test1  hello
(integer) 0


上一個終端的訂閱就收到了消息：
1) "message"
2) "test"
3) "hello"

二：redis持久化：

redis支持兩種數據持久化，一種是rdb，會在指定的數據間隔內把內存的數據快照到文件，aof將全部服務端執行的命令備查，在服務重啓的時候所有執行以此

rdb在斷電的時候會丟失部份內網，aof不會

一、使用rdb持久化redis數據：

配置文件的快照時間設置：

快照是fork出一個當前的進程在後臺進行快照，不會影響當前的進程，先寫一個臨時文件，寫完之後將臨時文件替換舊的文件。這個文件任什麼時候候都是一個完整的副本

 

save 900 1 #900秒之內有1個key發生變化就快照
save 300 10  #300秒之內有10個key發生變化就快照
save 60 10000 #60秒內有10000個key變化就快照

 

rdbcompression yes  #持久化到RDB文件時，是否壓縮，「yes」爲壓縮，「no」則反之

rdbchecksum yes  #讀取和寫入的時候是否支持CRC64校驗，默認是開啓的

dbfilename dump.rdb  #保存的文件的名稱

dir  /usr/local/redis  #快照文件的保存路徑

 

SAVE 與 BGSAVE:

SAVE:阻塞保存

BGSAVE：在後臺保存，不阻塞

 

KILL -9 會丟失自上次保存之後到如今的數據

 

二、使用AOF進行持久化：

appendonly yes #默認爲no，改成yes

appendfilename "appendonly.aof" #保存的文件名，路徑爲rdb指定的file目錄

關閉服務在啓動便可

 

三：redis主從：

一個主服務器能夠有多個從服務器，並且從服務器也能夠有主服務器，主從複製時非阻塞的，經過fork主進程

主服務器收到從服務器的sync命令，就在後臺執行bgsave，執行完畢後將保持的rdb文件發給客戶端，客戶端收到後將rdb載入到內存，新版本的支持增量備份

經過info命令查看默認配置：

# Replication
role:master    #默認較色爲主服務器
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

在從服務器執行命令：

127.0.0.1:6379> SLAVEOF 192.168.10.205 6379 #主服務器的地址和端口
OK

查看從服務器的info信息：

# Replication
role:slave #角色爲從服務器
master_host:192.168.10.205 #主服務器的地址
master_port:6379 #主服務器的端口
master_link_status:up
master_last_io_seconds_ago:8
master_sync_in_progress:0
slave_repl_offset:29
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

主服務器的狀態信息：

# Replication
role:master #角色爲主服務器
connected_slaves:1
slave0:ip=192.168.10.206,port=6379,state=online,offset=471,lag=0 #從服務器的地址和端口
master_repl_offset:471
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:470

從服務器沒法寫,以下：

127.0.0.1:6379> set a x
(error) READONLY You can't write against a read only slave.

 

三：redis集羣：

四種大方案：

一、客戶端分片，優點是比較靈活，不存在單點故障，缺點是添加節點須要從新配置，數據要手動同步

二、代理：代理分片，有proxy代理取數據，能夠爲proxy配置算法，如Twemproxy

三、Redis Cluster:在3.0版本之後支持，無中心，在某種狀況下會形成數據丟失

四、Coodis：豌豆莢的開源方案

 

 

使用redis cluster：

須要至少6個機器，3主3從，下面建立8個客戶端，另外2個作備用

先將配置文件複製到/opt，而後建立8個目錄並生成8個不一樣端口、不一樣pid和不一樣rdb文件的redis.conf配置文件：

[root@node5 ~]# cd /opt/
[root@node5 opt]# mkdir `seq 7001 7008`
[root@node5 opt]# cp /etc/redis/6379.conf  .
[root@node5 opt]# sed 's/6379/7001/g' 6379.conf >> 7001/redis.conf
[root@node5 opt]# sed 's/7001/7002/g' 6379.conf >> 7002/redis.conf
[root@node5 opt]# sed 's/7002/7003/g' 6379.conf >> 7003/redis.conf
[root@node5 opt]# sed 's/7003/7004/g' 6379.conf >> 7004/redis.conf
[root@node5 opt]# sed 's/7004/7005/g' 6379.conf >> 7005/redis.conf
[root@node5 opt]# sed 's/7005/7006/g' 6379.conf >> 7006/redis.conf
[root@node5 opt]# sed 's/7006/7007/g' 6379.conf >> 7007/redis.conf
[root@node5 opt]# sed 's/7007/7008/g' 6379.conf >> 7008/redis.conf

批量啓動redis：

[root@node5 opt]# for i in `seq 7001 7008`;do cd /opt/$i && /usr/local/redis/bin/redis-server  /opt/$i/redis.conf; done

 確認端口都已經啓動成功：

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7001
192.168.10.205:7001> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7002
192.168.10.205:7002> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7003
192.168.10.205:7003> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7004
192.168.10.205:7004> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7005
192.168.10.205:7005> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7006
192.168.10.205:7006> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7007
192.168.10.205:7007> 
[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7008
192.168.10.205:7008> 

安裝ruby管理工具：

[root@node5 opt]# yum install ruby rubygems -y
[root@node5 opt]# gem install redis
Successfully installed redis-3.2.2
1 gem installed
Installing ri documentation for redis-3.2.2...
Installing RDoc documentation for redis-3.2.2..

複製ruby的管理腳本：

[root@node5 ~]# cp /root/redis-3.0.7/src/redis-trib.rb  /usr/local/bin/redis-trib #是一個ruby腳本，方便redis管理

功能介紹：

root@node5 ~]# redis-trib  help
Usage: redis-trib <command> <options> <arguments ...>

  help            (show this help)
  del-node        host:port node_id #刪除節點
  reshard         host:port #從新分片
                  --timeout <arg>
                  --pipeline <arg>
                  --slots <arg>
                  --to <arg>
                  --yes
                  --from <arg>
  fix             host:port
                  --timeout <arg>
  create          host1:port1 ... hostN:portN #建立集羣
                  --replicas <arg>
  rebalance       host:port
                  --timeout <arg>
                  --simulate
                  --pipeline <arg>
                  --threshold <arg>
                  --use-empty-masters
                  --auto-weights
                  --weight <arg>
  call            host:port command arg arg .. arg
  add-node        new_host:new_port existing_host:existing_port #添加節點
                  --slave
                  --master-id <arg>
  check           host:port #檢測節點
  import          host:port
                  --replace
                  --copy
                  --from <arg>
  set-timeout     host:port milliseconds
  info            host:port

建立集羣：

 

[root@node5 ~]# redis-trib  create --replicas 1 192.168.10.205:7001 192.168.10.205:7002 192.168.10.205:7003 192.168.10.205:7004 192.168.10.205:7005 192.168.10.205:7006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.10.205:7001
192.168.10.205:7002
192.168.10.205:7003
Adding replica 192.168.10.205:7004 to 192.168.10.205:7001
Adding replica 192.168.10.205:7005 to 192.168.10.205:7002
Adding replica 192.168.10.205:7006 to 192.168.10.205:7003
M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001
   slots:0-5460 (5461 slots) master
M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002
   slots:5461-10922 (5462 slots) master
M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003
   slots:10923-16383 (5461 slots) master
S: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004
   replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4
S: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005
   replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6
S: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006
   replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 192.168.10.205:7001)
M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 #ID及IP和端口
   slots:0-5460 (5461 slots) master #本機對應的分片位置，0-5460
M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002
   slots:5461-10922 (5462 slots) master #本機對應的分片位置，5461-10922 #ID及IP和端口
M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003
   slots:10923-16383 (5461 slots) master #本機對應的分片位置，10923-16383 #ID及IP和端口
M: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004
   slots: (0 slots) master
   replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 #master的ID
M: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005
   slots: (0 slots) master
   replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 #master的ID
M: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006
   slots: (0 slots) master
   replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35 #master的ID
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

鏈接到集羣：

[root@node5 ~]# redis-cli  -h 192.168.10.205 -c -p 7001

查看集羣信息：

# Replication
role:master
connected_slaves:1
slave0:ip=192.168.10.205,port=7004,state=online,offset=407,lag=0
master_repl_offset:407
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:406

操做集羣：

192.168.10.205:7001> set key1 value1
-> Redirected to slot [9189] located at 192.168.10.205:7002
OK
192.168.10.205:7002> set key2 value2 
-> Redirected to slot [4998] located at 192.168.10.205:7001
OK
192.168.10.205:7001> set key3 value3
OK
192.168.10.205:7001> set key4 value4
-> Redirected to slot [13120] located at 192.168.10.205:7003
OK
192.168.10.205:7003> set key5 value5
-> Redirected to slot [9057] located at 192.168.10.205:7002
OK
192.168.10.205:7002> set key6 value6
-> Redirected to slot [4866] located at 192.168.10.205:7001
OK
192.168.10.205:7001> set key7 value7
OK
192.168.10.205:7001> set key8 value8
-> Redirected to slot [13004] located at 192.168.10.205:7003 #插入的數據是輪訓寫入到各個redis server
OK

 cluster nodes：列出當前的主從

192.168.10.205:7003> CLUSTER  nodes
8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 myself,master - 0 0 3 connected 10923-16383
5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458558870837 2 connected 5461-10922
32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458558869829 4 connected
840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458558868820 6 connected
19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458558870334 5 connected
9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458558869829 1 connected 0-5460

cluster info：當前的狀態：

192.168.10.205:7003> CLUSTER INFO 
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:3
cluster_stats_messages_sent:2911
cluster_stats_messages_received:2911

 向集羣中添加節點：

[root@node5 ~]# redis-trib  add-node 192.168.10.205:7007 192.168.10.205:7001

　　　　　　　　　　　　　　　　　　　　 要添加的節點和端口       添加到的目的節點和端口

[root@node5 ~]# redis-trib  add-node 192.168.10.205:7007 192.168.10.205:7001
>>> Adding node 192.168.10.205:7007 to cluster 192.168.10.205:7001
>>> Performing Cluster Check (using node 192.168.10.205:7001)
M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006
   slots: (0 slots) slave
   replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35
M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004
   slots: (0 slots) slave
   replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4
S: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005
   slots: (0 slots) slave
   replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.10.205:7007 to make it join the cluster.
[OK] New node added correctly.

 

添加主機之後須要從新分片：

[root@node5 ~]# redis-trib  reshard 192.168.10.205:7007

鏈接到新添加的節點查詢信息：

[root@node5 ~]# redis-cli -h 192.168.10.205 -p  7007
192.168.10.205:7007> KEYS *
(empty list or set)
192.168.10.205:7007> CLUSTER NODES
5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458560205983 2 connected 5728-10922
8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 master - 0 1458560204471 3 connected 11189-16383
19a5bb651f4b182a0e3dd49154323d31c1524167 192.168.10.205:7007 myself,master - 0 0 7 connected 0-265 5461-5727 10923-11188 #已經有了分片信息
19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458560204977 2 connected
840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458560205482 3 connected
9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458560204471 1 connected 266-5460
32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458560204977 1 connected

爲新添加的主節點添加從節點：

[root@node5 ~]# redis-trib  add-node 192.168.10.205:7008 192.168.10.205:7001 #添加一個節點

#登錄到從節點進行配置
[root@node5 ~]# redis-cli -h 192.168.10.205 -c -p 7008 #登錄到要備用的從節點
192.168.10.205:7008> CLUSTER replicate 19a5bb651f4b182a0e3dd49154323d31c1524167 #指定主節點的ID便可
OK

再次查看集羣信息：

192.168.10.205:7008> CLUSTER  nodes
19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458560717276 2 connected
ab146fb5881064a6375da76f861e205d2d702140 192.168.10.205:7008 myself,slave 19a5bb651f4b182a0e3dd49154323d31c1524167 0 0 0 connected   #已是從節點並鏈接了
32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458560716772 1 connected
8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 master - 0 1458560715764 3 connected 11189-16383
19a5bb651f4b182a0e3dd49154323d31c1524167 192.168.10.205:7007 master - 0 1458560717276 7 connected 0-265 5461-5727 10923-11188
9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458560717276 1 connected 266-5460
5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458560716267 2 connected 5728-10922
840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458560717780 3 connected