07：redis cluster（分佈式集羣）

redis cluster（分佈式集羣）

高性能：
一、在多分片節點中，將16384個槽位，均勻分佈到多個分片節點中
二、存數據時，將key作crc16(key),而後和16384進行取模，得出槽位值（0-16383之間）
三、根據計算得出的槽位值，找到相對應的分片節點的主節點，存儲到相應槽位上
四、若是客戶端當時鏈接的節點不是未來要存儲的分片節點，分片集羣會將客戶端鏈接切換至真正存儲節點進行數據存儲

高可用：
在搭建集羣時，會爲每個分片的主節點，對應一個從節點，實現slaveof的功能，同時當主節點down，實現相似於sentinel的自動failover的功能。

規劃、搭建過程：

6個redis實例，通常會放到3臺硬件服務器
注：在企業規劃中，一個分片的兩個分到不一樣的物理機，防止硬件主機宕機形成的整個分片數據丟失。

端口號：7000-7005

一、安裝集羣插件
EPEL源安裝ruby支持

yum install ruby rubygems -y

使用國內源
gem sources -l
gem sources -a http://mirrors.aliyun.com/rubygems/
gem sources --remove http://rubygems.org/
gem install redis -v 3.3.3
gem sources -l

或者：
gem sources -a http://mirrors.aliyun.com/rubygems/ --remove http://rubygems.org/

二、集羣節點準備

mkdir /nosql/700{0..5}

vim /nosql/7000/redis.conf
port 7000
daemonize yes
pidfile /nosql/7000/redis.pid
loglevel notice
logfile "/nosql/7000/redis.log"
dbfilename dump.rdb
dir /nosql/7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7001/redis.conf
port 7001
daemonize yes
pidfile /nosql/7001/redis.pid
loglevel notice
logfile "/nosql/7001/redis.log"
dbfilename dump.rdb
dir /nosql/7001
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7002/redis.conf
port 7002
daemonize yes
pidfile /nosql/7002/redis.pid
loglevel notice
logfile "/nosql/7002/redis.log"
dbfilename dump.rdb
dir /nosql/7002
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7003/redis.conf
port 7003
daemonize yes
pidfile /nosql/7003/redis.pid
loglevel notice
logfile "/nosql/7003/redis.log"
dbfilename dump.rdb
dir /nosql/7003
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7004/redis.conf
port 7004
daemonize yes
pidfile /nosql/7004/redis.pid
loglevel notice
logfile "/nosql/7004/redis.log"
dbfilename dump.rdb
dir /nosql/7004
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7005/redis.conf
port 7005
daemonize yes
pidfile /nosql/7005/redis.pid
loglevel notice
logfile "/nosql/7005/redis.log"
dbfilename dump.rdb
dir /nosql/7005
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

啓動節點：
redis-server /nosql/7000/redis.conf
redis-server /nosql/7001/redis.conf
redis-server /nosql/7002/redis.conf
redis-server /nosql/7003/redis.conf
redis-server /nosql/7004/redis.conf
redis-server /nosql/7005/redis.conf

[root@db01 ~]# ps -ef |grep redis
root 8854 1 0 03:56 ? 00:00:00 redis-server *:7000 [cluster]
root 8858 1 0 03:56 ? 00:00:00 redis-server *:7001 [cluster]
root 8860 1 0 03:56 ? 00:00:00 redis-server *:7002 [cluster]
root 8864 1 0 03:56 ? 00:00:00 redis-server *:7003 [cluster]
root 8866 1 0 03:56 ? 00:00:00 redis-server *:7004 [cluster]
root 8874 1 0 03:56 ? 00:00:00 redis-server *:7005 [cluster]

三、將節點加入集羣管理
redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

四、集羣狀態查看
集羣主節點狀態
redis-cli -p 7000 cluster nodes | grep master
集羣從節點狀態
redis-cli -p 7000 cluster nodes | grep slave

五、集羣節點管理

5.1 增長新的節點

mkdir /nosql/7006
mkdir /nosql/7007

vim /nosql/7006/redis.conf
port 7006
daemonize yes
pidfile /nosql/7006/redis.pid
loglevel notice
logfile "/nosql/7006/redis.log"
dbfilename dump.rdb
dir /nosql/7006
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7007/redis.conf
port 7007
daemonize yes
pidfile /nosql/7007/redis.pid
loglevel notice
logfile "/nosql/7007/redis.log"
dbfilename dump.rdb
dir /nosql/7007
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

redis-server /nosql/7006/redis.conf
redis-server /nosql/7007/redis.conf

5.2 添加主節點：
redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000

5.3 轉移slot（從新分片）
redis-trib.rb reshard 127.0.0.1:7000
計算如下應該分配多少個slot到新節點：；bc 16384/4 = 4096

移動多少個槽位
(1)How many slots do you want to move (from 1 to 16384)? 4096

誰來接受這些槽位
(2)What is the receiving node ID? dd76db8928575125e0046c995df5054f7acf69fa
從哪些節點去獲取這些槽位：
(3)Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all

最後確認分片計劃
(4)Do you want to proceed with the proposed reshard plan (yes/no)? yes

5.4 添加一個從節點
redis-trib.rb add-node --slave --master-id dd76db8928575125e0046c995df5054f7acf69fa 127.0.0.1:7007 127.0.0.1:7000

6.刪除節點

將須要刪除節點（7006）slot移動走

redis-trib.rb reshard 127.0.0.1:7000

（1）How many slots do you want to move (from 1 to 16384)? 4096
（2）What is the receiving node ID? 2d8667a8005786cc966dd18a07a442df471e9fe3
（3）Source node #1:dd76db8928575125e0046c995df5054f7acf69fa
（4）done
（5）Do you want to proceed with the proposed reshard plan (yes/no)? yes

刪除一個節點
redis-trib.rb del-node ip:port '<node-id>'
刪除master節點以前首先要使用reshard移除master的所有slot,而後再刪除當前節點

redis-trib.rb del-node 127.0.0.1:7006 dd76db8928575125e0046c995df5054f7acf69fa
redis-trib.rb del-node 127.0.0.1:7007 5c0b10501e4f6775eac3fd2e7c330f35fc81b1e2

---------------------
redis-cli -p 7000 cluster nodes | grep master
redis-cli -p 7000 cluster nodes | grep slave

------------------------------------------------------------------------------------------------------------------

擴展知識：

redis的多API支持
python爲例

tar xf Python-3.5.2.tar.xz
cd Python-3.5.2
./configure
make && make install

https://redis.io/clients

下載redis-py-master.zip

unzip redis-py-master.zip
cd redis-py-master

python3 setup.py install

安裝redis-cluser的客戶端程序

cd redis-py-cluster-unstable
python3 setup.py install

一、對redis的單實例進行鏈接操做
python3
>>>import redis
>>>r = redis.StrictRedis(host='localhost', port=6379, db=0,password='root')
>>>r.set('foo', 'bar')
True
>>>r.get('foo')
'bar'
--------------------

二、sentinel集羣鏈接並操做
redis-server /nosql/6380/redis.conf
redis-server /nosql/6381/redis.conf
redis-server /nosql/6382/redis.conf
redis-sentinel /nosql/26380/sentinel.conf &
--------------------------------
## 導入redis sentinel包
>>> from redis.sentinel import Sentinel
##指定sentinel的地址和端口號
>>> sentinel = Sentinel([('localhost', 26380)], socket_timeout=0.1)
##測試，獲取如下主庫和從庫的信息
>>> sentinel.discover_master('mymaster')
>>> sentinel.discover_slaves('mymaster')
##配置讀寫分離
#寫節點
>>> master = sentinel.master_for('mymaster', socket_timeout=0.1)
#讀節點
>>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
###讀寫分離測試 key
>>> master.set('oldboy', '123')
>>> slave.get('oldboy')
'123'

 

----------------------
redis cluster的鏈接並操做（python2.7.2以上版本才支持redis cluster，咱們選擇的是3.5）

https://github.com/Grokzen/redis-py-cluster

三、python鏈接rediscluster集羣測試
使用

python3
>>> from rediscluster import StrictRedisCluster
>>> startup_nodes = [{"host": "127.0.0.1", "port": "7000"}]
### Note: decode_responses must be set to True when used with python3
>>> rc = StrictRedisCluster(startup_nodes=startup_nodes, decode_responses=True)
>>> rc.set("foo", "bar")
True
>>> print(rc.get("foo"))
'bar'
----------------------