Redis 集羣安裝配置及調研記錄

時間 2019-11-05

標籤 redis 集羣安裝配置調研記錄欄目 Redis 简体版

原文原文鏈接

Redis簡介

REmote DIctionary Server(Redis) Redis是一個開源的使用ANSI C語言編寫、遵照BSD協議、支持網絡、可基於內存亦可持久化的日誌型、Key-Value數據庫，並提供多種語言的API。一般被稱爲數據結構服務器，值（value）能夠是字符串(String), 哈希(Map), 列表(list), 集合(sets) 和有序集合(sorted sets)等類型。支持主從模式的數據備份。擁有極高的讀寫性能，豐富的數據類型，原子性操做，還支持pub/sub，通知，key過時等特性。node

與其餘k-v存儲的區別，redis有更爲複雜的數據結構而且對他們提供原子性操做，redis運行在內存但支持持久化，持久化是以追加的方式產生的。mysql

Redis是一個開源的高性能鍵值對數據庫.它經過提供多種鍵值數據類型來適應不一樣場景下的存儲需求,而且藉助許多高層級的接口使其能夠勝任,如緩存、隊列系統的不一樣角色.將鍵值對數據類型存放在內存中的一個數據庫。c++

單機安裝

#掛載ISO操做系統光盤
[root@MySQLNODE02 /]# mount -t iso9660 /dev/cdrom  /mnt/
#確認是否掛載成功
[root@MySQLNODE02 /]# df -h
Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/centos_mysql-root   48G  7.7G   40G  17% /
devtmpfs                       903M     0  903M   0% /dev
tmpfs                          913M     0  913M   0% /dev/shm
tmpfs                          913M   17M  896M   2% /run
tmpfs                          913M     0  913M   0% /sys/fs/cgroup
/dev/sda1                      497M  125M  373M  25% /boot
tmpfs                          183M     0  183M   0% /run/user/0
/dev/sr0                       4.1G  4.1G     0 100% /mnt
#yum源配置，按如下步驟修改文件內容
[root@MySQLNODE02 /]#vi /etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
baseurl=file:///mnt/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
#清理yum配置
[root@MySQLNODE02 /]#yum  clean all
#安裝基礎依賴軟件包
yum -y install cpp binutils glibc glibc-kernheaders glibc-common glibc-devel gcc make gcc-c++ libstdc++-devel tcl
#解壓
[root@MySQLNODE02 local]# tar -zxf redis-3.2.3.tar.gz
#編譯安裝前
[root@MySQLNODE02 local]# cd /usr/local/redis-3.2.3
[root@MySQLNODE02 redis-3.2.3]# ls src/redis-*
src/redis-benchmark.c  src/redis-check-aof.c  src/redis-check-rdb.c  src/redis-cli.c  src/redis-trib.rb [root@MySQLNODE02 redis-3.2.3]# make && make install
……
    INSTALL install
    INSTALL install
    INSTALL install
    INSTALL install
    INSTALL install
make[1]: Leaving directory `/usr/local/redis-3.2.3/src'
#編譯安裝後
[root@MySQLNODE02 redis-3.2.3]# ls src/redis-*
src/redis-benchmark    src/redis-check-aof    src/redis-check-rdb    src/redis-cli    src/redis-sentinel
src/redis-benchmark.c  src/redis-check-aof.c  src/redis-check-rdb.c  src/redis-cli.c  src/redis-server
src/redis-benchmark.o  src/redis-check-aof.o  src/redis-check-rdb.o  src/redis-cli.o  src/redis-trib.rb
#利用默認配置啓動redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server
15968:C 25 Aug 11:19:34.214 # Warning: no config file specified, using the default config. In order to specify a config file use ./src/redis-server /path/to/redis.conf
15968:M 25 Aug 11:19:34.216 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 15968
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

15968:M 25 Aug 11:19:34.229 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15968:M 25 Aug 11:19:34.230 # Server started, Redis version 3.2.3
15968:M 25 Aug 11:19:34.230 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15968:M 25 Aug 11:19:34.230 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15968:M 25 Aug 11:19:34.230 * The server is now ready to accept connections on port 6379
#後臺進程啓動並簡單測試
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  >/dev/null  2>&1 &
[5] 16060
[root@MySQLNODE02 redis-3.2.3]# netstat -nlap | grep 6379
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      16060/./src/redis-s
tcp6       0      0 :::6379                 :::*                    LISTEN      16060/./src/redis-s
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli
127.0.0.1:6379> set foo bar
OK
127.0.0.1:6379> get foo
"bar"

簡單集羣配置

本集羣規模爲兩個redis節點和一個哨兵節點，配置兩個redis節點，過程基本一致，確認兩節點的主從狀態。redis

#節點222上的7000
[root@MySQLNODE02 redis-3.2.3]# cp redis.conf  clusters/redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# vi clusters/redis_7000.conf
port 7000
daemonize yes
bind 192.168.0.222
save ""
#save 900 1
#save 300 10
#save 60 10000
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
#節點224上的7000
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7000.conf
port 7000
daemonize yes
bind 192.168.0.224
save ""
#save 900 1
#save 300 10
#save 60 10000
slaveof 192.168.0.222 7000
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
#分別在兩個機器上檢查各自的主從狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.0.224,port=7000,state=online,offset=841,lag=1
master_repl_offset:841
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:840
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7000
# Replication
role:slave
master_host:192.168.0.222
master_port:7000
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:1023
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

哨兵

Redis的哨兵(sentinel) 系統用於管理多個 redis 服務器,該系統執行如下三個任務:算法

監控(Monitoring): 哨兵(sentinel) 會不斷地檢查你的Master,Slave,other sentinel是否運做正常；
提醒(Notification):當被監控的某個 Redis出現問題時, 哨兵(sentinel) 能夠經過 API 向管理員或者其餘應用程序發送通知；
自動故障遷移(Automatic failover):當一個Master不能正常工做時，哨兵(sentinel) 會開始一次自動故障遷移操做,它會將集羣中一個Slave升級爲新的Master,原master作爲slave; 當客戶端試圖鏈接失效的Master時,集羣也會向客戶端返回新Master的地址,使得集羣可使用新Master代替失效Master；

哨兵(sentinel) 是一個分佈式系統,你能夠在一個架構中運行多個哨兵(sentinel) 進程,這些進程使用流言協議(gossipprotocols)來接收關於Master是否下線的信息,並使用投票協議(agreement protocols)來決定是否執行自動故障遷移,以及選擇哪一個Slave做爲新的Master。sql

每一個哨兵(sentinel) 會向其它sentinel、master、slave定時發送消息,以確認對方是否」活」着,若是發現對方在指定時間(可配置)內未迴應,則暫時認爲對方已掛(所謂的」主觀認爲宕機」 Subjective Down,簡稱sdown)；若「哨兵羣」中的多數sentinel,都報告某一master沒響應,系統才認爲該master 「完全死亡」(即:客觀上的真正down機,Objective Down,簡稱odown),經過必定的vote算法,從剩下的slave節點中,選一臺提高爲master,而後自動修改相關配置。數據庫

雖然哨兵(sentinel) 是一個單獨的可執行文件 redis-sentinel ,但實際上它只是一個運行在特殊模式下的 Redis 服務器，你能夠在啓動一個普通 Redis 服務器時經過給定 --sentinel 選項來啓動哨兵(sentinel)。哨兵(sentinel) 的一些設計思路和zookeeper很是相似。json

哨兵配置文件在安裝reids過程當中已經建立，只須要修改相應的配置文件啓動便可。主要關注sentinel monitor配置項。centos

[root@MySQLNODE02 redis-3.2.3]# cp sentinel.conf  sentinel.conf.bak
[root@MySQLNODE02 redis-3.2.3]# vi sentinel.conf
port 27000
protected-mode no
sentinel monitor mymaster 192.168.0.222 7000 1
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-sentinel  sentinel.conf
18844:X 25 Aug 14:54:38.995 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 27000
 |    `-._   `._    /     _.-'    |     PID: 18844
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

18844:X 25 Aug 14:54:38.997 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
18844:X 25 Aug 14:54:38.999 # Sentinel ID is 94dd02d5be1379ade7b1fca1de96e9ad412dbe68
18844:X 25 Aug 14:54:38.999 # +monitor master mymaster 192.168.0.222 7000 quorum 1
18844:X 25 Aug 14:54:39.001 * +slave slave 192.168.0.224:7000 192.168.0.224 7000 @ mymaster 192.168.0.222 7000

實際配置中能夠將daemonize yes配置爲no，在啓動服務的時候用>/filename.log 2>&1 &的方式實如今後臺運行同時將對應運行日誌寫入指定文件，方便查看服務運行狀態。待系統運行穩定後能夠設置daemonize yes避免日誌過多佔用OS空間。緩存

測試哨兵模式下的集羣狀態變化過程，整個集羣中的哨兵，master,slave的配置文件在集羣狀態發生變化時內容會自動更新，因此從新啓動時若是要保持原始集羣的主從配置狀態，須要從新配置這些文件（特別是哨兵的配置文件sentinel.conf，不從新配置啓動時只能監控master節點）。在主從切換的過程當中，哨兵也會根據集羣的最新狀態更新相應的redis.conf配置文件。

複雜集羣配置

本集羣環境中配置5個Redis服務，3個哨兵，Redis的安裝參照單機安裝過程，如下爲本集羣各節點的配置過程。

#master 7000
[root@MySQLNODE02 redis-3.2.3]# cp redis.conf  clusters/redis_7000.conf
port 7000
daemonize yes
bind 192.168.0.222
save ""
#save 900 1
#save 300 10
#save 60 10000
#slave 7001
[root@MySQLNODE02 redis-3.2.3]# cp redis.conf  clusters/redis_7001.conf
port 7001
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.222
save ""
#save 900 1
#save 300 10
#save 60 10000
#slave 7002
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7002.conf
port 7002
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.224
save ""
#save 900 1
#save 300 10
#save 60 10000
#slave 7003
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7003.conf
port 7003
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.224
#slave 7004
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7004.conf
port 7004
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.224
save ""
#save 900 1
#save 300 10
#save 60 10000
如下爲各sentinel配置過程。
#sentinel 27000
[root@MySQLNODE02 redis-3.2.3]# cp sentinel.conf  clusters/sentinel_27000.conf
port 27000
protected-mode no
#2表示最低經過票數
sentinel monitor mymaster 192.168.0.222 7000 2
#sentinel 27001
[root@MySQLNODE02 redis-3.2.3]# cp sentinel.conf  clusters/sentinel_27001.conf
port 27001
protected-mode no
sentinel monitor mymaster 192.168.0.222 7000 2
#sentinel 27002
[root@MySQLNODE04 redis-3.2.3]# cp sentinel.conf  clusters/sentinel_27002.conf
port 27002
protected-mode no
sentinel monitor mymaster 192.168.0.222 7000 2
啓動各個redis節點，並經過客戶端工具查看集羣的狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7001.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root      20104      1  0 16:27 ?        00:00:00 ./src/redis-server 192.168.0.222:7000
root      20110      1  0 16:27 ?        00:00:00 ./src/redis-server 192.168.0.222:7001
root      20132  15583  0 16:28 pts/0    00:00:00 grep --color=auto redis
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7002.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7003.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7004.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     117642      1  0 16:28 ?        00:00:00 ./src/redis-server 192.168.0.224:7002
root     117646      1  0 16:28 ?        00:00:00 ./src/redis-server 192.168.0.224:7003
root     117650      1  0 16:28 ?        00:00:00 ./src/redis-server 192.168.0.224:7004
root     117654   6219  0 16:28 pts/0    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
192.168.0.222:7000> info replication
# Replication
role:master
connected_slaves:4
slave0:ip=192.168.0.222,port=7001,state=online,offset=197,lag=1
slave1:ip=192.168.0.224,port=7002,state=online,offset=197,lag=1
slave2:ip=192.168.0.224,port=7003,state=online,offset=197,lag=0
slave3:ip=192.168.0.224,port=7004,state=online,offset=197,lag=1
master_repl_offset:197
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:196
#依次在新的窗口啓動各個哨兵，啓動一會監聽到兩哨兵關掉
20938:X 25 Aug 17:24:14.116 # Sentinel ID is 1056bcf2bb85895e72338752e8a0fe7f1a233449
20938:X 25 Aug 17:24:14.116 # +monitor master mymaster 192.168.0.222 7000 quorum 2
20938:X 25 Aug 17:24:14.117 * +slave slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.119 * +slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.120 * +slave slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.122 * +slave slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.596 * +sentinel sentinel 8b66a575e24286486637d081030a49154436c5e6 192.168.0.224 27002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:15.138 * +sentinel sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.222 7000
驗證過程，先關閉其中一個slave節點，再啓動該節點；關閉master節點，再啓動該節點；關閉一個sentinel節點；收集個驗證過程當中集羣哨兵的日誌信息。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-sentinel  clusters/sentinel_27001.conf
20938:X 25 Aug 17:24:14.111 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 27001
 |    `-._   `._    /     _.-'    |     PID: 20938
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

20938:X 25 Aug 17:24:14.114 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
20938:X 25 Aug 17:24:14.116 # Sentinel ID is 1056bcf2bb85895e72338752e8a0fe7f1a233449
20938:X 25 Aug 17:24:14.116 # +monitor master mymaster 192.168.0.222 7000 quorum 2
20938:X 25 Aug 17:24:14.117 * +slave slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.119 * +slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.120 * +slave slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.122 * +slave slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.596 * +sentinel sentinel 8b66a575e24286486637d081030a49154436c5e6 192.168.0.224 27002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:15.138 * +sentinel sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.222 7000
#整個集羣啓動狀態
20938:X 25 Aug 17:29:10.069 # +sdown slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:29:46.571 * +reboot slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:29:46.622 # -sdown slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
#關閉一個slave，再啓動該slave節點的過程
20938:X 25 Aug 17:31:07.727 # +sdown master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.780 # +odown master mymaster 192.168.0.222 7000 #quorum 2/2
20938:X 25 Aug 17:31:07.780 # +new-epoch 1
20938:X 25 Aug 17:31:07.780 # +try-failover master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.781 # +vote-for-leader 1056bcf2bb85895e72338752e8a0fe7f1a233449 1
20938:X 25 Aug 17:31:07.783 # 2944fe768c00ff33c0dd2bbad24f432f384d946f voted for 1056bcf2bb85895e72338752e8a0fe7f1a233449 1
20938:X 25 Aug 17:31:07.783 # 8b66a575e24286486637d081030a49154436c5e6 voted for 1056bcf2bb85895e72338752e8a0fe7f1a233449 1
20938:X 25 Aug 17:31:07.845 # +elected-leader master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.845 # +failover-state-select-slave master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.938 # +selected-slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.938 * +failover-state-send-slaveof-noone slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.030 * +failover-state-wait-promotion slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.748 # +promoted-slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.749 # +failover-state-reconf-slaves master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.830 * +slave-reconf-sent slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.769 * +slave-reconf-inprog slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.769 * +slave-reconf-done slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.845 * +slave-reconf-sent slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.923 # -odown master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:10.814 * +slave-reconf-inprog slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:10.814 * +slave-reconf-done slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:10.870 * +slave-reconf-sent slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.817 * +slave-reconf-inprog slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.817 * +slave-reconf-done slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.875 # +failover-end master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.875 # +switch-master mymaster 192.168.0.222 7000 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:41.903 # +sdown slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002
#關閉Redis 的master節點，在多sentinel模式下會先選舉一個sentinel的leader節點
20938:X 25 Aug 17:32:31.091 # -sdown slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002
#啓動關閉的master節點
20938:X 25 Aug 17:32:41.044 * +convert-to-slave slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002

哨兵關閉一個，收集哨兵的日誌信息。被關閉的哨兵窗口會關閉，其餘兩個窗口輸出以下信息僅此而已，哨兵之間是平等關係的。

117740:X 25 Aug 17:53:34.956 # +sdown sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.224 7002

20938:X 25 Aug 17:53:35.183 # +sdown sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.224 7002

Redis命令操做

Hash

Redis的哈希是鍵值對的集合。 Redis的哈希值是字符串字段和字符串值之間的映射，所以它們被用來表示對象。

#在master節點上進行操做：
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> hset user:1 username zhangsan
(integer) 1
192.168.0.224:7002> hset user:1 password zhangsan123
(integer) 1
192.168.0.224:7002> hset user:2 username zhangsan
(integer) 1
192.168.0.224:7002> hdel user:2 username
(integer) 1
192.168.0.224:7002> hset user:2 username lisi
(integer) 1
192.168.0.224:7002> hset user:2 password lisi123
(integer) 1
192.168.0.224:7002> hmset user:3 username wangwu password wangwu123
OK
192.168.0.224:7002> hget user:1 username
"zhangsan"
192.168.0.224:7002> hmget user:1 username password
1) "zhangsan"
2) "zhangsan123"
192.168.0.224:7002> hgetall user:1
1) "username"
2) "zhangsan"
3) "password"
4) "zhangsan123"
192.168.0.224:7002> hgetall user:2;
(empty list or set)
192.168.0.224:7002> hgetall user:2
1) "username"
2) "lisi"
3) "password"
4) "lisi123"
#在slave節點上操做，slave只能查
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7001
192.168.0.222:7001> hgetall user:1
1) "username"
2) "zhangsan"
3) "password"
4) "zhangsan123"
192.168.0.222:7001> hgetall user:4
(empty list or set)
192.168.0.222:7001> hset user:4 username zhaoliu
(error) READONLY You can't write against a read only slave.

默認狀況下redis數據庫充當slave角色時是隻讀的不能進行寫操做。可經過修改配置項使從庫也具備寫功能。
slave-read-only yes

列表

一個簡單的字符串列表,排序插入順序,添加到頭部或者尾部,通常咱們都是存放json序列化後的數據。他的最大長度爲2^32 – 1。

192.168.0.224:7002> lpush list 1
(integer) 1
192.168.0.224:7002> rpush list 2
(integer) 2
192.168.0.224:7002> lpush list 3 4 5
(integer) 5
192.168.0.224:7002> llen list
(integer) 5
192.168.0.224:7002> lpop list
"5"
192.168.0.224:7002> rpop list
"2"
192.168.0.224:7002> llen list
(integer) 3

Redis持久化

RDB

RDB 持久化能夠在指定的時間間隔內生成數據集的時間點快照（point-in-time snapshot），在Redis配置文件中已經預置了3個條件知足其中任何一個條件就會進行快照。

默認的RDB的文件路徑是在當前目錄, 文件名是 : dump.rdb,能夠再配置文件中修改路徑和文件名,分別是dir和dbfilenameRedis啓動後會讀取RDB快照文件,將數據從硬盤中載入到內存,通常狀況下1GB的快照文件載入到內存的時間約爲20~30分鐘.(不一樣服務器會存在差別)。

BD文件是經過壓縮的, 能夠經過配置rdbcompression參數來禁用壓縮。

rdbcompression yes

若是要禁止Redis的RDB的持久化方式那麼只用吧上面的觸發條件禁止掉就OK了。

RDB的持久化過程

Redis是使用fork函數複製一份當前進程(父進程)的副本(子進程);
父進程繼續接收並處理客戶端發來的命令,而子進程開始將內存中的數據寫入到磁盤的臨時文件中;
當子進程寫入完全部的數據後會用該臨時文件替換掉舊的RDB文件;

因爲Redis使用fork來複制一份當前進程,那麼子進程就會佔有和主進程同樣的內存資源,若是主進程是8G內存,那麼在備份的時候必須保證有16GB的內存,要否則會啓用虛擬內存,性能很是的差,若是數據集很是巨大,也可能會影響到CPU。

AOF

AOF 持久化記錄服務器執行的全部寫操做命令,並在服務器啓動時,經過從新執行這些命令來還原數據集。AOF 文件中的命令所有以 Redis 協議的格式來保存,新命令會被追加到文件的末尾. Redis 還能夠在後臺對 AOF 文件進行重寫（rewrite）,使得 AOF 文件的體積不會超出保存數據集狀態所需的實際大小。

AOF持久化經過修改appendonly參數來設置默認是不開啓的

AOF文件的位置和RDB文件的位置是相同的,都是經過dir參數設置,默認的文件名是apendonly.aof,也能夠經過apendfilename參數修改。

# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
# appendfsync always
appendfsync everysec
# appendfsync no

auto-aof-rewrite-percentage：表示當前AOF文件大小超過上一次重寫時的AOF文件大小的百分之多少時會再次進行重寫,若是以前沒有重寫過,則以啓動時的AOF文件大小爲依據。上面的表示百分之一百。

uto-aof-rewrite-min-size：表示限制容許重寫的最小AOF文件大小,一般在AOF文件很小的時候即便其中有些冗餘的命令也是能夠忽略的。

文件寫入默認狀況下會先寫入到系統的緩存中,系統每30秒同步一次,纔是真正的寫入到硬盤中,若是在這30秒服務器宕機的話,那麼數據也會丟失,能夠經過修改Reids 配置參數來修改同步策略。

Redis主從架構

主從複製(讀寫分離)

redis的主從複製功能很是強大，一個master能夠擁有多個slave,而一個slave又能夠擁有多個slave,如此下去,造成了強大的多級服務器集羣架構. 能夠避免Redis單點故障,構建讀寫分離架構,知足讀多寫少的應用場景。

一個Master能夠有多個Slave；
Redis使用異步複製。從2.8開始，Slave會週期性（每秒一次）發起一個Ack確認複製流（replication stream）被處理進度；
不只主服務器能夠有從服務器，從服務器也能夠有本身的從服務器，多個從服務器之間能夠構成一個圖狀結構；
複製在Master端是非阻塞模式的，這意味着即使是多個Slave執行首次同步時，Master依然能夠提供查詢服務；
複製在Slave端也是非阻塞模式的：若是你在redis.conf作了設置，Slave在執行首次同步的時候仍可使用舊數據集提供查詢；你也能夠配置爲當Master與Slave失去聯繫時，讓Slave返回客戶端一個錯誤提示；
當Slave要刪掉舊的數據集，並從新加載新版數據時，Slave會阻塞鏈接請求（通常發生在與Master斷開重連後的恢復階段）；
複製功能能夠單純地用於數據冗餘（dataredundancy），也能夠經過讓多個從服務器處理只讀命令請求來提高擴展性（scalability）：好比說，繁重的 SORT 命令能夠交給附屬節點去運行。
能夠經過修改Master端的redis.config來避免在Master端執行持久化操做（Save），由Slave端來執行持久化。

Redis的主從配置能夠經過修改從庫的Redis.conf文件增長slaveof配置項，也能夠在全部節點都啓動後，在從庫的redis.cli窗口執行slaveof命令配置，但此種配置重啓後會丟失配置信息。

主從從架構，的配置將slaveof配置爲上層從庫的IP和端口便可。在主庫set的數據在從庫都能get到。

數據同步過程

當從庫和主庫創建MS關係後，會向主數據庫發送SYNC命令(同步命令);

主庫接收到SYNC命令後會開始在後臺保存快照（RDB持久化過程），並將期間接收到的寫命令緩存起來;因此就算關掉了RDB持久化方式,在他們同步的時候也會產生RDB文件

以上主庫只作了2件事情將接收的命令進行RDB持久化；在RDB持久化中,將接收的命令緩存起來

當快照完成後，主Redis會將快照文件和全部緩存的寫命令發送給從Redis;

從Redis接收到後，會載入快照文件而且執行收到的緩存的命令;

以後，主Redis每當接收到寫命令時就會將命令發送從Redis，從而保證數據的一致;

在這個過程當中主從同步是經過RDB來同步數據, 即便禁用了RDB也沒有用,那麼就會產生IO問題,在這個複製過程可能就會出現瓶頸. Redis在2.8.18版本開始實現了無磁盤複製功能

Redis在與從庫進行復制初始化時將不會吧快照儲存到磁盤,而是直接經過網絡發送給從庫,避免了IO性能問題

修改repl-diskless-sync 爲 yes

# With slow disks and fast (large bandwidth) networks, diskless replication
# works better.
repl-diskless-sync no

宕機處理

在主從架構中出現了宕機的狀況

Slave 宕機

在Redis中,從庫從新啓動會自動加入到主從架構中,自動完成同步數據；

Redis 2.8以後,在從庫有作持久化的前提下,若是從庫在斷開的期間,主庫的變化不大,從庫再次啓動後,主庫不會將或有的數據進行RDB操做,而是進行增量複製；

Master 宕機

在Slave中執行SLAVEOF NO ONE 命令,斷開主從關係而且提高爲主庫繼續服務；

將主庫從新啓動後,執行 SLAVEOF命令,將其設置爲其餘Redis的從庫,這個時候數據就同步回來了；

Redis3.0集羣

雖然咱們搭建了一個主從架構,可是每一個Redis都要保存相同的數據,這樣容易形成水桶效應。並且主從架構頻繁TCP鏈接斷開也可能會對服務器和網絡帶來很大負擔。Redis 3.0版本最大的更新就是支持集羣。集羣具備如下特徵。

節點自動發現

slave->master 選舉,集羣容錯

Hot resharding:在線分片

進羣管理:clusterxxx

基於配置(nodes-port.conf)的集羣管理

ASK轉向/MOVED 轉向機制

何時整個集羣不可用(cluster_state:fail),當集羣不可用時,全部對集羣的操做作都不可用，收到((error)CLUSTERDOWN The cluster is down)錯誤；若是集羣任意master掛掉,且當前master沒有slave.集羣進入fail狀態,也能夠理解成進羣的slot映射[0-16383]不完成時進入fail狀態；若是進羣超過半數以上master掛掉，不管是否有slave集羣進入fail狀態。

集羣搭建

搭建不帶slave的三節點master集羣。修改各節點的配置文件。

#若是配置了slaveof須要關閉
#7000節點
cluster-enabled yes
cluster-config-file nodes-7000.conf
port 7000
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
#7001節點
cluster-enabled yes
cluster-config-file nodes-7001.conf
port 7001
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
#7002節點
cluster-enabled yes
cluster-config-file nodes-7002.conf
port 7002
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
依次啓動確認各節點的狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7001.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root      76990      1  0 17:36 ?        00:00:00 ./src/redis-server 192.168.0.222:7000 [cluster]
root      76996      1  0 17:36 ?        00:00:00 ./src/redis-server 192.168.0.222:7001 [cluster]
root      77006  70424  0 17:36 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7002.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     121497      1  0 17:37 ?        00:00:00 ./src/redis-server 192.168.0.224:7002 [cluster]
root     121501 121037  0 17:37 pts/1    00:00:00 grep --color=auto redis
Redis3.0之後版本配置集羣時使用redis-trib.rb工具，該工具的使用須要安裝依賴的基礎軟件包，此軟件包（redis-3.2.1.gem）在ISO光盤中並不存在，須要下載上傳服務器後安裝。
[root@MySQLNODE02 redis-3.2.3]# yum -y install zlib ruby rubygems
[root@MySQLNODE02 redis-3.2.3]# gem install -l ./redis-3.2.1.gem
Successfully installed redis-3.2.1
Parsing documentation for redis-3.2.1
Installing ri documentation for redis-3.2.1
1 gem installed
集羣的配置要求各個Redis節點都是最新的，不然會報一下錯誤，能夠從新安裝配置Redis再配置（也能夠刪除對應node-port.conf文件）。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb create --replicas 0 192.168.0.222:7000 192.168.0.222:7001 192.168.0.224:7002
>>> Creating cluster
[ERR] Node 192.168.0.222:7000 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
經過redis-trib.rb工具配置集羣環境
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb create --replicas 0 192.168.0.222:7000 192.168.0.222:7001 192.168.0.224:7002
>>> Creating cluster
>>> Performing hash slots allocation on 3 nodes...
Using 3 masters:
192.168.0.222:7000
192.168.0.224:7002
192.168.0.222:7001
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 192.168.0.222:7000)
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
由於配置Redis集羣以後每次設置值的時候都要先計算key的值獲取該key 的插槽值,而後在設置到集羣中對應插槽值的Redis實例中，若是沒有換一個key試試，在set 和 get 的時候都報錯。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
192.168.0.222:7000> set test test
(error) MOVED 6918 192.168.0.224:7002
192.168.0.222:7000> get test
(error) MOVED 6918 192.168.0.224:7002
#能夠去指定節點上操做
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> get test
(nil)
192.168.0.224:7002> set test AAAA
OK
192.168.0.224:7002> get test
"AAAA"
redis-cli 提供了一個參數–c，指定了這個參數以後,redis-cli會根據插槽值作一個重定向,鏈接到指定的redis實例上面。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> get test
-> Redirected to slot [6918] located at 192.168.0.224:7002
"AAAA"
#查看集羣的節點信息
192.168.0.224:7002> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 master - 0 1503915810894 1 connected 0-5460
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 myself,master - 0 0 3 connected 5461-10922
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1503915811914 2 connected 10923-16383

Redis集羣的數據是根據插槽值來設置進具體的節點中的.可是若是這個key的插槽值不是在當前redis實例的話,他就須要進行重定向.這樣一次操做就變成了2次，這個就是Redis3.0裏面所說的ASK 轉向/MOVED 轉向機制。

整個Redis提供了16384個插槽，也就是說集羣中的每一個節點分得的插槽數總和爲16384。redis-trib.rb腳本實現了是將16384個插槽平均分配給了N個節點。

key的有效部分使用CRC16算法計算出哈希值，再將哈希值對16384取餘，獲得插槽值。

新增節點

新增一個節點的配置文件，並結合集羣中各節點的配置修改相應配置項。

7003
cluster-enabled yes
cluster-config-file nodes-7003.conf
port 7003
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
#啓動該節點
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7003.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     124626      1  0 18:18 ?        00:00:01 ./src/redis-server 192.168.0.224:7002 [cluster]
root     124640      1  0 18:27 ?        00:00:00 ./src/redis-server 192.168.0.224:7003 [cluster]
root     124644 121037  0 18:27 pts/1    00:00:00 grep --color=auto redis
在集羣中的任意節點上利用redis-trib.rb工具配置新增節點，
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-trib.rb add-node 192.168.0.224:7003 192.168.0.224:7002
>>> Adding node 192.168.0.224:7003 to cluster 192.168.0.224:7002
>>> Performing Cluster Check (using node 192.168.0.224:7002)
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
   0 additional replica(s)
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.224:7003 to make it join the cluster.
[OK] New node added correctly.
添加成功後，查看集羣個節點的狀態信息，發現新增長的節點沒有分配插槽值。
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 master - 0 1503916248543 1 connected 0-5460
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 myself,master - 0 0 3 connected 5461-10922
f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003 master - 0 1503916249565 0 connected
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1503916250585 2 connected 10923-16383
經過命令手動給新增長節點分配插槽值
[root@MySQLNODE02 redis-3.2.3]# ./redis-trib.rb reshard 192.168.0.224:7003
#指定要轉移的插槽數
#接收插槽的節點ID
#指定插槽轉移方式，all(所有重分配),done(從指定節點轉移)
#yes確認轉移
#轉移完成後查看集羣節點的狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1503916592510 2 connected 10923-16383
f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003 master - 0 1503916590480 4 connected 5461-6460
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 master - 0 1503916591496 3 connected 6461-10922

刪除節點

若是要刪除某個節點的話, 咱們要先把該節點上面的插槽數給轉移到別的節點上面,而後在經過redis-trib.rb腳原本刪除節點,否則這個插槽數就沒有咯,而對應的key也失效。

#轉移要刪除節點的插槽
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb reshard 192.168.0.224:7003
>>> Performing Cluster Check (using node 192.168.0.224:7003)
M: f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003
   slots:5461-6460 (1000 slots) master
   0 additional replica(s)
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:6461-10922 (4462 slots) master
   0 additional replica(s)
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
#指定要轉移插槽數
How many slots do you want to move (from 1 to 16384)? 1000
#指定接收者
What is the receiving node ID? 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
#指定要轉移的節點
Source node #1:f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac
Source node #2:done  
......
#確認轉移
Do you want to proceed with the proposed reshard plan (yes/no)? yes
#查看轉移後的集羣狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1504058169010 2 connected 10923-16383
f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003 master - 0 1504058170031 4 connected
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 master - 0 1504058167991 5 connected 5461-10922
#刪除節點
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb   del-node 192.168.0.224:7003 f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac
>>> Removing node f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac from cluster 192.168.0.224:7003
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
[root@MySQLNODE02 redis-3.2.3]#
#查看刪除後的狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1504058318129 2 connected 10923-16383
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 master - 0 1504058317110 5 connected 5461-10922

高可用集羣

能夠搭建一個只有master節點的集羣,可是若是咱們其中的一個master宕機了,那麼他對應插槽值的key所有都會失效,集羣就不可用.Redis集羣爲咱們提供了故障機制，能夠爲集羣中的每一個master節點指定一個slave節點實現高可用集羣環境。

集羣中的每一個節點都會按期的向其它節點發送PING命令，而且經過有沒有收到回覆判斷目標節點是否下線

集羣中每一秒就會隨機選擇幾個節點，而後選擇其中最久沒有響應的節點放PING命令

若是必定時間內目標節點都沒有響應，那麼該節點就認爲目標節點疑似下線

當集羣中的節點超過半數認爲該目標節點疑似下線，那麼該節點就會被標記爲下線

當集羣中的任何一個節點下線，就會致使插槽區有空檔，不完整，那麼該集羣將不可用

若是該下線的節點使用了主從模式,那麼該節點(master)宕機後,集羣會將該節點的從庫(slave)提高爲(master)繼續完成集羣服務。

按照集羣模式準備好6個redis節點的配置文件，清理原來啓用過得節點文件（不然啓動時會提示節點不爲空），依次啓動各節點。

[root@MySQLNODE02 redis-3.2.3]# ll redis_*
-rw-r--r-- 1 root root 46727 Aug 30 10:23 redis_7000.conf
-rw-r--r-- 1 root root 46727 Aug 28 17:32 redis_7001.conf
-rw-r--r-- 1 root root 46727 Aug 30 10:25 redis_7005.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7001.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7005.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111838      1  0 10:31 ?        00:00:00 ./src/redis-server 192.168.0.222:7000 [cluster]
root     111844      1  0 10:31 ?        00:00:00 ./src/redis-server 192.168.0.222:7001 [cluster]
root     111848      1  0 10:31 ?        00:00:00 ./src/redis-server 192.168.0.222:7005 [cluster]
root     111854  70424  0 10:32 pts/1    00:00:00 grep --color=auto redis

[root@MySQLNODE04 redis-3.2.3]# ll redis_*
-rw-r--r-- 1 root root 46727 Aug 28 17:34 redis_7002.conf
-rw-r--r-- 1 root root 46727 Aug 28 18:26 redis_7003.conf
-rw-r--r-- 1 root root 46755 Aug 30 10:29 redis_7004.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7002.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7003.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7004.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     126824      1  0 10:32 ?        00:00:00 ./src/redis-server 192.168.0.224:7002 [cluster]
root     126828      1  0 10:32 ?        00:00:00 ./src/redis-server 192.168.0.224:7003 [cluster]
root     126832      1  0 10:32 ?        00:00:00 ./src/redis-server 192.168.0.224:7004 [cluster]
root     126836 121037  0 10:32 pts/1    00:00:00 grep --color=auto redis
利用redis-trib.rb文件配置高可用集羣，redis會自動指定主從節點，經過實際配置大體應該是從小到大依次分配主從節點。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb  create --replicas 1 192.168.0.222:7000 192.168.0.222:7001 192.168.0.222:7005 192.168.0.224:7002 192.168.0.224:7003 192.168.0.224:7004
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.0.222:7000
192.168.0.224:7002
192.168.0.222:7001
Adding replica 192.168.0.224:7003 to 192.168.0.222:7000
Adding replica 192.168.0.222:7005 to 192.168.0.224:7002
Adding replica 192.168.0.224:7004 to 192.168.0.222:7001
M: 60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
S: 91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005
   replicates e154234f624b5f4f44cd3dc029b87ecabb29e564
M: e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
S: e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003
   replicates 60b53e792f10a1fab087e73713180aa56554858a
S: db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004
   replicates 3ef887b183685eb4687085bea40352af018f07ea
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 192.168.0.222:7000)
M: 60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
M: 91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005
   slots: (0 slots) master
   replicates e154234f624b5f4f44cd3dc029b87ecabb29e564
M: e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
M: e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003
   slots: (0 slots) master
   replicates 60b53e792f10a1fab087e73713180aa56554858a
M: db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004
   slots: (0 slots) master
   replicates 3ef887b183685eb4687085bea40352af018f07ea
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
#查看集羣的狀態
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504060793611 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504060790556 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master - 0 1504060789541 4 connected 5461-10922
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 slave e154234f624b5f4f44cd3dc029b87ecabb29e564 0 1504060791568 4 connected
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504060792591 5 connected
192.168.0.222:7000>
測試集羣環境下的數據操做
#操做時仍是須要重定向
192.168.0.222:7000> set one aaaa
(error) MOVED 9084 192.168.0.224:7002
192.168.0.222:7000> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> set one AAAAAA
-> Redirected to slot [9084] located at 192.168.0.224:7002
OK
192.168.0.224:7002> get one
"AAAAAA"
192.168.0.224:7002> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> get one
"AAAAAA"
192.168.0.224:7002> exit
#7002的從節點上沒法直接獲取數據，但有該key信息
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7005
192.168.0.222:7005> get one
(error) MOVED 9084 192.168.0.224:7002
192.168.0.222:7005> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7005
192.168.0.222:7005> get one
-> Redirected to slot [9084] located at 192.168.0.224:7002
"AAAAAA"
192.168.0.224:7002> keys *
1) "one"
測試集羣中節點的啓停對整個集羣的影響狀況。
#一個slave故障對集羣的操做無影響
[root@MySQLNODE02 redis-3.2.3]# kill -9 111848
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111838      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7000 [cluster]
root     111844      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7001 [cluster]
root     112628  70424  0 11:24 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504063464906 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504063467964 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master - 0 1504063465922 4 connected 5461-10922
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 slave,fail e154234f624b5f4f44cd3dc029b87ecabb29e564 1504063447033 1504063442442 4 disconnected
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504063466942 5 connected
192.168.0.222:7000> set two BBBBB
OK
192.168.0.222:7001> get two
-> Redirected to slot [2127] located at 192.168.0.222:7000
"BBBBB"
#重啓slave節點，對集羣的操做無影響
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7005.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111838      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7000 [cluster]
root     111844      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7001 [cluster]
root     112663      1  0 11:26 ?        00:00:00 ./src/redis-server 192.168.0.222:7005 [cluster]
root     112667  70424  0 11:26 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504063622035 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504063624074 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master - 0 1504063620413 4 connected 5461-10922
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 slave e154234f624b5f4f44cd3dc029b87ecabb29e564 0 1504063625093 4 connected
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504063623054 5 connected
192.168.0.222:7000> set three CCCCCC
-> Redirected to slot [13861] located at 192.168.0.222:7001
OK
192.168.0.222:7001> get three
"CCCCCC"
#關閉7002master節點
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     126824      1  0 10:32 ?        00:00:08 ./src/redis-server 192.168.0.224:7002 [cluster]
root     126828      1  0 10:32 ?        00:00:09 ./src/redis-server 192.168.0.224:7003 [cluster]
root     126832      1  0 10:32 ?        00:00:09 ./src/redis-server 192.168.0.224:7004 [cluster]
root     126888 121037  0 11:30 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE04 redis-3.2.3]# kill -9 126824
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504063851577 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504063846500 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master,fail - 1504063834998 1504063829584 4 disconnected
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 master - 0 1504063853607 7 connected 5461-10922
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504063852599 5 connected
#原7002上的數重定向到7005新的master上
192.168.0.222:7000> get one
-> Redirected to slot [9084] located at 192.168.0.222:7005
"AAAAAA"
#重啓7002，做爲7005的slave節點
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7002.conf
192.168.0.222:7005> cluster nodes
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504064034546 5 connected
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 myself,master - 0 0 7 connected 5461-10922
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504064031480 2 connected 10923-16383
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 slave 91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 0 1504064035568 7 connected
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504064036591 6 connected
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 master - 0 1504064037615 1 connected 0-5460
#當其中一個節點掛掉，那麼久沒法經過該節點端口鏈接到redis集羣
[root@MySQLNODE02 redis-3.2.3]# kill -9 111838
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111844      1  0 10:31 ?        00:00:25 ./src/redis-server 192.168.0.222:7001 [cluster]
root     112663      1  0 11:26 ?        00:00:17 ./src/redis-server 192.168.0.222:7005 [cluster]
root     114158  70424  0 13:22 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
Could not connect to Redis at 192.168.0.222:7000: Connection refused
Could not connect to Redis at 192.168.0.222:7000: Connection refused
not connected> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.224 -p 7003
192.168.0.224:7003> get two
"BBBBB"

多鍵的命令操做（如MGET、MSET），若是每一個鍵都位於同一個節點，則能夠正常支持，不然會提示錯誤。

集羣中的節點只能使用0號數據庫，若是執行SELECT切換數據庫會提示錯誤。

Codis

Codis 是一個分佈式 Redis 解決方案, 對於上層的應用來講, 鏈接到 CodisProxy 和鏈接原生的 Redis Server 沒有明顯的區別 (不支持的命令列表), 上層應用能夠像使用單機的 redis 同樣使用, Codis 底層會處理請求的轉發, 不停機的數據遷移等工做, 全部後邊的一切事情, 對於前面的客戶端來講是透明的, 能夠簡單的認爲後邊鏈接的是一個內存無限大的 Redis 服務。

Codis 由四部分組成:

Codis Proxy (codis-proxy) 實現redis協議，因爲自己是無狀態的，所以能夠部署不少個節點
Codis Manager (codis-config) 是codis的管理工具，包括添加/刪除redis節點添加刪除proxy節點，發起數據遷移等操做，自帶httpserver，支持管理後臺方式管理配置
Codis Redis (codis-server) 是codis維護的redis分支，基於2.8.21分支，加入了slot的支持和原子的數據遷移命令;codis-proxy和 codis-config只能和這個版本的redis交互才能正常運行
ZooKeeper 用於codis集羣元數據的存儲,維護codis集羣節點

Codis優缺點

優勢