Redis服務之Redis Cluster
在上一篇博客中咱們聊到了redis的高可用組件sentinel的相關配置，回顧請參考http://www.javashuo.com/article/p-kronkjpt-nq.html；sentinel在redis主從同步架構中主要起到了監控集羣master是否正常，若是master不正常，或者宕機，那麼sentinel會提高一個slave當選新的master，從而保證了redis服務的正常使用；但對於redis的單機寫入問題仍是一直存在；在sentinel+主從同步架構中，程序寫數據，始終是把讀寫請求發送給master（固然，若是有語句路由器讀寫請求是能夠分開的）；這樣一來對於master來說它就承擔了全部的寫操做，很顯然這種在寫操做很是的頻繁的場景，單臺master確定沒法承受這麼大的壓力；爲了解決單機master寫入數據的瓶頸問題， redis 官方在 redis 3.0 版本以後推出了無中心架構的 redis cluster 機制；所謂無中心架構的cluster意思是集羣中的每個成員都是中心節點，每一個成員擁有整個集羣的狀態元數據，但各自又只存儲整個集羣的部分數據；對於redis cluster這種架構來說，全部redis節點使用（ping機制）互聯，若是集羣中某個節點的失敗，是須要整個集羣中超過半數的節點監測都失效纔算真正的失效；客戶端不須要 proxy 便可直接鏈接 redis，應用程序須要寫所有的 redis 服務器 IP。redis cluster 把全部的 redis node 映射到 0-16383 個槽位(slot)上，讀寫須要到指定的 redis node 上進行操做，所以有多少個 reids node 至關於 redis 併發擴展了多少倍。Redis cluster 預先分配 16384 個(slot)槽位，當須要在 redis 集羣中寫入一個 key -value 的時候，會使用 CRC16(key) mod 16384 以後的值，決定將 key 寫入值哪個槽位從而決定寫入哪個 Redis 節點上，從而有效解決redis單機讀寫瓶頸。以下所示html
提示：客戶端每次讀寫都要通過CRC16對讀寫的KEY作hash計算，而後把hash後的結果再和16384作取模運算，最終結果會落到0-16383的某一個槽位上；咱們事先在定義集羣時，就把0-5460號槽位都分給了master01，意思就是隻要讀寫的數據的KEY經過CRC16hash計算取得的結果%16384，最後的結果在0-5460這個範圍內，就把本次的讀/寫請求調度到master01上；以此類推，範圍在5461-10922就把讀/寫請求調度到master02,範圍在10923-16383就把讀/寫請求調度到master03;經過上面的圖咱們也能看出各個master都只保存整個集羣的一部分數據，若是某一個master宕機，那麼就會致使對應master上分的槽位對應數據丟失；爲了解決各個master單點的問題，咱們還須要對每一個master作主從，以下圖node
提示：爲了防止某個master宕機後致使數據丟失和寫操做不能執行，對應slave應該可以及時的提高爲master，相似sentinel的功能，可是咱們不須要在部署sentinel了，由於redis cluster就支持主從切換；這樣一來咱們的集羣就變成了3主3從的集羣；接下咱們就準備一個實驗環境來部署上面的架構；python
部署redis集羣linux
環境說明web
| 角色 | ip地址 | 端口 |
| Master01 | 192.168.0.41 | 6379 |
| Slave01 | 192.168.0.42 | 6380 |
| Master02 | 192.168.0.42 | 6379 |
| Slave02 | 192.168.0.43 | 6380 |
| Master03 | 192.168.0.43 | 6379 |
| Slave03 | 192.168.0.41 | 6380 |
準備啓動3臺服務器，每一個服務器上啓動兩個redis實例，端口分別是6379（master）和6380（slave)，爲了防止master和slave落在同一個節點，有意將他們交叉組合，如上表格；redis
建立redis cluster咱們須要注意每一個 redis node 節點儘可能採用相同的硬件配置，相同的密碼；全部 redis 服務器必須沒有任何數據；json
在node01上編譯安裝redis，有關redis的編譯安裝和部署請參考http://www.javashuo.com/article/p-ckvacwaq-kp.html；api
建立目錄結構ruby
[root@node01 ~]# tree /usr/local/redis/
/usr/local/redis/
├── 6379
│ ├── etc
│ │ ├── redis.conf
│ │ └── sentinel.conf
│ ├── logs
│ └── run
├── 6380
│ ├── etc
│ │ ├── redis.conf
│ │ └── sentinel.conf
│ ├── logs
│ └── run
└── bin
├── redis-benchmark
├── redis-check-aof
├── redis-check-rdb
├── redis-cli
├── redis-sentinel -> redis-server
└── redis-server
9 directories, 10 files
[root@node01 ~]#
提示：在/usr/local/redis目錄下分別建立6379和6380,而後在其下都建立etc,logs,run 等子目錄，如上所示bash
修改配置文件
[root@node01 ~]# grep -E "^(bind|requirepass|masterauth|cluster)" /usr/local/redis/6379/etc/redis.conf bind 0.0.0.0 masterauth admin requirepass admin cluster-enabled yes cluster-config-file redis-cluster_6379.conf [root@node01 ~]# grep -E "^(bind|requirepass|masterauth|cluster)" /usr/local/redis/6380/etc/redis.conf bind 0.0.0.0 masterauth admin requirepass admin cluster-enabled yes cluster-config-file redis-cluster_6380.conf [root@node01 ~]#
提示：6379和6380的配置文件內容都是同樣的，惟一區別是端口不一樣，主要修改bind 監聽地址，設置密碼，開啓集羣以及指定對應集羣生成的配置文件名稱；
把node01上的/usr/local/redis複製給node02和node03並保存到/usr/local/目錄下
提示：我上面的三臺主機都是作了免密登陸，因此互相拷貝數據不須要輸入密碼；有關免密登陸的說明和配置能夠參考http://www.javashuo.com/article/p-rjibahxc-bk.html；
分別在node01 node02 node03上啓動redis
到此咱們就把6個redis實例啓動了；接下來咱們來建立集羣；Redis 3 和 4 版本須要使用到集羣管理工具 redis-trib.rb，這個工具是 redis 官方推出的管理 redis 集羣的工具，集成在redis 的源碼 src 目錄下，是基於 redis 提供的集羣命令封裝成簡單、便捷、實用的操做工具，redis-trib.rb是 redis 做者用 ruby 開發完成的，因此咱們要使用redis-trib.rb首先要解決ruby環境的問題；
安裝ruby環境
[root@node01 ~]# rz
rz waiting to receive.
zmodem trl+C ȡ
100% 15621 KB 15621 KB/s 00:00:01 0 Errors
[root@node01 ~]# ls
redis-4.0.9.tar.gz ruby-2.5.5.tar.gz
[root@node01 ~]# tar xf ruby-2.5.5.tar.gz -C /usr/local/src/
[root@node01 ~]# cd /usr/local/src/
[root@node01 src]# cd ruby-2.5.5/
[root@node01 ruby-2.5.5]# ./configure
checking for ruby... false
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
……省略部份內容……
checking for setjmp type... __builtin_setjmp
checking for prefix of external symbols... NONE
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking if make is GNU make... yes
.ext/include/x86_64-linux/ruby/config.h updated
configure: ruby library version = 2.5.0
configure: creating ./config.status
config.status: creating GNUmakefile
config.status: creating Makefile
config.status: creating ruby-2.5.pc
---
Configuration summary for ruby version 2.5.5
* Installation prefix: /usr/local
* exec prefix: ${prefix}
* arch: x86_64-linux
* site arch: ${arch}
* RUBY_BASE_NAME: ruby
* ruby lib prefix: ${libdir}/${RUBY_BASE_NAME}
* site libraries path: ${rubylibprefix}/${sitearch}
* vendor path: ${rubylibprefix}/vendor_ruby
* target OS: linux
* compiler: gcc
* with pthread: yes
* enable shared libs: no
* dynamic library ext: so
* CFLAGS: ${optflags} ${debugflags} ${warnflags}
* LDFLAGS: -L. -fstack-protector -rdynamic \
-Wl,-export-dynamic
* optflags: -O3
* debugflags: -ggdb3
* warnflags: -Wall -Wextra -Wno-unused-parameter \
-Wno-parentheses -Wno-long-long \
-Wno-missing-field-initializers \
-Wno-tautological-compare \
-Wno-parentheses-equality \
-Wno-constant-logical-operand -Wno-self-assign \
-Wunused-variable -Wimplicit-int -Wpointer-arith \
-Wwrite-strings -Wdeclaration-after-statement \
-Wimplicit-function-declaration \
-Wdeprecated-declarations \
-Wno-packed-bitfield-compat \
-Wsuggest-attribute=noreturn \
-Wsuggest-attribute=format
* strip command: strip -S -x
* install doc: yes
* man page type: doc
---
[root@node01 ruby-2.5.5]#
提示：redis-trib.rb這個工具在集羣任意一臺安裝便可，不須要每臺都安裝；
編譯
[root@node01 ruby-2.5.5]# make -j 2
CC = gcc
LD = ld
LDSHARED = gcc -shared
CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -std=gnu99
XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE
CPPFLAGS = -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0
DLDFLAGS = -fstack-protector -pie
SOLIBS =
LANG = en_US.UTF-8
LC_ALL =
LC_CTYPE =
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
compiling ./main.c
compiling dmydln.c
compiling miniinit.c
compiling dmyext.c
compiling miniprelude.c
making dummy probes.h
compiling bignum.c
compiling class.c
compiling compar.c
compiling compile.c
compiling complex.c
compiling cont.c
compiling debug.c
compiling debug_counter.c
compiling dir.c
compiling dln_find.c
compiling encoding.c
compiling enum.c
compiling enumerator.c
……省略部份內容……
*** Fix the problems, then remove these directories and try again if you want.
make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'
Generating RDoc documentation
Parsing sources...
100% [871/871] vsnprintf.c
Generating RI format into /usr/local/src/ruby-2.5.5/.ext/rdoc...
Files: 871
Classes: 1324 ( 565 undocumented)
Modules: 286 ( 121 undocumented)
Constants: 2181 ( 555 undocumented)
Attributes: 1066 ( 251 undocumented)
Methods: 10080 (2161 undocumented)
Total: 14937 (3653 undocumented)
75.54% documented
Elapsed: 25.0s
[root@node01 ruby-2.5.5]#
安裝ruby環境
[root@node01 ruby-2.5.5]# make install
CC = gcc
LD = ld
LDSHARED = gcc -shared
CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -std=gnu99
XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE
CPPFLAGS = -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0
DLDFLAGS = -fstack-protector -pie
SOLIBS =
LANG = en_US.UTF-8
LC_ALL =
LC_CTYPE =
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
generating enc.mk
making srcs under enc
make[1]: Entering directory `/usr/local/src/ruby-2.5.5'
make[1]: Nothing to be done for `srcs'.
make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'
generating transdb.h
transdb.h unchanged
generating makefiles ext/configure-ext.mk
ext/configure-ext.mk updated
make[1]: Entering directory `/usr/local/src/ruby-2.5.5'
make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'
generating makefile exts.mk
exts.mk updated
make[1]: Entering directory `/usr/local/src/ruby-2.5.5'
make[2]: Entering directory `/usr/local/src/ruby-2.5.5/ext/fcntl'
make[2]: Leaving directory `/usr/local/src/ruby-2.5.5/ext/fcntl'
make[2]: Entering directory `/usr/local/src/ruby-2.5.5/ext/rubyvm'
……省略部份內容……
installing binary commands: /usr/local/bin
installing base libraries: /usr/local/lib
installing arch files: /usr/local/lib/ruby/2.5.0/x86_64-linux
installing pkgconfig data: /usr/local/lib/pkgconfig
installing command scripts: /usr/local/bin
installing library scripts: /usr/local/lib/ruby/2.5.0
installing common headers: /usr/local/include/ruby-2.5.0
installing manpages: /usr/local/share/man/man1
installing extension objects: /usr/local/lib/ruby/2.5.0/x86_64-linux
installing extension objects: /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux
installing extension objects: /usr/local/lib/ruby/vendor_ruby/2.5.0/x86_64-linux
installing extension headers: /usr/local/include/ruby-2.5.0/x86_64-linux
installing extension scripts: /usr/local/lib/ruby/2.5.0
installing extension scripts: /usr/local/lib/ruby/site_ruby/2.5.0
installing extension scripts: /usr/local/lib/ruby/vendor_ruby/2.5.0
installing extension headers: /usr/local/include/ruby-2.5.0/ruby
installing default gems from lib: /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications)
cmath 1.0.0
csv 1.0.0
fileutils 1.0.2
ipaddr 1.2.0
rdoc 6.0.1
scanf 1.0.0
webrick 1.4.2
installing default gems from ext: /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications)
bigdecimal 1.3.4
date 1.0.0
etc 1.0.0
fcntl 1.0.0
fiddle 1.0.0
io-console 0.4.6
json 2.1.0
psych 3.0.2
sdbm 1.0.0
stringio 0.0.1
strscan 1.0.0
installing bundled gems: /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications)
minitest 5.10.3
rake 12.3.0
xmlrpc 0.3.0
did_you_mean 1.2.0
net-telnet 0.1.1
power_assert 1.1.1
test-unit 3.2.7
installing rdoc: /usr/local/share/ri/2.5.0/system
installing capi-docs: /usr/local/share/doc/ruby
[root@node01 ruby-2.5.5]#
提示：到此ruby環境就安裝好了，這裏注意一點，yum安裝的ruby版本有點低，因此咱們這裏選擇編譯安裝；
gem安裝redis包
[root@node01 ruby-2.5.5]# gem install redis
ERROR: Loading command: install (LoadError)
cannot load such file -- zlib
ERROR: While executing gem ... (NoMethodError)
undefined method `invoke_with_build_args' for nil:NilClass
[root@node01 ruby-2.5.5]#
提示：gem安裝提示咱們沒有zlib，遇到這個問題咱們用yum安裝zlib-devel包，而後在把zlib集成到ruby環境便可；
[root@node01 ruby-2.5.5]# yum install zlib-devel Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Resolving Dependencies --> Running transaction check ---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed --> Processing Dependency: zlib = 1.2.7-18.el7 for package: zlib-devel-1.2.7-18.el7.x86_64 --> Running transaction check ---> Package zlib.x86_64 0:1.2.7-17.el7 will be updated ---> Package zlib.x86_64 0:1.2.7-18.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved =========================================================================================================== Package Arch Version Repository Size =========================================================================================================== Installing: zlib-devel x86_64 1.2.7-18.el7 base 50 k Updating for dependencies: zlib x86_64 1.2.7-18.el7 base 90 k Transaction Summary =========================================================================================================== Install 1 Package Upgrade ( 1 Dependent package) Total download size: 140 k Is this ok [y/d/N]: y Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. (1/2): zlib-devel-1.2.7-18.el7.x86_64.rpm | 50 kB 00:00:00 /2): zlib-1.2.7-18.el7.x86_64.rpm | 90 kB 00:00:00 ----------------------------------------------------------------------------------------------------------- Total 509 kB/s | 140 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : zlib-1.2.7-18.el7.x86_64 1/3 Installing : zlib-devel-1.2.7-18.el7.x86_64 2/3 Cleanup : zlib-1.2.7-17.el7.x86_64 3/3 Verifying : zlib-devel-1.2.7-18.el7.x86_64 1/3 Verifying : zlib-1.2.7-18.el7.x86_64 2/3 Verifying : zlib-1.2.7-17.el7.x86_64 3/3 Installed: zlib-devel.x86_64 0:1.2.7-18.el7 Dependency Updated: zlib.x86_64 0:1.2.7-18.el7 Complete! [root@node01 ruby-2.5.5]#
集成zlib庫到Ruby環境
[root@node01 ruby-2.5.5]# cd ext/zlib/ [root@node01 zlib]# ruby extconf.rb checking for deflateReset() in -lz... yes checking for zlib.h... yes checking for crc32_combine() in zlib.h... yes checking for adler32_combine() in zlib.h... yes checking for z_crc_t in zlib.h... yes creating Makefile [root@node01 zlib]# make make: *** No rule to make target `/include/ruby.h', needed by `zlib.o'. Stop. [root@node01 zlib]#
提示：遇到這種問題，咱們須要打開上面生成的Makefile 把zlib.o: $(top_srcdir)/include/ruby.h替換爲zlib.o: ../../include/ruby.h便可
提示：修改好Makefile，再次make
"Makefile" 282L, 8468C written [root@node01 zlib]# make compiling zlib.c linking shared-object zlib.so [root@node01 zlib]# make install /usr/bin/install -c -m 0755 zlib.so /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux [root@node01 zlib]#
再次使用gem安裝redis包
[root@node01 zlib]# gem install redis
ERROR: While executing gem ... (Gem::Exception)
Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources
[root@node01 zlib]#
提示：它提示咱們缺乏openssl庫，處理方法安裝openssl-devel包，而後把openssl庫集成到ruby環境中；
安裝openssl-devel包
[root@node01 zlib]# yum install openssl-devel Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Resolving Dependencies --> Running transaction check ---> Package openssl-devel.x86_64 1:1.0.2k-19.el7 will be installed --> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-19.el7 for package: 1:openssl-devel-1.0.2k-19.el7.x86_64 --> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-19.el7.x86_64 --> Running transaction check ---> Package krb5-devel.x86_64 0:1.15.1-46.el7 will be installed ……省略部份內容…… Installed: openssl-devel.x86_64 1:1.0.2k-19.el7 Dependency Installed: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 krb5-devel.x86_64 0:1.15.1-46.el7 libcom_err-devel.x86_64 0:1.42.9-17.el7 libkadm5.x86_64 0:1.15.1-46.el7 libselinux-devel.x86_64 0:2.5-15.el7 libsepol-devel.x86_64 0:2.5-10.el7 libverto-devel.x86_64 0:0.2.5-4.el7 pcre-devel.x86_64 0:8.32-17.el7 Dependency Updated: e2fsprogs.x86_64 0:1.42.9-17.el7 e2fsprogs-libs.x86_64 0:1.42.9-17.el7 krb5-libs.x86_64 0:1.15.1-46.el7 libcom_err.x86_64 0:1.42.9-17.el7 libselinux.x86_64 0:2.5-15.el7 libselinux-python.x86_64 0:2.5-15.el7 libselinux-utils.x86_64 0:2.5-15.el7 libsepol.x86_64 0:2.5-10.el7 libss.x86_64 0:1.42.9-17.el7 openssl.x86_64 1:1.0.2k-19.el7 openssl-libs.x86_64 1:1.0.2k-19.el7 Complete! [root@node01 zlib]#
集成openssl到ruby環境中
[root@node01 zlib]# cd .. [root@node01 ext]# ls bigdecimal date fcntl json psych ripper Setup.nt -test- cgi dbm fiber nkf pty rubyvm socket win32 configure-ext.mk digest fiddle objspace racc sdbm stringio win32ole continuation etc gdbm openssl rbconfig Setup strscan zlib coverage extmk.rb io pathname readline Setup.atheos syslog [root@node01 ext]# cd openssl/ [root@node01 openssl]# ls depend ossl_asn1.c ossl_digest.c ossl_ocsp.c ossl_pkey_rsa.c ossl_x509ext.c deprecation.rb ossl_asn1.h ossl_digest.h ossl_ocsp.h ossl_rand.c ossl_x509.h extconf.rb ossl_bio.c ossl_engine.c ossl_pkcs12.c ossl_rand.h ossl_x509name.c exts.mk ossl_bio.h ossl_engine.h ossl_pkcs12.h ossl_ssl.c ossl_x509req.c History.md ossl_bn.c ossl.h ossl_pkcs7.c ossl_ssl.h ossl_x509revoked.c lib ossl_bn.h ossl_hmac.c ossl_pkcs7.h ossl_ssl_session.c ossl_x509store.c Makefile ossl.c ossl_hmac.h ossl_pkey.c ossl_version.h ruby_missing.h mkmf.log ossl_cipher.c ossl_kdf.c ossl_pkey_dh.c ossl_x509attr.c openssl.gemspec ossl_cipher.h ossl_kdf.h ossl_pkey_dsa.c ossl_x509.c openssl_missing.c ossl_config.c ossl_ns_spki.c ossl_pkey_ec.c ossl_x509cert.c openssl_missing.h ossl_config.h ossl_ns_spki.h ossl_pkey.h ossl_x509crl.c [root@node01 openssl]# ruby extconf.rb checking for t_open() in -lnsl... no checking for socket() in -lsocket... no checking for openssl/ssl.h... yes checking for OpenSSL version is 1.0.1 or later... yes checking for RAND_egd()... yes ……省略部份內容…… checking for X509_get0_notBefore()... no checking for SSL_SESSION_get_protocol_version()... no checking for EVP_PBE_scrypt()... no creating extconf.h creating Makefile [root@node01 openssl]# make compiling openssl_missing.c make: *** No rule to make target `/include/ruby.h', needed by `ossl.o'. Stop. [root@node01 openssl]#
提示：這個錯誤和剛纔繼承zlib庫到ruby環境中相似，處理方式同上面的處理方式同樣修改Makefile文件，在makefile定義變量的位置增長top_srcdir = ../..
再次make
Makefile" 1458L, 49182C written [root@node01 openssl]# make compiling ossl.c compiling ossl_asn1.c compiling ossl_bio.c compiling ossl_bn.c compiling ossl_cipher.c compiling ossl_config.c compiling ossl_digest.c compiling ossl_engine.c compiling ossl_hmac.c compiling ossl_kdf.c compiling ossl_ns_spki.c compiling ossl_ocsp.c compiling ossl_pkcs12.c compiling ossl_pkcs7.c compiling ossl_pkey.c compiling ossl_pkey_dh.c compiling ossl_pkey_dsa.c compiling ossl_pkey_ec.c compiling ossl_pkey_rsa.c compiling ossl_rand.c compiling ossl_ssl.c compiling ossl_ssl_session.c compiling ossl_x509.c compiling ossl_x509attr.c compiling ossl_x509cert.c compiling ossl_x509crl.c compiling ossl_x509ext.c compiling ossl_x509name.c compiling ossl_x509req.c compiling ossl_x509revoked.c compiling ossl_x509store.c linking shared-object openssl.so [root@node01 openssl]# make install /usr/bin/install -c -m 0755 openssl.so /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux installing default openssl libraries [root@node01 openssl]#
再次用gem安裝redis包
[root@node01 openssl]# gem install redis Fetching: redis-4.2.1.gem (100%) Successfully installed redis-4.2.1 Parsing documentation for redis-4.2.1 Installing ri documentation for redis-4.2.1 Done installing documentation for redis after 0 seconds 1 gem installed [root@node01 openssl]#
提示：到此ruby環境就準備好了，redis包也安裝好了，接下來咱們把redis-trib.rb鏈接到/usr/bin/下，而後執行redis-trib.rb，若是可以正常執行說明ruby環境沒有問題，若是不能則還須要調整
軟鏈接redis-trib.rb到/usr/bin/
[root@node01 openssl]# find /usr/local/src/redis-4.0.9/ -name "redis-trib.rb"
/usr/local/src/redis-4.0.9/src/redis-trib.rb
[root@node01 openssl]# ln -s /usr/local/src/redis-4.0.9/src/redis-trib.rb /usr/bin/
[root@node01 openssl]# redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...>
create host1:port1 ... hostN:portN
--replicas <arg>
check host:port
info host:port
fix host:port
--timeout <arg>
reshard host:port
--from <arg>
--to <arg>
--slots <arg>
--yes
--timeout <arg>
--pipeline <arg>
rebalance host:port
--weight <arg>
--auto-weights
--use-empty-masters
--timeout <arg>
--simulate
--pipeline <arg>
--threshold <arg>
add-node new_host:new_port existing_host:existing_port
--slave
--master-id <arg>
del-node host:port node_id
set-timeout host:port milliseconds
call host:port command arg arg .. arg
import host:port
--from <arg>
--copy
--replace
help (show this help)
For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
[root@node01 openssl]#
提示：能夠看到redis-trib.rb能夠正常打印出幫助信息，說明當前ruby環境可以支持redis-trib.rb運行；
添加密碼到剛纔下載的redis包的庫文件中
提示：這裏須要把對應密碼字符串用引號把它引發來；
建立集羣
[root@node01 ~]# redis-trib.rb create --replicas 1 192.168.0.41:6379 192.168.0.42:6379 192.168.0.43:6379 192.168.0.41:6380 192.168.0.42:6380 192.168.0.43:6380 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 192.168.0.41:6379 192.168.0.42:6379 192.168.0.43:6379 Adding replica 192.168.0.42:6380 to 192.168.0.41:6379 Adding replica 192.168.0.43:6380 to 192.168.0.42:6379 Adding replica 192.168.0.41:6380 to 192.168.0.43:6379 M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379 slots:0-5460 (5461 slots) master M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379 slots:5461-10922 (5462 slots) master M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379 slots:10923-16383 (5461 slots) master S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380 replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380 replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380 replicates e99b0b450e78719d63520cb6efc068d5e8d4d081 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join...... >>> Performing Cluster Check (using node 192.168.0.41:6379) M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380 slots: (0 slots) slave replicates e99b0b450e78719d63520cb6efc068d5e8d4d081 M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380 slots: (0 slots) slave replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380 slots: (0 slots) slave replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. [root@node01 ~]#
提示：從上面的信息能夠了解到，咱們給指定了6個redis實例的地址和端口，它給咱們建立了3個master3個slave，而且在3個master上平均分配了16384個槽位；若是能看到後面的ok all 16384 slots covered,說明集羣建立成功；--replicates用來指定集羣中每一個master的副本數量，1表示每一個master有一個副本；
查看集羣狀態信息
[root@node01 ~]# redis-trib.rb info 192.168.0.41:6379 192.168.0.41:6379 (8c785e6e...) -> 0 keys | 5461 slots | 1 slaves. 192.168.0.42:6379 (e99b0b45...) -> 0 keys | 5462 slots | 1 slaves. 192.168.0.43:6379 (a7ace08c...) -> 0 keys | 5461 slots | 1 slaves. [OK] 0 keys in 3 masters. 0.00 keys per slot on average. [root@node01 ~]# redis-trib.rb check 192.168.0.41:6379 >>> Performing Cluster Check (using node 192.168.0.41:6379) M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380 slots: (0 slots) slave replicates e99b0b450e78719d63520cb6efc068d5e8d4d081 M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380 slots: (0 slots) slave replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380 slots: (0 slots) slave replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. [root@node01 ~]#
提示：從上面的信息能夠看到192.168.0.41/42/43的6379端口都是master，而且都有一個slave節點；
確認master狀態
[root@node01 ~]# redis-cli -h 192.168.0.41 192.168.0.41:6379> AUTH admin OK 192.168.0.41:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:606 cluster_stats_messages_pong_sent:586 cluster_stats_messages_sent:1192 cluster_stats_messages_ping_received:581 cluster_stats_messages_pong_received:606 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:1192 192.168.0.41:6379> info replication # Replication role:master connected_slaves:1 slave0:ip=192.168.0.42,port=6380,state=online,offset=854,lag=0 master_replid:303943ae1e9f54e3525abda2fd1efff905bf08e3 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:854 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:854 192.168.0.41:6379> quit [root@node01 ~]# redis-cli -h 192.168.0.42 192.168.0.42:6379> auth admin OK 192.168.0.42:6379> info replication # Replication role:master connected_slaves:1 slave0:ip=192.168.0.43,port=6380,state=online,offset=896,lag=0 master_replid:f4e1b75b8f30be622868814fc4618a4584f3aa32 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:896 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:896 192.168.0.42:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:2 cluster_stats_messages_ping_sent:662 cluster_stats_messages_pong_sent:645 cluster_stats_messages_meet_sent:5 cluster_stats_messages_sent:1312 cluster_stats_messages_ping_received:644 cluster_stats_messages_pong_received:667 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:1312 192.168.0.42:6379> quit [root@node01 ~]# redis-cli -h 192.168.0.43 192.168.0.43:6379> AUTH admin OK 192.168.0.43:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:3 cluster_stats_messages_ping_sent:683 cluster_stats_messages_pong_sent:787 cluster_stats_messages_meet_sent:4 cluster_stats_messages_sent:1474 cluster_stats_messages_ping_received:786 cluster_stats_messages_pong_received:687 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:1474 192.168.0.43:6379> info replication # Replication role:master connected_slaves:1 slave0:ip=192.168.0.41,port=6380,state=online,offset=952,lag=1 master_replid:7cbc7d52452717cd49a0e543f6efdccafe5fe1bd master_replid2:0000000000000000000000000000000000000000 master_repl_offset:952 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:952 192.168.0.43:6379> quit [root@node01 ~]#
提示：能夠看到集羣狀態都是正常ok的，而且41的6379對應42的6380，42的6379對應43的6380，43的6379對應41的6380，恰好master和slave都是錯開的；
查看集羣各node對應關係
提示：上面的node對應是依據id號來的。到此redis集羣就正常的跑起來了；
驗證：在master上任意寫入一個數據看看會發生什麼？
提示：在master01上寫入k1，它提示咱們到192.168.0.43：6379上去寫；這是由於咱們寫入k1計算的槽位是12706，這個槽位是在master03上，因此在master01上就不能正常的寫入；這同時也告訴咱們，要使用redis集羣，客戶端還必須可以智能去理解redis協議，經過集羣返回的信息，把對應請求調度到對應master上執行；
驗證讀請求
提示：從上面的信息能夠肯定一點，這三個主節點他們互相數據是徹底隔離的，也就說master01不知道master02上到底有沒有對應的數據，它只知道應該去master02上去查看才知道；這就是咱們以前說的，每一個節點擁有整個集羣的所有狀態信息（元數據），而每一個節點只擁有部分數據在本節點；
驗證：把任意一個master節點宕機，看看對應slave是否可以被提高爲master，接管master呢？
提示：能夠看到當master01宕機之後，對應slave就被提高爲master；
查看master01的配置文件是否有改動？
[root@node01 ~]# cat /usr/local/redis/redis-cluster_6379.conf 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380@16380 master - 0 1596648345000 8 connected 5461-10922 dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380@16380 slave 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 0 1596648345510 9 connected 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380@16380 slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596648343893 4 connected e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379@16379 slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596648344903 8 connected 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379@16379 myself,master - 0 1596648345000 9 connected 0-5460 a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379@16379 master - 0 1596648345000 3 connected 10923-16383 vars currentEpoch 9 lastVoteEpoch 0 [root@node01 ~]# ps -ef |grep redis root 15980 1 0 Aug05 ? 00:00:17 redis-server 0.0.0.0:6380 [cluster] root 30867 1 0 01:19 ? 00:00:02 redis-server 0.0.0.0:6379 [cluster] root 30949 14690 0 01:30 pts/0 00:00:00 grep --color=auto redis [root@node01 ~]# redis-cli -a admin 127.0.0.1:6379> info replication # Replication role:master connected_slaves:1 slave0:ip=192.168.0.42,port=6380,state=online,offset=4476,lag=0 master_replid:ed690ba2766e6e334ffc298fc6aba8be80465aa8 master_replid2:776b5f996d78171e7c5ab5419d65c6c14a253bda master_repl_offset:4476 second_repl_offset:4071 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:3595 repl_backlog_histlen:882 127.0.0.1:6379> quit [root@node01 ~]# kill -9 30867 [root@node01 ~]# redis-cli -h 192.168.0.42 -p 6380 192.168.0.42:6380> AUTH admin OK 192.168.0.42:6380> info replication # Replication role:master connected_slaves:0 master_replid:d46c0a3761b61352b1fc75986ab3343e04a89b83 master_replid2:ed690ba2766e6e334ffc298fc6aba8be80465aa8 master_repl_offset:4518 second_repl_offset:4519 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:4071 repl_backlog_histlen:448 192.168.0.42:6380> quit [root@node01 ~]# cat /usr/local/redis/redis-cluster_6379.conf 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380@16380 master - 0 1596648345000 8 connected 5461-10922 dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380@16380 slave 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 0 1596648345510 9 connected 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380@16380 slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596648343893 4 connected e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379@16379 slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596648344903 8 connected 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379@16379 myself,master - 0 1596648345000 9 connected 0-5460 a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379@16379 master - 0 1596648345000 3 connected 10923-16383 vars currentEpoch 9 lastVoteEpoch 0 [root@node01 ~]# grep slaveof /usr/local/redis/6379/etc/redis.conf # Master-Slave replication. Use slaveof to make a Redis instance a copy of # slaveof <masterip> <masterport> [root@node01 ~]#
提示：從上面的信息能夠看到，在master01沒有宕機前和宕機之後slave被提高爲master後，對應的配置信息都沒有發生變化；我上面爲了驗證配置文件是否發生變化，先是把master01上線，而後把對應slave下線在上線，還原master01爲master角色；
從新把master上線，看看它的配置文件有什麼變化？角色還會是master嗎？
[root@node01 ~]# redis-server /usr/local/redis/6379/etc/redis.conf [root@node01 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 511 *:6379 *:* LISTEN 0 511 *:6380 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 511 *:16379 *:* LISTEN 0 511 *:16380 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node01 ~]# cat /usr/local/redis/redis-cluster_6379.conf 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380@16380 master - 1596649311329 1596649311326 8 connected 5461-10922 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379@16379 myself,slave dbfff4c49a94c0ee55d14401ccc9245af3655427 0 1596649311326 9 connected e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379@16379 slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596649311331 8 connected a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379@16379 master - 0 1596649311332 3 connected 10923-16383 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380@16380 slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596649311331 4 connected dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380@16380 master - 0 1596649311332 10 connected 0-5460 vars currentEpoch 10 lastVoteEpoch 0 [root@node01 ~]# redis-cli 127.0.0.1:6379> AUTH admin OK 127.0.0.1:6379> info replication # Replication role:slave master_host:192.168.0.42 master_port:6380 master_link_status:up master_last_io_seconds_ago:5 master_sync_in_progress:0 slave_repl_offset:4686 slave_priority:100 slave_read_only:1 connected_slaves:0 master_replid:d46c0a3761b61352b1fc75986ab3343e04a89b83 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:4686 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:4519 repl_backlog_histlen:168 127.0.0.1:6379>
提示：能夠看到，從新把master01上線之後，它自動降級爲slave了，而且配置文件也從master需改爲slave了；這應該是redis cluster在檢測到192.168.0.41：6379上線後，觸發把內存中的配置同步到對應文件，使得對應配置文件也被修改了；到此一個redis cluster 就搭建測試完成了；
相關文章
- 1. redis之Cluster
- 2. redis之cluster
- 3. Redis學習之Redis Cluster（三）
- 4. Redis 集羣之 Redis-Cluster
- 5. redis之redis-cluster配置
- 6. redis集羣之REDIS CLUSTER
- 7. Redis學習之七 —— Redis Cluster
- 8. redis（9）、redis集羣之redis Cluster使用
- 9. Redis --- Redis Cluster
- 10. redis & redis-cluster
- 更多相關文章...
- • 探索Redis事務回滾 - Redis教程
- • 啓動MySQL服務 - MySQL教程
- • Spring Cloud 微服務實戰(三) - 服務註冊與發現
- • 互聯網組織的未來：剖析GitHub員工的任性之源
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
