redis cluster單機僞分佈式搭建--- 3主3從3哨兵集羣

時間 2020-10-04

標籤 node redis 數據庫 ruby bash 服務器 app 框架 curl 分佈式欄目 Redis 简体版

原文原文鏈接

redis cluster單機僞分佈式搭建--- 3主3從3哨兵集羣

最近公司引進微服務框架,以前的一臺redis的預存60G已經沒法知足如今的260G業務須要,通過一番考慮搭建了這套集羣 . 爲了方便我就用一臺服務器演示,生產環境中不建議這麼作(沒啥用),只爲記錄一下過程,至於精細化的配置須要在生產中自行研究

演示環境

[root@localhost ~]# cat /etc/redhat-release   
CentOS Linux release 7.4.1708 (Core)

redis服務部署過程

我下載的是redis-4.0.14.tar.gz版本的包
解壓安裝
tar -zxvf redis-4.0.14.tar.gz
make install
啓動服務
./redis-server

驗證,啓動客戶端測試

./redis-cli  
127.0.0.1:6379> set ab b  
OK  
127.0.0.1:6379> get ab  
"b"  
127.0.0.1:6379> keys *  
"ab"  
"foo"

關閉redis服務
./redis-cli shutdown
設置redis後臺啓動,在/usr/local/bin目錄下
cp /usr/local/redis/redis-4.0.8/redis.conf ./
修改redis.conf這個配置文件，找到 daemonize no 這一行，將 no 改爲 yes 保存
而後啓動redis時攜帶配置文件,./redis-server redis.conf
在/usr/local/bin目錄下修改好redis.conf配置（主要是設置後臺啓動、bind、密碼、保護模式等）後
mkdir /etc/redis
cp /usr/local/bin/redis.conf /etc/redis/6379.conf
將redis自帶的啓動腳本複製到/etc/init.d目錄下，命名爲redisd（一般都以d結尾表示是後臺自啓動服務）
cp /usr/local/redis/redis-4.0.8/utils/redis_init_script /etc/init.d/redisd
設置爲開機自啓動 chkconfig redisd on
若是報錯redisd 不支持 chkconfig,在/etc/init.d/redisd中增長以下片斷，便可再設置開機自啓動
```
#!/bin/sh  
#chkconfig:   2345 90 10  
#description:  Redis is a persistent key-value database
```
chkconfig redisd on 成功以後便可以使用啓動腳原本啓動redis
僞分佈式集羣的搭建

搭建一個最小的集羣須要6臺機器（由節點投票機制決定至少須要3臺主節點，另外爲了高可用每一個主節點至少要備一個從節點，因此總共須要6個節點。這裏6個節點所有設在一臺機上，經過端口號來分別。實現僞分佈式。）
在/usr/local/目錄下建立文件夾redisCluster，並在redisCluster目錄下建立6個文件夾
mkdir /usr/local/redisCluster
mkdir /usr/local/redisCluster/redis0{1,2,3,4,5,6}

複製單機單實例下可正常啓動redis狀況下的 /usr/local/bin 目錄下的全部文件分別到上面建立的6個文件夾中

cp /usr/local/bin/* /usr/local/redisCluster/redis01/  
cp /usr/local/bin/* /usr/local/redisCluster/redis02/  
cp /usr/local/bin/* /usr/local/redisCluster/redis03/  
cp /usr/local/bin/* /usr/local/redisCluster/redis04/  
cp /usr/local/bin/* /usr/local/redisCluster/redis05/  
cp /usr/local/bin/* /usr/local/redisCluster/redis06/

再對每一個實例裏面的redis.conf進行修改（這裏修改其中一份，而後其餘5份進行復制以後修改下端口號便可)

vi /usr/local/redisCluster/redis01/redis.conf  
第1處： port 6379                  ————> 值 改爲 7001  
第2處： daemonize no               ————> 值 改爲 yes  
第3處： pidfile /var/run/redis_6379.pid        ————> 值 改爲 ./redis_7001.pid  
第4處： logfile ""                     ————> 值 改爲 "./logs/redis_7001.log"  
第5處： bind 127.0.0.1                 ————> 值 改爲 本機IP（如我這裏是192.168.25.129）  
第6處： protected-mode yes                 ————> 值 改爲 no  
第7處： # cluster-enabled yes          ————> 去掉註釋  
第8處： # cluster-config-file nodes-6379.conf    ————> 去掉註釋,並把數字改爲對應端口號，如這裏是7001  
第9處： # cluster-node-timeout 15000               ————> 去掉註釋   
第10處：  appendonly no                ————> 值 改爲 yes

將上面編輯號保存的/usr/local/redisCluster/redis01/目錄下的redis.conf文件重命名爲7001.conf，而後複製5份分別放到redis0二、redis0三、redis0四、redis0五、redis06文件夾下面（可刪除掉原文件夾中的redis.conf文件），而後再對每一個文件夾下的.conf文件重命名爲對應端口號.conf，接着編輯裏面的內容，將以前修改成7001的所有修改爲對應端口號保存便可
將ruby腳本拷貝到redisCluster目錄下
cp /usr/local/redis/redis-4.0.8/src/redis-trib.rb /usr/local/redisCluster/

建立redis集羣啓動、中止的腳本以及ruby建立集羣的腳本並受權

touch /usr/local/redisCluster/startCluster.sh  
touch /usr/local/redisCluster/shutdownCluster.sh  
touch /usr/local/redisCluster/createCluster.sh  
chmod 777 /usr/local/redisCluster/*.sh

startcluster.sh 內容以下: node

/usr/local/rediscluster/redis01/redis-server /usr/local/rediscluster/redis01/7001.conf  
/usr/local/rediscluster/redis02/redis-server /usr/local/rediscluster/redis02/7002.conf  
/usr/local/rediscluster/redis03/redis-server /usr/local/rediscluster/redis03/7003.conf  
/usr/local/rediscluster/redis04/redis-server /usr/local/rediscluster/redis04/7004.conf  
/usr/local/rediscluster/redis05/redis-server /usr/local/rediscluster/redis05/7005.conf  
/usr/local/rediscluster/redis06/redis-server /usr/local/rediscluster/redis06/7006.conf

shutdowncluster.sh 內容以下: redis

/usr/local/rediscluster/redis01/redis-cli -c -h 10.11.1.103 -p 7001 shutdown  
/usr/local/rediscluster/redis02/redis-cli -c -h 10.11.1.103 -p 7002 shutdown  
/usr/local/rediscluster/redis03/redis-cli -c -h 10.11.1.103 -p 7003 shutdown  
/usr/local/rediscluster/redis04/redis-cli -c -h 10.11.1.103 -p 7004 shutdown  
/usr/local/rediscluster/redis05/redis-cli -c -h 10.11.1.103 -p 7005 shutdown  
/usr/local/rediscluster/redis06/redis-cli -c -h 10.11.1.103 -p 7006 shutdown

createcluster.sh 內容以下:
ruby redis-trib.rb create --replicas 1 10.11.1.103:7001 10.11.1.103:7002 10.11.1.103:7003 10.11.1.103:7004 10.11.1.103:7005 10.11.1.103:7006 數據庫

用剛剛建立的腳本啓動6個實例，再用ruby腳本建立集羣

cd /usr/local/redisCluster/  
./startCluster.sh  
./createcluster.sh(輸入yes)

驗證,查看進程

[root@localhost rediscluster]# ps -ef |  grep redis|grep -v grep  
root     23716     1  0 Aug08 ?        00:10:55 /usr/local/rediscluster/redis01/redis-server *:7001 [cluster]  
root     23721     1  0 Aug08 ?        00:10:53 /usr/local/rediscluster/redis02/redis-server *:7002 [cluster]  
root     23723     1  0 Aug08 ?        00:10:51 /usr/local/rediscluster/redis03/redis-server *:7003 [cluster]  
root     23731     1  0 Aug08 ?        00:13:37 /usr/local/rediscluster/redis04/redis-server *:7004 [cluster]  
root     23736     1  0 Aug08 ?        00:10:55 /usr/local/rediscluster/redis05/redis-server *:7005 [cluster]  
root     23741     1  0 Aug08 ?        00:08:59 /usr/local/rediscluster/redis06/redis-server *:7006 [cluster]

集羣就搭建成功了。登陸任意一個節點客戶端驗證(-c 表示以集羣方式登陸)，也能夠用redis客戶端工具RedisDesktopManager來可視化地管理redis數據庫。

cd /usr/local/redisCluster/redis01/  
./redis-cli -h 10.11.1.103 -p 7001 -c  
10.11.1.103:7001> set aaaa b  
OK  
10.11.1.103:7002> get aaaa  
-> Redirected to slot [1953] located at 10.11.1.103:7001  
"b"  
10.11.1.103:7001> cluster info  
cluster_state:ok  
cluster_slots_assigned:16384  
cluster_slots_ok:16384  
cluster_slots_pfail:0  
cluster_slots_fail:0  
cluster_known_nodes:6  
cluster_size:3  
cluster_current_epoch:6  
cluster_my_epoch:1  
cluster_stats_messages_ping_sent:367555  
cluster_stats_messages_pong_sent:342015  
cluster_stats_messages_publish_sent:903835  
cluster_stats_messages_sent:1613405  
cluster_stats_messages_ping_received:342015  
cluster_stats_messages_pong_received:367550  
cluster_stats_messages_publish_received:1084495  
cluster_stats_messages_received:1794060  
10.11.1.103:7001>

在redis01 02 03 裏配置哨兵,這裏只配置了redis01的剩餘的按照這個自行配置便可,須要注意的是監視的名稱不可重複(已標出)
sentinel.conf 配置以下:

protected-mode no  
port 27001   
sentinel deny-scripts-reconfig yes  
sentinel monitor **mymaster1** 10.11.1.103 7001 2  
sentinel down-after-milliseconds **mymaster1** 10000  
logfile "/var/log/sentinel1.log"  
dir "/usr/local/rediscluster/redis01"  
sentinel failover-timeout **mymaster1** 15000  
sentinel config-epoch **mymaster1** 0  
sentinel leader-epoch **mymaster1** 0  
sentinel known-slave **mymaster1 10.11.1.103 7004**  
sentinel current-epoch 0

在rediscluster目錄下面建立哨兵啓動和關閉的腳本
startsentinel.sh 的內容以下

/usr/local/rediscluster/redis01/redis-sentinel /usr/local/rediscluster/redis01/sentinel.conf &  
/usr/local/rediscluster/redis02/redis-sentinel /usr/local/rediscluster/redis02/sentinel.conf &  
/usr/local/rediscluster/redis03/redis-sentinel /usr/local/rediscluster/redis03/sentinel.conf &

stopsentinel.sh 腳本的內容以下: ruby

kill -9 `ps -ef |  grep sentinel| grep -v grep|awk '{print $2}'`

再次查看進程

[root@localhost rediscluster]# ps -ef |  grep redis|grep -v grep  
root     23716     1  0 Aug08 ?        00:10:55 /usr/local/rediscluster/redis01/redis-server *:7001 [cluster]  
root     23721     1  0 Aug08 ?        00:10:53 /usr/local/rediscluster/redis02/redis-server *:7002 [cluster]  
root     23723     1  0 Aug08 ?        00:10:51 /usr/local/rediscluster/redis03/redis-server *:7003 [cluster]  
root     23731     1  0 Aug08 ?        00:13:37 /usr/local/rediscluster/redis04/redis-server *:7004 [cluster]  
root     23736     1  0 Aug08 ?        00:10:55 /usr/local/rediscluster/redis05/redis-server *:7005 [cluster]  
root     23741     1  0 Aug08 ?        00:08:59 /usr/local/rediscluster/redis06/redis-server *:7006 [cluster]  
root     23924     1  0 Aug08 ?        00:09:55 /usr/local/rediscluster/redis01/redis-sentinel *:27001 [sentinel]  
root     23925     1  0 Aug08 ?        00:09:38 /usr/local/rediscluster/redis02/redis-sentinel *:27002 [sentinel]  
root     23926     1  0 Aug08 ?        00:10:22 /usr/local/rediscluster/redis03/redis-sentinel *:27003 [sentinel]

這樣就完成搭建,如下是須要聯網安裝的軟件

yum -y install gcc                    ## make install 命令須要此環境  
yum -y intall ruby                    ## 安裝ruby（此時安裝的可能使低版本的）  
curl -L get.rvm.io | bash -s stable   ## 安裝rvm  
source /usr/local/rvm/scripts/rvm     ## 使安裝的rvm當即生效  
rvm install 2.4.1                 ## 經過rvm來安裝ruby高級版本  
gem install redis                     ## 經過ruby的工具包gem來安裝redis接口