一:phpredisadmin工具:相似於mysqladmin管理mysql同樣php
安裝環境:html
[root@node5 ~]# yum install httpd php php-redis -y
下載管理包:前端
[root@node5 html]# git clone https://github.com/erikdubbelboer/phpRedisAdmin.git
更更名稱並解決依賴:java
[root@node5 html]# mv phpRedisAdmin admin [root@node5 html]# cd admin [root@node5 admin]# git clone https://github.com/nrk/predis.git vendor #解決依賴包
啓動http,啓動前確認80端口沒有被佔用,若是被佔用能夠修改監聽的默認端口爲80覺得的其餘端口:node
[root@node5 html]# /etc/init.d/httpd restart Stopping httpd: [FAILED] Starting httpd: [ OK ]
[root@node5 html]# chkconfig httpd on
訪問http://ServerIP/admin:python
報錯,沒法打開,查看apache錯誤日誌以下:mysql
[Mon Mar 21 21:24:47 2016] [error] [client 192.168.10.1] PHP Fatal error: require(): Failed opening required '/var/www/html/admin/includes/../vendor/autoload.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/www/html/admin/includes/common.inc.php on line 2
解決過程:linux
[root@node5 admin]# cd includes/ [root@node5 includes]# ls config.sample.inc.php config.inc.php [root@node5 includes]# vim config.inc.php $config = array( 'servers' => array( array( 'name' => 'local server', // Optional name. 'host' => '192.168.10.205', #配置爲本身的主機IP 'port' => 6379, 'filter' => '*', // Optional Redis authentication. //'auth' => 'redispasswordhere' // Warning: The password is sent in plain-text to the Redis server. ),
保存退出後從新啓動httpd:
[root@node5 includes]# /etc/init.d/httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]git
再次測試仍是不能訪問,報錯以下:github
[Mon Mar 21 21:55:27 2016] [error] [client 192.168.10.1] PHP Fatal error: Call to undefined function mb_internal_encoding() in /var/www/html/admin/includes/common.inc.php on line 59
解決辦法:
[root@node5 admin]# vim /var/www/html/admin/includes/common.inc.php mb_internal_encoding('utf-8'); 將此行改成註釋,以下: #mb_internal_encoding('utf-8');
訪問:
配置管理多個reids Server:
[root@node5 includes]# pwd /var/www/html/admin/includes [root@node5 includes]# vim config.inc.php $config = array( 'servers' => array( array( 'name' => '182.168.10.205', // Optional name. 'host' => '192.168.10.205', 'port' => 6379, 'filter' => '*', // Optional Redis authentication. //'auth' => 'redispasswordhere' // Warning: The password is sent in plain-text to the Redis server. ), array( #一個主機一個配置,名稱自定義,IP和端口是Redis的IP和端口 'name' => '7001', // Optional name.name支持中文 'host' => '192.168.10.205', 'port' => 7001, ), array( 'name' => '7002', // Optional name. 'host' => '192.168.10.205', 'port' => 7002, ), array( 'name' => '7003', // Optional name. 'host' => '192.168.10.205', 'port' => 7003, ),
更改完成之後無需重啓apache,若是瀏覽器已經打開管理地址,刷新一下便可:
二:rdbtools管理工具:
使用pip命令安裝,若是沒有pip命令則先安裝pip:
[root@node5 ~]# yum install python-pip
使用pip安裝rdbtools: pip install rdbtools
[root@node5 ~]# pip install rdbtools /usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning You are using pip version 7.1.0, however version 8.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting rdbtools /usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Downloading rdbtools-0.1.6.tar.gz Installing collected packages: rdbtools Running setup.py install for rdbtools Successfully installed rdbtools-0.1.6
導出rdb中的key信息:
[root@node5 7001]# rdb -c memory dump_7001.rdb > memory.csv #而後將memory.csv文件下載到本地,便可用excel打開分析了,格式以下圖:
三:經過saltstack管理redis:
模塊名稱:redismod
官方文檔地址:https://docs.saltstack.com/en/latest/ref/modules/all/index.html
四:經過codis完成redis管理:
1.codis簡介:
Codis 是一個分佈式 Redis 解決方案, 對於上層的應用來講, 鏈接到 Codis Proxy 和鏈接原生的 Redis Server 沒有明顯的區別 (不支持的命令列表), 上層應用能夠像使用單機的 Redis 同樣使用, Codis 底層會處理請求的轉發, 不停機的數據遷移等工做, 全部後邊的一切事情, 對於前面的客戶端來講是透明的, 能夠簡單的認爲後邊鏈接的是一個內存無限大的 Redis 服務.其架構以下:
Codis 由四部分組成:
codis-proxy 是客戶端鏈接的 Redis 代理服務, codis-proxy 自己實現了 Redis 協議, 表現得和一個原生的 Redis 沒什麼區別 (就像 Twemproxy), 對於一個業務來講, 能夠部署多個 codis-proxy, codis-proxy 自己是無狀態的.
codis-config 是 Codis 的管理工具, 支持包括, 添加/刪除 Redis 節點, 添加/刪除 Proxy 節點, 發起數據遷移等操做. codis-config 自己還自帶了一個 http server, 會啓動一個 dashboard, 用戶能夠直接在瀏覽器上觀察 Codis 集羣的運行狀態.
2.準備基礎環境:
安裝go語言並設置環境變量:
[root@node5 opt]# yum install golang -y #codis是基於go語言編寫的,所以要安裝go語言環境 [root@node5 opt]# mkdir /opt/gopath #保存go環境的路徑 [root@node5 opt]# vim /etc/profile export GOPATH=/opt/gopath
[root@node5 opt]# source /etc/profile
安裝配置zookeeper,須要有java環境:
[root@node5 ~]# wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz [root@node5 ~]# mv zookeeper-3.4.6 /usr/local/zookeeper #移動到目錄 [root@node5 ~]# cd /usr/local/zookeeper/conf/ #進入到目錄 [root@node5 conf]# cp zoo_sample.cfg /opt/zoo.cfg #複製配置文件 [root@node5 conf]# cd /opt/ [root@node5 opt]# mkdir zk1 zk2 zk3 #準備zookeeper 服務ID,每一個服務器的ID是不一樣的 [root@node5 opt]# echo 1 > zk1/myid [root@node5 opt]# echo 2 > zk2/myid [root@node5 opt]# echo 3 > zk3/myid [root@node5 opt]# vim zoo.cfg #編輯配置文件 dataDir=/opt/zk1 #保存數據的命令 clientPort=2181 #客戶端鏈接的端口 server.1 = 192.168.10.205:2887:3887 server.2 = 192.168.10.205:2888:3888 server.3 = 192.168.10.205:2889:3889 [root@node5 opt]# cp zoo.cfg zk1/zk1.cfg [root@node5 opt]# cp zoo.cfg zk2/zk2.cfg [root@node5 opt]# cp zoo.cfg zk3/zk3.cfg [root@node5 opt]# vim zk2/zk2.cfg #每一個服務對應不一樣的目錄 dataDir=/opt/zk2
clientPort=2182 [root@node5 opt]# vim zk3/zk3.cfg dataDir=/opt/zk3
clientPort=2183
詳細配置:
[root@node5 opt]# grep '^[a-z]' zk3/zk3.cfg tickTime=2000 #服務器和客戶端的心跳維持間隔,間隔多久發送心跳 initLimit=10 #選舉的時候的時間間隔是10次,10次 * 2000微秒 syncLimit=5 # Leader 與 Follower 之間發送消息,請求和應答時間長度,最長不能超過多少個 tickTime 的時間長度 dataDir=/opt/zk3 #數據保存目錄 clientPort=2181 #客戶端鏈接的端口 server.1 = 192.168.10.205:2887:3887 #集羣端口和ID配置 server.2 = 192.168.10.205:2888:3888 server.3 = 192.168.10.205:2889:3889 詳細解釋: tickTime:這個時間是做爲 Zookeeper 服務器之間或客戶端與服務器之間維持心跳的時間間隔,也就是每一個 tickTime 時間就會發送一個心跳。 dataDir:顧名思義就是 Zookeeper 保存數據的目錄,默認狀況下,Zookeeper 將寫數據的日誌文件也保存在這個目錄裏。 clientPort:這個端口就是客戶端鏈接 Zookeeper 服務器的端口,Zookeeper 會監聽這個端口,接受客戶端的訪問請求。 initLimit:這個配置項是用來配置 Zookeeper 接受客戶端(這裏所說的客戶端不是用戶鏈接 Zookeeper 服務器的客戶端,而是 Zookeeper 服務器集羣中鏈接到 Leader 的 Follower 服務器)初始化鏈接時最長能忍受多少個心跳時間間隔數。當已經超過 5個心跳的時間(也就是 tickTime)長度後 Zookeeper 服務器尚未收到客戶端的返回信息,那麼代表這個客戶端鏈接失敗。總的時間長度就是 5*2000=10 秒 syncLimit:這個配置項標識 Leader 與 Follower 之間發送消息,請求和應答時間長度,最長不能超過多少個 tickTime 的時間長度,總的時間長度就是 2*2000=4 秒 server.A=B:C:D:其中 A 是一個數字,表示這個是第幾號服務器;B 是這個服務器的 ip 地址;C 表示的是這個服務器與集羣中的 Leader 服務器交換信息的端口;D 表示的是萬一集羣中的 Leader 服務器掛了,須要一個端口來從新進行選舉,選出一個新的 Leader,而這個端口就是用來執行選舉時服務器相互通訊的端口。若是是僞集羣的配置方式,因爲 B 都是同樣,因此不一樣的 Zookeeper 實例通訊端口號不能同樣,因此要給它們分配不一樣的端口號。
啓動每一個zookeeper 服務:
[root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk1/zk1.cfg JMX enabled by default Using config: /opt/zk1/zk1.cfg Starting zookeeper ... STARTED [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk2/zk2.cfg JMX enabled by default Using config: /opt/zk2/zk2.cfg Starting zookeeper ... STARTED [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk3/zk3.cfg JMX enabled by default Using config: /opt/zk3/zk3.cfg Starting zookeeper ... STARTED
確認端口啓動成功:
[root@node5 opt]# ss -tnl | grep 2181 LISTEN 0 50 :::2181 :::* [root@node5 opt]# ss -tnl | grep 2182 LISTEN 0 50 :::2182 :::* [root@node5 opt]# ss -tnl | grep 2183 LISTEN 0 50 :::2183 :::* [root@node5 opt]#
查看每一個節點的狀態:
[root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk1/zk1.cfg JMX enabled by default Using config: /opt/zk1/zk1.cfg Mode: follower [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk2/zk2.cfg JMX enabled by default Using config: /opt/zk2/zk2.cfg Mode: leader #這是主的,另外兩個是從的,表示已經啓動完成 [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk3/zk3.cfg JMX enabled by default Using config: /opt/zk3/zk3.cfg Mode: follower
鏈接到zookeeper節點:
[root@node5 opt]# /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.205:2181
[root@node5 opt]# /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.205:2181Connecting to 192.168.10.205:2181 2016-03-22 01:27:48,325 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2016-03-22 01:27:48,331 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node5.a.com 2016-03-22 01:27:48,332 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.7.0_45 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper/bin/../src/java/lib/*.jar:/usr/local/zookeeper/bin/../conf: 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2016-03-22 01:27:48,336 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/opt 2016-03-22 01:27:48,337 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=192.168.10.205:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@6ade299f Welcome to ZooKeeper! 2016-03-22 01:27:48,393 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@975] - Opening socket connection to server 192.168.10.205/192.168.10.205:2181. Will not attempt to authenticate using SASL (unknown error) 2016-03-22 01:27:48,424 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@852] - Socket connection established to 192.168.10.205/192.168.10.205:2181, initiating session JLine support is enabled 2016-03-22 01:27:48,621 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server 192.168.10.205/192.168.10.205:2181, sessionid = 0x1539a349fc40000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
進入到目錄執行編譯:
[root@node5 codis]# pwd /opt/gopath/src/github.com/CodisLabs/codis [root@node5 codis]# ls cmd docker genver.sh MIT-LICENSE.txt test config.ini Dockerfile Godeps pkg vitess_license doc extern Makefile README.md wandoujia_licese.txt [root@node5 codis]# make
[root@node5 codis]# make GO15VENDOREXPERIMENT=0 GOPATH=`godep path` godep restore godep: [WARNING]: godep should only be used inside a valid go package directory and godep: [WARNING]: may not function correctly. You are probably outside of your $GOPATH. godep: [WARNING]: Current Directory: /opt/gopath/src/github.com/CodisLabs/codis godep: [WARNING]: $GOPATH: /opt/gopath/src/github.com/CodisLabs/codis/Godeps/_workspace GOPATH=`godep path`:$GOPATH go build -o bin/codis-proxy ./cmd/proxy GOPATH=`godep path`:$GOPATH go build -o bin/codis-config ./cmd/cconfig make -j4 -C extern/redis-2.8.21/ make[1]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21' cd src && make all make[2]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/src' rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html (cd ../deps && make distclean) make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (cd hiredis && make clean) > /dev/null || true (cd linenoise && make clean) > /dev/null || true (cd lua && make clean) > /dev/null || true (cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true (rm -f .make-*) make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (rm -f .make-*) echo STD=-std=c99 -pedantic >> .make-settings echo WARN=-Wall -W >> .make-settings echo OPT=-O2 >> .make-settings echo MALLOC=jemalloc >> .make-settings echo CFLAGS= >> .make-settings echo LDFLAGS= >> .make-settings echo REDIS_CFLAGS= >> .make-settings echo REDIS_LDFLAGS= >> .make-settings echo PREV_FINAL_CFLAGS=-std=c99 -pedantic -Wall -W -O2 -g -ggdb -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -DUSE_JEMALLOC -I../deps/jemalloc/include >> .make-settings echo PREV_FINAL_LDFLAGS= -g -ggdb -rdynamic >> .make-settings (cd ../deps && make hiredis linenoise lua jemalloc) make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (cd hiredis && make clean) > /dev/null || true (cd linenoise && make clean) > /dev/null || true (cd lua && make clean) > /dev/null || true (cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true (rm -f .make-*) (echo "" > .make-ldflags) (echo "" > .make-cflags) MAKE hiredis cd hiredis && make static MAKE linenoise cd linenoise && make MAKE lua make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise' cc -Wall -Os -g -c linenoise.c cd lua/src && make all CFLAGS="-O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL " MYLDFLAGS="" AR="ar rcu" MAKE jemalloc cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src' cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lapi.o lapi.c make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis' cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb net.c checking for xsltproc... /usr/bin/xsltproc checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb hiredis.c checking whether we are cross compiling... no checking for suffix of object files... make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise' cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb sds.c o checking whether we are using the GNU C compiler... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lcode.o lcode.c yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb async.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldebug.o ldebug.c yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldo.o ldo.c yes ldo.c: In function ‘f_parser’: ldo.c:496: warning: unused variable ‘c’ cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldump.o ldump.c checking for inttypes.h... yes checking for stdint.h... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lfunc.o lfunc.c yes checking for unistd.h... ar rcs libhiredis.a net.o hiredis.o sds.o async.o cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lgc.o lgc.c yes checking whether byte ordering is bigendian... make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis' cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o llex.o llex.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmem.o lmem.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lobject.o lobject.c no checking size of void *... 8 checking size of int... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lopcodes.o lopcodes.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lparser.o lparser.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstate.o lstate.c 4 checking size of long... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstring.o lstring.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltable.o ltable.c 8 checking size of intmax_t... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltm.o ltm.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lundump.o lundump.c 8 checking build system type... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lvm.o lvm.c x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking whether pause instruction is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lzio.o lzio.c yes checking whether SSE2 intrinsics is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o strbuf.o strbuf.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o fpconv.o fpconv.c yes checking for ar... ar checking whether __attribute__ syntax is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lauxlib.o lauxlib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lbaselib.o lbaselib.c yes checking whether compiler supports -fvisibility=hidden... yes checking whether compiler supports -Werror... yes checking whether tls_model attribute is compilable... yes checking for a BSD-compatible install... /usr/bin/install -c checking for ranlib... ranlib checking for ld... /usr/bin/ld checking for autoconf... /usr/bin/autoconf checking for memalign... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldblib.o ldblib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o liolib.o liolib.c yes checking for valloc... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmathlib.o lmathlib.c yes checking configured backtracing method... N/A checking for sbrk... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loslib.o loslib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltablib.o ltablib.c yes checking whether utrace(2) is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstrlib.o lstrlib.c no checking whether valgrind is compilable... no checking STATIC_PAGE_SHIFT... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loadlib.o loadlib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o linit.o linit.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cjson.o lua_cjson.c 12 checking pthread.h usability... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_struct.o lua_struct.c yes checking pthread.h presence... yes checking for pthread.h... yes checking for pthread_create in -lpthread... yes checking for _malloc_thread_cleanup... no checking for _pthread_mutex_init_calloc_cb... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cmsgpack.o lua_cmsgpack.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_bit.o lua_bit.c no checking for TLS... yes checking whether a program using ffsl is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua.o lua.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o luac.o luac.c yes checking whether atomic(9) is compilable... no checking whether Darwin OSAtomic*() is compilable... no checking whether to force 32-bit __sync_{add,sub}_and_fetch()... no checking whether to force 64-bit __sync_{add,sub}_and_fetch()... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o print.o print.c no checking whether Darwin OSSpin*() is compilable... ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o strbuf.o fpconv.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o lua_bit.o # DLL needs all object files ranlib liblua.a no checking for stdbool.h that conforms to C99... cc -o lua lua.o liblua.a -lm yes checking for _Bool... liblua.a(loslib.o): In function `os_tmpname': loslib.c:(.text+0x35): warning: the use of `tmpnam' is dangerous, better use `mkstemp' cc -o luac luac.o print.o liblua.a -lm make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src' yes configure: creating ./config.status config.status: creating Makefile config.status: creating doc/html.xsl config.status: creating doc/manpages.xsl config.status: creating doc/jemalloc.xml config.status: creating include/jemalloc/jemalloc_macros.h config.status: creating include/jemalloc/jemalloc_protos.h config.status: creating include/jemalloc/internal/jemalloc_internal.h config.status: creating test/test.sh config.status: creating test/include/test/jemalloc_test.h config.status: creating config.stamp config.status: creating bin/jemalloc.sh config.status: creating include/jemalloc/jemalloc_defs.h config.status: creating include/jemalloc/internal/jemalloc_internal_defs.h config.status: creating test/include/test/jemalloc_test_defs.h config.status: executing include/jemalloc/internal/private_namespace.h commands config.status: executing include/jemalloc/internal/private_unnamespace.h commands config.status: executing include/jemalloc/internal/public_symbols.txt commands config.status: executing include/jemalloc/internal/public_namespace.h commands config.status: executing include/jemalloc/internal/public_unnamespace.h commands config.status: executing include/jemalloc/internal/size_classes.h commands config.status: executing include/jemalloc/jemalloc_protos_jet.h commands config.status: executing include/jemalloc/jemalloc_rename.h commands config.status: executing include/jemalloc/jemalloc_mangle.h commands config.status: executing include/jemalloc/jemalloc_mangle_jet.h commands config.status: executing include/jemalloc/jemalloc.h commands =============================================================================== jemalloc version : 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340 library revision : 1 CC : gcc CPPFLAGS : -D_GNU_SOURCE -D_REENTRANT CFLAGS : -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -fvisibility=hidden LDFLAGS : EXTRA_LDFLAGS : LIBS : -lpthread RPATH_EXTRA : XSLTPROC : /usr/bin/xsltproc XSLROOT : /usr/share/sgml/docbook/xsl-stylesheets PREFIX : /usr/local BINDIR : /usr/local/bin INCLUDEDIR : /usr/local/include LIBDIR : /usr/local/lib DATADIR : /usr/local/share MANDIR : /usr/local/share/man srcroot : abs_srcroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/ objroot : abs_objroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/ JEMALLOC_PREFIX : je_ JEMALLOC_PRIVATE_NAMESPACE : je_ install_suffix : autogen : 0 experimental : 1 cc-silence : 1 debug : 0 code-coverage : 0 stats : 1 prof : 0 prof-libunwind : 0 prof-libgcc : 0 prof-gcc : 0 tcache : 1 fill : 1 utrace : 0 valgrind : 0 xmalloc : 0 mremap : 0 munmap : 0 dss : 0 lazy_lock : 0 tls : 1 =============================================================================== cd jemalloc && make CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" lib/libjemalloc.a make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc' gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/jemalloc.o src/jemalloc.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/arena.o src/arena.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/atomic.o src/atomic.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/base.o src/base.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/bitmap.o src/bitmap.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk.o src/chunk.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_dss.o src/chunk_dss.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_mmap.o src/chunk_mmap.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ckh.o src/ckh.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ctl.o src/ctl.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/extent.o src/extent.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/hash.o src/hash.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/huge.o src/huge.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mb.o src/mb.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mutex.o src/mutex.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/prof.o src/prof.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/quarantine.o src/quarantine.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/rtree.o src/rtree.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/stats.o src/stats.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tcache.o src/tcache.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/util.o src/util.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tsd.o src/tsd.c ar crus lib/libjemalloc.a src/jemalloc.o src/arena.o src/atomic.o src/base.o src/bitmap.o src/chunk.o src/chunk_dss.o src/chunk_mmap.o src/ckh.o src/ctl.o src/extent.o src/hash.o src/huge.o src/mb.o src/mutex.o src/prof.o src/quarantine.o src/rtree.o src/stats.o src/tcache.o src/util.o src/tsd.o make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc' make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' CC adlist.o CC ae.o CC anet.o CC dict.o anet.c: In function ‘anetSockName’: anet.c:565: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:563: note: initialized from here anet.c:569: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:567: note: initialized from here anet.c: In function ‘anetPeerToString’: anet.c:543: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:541: note: initialized from here anet.c:547: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:545: note: initialized from here anet.c: In function ‘anetTcpAccept’: anet.c:511: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:509: note: initialized from here anet.c:515: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:513: note: initialized from here CC redis.o CC sds.o CC zmalloc.o CC lzf_c.o CC lzf_d.o CC pqsort.o CC zipmap.o CC sha1.o CC ziplist.o CC release.o CC networking.o CC util.o CC object.o CC db.o CC replication.o CC rdb.o CC t_string.o db.c: In function ‘scanGenericCommand’: db.c:454: warning: ‘pat’ may be used uninitialized in this function db.c:455: warning: ‘patlen’ may be used uninitialized in this function CC t_list.o CC t_set.o CC t_zset.o CC t_hash.o CC config.o CC aof.o CC pubsub.o CC multi.o CC debug.o CC sort.o CC intset.o CC syncio.o CC migrate.o CC endianconv.o CC slowlog.o CC scripting.o CC bio.o CC rio.o CC rand.o CC memtest.o CC crc64.o CC crc32.o CC bitops.o CC sentinel.o CC notify.o CC setproctitle.o CC hyperloglog.o CC latency.o CC sparkline.o CC slots.o CC redis-cli.o CC redis-benchmark.o CC redis-check-dump.o CC redis-check-aof.o LINK redis-benchmark LINK redis-check-dump LINK redis-check-aof LINK redis-server INSTALL redis-sentinel LINK redis-cli Hint: It's a good idea to run 'make test' ;)
[root@node5 codis]# make gotest GOPATH=`godep path`:$GOPATH go test ./pkg/... ./cmd/... ok github.com/CodisLabs/codis/pkg/models 17.324s ok github.com/CodisLabs/codis/pkg/proxy 6.296s ok github.com/CodisLabs/codis/pkg/proxy/redis 1.338s ok github.com/CodisLabs/codis/pkg/proxy/router 0.654s ? github.com/CodisLabs/codis/pkg/utils [no test files] ? github.com/CodisLabs/codis/pkg/utils/assert [no test files] ? github.com/CodisLabs/codis/pkg/utils/atomic2 [no test files] ok github.com/CodisLabs/codis/pkg/utils/bytesize 0.002s ? github.com/CodisLabs/codis/pkg/utils/errors [no test files] ? github.com/CodisLabs/codis/pkg/utils/log [no test files] ? github.com/CodisLabs/codis/pkg/utils/trace [no test files] ? github.com/CodisLabs/codis/cmd/cconfig [no test files] ? github.com/CodisLabs/codis/cmd/proxy [no test files]
執行所有指令後,會在 bin 文件夾內生成 codis-config、codis-proxy、codis-server三個可執行文件。另外, bin/assets 文件夾是 codis-config 的 dashboard http 服務須要的前端資源, 須要和 codis-config 放置在同一文件夾下)
[root@node5 codis]# cd bin/ [root@node5 bin]# ls assets codis-config codis-proxy codis-server
可執行文件使用方式:
[root@node5 bin]# ./codis-config -h
[root@node5 bin]# ./codis-config -h
usage: codis-config [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>] <command> [<args>...] options: -c #配置文件地址 -L #日誌輸出文件地址 --log-level=<loglevel> #輸出日誌級別 (debug < info (default) < warn < error < fatal) commands: server #redis 服務器組管理 slot #slot 管理 dashboard #啓動 dashboard 服務 action #事件管理 (目前只有刪除歷史事件的日誌) proxy #proxy 管理
[root@node5 bin]# ./codis-proxy -h
$ bin/codis-proxy -h usage: codis-proxy [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>] [--cpu=<cpu_num>] [--addr=<proxy_listen_addr>] [--http-addr=<debug_http_server_addr>] options: -c #配置文件地址 -L #日誌輸出文件地址 --log-level=<loglevel> #輸出日誌級別 (debug < info (default) < warn < error < fatal) --cpu=<cpu_num> #proxy佔用的 cpu 核數, 默認1, 最好設置爲機器的物理cpu數的一半到2/3左右 --addr=<proxy_listen_addr> #proxy 的 redis server 監聽的地址, 格式 <ip or hostname>:<port>, 如: localhost:9000, :9001 --http-addr=<debug_http_server_addr> #proxy 的調試信息啓動的http server, 能夠訪問 http://debug_http_server_addr/debug/vars
[root@node5 bin]# ./codis-server -h
[root@node5 bin]# ./codis-server -h Usage: ./redis-server [/path/to/redis.conf] [options] ./redis-server - (read config from stdin) #指定redis配置文件 ./redis-server -v or --version #顯示版本 ./redis-server -h or --help #獲取幫助 ./redis-server --test-memory <megabytes> #單位爲字節 Examples: ./redis-server (run the server with default conf) ./redis-server /etc/redis/6379.conf ./redis-server --port 7777 ./redis-server --port 7777 --slaveof 127.0.0.1 8888 ./redis-server /etc/myredis.conf --loglevel verbose
開始部署:
[root@node5 bin]# cd ..
[root@node5 codis]# vim config.ini
zk=192.168.10.205:2181
dashboard_addr=192.168.10.205:18087
配置並啓動 dashboard:
[root@node5 codis]# cp config.ini bin/ #將修改後的文件複製至bin目錄,由於bin裏面你的命令默認讀取當前目錄的config.ini文件,或可使用 -c指定也能夠
[root@node5 bin]# ./codis-config dashboard & #該命令會啓動 dashboard
啓動成功就能夠經過 http://ServerIP:18087訪問
執行./codis-config slot init
,該命令會在zookeeper上建立slot相關信息,若是提示已經初始化能夠加 -f 強制
[root@node5 bin]# ./codis-config slot init -f { "msg": "OK", "ret": 0 }
若是已經啓動能夠跳過,若是還未啓動就執行:
[root@node5 bin]# /etc/init.d/redisd start
每個 Server Group 做爲一個 Redis 服務器組存在, 只容許有一個 master, 能夠有多個 slave, group id 僅支持大於等於1的整數
添加一個master:
[root@node5 bin]# ./codis-config server add 1 localhost:6379 master { "msg": "OK", "ret": 0 }
爲master添加一個slave:
[root@node5 bin]# ./codis-config server add 1 192.168.10.205:7009 slave
[root@node5 bin]# ./codis-config server add 1 192.168.10.205:7009 slave 2016/03/22 03:37:27 main.go:154: [PANIC] run sub-command failed [error]: http status code 500, ERR SLAVEOF not allowed in cluster mode. 4 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/utils.go:66 main.callApi 3 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/server_group.go:94 main.runAddServerToGroup 2 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/server_group.go:54 main.cmdServer 1 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:87 main.runCommand 0 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:152 main.main ... ... [stack]: 0 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:154 main.main ... ...
查看列表信息:
[root@node5 bin]# ./codis-config server list [ { "id": 1, "product_name": "test", "servers": [ { "addr": "localhost:6379", "group_id": 1, "type": "master" }, { "addr": "192.168.10.205:7009", "group_id": 1, "type": "slave" } ] } ]
在瀏覽器驗證:
Codis 採用 Pre-sharding 的技術來實現數據的分片, 默認分紅 1024 個 slots (0-1023), 對於每一個key來講, 經過如下公式肯定所屬的 Slot Id : SlotId = crc32(key) % 1024 每個 slot 都會有一個且必須有一個特定的 server group id 來表示這個 slot 的數據由哪一個 server group 來提供:
如:
設置編號爲[0, 511]的 slot 由 server group 1 提供服務, 編號 [512, 1023] 的 slot 由 server group 2 提供服務
[root@node5 bin]# ./codis-config slot range-set 0 511 1 online { "msg": "OK", "ret": 0 } [root@node5 bin]# ./codis-config slot range-set 512 1023 2 online { "msg": "OK", "ret": 0 }
[root@node5 bin]# ./codis-proxy -c config.ini -L ./log/proxy.log --cpu=8 --addr=0.0.0.0:19000 --http-addr=0.0.0.0:11000 _____ ____ ____/ / (_) _____ / ___/ / __ \ / __ / / / / ___/ / /__ / /_/ / / /_/ / / / (__ ) \___/ \____/ \__,_/ /_/ /____/
剛啓動的 codis-proxy 默認是處於 offline狀態的, 而後設置 proxy 爲 online 狀態, 只有處於 online 狀態的 proxy 纔會對外提供服務
[root@node5 bin]# ./codis-config -c config.ini proxy online proxy_1 { "msg": "OK", "ret": 0 }
注:命令格式爲./codis-config -c config.ini proxy online <proxy_name> <---- proxy的id, 如 proxy_1,能夠在瀏覽器看到