消息中間件metaq

消息中間件metaq安裝並註冊到zookper集羣

項目地址

https://github.com/killme2008/Metamorphosis

Memorphosis是一個消息中間件，它是linkedin開源MQ——kafka的Java版本，針對淘寶內部應用作了定製和優化。Metamorphosis的設計原則

• 消息都是持久的，保存在磁盤
• 吞吐量第一
• 消費狀態保存在客戶端
•  分佈式，生產者、服務器和消費者均可分佈

Metamorphosis的部署結構

[root@q tools]# tar xf metaq-server-1.4.6.2.tar.gz 
[root@q tools]# cd taobao/
[root@q taobao]# ls
COPYING.txt  metamorphosis-server-wrapper  README.md

[root@q taobao]# cd metamorphosis-server-wrapper/
[root@q metamorphosis-server-wrapper]# ls
bin  conf  lib  provided

其中bin目錄包含啓動腳本（目前只有linux下的啓動腳本），conf下爲配置文件，lib下爲meta服務器的全部jar包,logs爲日誌所在目錄。

[root@q metamorphosis-server-wrapper]# cd conf/
[root@q conf]# ls
async_slave.properties   jettyBroker.properties  notifySlave.properties  samsa_master.properties  timetunnel.properties
gregor_slave.properties  notifyadaper            sample.server.ini       server.ini
[root@q conf]# vim server.ini 

[system]
brokerId=0
numPartitions=1
serverPort=8123
dashboardHttpPort=8120
unflushThreshold=0
unflushInterval=10000
maxSegmentSize=1073741824
maxTransferSize=1048576
deletePolicy=delete,168
deleteWhen=0 0 6,18 * * ?
flushTxLogAtCommit=1
stat=true

;; Update consumers offsets to current max offsets when consumers offsets are out of range of current broker's messages.
;; It must be false in production.But recommend to be true in development or test.
updateConsumerOffsets=true

[zookeeper]
zk.zkConnect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
zk.zkSessionTimeoutMs=30000
zk.zkConnectionTimeoutMs=30000
zk.zkSyncTimeMs=5000

;; Topics section
[topic=test]

[topic=meta-test]

[topic=xl]   #自定義的消息xl的隊列

啓動

[root@q conf]# cd ..
[root@q metamorphosis-server-wrapper]# ls
bin  conf  lib  logs  provided
[root@q metamorphosis-server-wrapper]# cd bin/
[root@q bin]# sh metaServer.sh start &

查看狀態

[root@q bin]# sh metaServer.sh stats

測試

[root@q bin]# ./zkCli.sh -server 127.0.0.1:2181 [zk: 127.0.0.1:2181(CONNECTING) 0] ls /
2017-01-01 19:19:42,232 [myid:] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x159591602330000, negotiated timeout = 30000
WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zookeeper, meta]  #在此目錄生成meta
[zk: 127.0.0.1:2181(CONNECTED) 1] ls /meta
[brokers]
[zk: 127.0.0.1:2181(CONNECTED) 2] ls /meta/brokers
[topics-sub, ids, topics-pub, topics]
 [zk: 127.0.0.1:2181(CONNECTED) 3] ls /meta/brokers/topics
[meta-test, test, xl]
[zk: 127.0.0.1:2181(CONNECTED) 4] ls /meta/brokers/ids  #brok節點 
[0]
[zk: 127.0.0.1:2181(CONNECTED) 5] get  /meta/brokers/ids/0
meta://192.168.179.161:8123
cZxid = 0x200000046
ctime = Sun Jan 01 16:44:09 CST 2017
mZxid = 0x200000048
mtime = Sun Jan 01 16:44:09 CST 2017
pZxid = 0x200000049
cversion = 2
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 27
numChildren = 2
[zk: 127.0.0.1:2181(CONNECTED) 6]

部署另一臺

在同一臺模擬部署

[root@q tools]# cp -rf taobao/ taobao1
[root@q bin]# pwd
/home/zsq/tools/taobao1/metamorphosis-server-wrapper/bin
[root@q bin]# ls
env.bat  env.sh  localMetaServer.bat  log4j.properties  metaServer.bat  metaServer.sh  tools_log4j.properties

修改腳本metaServer.sh的pid文件

PID_FILE="$PID_DIR/.run1.pid"

修改env.sh，默認是9123端口

export JMX_PORT=9124

啓動

[root@q bin]# sh metaServer.sh start $

[zk: 127.0.0.1:2181(CONNECTED) 12] ls  /meta/brokers/ids  #有兩個broker就表面集羣部署成功
[0, 1]

[zk: 127.0.0.1:2181(CONNECTED) 18] get  /meta/brokers/ids/1
meta://192.168.179.161:8124
cZxid = 0x200000092
ctime = Sun Jan 01 21:05:26 CST 2017
mZxid = 0x200000094
mtime = Sun Jan 01 21:05:26 CST 2017
pZxid = 0x200000095
cversion = 2
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 27
numChildren = 2
[zk: 127.0.0.1:2181(CONNECTED) 19]

消息中間件metaq的配置啓動及相關維護操做