ZooKeeper-3.3.4集羣安裝配置(轉載)

ZooKeeper是一個分佈式開源框架,提供了協調分佈式應用的基本服務,它向外部應用暴露一組通用服務——分佈式同步(Distributed Synchronization)、命名服務(Naming Service)、集羣維護(Group Maintenance)等,簡化分佈式應用協調及其管理的難度,提供高性能的分佈式服務。ZooKeeper自己能夠以Standalone模式安裝運行,不過它的長處在於經過分佈式ZooKeeper集羣(一個Leader,多個Follower),基於必定的策略來保證ZooKeeper集羣的穩定性和可用性,從而實現分佈式應用的可靠性。 有關ZooKeeper的介紹,網上不少,也能夠參考文章後面,我整理的一些相關連接。 ZooKeeper的安裝配置還算比較容易的,下面,咱們簡單說明一下ZooKeeper的配置。html

ZooKeeper Standalone模式java

從Apache網站上(zookeeper.apache.org)下載ZooKeeper軟件包,我選擇了3.3.4版本的(zookeeper-3.3.4.tar.gz),在一臺Linux機器上安裝很是容易,只須要解壓縮後,簡單配置一下便可以啓動ZooKeeper服務器進程。 將zookeeper-3.3.4/conf目錄下面的 zoo_sample.cfg修改成zoo.cfg,配置文件內容以下所示:算法

1 tickTime=2000
2 dataDir=/home/hadoop/storage/zookeeper
3 clientPort=2181
4 initLimit=5
5 syncLimit=2

上面各個配置參數的含義也很是簡單,引用以下所示:apache

tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime. dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. clientPort —— the port to listen for client connections服務器

下面啓動ZooKeeper服務器進程:session

cd zookeeper-3.3.4/
bin/zkServer.sh start

 經過jps命令能夠查看ZooKeeper服務器進程,名稱爲QuorumPeerMain。
在客戶端鏈接ZooKeeper服務器,執行以下命令:架構

bin/zkCli.sh -server dynamic:2181

 客戶端鏈接信息以下所示:app

shirdrn@master:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server dynamic:2181
Connecting to dynamic:2181
2013-10-28 21:30:06,178 - INFO  [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2013-10-28 21:30:06,188 - INFO  [main:Environment@97] - Client environment:host.name=master
2013-10-28 21:30:06,191 - INFO  [main:Environment@97] - Client environment:java.version=1.6.0_30
2013-10-28 21:30:06,194 - INFO  [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2013-10-28 21:30:06,200 - INFO  [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2013-10-28 21:30:06,203 - INFO  [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2013-10-28 21:30:06,206 - INFO  [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2013-10-28 21:30:06,213 - INFO  [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2013-10-28 21:30:06,216 - INFO  [main:Environment@97] - Client environment:java.compiler=<NA>
2013-10-28 21:30:06,235 - INFO  [main:Environment@97] - Client environment:os.name=Linux
2013-10-28 21:30:06,244 - INFO  [main:Environment@97] - Client environment:os.arch=i386
2013-10-28 21:30:06,246 - INFO  [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2013-10-28 21:30:06,251 - INFO  [main:Environment@97] - Client environment:user.name=hadoop
2013-10-28 21:30:06,254 - INFO  [main:Environment@97] - Client environment:user.home=/home/hadoop
2013-10-28 21:30:06,255 - INFO  [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2013-10-28 21:30:06,264 - INFO  [main:ZooKeeper@379] - Initiating client connection, connectString=dynamic:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2013-10-28 21:30:06,339 - INFO  [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server dynamic/192.168.0.107:2181
Welcome to ZooKeeper!
2013-10-28 21:30:06,397 - INFO  [main-SendThread(dynamic:2181):ClientCnxn$SendThread@950] - Socket connection established to dynamic/192.168.0.107:2181, initiating session
JLine support is enabled
2013-10-28 21:30:06,492 - INFO  [main-SendThread(dynamic:2181):ClientCnxn$SendThread@739] - Session establishment complete on server dynamic/192.168.0.107:2181, sessionid = 0x134b9b714f9000c, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: dynamic:2181(CONNECTED) 0] 

 接着,可使用help查看Zookeeper客戶端可使用的基本操做命令。框架

ZooKeeper Distributed模式less

ZooKeeper分佈式模式安裝(ZooKeeper集羣)也比較容易,這裏說明一下基本要點。 首先要明確的是,ZooKeeper集羣是一個獨立的分佈式協調服務集羣,「獨立」的含義就是說,若是想使用ZooKeeper實現分佈式應用的協調與管理,簡化協調與管理,任何分佈式應用均可以使用,這就要歸功於Zookeeper的數據模型(Data Model)和層次命名空間(Hierarchical Namespace)結構,詳細能夠參考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在設計你的分佈式應用協調服務時,首要的就是考慮如何組織層次命名空間。 下面說明分佈式模式的安裝配置,過程以下所示:

第一步:主機名稱到IP地址映射配置 ZooKeeper集羣中具備兩個關鍵的角色:Leader和Follower。集羣中全部的結點做爲一個總體對分佈式應用提供服務,集羣中每一個結點之間都互相鏈接,因此,在配置的ZooKeeper集羣的時候,每個結點的host到IP地址的映射都要配置上集羣中其它結點的映射信息。

例如,個人ZooKeeper集羣中每一個結點的配置,以zk-01爲例,/etc/hosts內容以下所示:

192.168.0.179   zk-01
192.168.0.178   zk-02
192.168.0.177   zk-03

 ZooKeeper採用一種稱爲Leader election的選舉算法。在整個集羣運行過程當中,只有一個Leader,其餘的都是Follower,若是ZooKeeper集羣在運行過程當中Leader出了問題,系統會採用該算法從新選出一個Leader。所以,各個結點之間要可以保證互相鏈接,必須配置上述映射。
ZooKeeper集羣啓動的時候,會首先選出一個Leader,在Leader election過程當中,某一個知足選舉算的結點就能成爲Leader。整個集羣的架構能夠參考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。
第二步:修改ZooKeeper配置文件
在其中一臺機器(zk-01)上,解壓縮zookeeper-3.3.4.tar.gz,修改配置文件conf/zoo.cfg,內容以下所示:

tickTime=2000
dataDir=/home/hadoop/storage/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zk-01:2888:3888
server.2=zk-02:2888:3888
server.3=zk-03:2888:3888

 上述配置內容說明,能夠參考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper
第三步:遠程複製分發安裝文件
上面已經在一臺機器zk-01上配置完成ZooKeeper,如今能夠將該配置好的安裝文件遠程拷貝到集羣中的各個結點對應的目錄下:

cd /home/hadoop/installation/
scp -r zookeeper-3.3.4/ shirdrn@zk-02:/home/hadoop/installation/
scp -r zookeeper-3.3.4/ shirdrn@zk-03:/home/hadoop/installation/

 第四步:設置myid
在咱們配置的dataDir指定的目錄下面,建立一個myid文件,裏面內容爲一個數字,用來標識當前主機,conf/zoo.cfg文件中配置的server.X中X爲何數字,則myid文件中就輸入這個數字,例如:

shirdrn@zk-01:~/installation/zookeeper-3.3.4$ echo "1" > /home/hadoop/storage/zookeeper/myid
shirdrn@zk-02:~/installation/zookeeper-3.3.4$ echo "2" > /home/hadoop/storage/zookeeper/myid
shirdrn@zk-03:~/installation/zookeeper-3.3.4$ echo "3" > /home/hadoop/storage/zookeeper/myid

 按照上述進行配置便可。
第五步:啓動ZooKeeper集羣
在ZooKeeper集羣的每一個結點上,執行啓動ZooKeeper服務的腳本,以下所示:

shirdrn@zk-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
shirdrn@zk-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
shirdrn@zk-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start

 以結點zk-01爲例,日誌以下所示:

shirdrn@zk-01:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out
2013-10-28 06:51:19,117 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
2013-10-28 06:51:19,133 - INFO  [main:QuorumPeerConfig@310] - Defaulting to majority quorums
2013-10-28 06:51:19,167 - INFO  [main:QuorumPeerMain@119] - Starting quorum peer
2013-10-28 06:51:19,227 - INFO  [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:2181
2013-10-28 06:51:19,277 - INFO  [main:QuorumPeer@819] - tickTime set to 2000
2013-10-28 06:51:19,278 - INFO  [main:QuorumPeer@830] - minSessionTimeout set to -1
2013-10-28 06:51:19,279 - INFO  [main:QuorumPeer@841] - maxSessionTimeout set to -1
2013-10-28 06:51:19,281 - INFO  [main:QuorumPeer@856] - initLimit set to 5
2013-10-28 06:51:19,347 - INFO  [Thread-1:QuorumCnxManager$Listener@473] - My election bind port: 3888
2013-10-28 06:51:19,393 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@621] - LOOKING
2013-10-28 06:51:19,396 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@663] - New election. My id =  1, Proposed zxid = 0
2013-10-28 06:51:19,400 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
2013-10-28 06:51:19,416 - WARN  [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 2 at election address zk-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
        at java.lang.Thread.run(Thread.java:662)
2013-10-28 06:51:19,420 - WARN  [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
        at java.lang.Thread.run(Thread.java:662)
2013-10-28 06:51:19,612 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address zk-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:19,615 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:19,616 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 400
2013-10-28 06:51:20,019 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address zk-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:20,021 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:20,022 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 800
2013-10-28 06:51:20,825 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address zk-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:20,827 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:20,828 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 1600
2013-10-28 06:51:22,435 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address zk-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:22,439 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:51:22,441 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 3200
2013-10-28 06:51:22,945 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)
2013-10-28 06:51:22,946 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@721] - Updating proposal
2013-10-28 06:51:22,949 - WARN  [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
        at java.lang.Thread.run(Thread.java:662)
2013-10-28 06:51:22,951 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
2013-10-28 06:51:23,156 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@643] - FOLLOWING
2013-10-28 06:51:23,170 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@80] - TCP NoDelay set to: true
2013-10-28 06:51:23,206 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2013-10-28 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:host.name=zk-01
2013-10-28 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.version=1.6.0_30
2013-10-28 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.vendor=Sun Microsystems Inc.
2013-10-28 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2013-10-28 06:51:23,209 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2013-10-28 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2013-10-28 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.io.tmpdir=/tmp
2013-10-28 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.compiler=<NA>
2013-10-28 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.name=Linux
2013-10-28 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.arch=i386
2013-10-28 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.version=3.0.0-14-generic
2013-10-28 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.name=hadoop
2013-10-28 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.home=/home/hadoop
2013-10-28 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2013-10-28 06:51:23,223 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@151] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
2013-10-28 06:51:23,339 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@294] - Getting a snapshot from leader
2013-10-28 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@325] - Setting leader epoch 1
2013-10-28 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FileTxnSnapLog@254] - Snapshotting: 0
2013-10-28 06:51:25,511 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
2013-10-28 06:51:42,584 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)

 我啓動的順序是zk-01>zk-02>zk-03,因爲ZooKeeper集羣啓動的時候,每一個結點都試圖去鏈接集羣中的其它結點,先啓動的確定連不上後面還沒啓動的,因此上面日誌前面部分的異常是能夠忽略的。經過後面部分能夠看到,集羣在選出一個Leader後,最後穩定了。
其餘結點可能也出現相似問題,屬於正常。
第六步:安裝驗證
能夠經過ZooKeeper的腳原本查看啓動狀態,包括集羣中各個結點的角色(或是Leader,或是Follower),以下所示,是在ZooKeeper集羣中的每一個結點上查詢的結果:

shirdrn@zk-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: follower

shirdrn@zk-02:~/installation/zookeeper-3.3.4$  bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: leader

shirdrn@zk-03:~/installation/zookeeper-3.3.4$  bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: follower

 經過上面狀態查詢結果可見,zk-02是集羣的Leader,其他的兩個結點是Follower。
另外,能夠經過客戶端腳本,鏈接到ZooKeeper集羣上。對於客戶端來講,ZooKeeper是一個總體(ensemble),鏈接到ZooKeeper集羣實際上感受在獨享整個集羣的服務,因此,你能夠在任何一個結點上創建到服務集羣的鏈接,例如:

shirdrn@zk-03:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server zk-01:2181
Connecting to zk-01:2181
2013-10-28 07:14:21,068 - INFO  [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2013-10-28 07:14:21,080 - INFO  [main:Environment@97] - Client environment:host.name=zk-03
2013-10-28 07:14:21,085 - INFO  [main:Environment@97] - Client environment:java.version=1.6.0_30
2013-10-28 07:14:21,089 - INFO  [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2013-10-28 07:14:21,095 - INFO  [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2013-10-28 07:14:21,104 - INFO  [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2013-10-28 07:14:21,111 - INFO  [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2013-10-28 07:14:21,116 - INFO  [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2013-10-28 07:14:21,124 - INFO  [main:Environment@97] - Client environment:java.compiler=<NA>
2013-10-28 07:14:21,169 - INFO  [main:Environment@97] - Client environment:os.name=Linux
2013-10-28 07:14:21,175 - INFO  [main:Environment@97] - Client environment:os.arch=i386
2013-10-28 07:14:21,177 - INFO  [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2013-10-28 07:14:21,185 - INFO  [main:Environment@97] - Client environment:user.name=hadoop
2013-10-28 07:14:21,188 - INFO  [main:Environment@97] - Client environment:user.home=/home/hadoop
2013-10-28 07:14:21,190 - INFO  [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2013-10-28 07:14:21,197 - INFO  [main:ZooKeeper@379] - Initiating client connection, connectString=zk-01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2013-10-28 07:14:21,305 - INFO  [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server zk-01/192.168.0.179:2181
Welcome to ZooKeeper!
2013-10-28 07:14:21,376 - INFO  [main-SendThread(zk-01:2181):ClientCnxn$SendThread@950] - Socket connection established to zk-01/192.168.0.179:2181, initiating session
JLine support is enabled
[zk: zk-01:2181(CONNECTING) 0] 2013-10-28 07:14:21,872 - INFO  [main-SendThread(zk-01:2181):ClientCnxn$SendThread@739] - Session establishment complete on server zk-01/192.168.0.179:2181, sessionid = 0x134bdcd6b730000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

[zk: zk-01:2181(CONNECTED) 0] ls /
[zookeeper]

 當前根路徑爲/zookeeper。

總結說明

主機名與IP地址映射配置問題 啓動ZooKeeper集羣時,若是ZooKeeper集羣中zk-01結點的日誌出現以下錯誤:

java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:37:46,026 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 6400
2013-10-28 06:37:57,431 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address zk-02/202.106.199.35:3888
java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2013-10-28 06:38:02,442 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address zk-03/202.106.199.35:3888

 

很顯然,zk-01在啓動時鏈接集羣中其餘結點(zk-0二、zk-03)時,主機名映射的IP與咱們實際配置的不一致,因此集羣中各個結點之間沒法創建鏈路,整個ZooKeeper集羣啓動是失敗的。 上面錯誤日誌中zk-02/202.106.199.35:3888實際應該是zk-02/202.192.168.0.178:3888就對了,可是在進行域名解析的時候映射有問題,修改每一個結點的/etc/hosts文件,將ZooKeeper集羣中全部結點主機名到IP地址的映射配置上。

參考連接

下面是我整理蒐集的有關ZooKeeper相關內容的網址,能夠學習參考。 中文連接:

英文連接:

本文基於署名-非商業性使用-相同方式共享 4.0許可協議發佈,歡迎轉載、使用、從新發布,但務必保留文章署名時延軍(包含連接:http://shiyanjun.cn),不得用於商業目的,基於本文修改後的做品務必以相同的許可發佈。若有任何疑問,請與我聯繫。
相關文章
相關標籤/搜索