20190426 部署Zookeeper集羣

kafka是製做消息隊列的，與redis類似，可是優於redis，支持消息並列與通知，

實驗:部署zookeeper集羣：

安裝思路及環境：先安裝Java環境，再安裝zookeeper，後安裝kafka，：安裝思路及環境：先安裝Java環境，再安裝zookeeper，後安裝kafka，
zookeeper版本下載網址： http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
http://zookeeper.apache.org/doc/current/zookeeperStarted.html中有：
server.1=172.18.9.203:2888:3888 server.2=172.18.9.204:2888:3888 server.3=172.18.9.205:2888:3888
製做zookeeper集羣需三臺服務器：172.18.9.203,172.18.9.204,172.18.9.205
三臺服務器分別配置hosts文件：
#cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.18.9.203 linux-host1.exmaple.com
172.18.9.204 linux-host2.exmaple.com
172.18.9.205 linux-host3.exmaple.com

1、安裝Java環境：[root@203/204/205 src]#yum -y install jdk-8u191-linux-x64.rpm

2、安裝zookeeper：[root@203/204/205 src]#wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

[root@203/204/205 src]#tar xf zookeeper-3.4.14.tar.gz

3、製做zookeeper配置文件：

[root@203/204/205 src]#cd zookeeper-3.4.14/conf
[root@203/204/205 conf]#ll
total 12
-rw-rw-r--. 1 2002 2002 535 Mar 7 00:50 configuration.xsl
-rw-rw-r--. 1 2002 2002 2161 Mar 7 00:50 log4j.properties
-rw-rw-r--. 1 2002 2002 922 Mar 7 00:50 zoo_sample.cfg
[root@203/204/205 conf]#cp zoo_sample.cfg zoo.cfg #把模板cp成配置文件

4、修改配置文件並作軟鏈接並建立數據存放目錄：

[root@203/204/205 conf]#vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/data
clientPort=2181
maxClientCnxns=4096
server.1=172.18.9.203:2888:3888
server.2=172.18.9.204:2888:3888
server.3=172.18.9.205:2888:3888
[root@203/204/205 conf]#mkdir /usr/local/zookeeper/data
[root@203/204/205 conf]#ln -s /usr/local/src/zookeeper-3.4.14 /usr/local/zookeeper

5、開啓並查看狀態：[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh start

ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
出現此錯的緣由是zookeeper的pid沒有生成！

6、將server的編號寫入myid文件：

[root@203/204/205 zookeeper-3.4.14]#echo 1 2 3 > /usr/local/zookeeper/data/myid
[root@203/204/205 zookeeper-3.4.14]#

7、從新開啓服務並查看端口：

[root@203/204/205 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh start
[root@203/204/205 zookeeper-3.4.14]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 50 :::2181 :::*

8、查看集羣狀態：leader/follower 一目瞭然：

[root@203 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@204 conf]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower

9、將開啓服務命令寫入腳本、令其開機自啓：

[root@203/204/205 zookeeper-3.4.14]#vim /etc/rc.d/rc.local
/usr/local/zookeeper/bin/zkServer.sh start
此時，zookeeper已完成安裝，它沒有帳號認證，任何普戶都能用！

10、安裝kafka：

[root@203/204/205 src]#tar xf kafka_2.12-2.2.0.tar.gz
[root@203/204/205 src]#ln -s /usr/local/src/kafka_2.12-2.2.0 /usr/local/kafka
[root@203/204/205 src]#vim /usr/local/kafka/config/server.properties
21 broker.id=1 2 3 #設置每一個代理全局惟一的整數ID
31 listeners=PLAINTEXT://172.18.9.203:9092
103 log.retention.hours=24 #保留指定小時的日誌內容
123 zookeeper.connect=172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 #全部的zookeeper地址

11、啓動kafka，驗證進程「jps」：

[root@203/204/205 ~]#/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties
[root@203 src]#jps
26632 Jps
26602 Kafka
23453 QuorumPeerMain
[root@204 src]#jps
41969 Jps
41939 Kafka
34490 QuorumPeerMain
[root@205 src]#jps
19785 QuorumPeerMain
23066 Kafka
23147 Jps

12、下載安裝kafka客戶端插件：點擊並安裝：

http://www.kafkatool.com/download.html Windows 64-Bit download

十3、one：測試建立topic：建立名爲logstashtest，partitions(分區)爲3，replication(複製)爲3的topic(主題)：在任意kafaka服務器操做：

[root@204 src]#/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --partitions 3 --replication-factor 3 --topic logstashtest
Created topic logstashtest.

two：測試獲取topic：可在任意一臺服務器進行測試：
[root@205 src]#/usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --topic logstashtest
Topic:logstashtest PartitionCount:3 ReplicationFactor:3 Configs:
Topic: logstashtest Partition: 0 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
Topic: logstashtest Partition: 1 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2
Topic: logstashtest Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
狀態說明：logstashtest有三個分區分別爲0、一、2，分區0的leader是3（broker.id），分區0有三個副本，而且狀態都爲lsr（ln-sync，表示能夠參加選舉成爲leader）。

three：刪除topic：[root@205 src]#/usr/local/kafka/bin/kafka-topics.sh --delete --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --topic logstashtest
Topic logstashtest is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

four：獲取全部的topic：[root@205 src]#/usr/local/kafka/bin/kafka-topics.sh --list --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181logstashtest #此爲topic的內容