Kafka集羣搭建

一、環境：

centos7

kafka_2.10-0.10.2.1

zookeeper-3.5.2

jdk1.8

二、前提：

安裝JDK直播帶貨

安裝zookeeper

三、在master上將kafka安裝包上傳到Linu的/opt/softWare/kafka目錄下並解壓

四、將/opt/softWare/kafka分發到其餘服務器

scp -r kafka slaves1:/opt/softWare/

scp -r kafka slaves2:/opt/softWare/

五、分別配置master、slaves一、slaves2中kafka中config下的server.properties

 broker.id=0
 port=6667
 host.name=master

 zookeeper.connect=master:2181,slaves1:2181,slaves2:2181

 broker.id=1
 port=6667
 host.name=slaves1

 zookeeper.connect=master:2181,slaves1:2181,slaves2:2181

 broker.id=2
 port=6667
 host.name=slaves2

 zookeeper.connect=master:2181,slaves1:2181,slaves2:2181

六、啓動zookeeper

八、啓動kafka

[root@master bin]# nohup ./kafka-server-start.sh ../config/server.properties &

[root@slaves1 bin]# nohup ./kafka-server-start.sh ../config/server.properties &

[root@slaves2 bin]# nohup ./kafka-server-start.sh ../config/server.properties &

九、建立主題

[root@master bin]# ./kafka-topics.sh --create --zookeeper master:2181,slaves1:2181,slaves2:2181 --replication-factor 3 --partitions 1 --topic mytest
Created topic "mytest".
[root@master bin]# ./kafka-topics.sh --list --zookeeper master:2181,slaves1:2181,slaves2:2181
mytest

十、測試：master生產數據、slaves一、slaves2消費數據

[root@master bin]# ./kafka-console-producer.sh --broker-list master:6667 --topic mytest
test01
test02
test02
test03

[root@slaves1 bin]# ./kafka-console-consumer.sh --zookeeper master:2181,slaves1:2181,slaves2:2181 --topic mytest --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
test01
test02
test02
test03

[root@slaves2 bin]# ./kafka-console-consumer.sh --zookeeper master:2181,slaves1:2181,slaves2:2181 --topic mytest --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
test01
test02
test02
test03