Kafka集羣安裝和配置（二）

（一）、環境介紹
一、服務器的相關設置：
一、kafka1:172.20.67.52
二、kafka2:172.20.67.56
三、kafka3:172.20.67.57

二、zookeeper所需的三個端口
一、2181
二、2888
三、3888
三個端口的做用：
2181:對客戶端提供服務
2888:集羣內機器通信使用（Leader監聽此端口）
3888:選舉leader使用

三、kafka 所需的端口
9092：kafka所需的端口

（二）、kafka集羣的安裝配置
(1）、kafka1172.20.67.52的安裝及相關配置

一、去官網下載相應的版本（https://kafka.apache.org/downloads )。我已下載好直接使用kafka_2.11-0.11.0.3版本

二、解壓並放到相應的目錄

[root@localhost opt]# tar xf kafka_2.11-0.11.0.3.tgz
    [root@localhost opt]# mv kafka_2.11-0.11.0.3  /usr/local/kafka

三、zookeeper的相關配置

[root@localhost kafka]# vim /usr/local/kafka/config/zookeeper.properties
dataDir=/data/zookeeper
clientPort=2181
maxClientCnxns=0
tickTime=2000
initLimit=20
syncLimit=10
server.1=172.20.67.52:2888:3888
server.2=172.20.67.56:2888:3888
server.3=172.20.67.57:2888:3888

四、kafka的相關配置

[root@localhost kafka]# vim /usr/local/kafka/config/server.properties 
broker.id=1
host.name=172.20.67.52
listeners=PLAINTEXT://172.20.67.52:9092
log.dirs=/data/kafka-logs
log.retention.check.interval.ms=300000
log.retention.hours=1
log.segment.bytes=1073741824
num.io.threads=4
default.replication.factor=2
num.network.threads=3
num.partitions=30
num.recovery.threads.per.data.dir=1
port=9092
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
zookeeper.connect=172.20.67.52:2181,172.20.67.56:2181,172.20.67.57:2181
zookeeper.connection.timeout.ms=10000
fetch.message.max.bytes=52428700
replica.fetch.max.bytes=52428700

五、啓動時加入到/etc/rc.local

[root@localhost config]# vim /etc/rc.local
cd /var/tmp;nohup /usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties &
cd /tmp;nohup /usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties &
touch /var/lock/subsys/local

六、依次建立zookeeper的數據目錄/data/zookeeper和文件/data/zookeeper/myid,此時咱們須要對應的Zookeeper服務器上在dataDir的目錄下面新建myid文件，同時把對應的一、二、3寫進去。

[root@localhost config]# mkdir -p /data/zookeeper
[root@localhost config]# vim /data/zookeeper/myid
1

七、依次啓動集羣的kafka一、kafka二、kafka3 的zookeeper

[root@localhost config]# /usr/local/kafka/bin/zookeeper-server-start.sh    /usr/local/kafka/config/zookeeper.properties  &

八、而後在依次啓動集羣的kafka一、kafka二、kafka3 的kafka服務

[root@localhost config]# cd /tmp;nohup /usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties &

九、最後依次按照二、三、四、五、六、七、8的步驟依次在kafka2（172.20.67.56）和kafka3（172.20.67.57）依次安裝配置zookeeper和kafka的配置文件和啓動。

[root@Kafka1 opt]# jps
31234 Jps
30298 QuorumPeerMain
30573 Kafka

十、查看集羣所啓動的端口

kafka1
[root@Kafka1 ~]# netstat -lntp|grep java
tcp6       0      0 172.20.67.52:3888       :::*                    LISTEN      20184/java          
tcp6       0      0 127.0.0.1:9600          :::*                    LISTEN      20143/java          
tcp6       0      0 172.20.67.52:9092       :::*                    LISTEN      20649/java          
tcp6       0      0 :::35300                :::*                    LISTEN      20649/java          
tcp6       0      0 :::2181                 :::*                    LISTEN      20184/java          
tcp6       0      0 :::46471                :::*                    LISTEN      20184/java 

kafka2
[root@Kafka2 ~]# netstat -lntp|grep java
tcp6       0      0 172.20.67.56:3888       :::*                    LISTEN      30298/java          
tcp6       0      0 :::33395                :::*                    LISTEN      30298/java          
tcp6       0      0 :::43031                :::*                    LISTEN      30573/java          
tcp6       0      0 172.20.67.56:9092       :::*                    LISTEN      30573/java          
tcp6       0      0 :::2181                 :::*                    LISTEN      30298/java  

kafka3
[root@Kafka3 ~]# netstat -lntp|grep java
tcp6       0      0 172.20.67.57:3888       :::*                    LISTEN      26313/java          
tcp6       0      0 :::44194                :::*                    LISTEN      26313/java          
tcp6       0      0 172.20.67.57:9092       :::*                    LISTEN      26883/java          
tcp6       0      0 :::2181                 :::*                    LISTEN      26313/java          
tcp6       0      0 172.20.67.57:2888       :::*                    LISTEN      26313/java          
tcp6       0      0 :::36041                :::*                    LISTEN      26883/java