Zookeeper和Kafka環境搭建總結

前言

因爲項目須要涉及到zookeeper和Kafka的使用，快速作了一篇筆記，方便小夥伴們搭建環境。

zookeeper

官方定義

What is ZooKeeper?

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

ZooKeeper是一個分佈式的，開放源碼的分佈式應用程序協調服務，是Google的Chubby一個開源的實現，是Hadoop和Hbase的重要組件。它是一個爲分佈式應用提供一致性服務的軟件，提供的功能包括：配置維護、域名服務、分佈式同步、組服務等。 ZooKeeper的目標就是封裝好複雜易出錯的關鍵服務，將簡單易用的接口和性能高效、功能穩定的系統提供給用戶。

zookeeper經常使用端口號：2181

環境搭建

0. 下載zookeeper：

   下載地址：mirrors.hust.edu.cn/apache/zook…

1. 在zookeeper-3.4.10目錄下，新建文件夾，並命名，例如data。(路徑爲：D:\zookeeper-3.4.10\conf\data)

2. 複製「zoo_sample.cfg」副本à並將副本重命名爲「zoo.cfg，在任意文本編輯器（eg：記事本）中打開zoo.cfg，找到並編輯dataDir=D:\\zookeeper-3.4.10\\data

3. windows的話，在cmd下運行：D:\zookeeper-3.4.13\bin>zkServer.cmd

D:\zookeeper-3.4.13\bin>zkServer.cmd

D:\zookeeper-3.4.13\bin>call "C:\Program Files\Java\jdk1.8.0_181"\bin\java "-Dzookeeper.log.dir=D:\zookeeper-3.4.13\bin\.." "-Dzookeeper.root.logger=INFO,CONSOLE" -cp "D:\zookeeper-3.4.13\bin\..\build\classes;D:\zookeeper-3.4.13\bin\..\build\lib\*;D:\zookeeper-3.4.13\bin\..\*;D:\zookeeper-3.4.13\bin\..\lib\*;D:\zookeeper-3.4.13\bin\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "D:\zookeeper-3.4.13\bin\..\conf\zoo.cfg"
2019-03-18 22:07:46,267 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: D:\zookeeper-3.4.13\bin\..\conf\zoo.cfg
2019-03-18 22:07:46,276 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2019-03-18 22:07:46,276 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2019-03-18 22:07:46,277 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2019-03-18 22:07:46,280 [myid:] - WARN  [main:QuorumPeerMain@116] - Either no config or no quorum defined in config, running  in standalone mode
2019-03-18 22:07:46,397 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: D:\zookeeper-3.4.13\bin\..\conf\zoo.cfg
2019-03-18 22:07:46,398 [myid:] - INFO  [main:ZooKeeperServerMain@98] - Starting server
2019-03-18 22:07:46,422 [myid:] - INFO  [main:Environment@100] - Server environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
2019-03-18 22:07:46,422 [myid:] - INFO  [main:Environment@100] - Server environment:host.name=DESKTOP-D5MA1P9
2019-03-18 22:07:46,422 [myid:] - INFO  [main:Environment@100] - Server environment:java.version=1.8.0_181
2019-03-18 22:07:46,423 [myid:] - INFO  [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
2019-03-18 22:07:46,423 [myid:] - INFO  [main:Environment@100] - Server environment:java.home=C:\Program Files\Java\jdk1.8.0_181\jre
2019-03-18 22:07:46,423 [myid:] - INFO  [main:Environment@100] - Server environment:java.class.path=D:\zookeeper-3.4.13\bin\..\build\classes;D:\zookeeper-3.4.13\bin\..\build\lib\*;D:\zookeeper-3.4.13\bin\..\zookeeper-3.4.13.jar;D:\zookeeper-3.4.13\bin\..\lib\audience-annotations-0.5.0.jar;D:\zookeeper-3.4.13\bin\..\lib\jline-0.9.94.jar;D:\zookeeper-3.4.13\bin\..\lib\log4j-1.2.17.jar;D:\zookeeper-3.4.13\bin\..\lib\netty-3.10.6.Final.jar;D:\zookeeper-3.4.13\bin\..\lib\slf4j-api-1.7.25.jar;D:\zookeeper-3.4.13\bin\..\lib\slf4j-log4j12-1.7.25.jar;D:\zookeeper-3.4.13\bin\..\conf
2019-03-18 22:07:46,424 [myid:] - INFO  [main:Environment@100] - Server environment:java.library.path=C:\Program Files\Java\jdk1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\nodejs\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Program Files\Java\jre1.8.0_191\bin;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Users\yzd\AppData\Local\Microsoft\WindowsApps;C:\Users\yzd\AppData\Roaming\npm;.
2019-03-18 22:07:46,425 [myid:] - INFO  [main:Environment@100] - Server environment:java.io.tmpdir=C:\Users\yzd\AppData\Local\Temp\
2019-03-18 22:07:46,425 [myid:] - INFO  [main:Environment@100] - Server environment:java.compiler=<NA>
2019-03-18 22:07:46,427 [myid:] - INFO  [main:Environment@100] - Server environment:os.name=Windows 10
2019-03-18 22:07:46,427 [myid:] - INFO  [main:Environment@100] - Server environment:os.arch=amd64
2019-03-18 22:07:46,427 [myid:] - INFO  [main:Environment@100] - Server environment:os.version=10.0
2019-03-18 22:07:46,428 [myid:] - INFO  [main:Environment@100] - Server environment:user.name=yzd
2019-03-18 22:07:46,428 [myid:] - INFO  [main:Environment@100] - Server environment:user.home=C:\Users\yzd
2019-03-18 22:07:46,428 [myid:] - INFO  [main:Environment@100] - Server environment:user.dir=D:\zookeeper-3.4.13\bin
2019-03-18 22:07:46,438 [myid:] - INFO  [main:ZooKeeperServer@836] - tickTime set to 2000
2019-03-18 22:07:46,438 [myid:] - INFO  [main:ZooKeeperServer@845] - minSessionTimeout set to -1
2019-03-18 22:07:46,439 [myid:] - INFO  [main:ZooKeeperServer@854] - maxSessionTimeout set to -1
2019-03-18 22:07:46,647 [myid:] - INFO  [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2019-03-18 22:07:46,651 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
複製代碼

搭建成功

zookeeper和eureka比較

ZooKeeper、Eureka對比：

www.cnblogs.com/jieqing/p/8…

CAP定理（C-數據一致性；A-服務可用性；P-服務對網絡分區故障的容錯性，這三個特性在任何分佈式系統中不能同時知足，最多同時知足兩個）

ZooKeeper基於CP，不保證高可用，若是zookeeper正在選主，或者Zookeeper集羣中半數以上機器不可用，那麼將沒法得到數據。Eureka基於AP，能保證高可用，即便全部機器都掛了，也能拿到本地緩存的數據。做爲註冊中心，其實配置是不常常變更的，只有發版和機器出故障時會變。對於不常常變更的配置來講，CP是不合適的，而AP在遇到問題時能夠用犧牲一致性來保證可用性，既返回舊數據，緩存數據。

因此理論上Eureka是更適合做註冊中心。而現實環境中大部分項目可能會使用ZooKeeper，那是由於集羣不夠大，而且基本不會遇到用作註冊中心的機器一半以上都掛了的狀況。因此實際上也沒什麼大問題。

kfaka

Kafka® is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.

Kafka是由Apache軟件基金會開發的一個開源流處理平臺，由Scala和Java編寫。Kafka是一種高吞吐量的分佈式發佈訂閱消息系統，它能夠處理消費者規模的網站中的全部動做流數據。 這種動做（網頁瀏覽，搜索和其餘用戶的行動）是在現代網絡上的許多社會功能的一個關鍵因素。 這些數據一般是因爲吞吐量的要求而經過處理日誌和日誌聚合來解決。 對於像Hadoop同樣的日誌數據和離線分析系統，但又要求實時處理的限制，這是一個可行的解決方案。Kafka的目的是經過Hadoop的並行加載機制來統一線上和離線的消息處理，也是爲了經過集羣來提供實時的消息。

kafka包內自帶整合好的zookeeper，因此若是要搭建kafka能夠省去單獨下載zookeeper的步驟。想起了欲練武功必先自宮的笑話，哈哈哈。

環境搭建

0. 下載地址：kafka.apache.org/downloads.h…
1. 解壓縮，創建一個空文件夾 logs. eg: D:\kafka_2.11-1.0.0\logs
2. 進入config目錄，編輯 server.properties文件(eg: 用「寫字板」打開)。找到並編輯log.dirs= D:\kafka_2.11-1.0.0\logs
3. 請確保在啓動Kafka服務器前，Zookeeper實例已經準備好並開始運行。
4. 在此處打開Powershell窗口，運行：.\bin\windows\kafka-server-start.bat .\config\server.properties

PS D:\kafka_2.12-2.1.1> .\bin\windows\kafka-server-start.bat .\config\server.properties
[2019-03-18 22:15:34,866] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2019-03-18 22:15:35,503] INFO starting (kafka.server.KafkaServer)
[2019-03-18 22:15:35,505] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2019-03-18 22:15:35,530] INFO [ZooKeeperClient] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2019-03-18 22:15:35,536] INFO Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,536] INFO Client environment:host.name=DESKTOP-D5MA1P9 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,536] INFO Client environment:java.version=1.8.0_181 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,537] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,537] INFO Client environment:java.home=C:\Program Files\Java\jdk1.8.0_181\jre (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,537] INFO Client environment:java.class.path=D:\kafka_2.12-2.1.1\libs\activation-1.1.1.jar;D:\kafka_2.12-2.1.1\libs\aopalliance-repackaged-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\argparse4j-0.7.0.jar;D:\kafka_2.12-2.1.1\libs\audience-annotations-0.5.0.jar;D:\kafka_2.12-2.1.1\libs\commons-lang3-3.8.1.jar;D:\kafka_2.12-2.1.1\libs\connect-api-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-basic-auth-extension-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-file-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-json-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-runtime-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-transforms-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\guava-20.0.jar;D:\kafka_2.12-2.1.1\libs\hk2-api-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\hk2-locator-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\hk2-utils-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\jackson-annotations-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-core-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-databind-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-jaxrs-base-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-jaxrs-json-provider-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-module-jaxb-annotations-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\javassist-3.22.0-CR2.jar;D:\kafka_2.12-2.1.1\libs\javax.annotation-api-1.2.jar;D:\kafka_2.12-2.1.1\libs\javax.inject-1.jar;D:\kafka_2.12-2.1.1\libs\javax.inject-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\javax.servlet-api-3.1.0.jar;D:\kafka_2.12-2.1.1\libs\javax.ws.rs-api-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\javax.ws.rs-api-2.1.jar;D:\kafka_2.12-2.1.1\libs\jaxb-api-2.3.0.jar;D:\kafka_2.12-2.1.1\libs\jersey-client-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-common-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-container-servlet-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-container-servlet-core-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-hk2-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-media-jaxb-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-server-2.27.jar;D:\kafka_2.12-2.1.1\libs\jetty-client-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-continuation-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-http-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-io-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-security-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-server-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-servlet-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-servlets-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-util-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jopt-simple-5.0.4.jar;D:\kafka_2.12-2.1.1\libs\kafka-clients-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-log4j-appender-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-examples-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-scala_2.12-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-test-utils-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-tools-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-javadoc.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-javadoc.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-scaladoc.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-scaladoc.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-sources.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-sources.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test-sources.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test-sources.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1.jar.asc;D:\kafka_2.12-2.1.1\libs\log4j-1.2.17.jar;D:\kafka_2.12-2.1.1\libs\lz4-java-1.5.0.jar;D:\kafka_2.12-2.1.1\libs\maven-artifact-3.6.0.jar;D:\kafka_2.12-2.1.1\libs\metrics-core-2.2.0.jar;D:\kafka_2.12-2.1.1\libs\osgi-resource-locator-1.0.1.jar;D:\kafka_2.12-2.1.1\libs\plexus-utils-3.1.0.jar;D:\kafka_2.12-2.1.1\libs\reflections-0.9.11.jar;D:\kafka_2.12-2.1.1\libs\rocksdbjni-5.14.2.jar;D:\kafka_2.12-2.1.1\libs\scala-library-2.12.7.jar;D:\kafka_2.12-2.1.1\libs\scala-logging_2.12-3.9.0.jar;D:\kafka_2.12-2.1.1\libs\scala-reflect-2.12.7.jar;D:\kafka_2.12-2.1.1\libs\slf4j-api-1.7.25.jar;D:\kafka_2.12-2.1.1\libs\slf4j-log4j12-1.7.25.jar;D:\kafka_2.12-2.1.1\libs\snappy-java-1.1.7.2.jar;D:\kafka_2.12-2.1.1\libs\validation-api-1.1.0.Final.jar;D:\kafka_2.12-2.1.1\libs\zkclient-0.11.jar;D:\kafka_2.12-2.1.1\libs\zookeeper-3.4.13.jar;D:\kafka_2.12-2.1.1\libs\zstd-jni-1.3.7-1.jar (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,538] INFO Client environment:java.library.path=C:\Program Files\Java\jdk1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\nodejs\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Program Files\Java\jre1.8.0_191\bin;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Users\yzd\AppData\Local\Microsoft\WindowsApps;C:\Users\yzd\AppData\Roaming\npm;. (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,539] INFO Client environment:java.io.tmpdir=C:\Users\yzd\AppData\Local\Temp\ (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,542] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,542] INFO Client environment:os.name=Windows 10 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,543] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,543] INFO Client environment:os.version=10.0 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,547] INFO Client environment:user.name=yzd (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,547] INFO Client environment:user.home=C:\Users\yzd (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,547] INFO Client environment:user.dir=D:\kafka_2.12-2.1.1 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,549] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@3e2e18f2 (org.apache.zookeeper.ZooKeeper)
[2019-03-18 22:15:35,570] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2019-03-18 22:15:35,570] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2019-03-18 22:15:35,574] INFO Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2019-03-18 22:15:35,598] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x1000ae3e3690000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2019-03-18 22:15:35,602] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.ZooKeeperClient)
[2019-03-18 22:15:35,974] INFO Cluster ID = brvhJ2K3R4qZnGJ6CQhghA (kafka.server.KafkaServer)
[2019-03-18 22:15:35,980] WARN No meta.properties file under dir D:\kafka_2.12-2.1.1\logs\meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2019-03-18 22:15:36,049] INFO KafkaConfig values:
        advertised.host.name = null
        ...(省略若干)
        zookeeper.sync.time.ms = 2000
 (kafka.server.KafkaConfig)
[2019-03-18 22:15:36,092] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2019-03-18 22:15:36,092] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2019-03-18 22:15:36,094] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2019-03-18 22:15:36,132] INFO Loading logs. (kafka.log.LogManager)
[2019-03-18 22:15:36,143] INFO Logs loading complete in 11 ms. (kafka.log.LogManager)
[2019-03-18 22:15:36,161] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2019-03-18 22:15:36,166] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2019-03-18 22:15:36,486] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2019-03-18 22:15:36,534] INFO [SocketServer brokerId=0] Started 1 acceptor threads (kafka.network.SocketServer)
[2019-03-18 22:15:36,562] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-18 22:15:36,563] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-18 22:15:36,563] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-18 22:15:36,581] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2019-03-18 22:15:36,607] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)
[2019-03-18 22:15:36,612] INFO Result of znode creation at /brokers/ids/0 is: OK (kafka.zk.KafkaZkClient)
[2019-03-18 22:15:36,613] INFO Registered broker 0 at path /brokers/ids/0 with addresses: ArrayBuffer(EndPoint(DESKTOP-D5MA1P9,9092,ListenerName(PLAINTEXT),PLAINTEXT)) (kafka.zk.KafkaZkClient)
[2019-03-18 22:15:36,615] WARN No meta.properties file under dir D:\kafka_2.12-2.1.1\logs\meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2019-03-18 22:15:36,682] INFO [ExpirationReaper-0-topic]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-18 22:15:36,684] INFO [ExpirationReaper-0-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-18 22:15:36,685] INFO [ExpirationReaper-0-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-18 22:15:36,689] INFO Successfully created /controller_epoch with initial epoch 0 (kafka.zk.KafkaZkClient)
[2019-03-18 22:15:36,714] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.group.GroupCoordinator)
[2019-03-18 22:15:36,716] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.group.GroupCoordinator)
[2019-03-18 22:15:36,731] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 15 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2019-03-18 22:15:36,748] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 (kafka.coordinator.transaction.ProducerIdManager)
[2019-03-18 22:15:36,780] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-03-18 22:15:36,782] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-03-18 22:15:36,791] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2019-03-18 22:15:36,875] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2019-03-18 22:15:36,892] INFO [SocketServer brokerId=0] Started processors for 1 acceptors (kafka.network.SocketServer)
[2019-03-18 22:15:36,901] INFO Kafka version : 2.1.1 (org.apache.kafka.common.utils.AppInfoParser)
[2019-03-18 22:15:36,919] INFO Kafka commitId : 21234bee31165527 (org.apache.kafka.common.utils.AppInfoParser)
[2019-03-18 22:15:36,922] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
複製代碼

搭建成功。

測試kafka生產者消費者

0. 開啓zookeeper和kafka服務

1. 建立topic：

> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
複製代碼

看一眼剛纔建立的主題

> bin/kafka-topics.sh --list --zookeeper localhost:2181
test
複製代碼

2. 發消息：接收者

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
test
複製代碼

3. 收消息：消費者

> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
This is a message
This is another message
複製代碼

參考

blog.csdn.net/tianmanchn/…

www.jianshu.com/p/ca60bc8c5…

關注我

我是蠻三刀把刀，碩士，後端開發工程師。主要關注後端開發，數據安全，爬蟲等方向。微信：yangzd1102

Github我的主頁：

github.com/qqxx6661

原創博客主要內容

• Java知識點複習全手冊
• Leetcode算法題解析
• 劍指offer算法題解析
• Python爬蟲相關技術實戰
• 後端開發相關技術實戰
• SpringCloud實戰

我的公衆號：Rude3Knife

若是文章對你有幫助，不妨收藏起來並轉發給您的朋友們~