Docker部署zookeeper集羣和kafka集羣，實現互聯

本文介紹在單機上經過docker部署zookeeper集羣和kafka集羣的可操做方案。

0、準備工做

建立zk目錄，在該目錄下建立生成zookeeper集羣和kafka集羣的yml文件，以及用於在該目錄下掛載zookeeper和kafka的主要目錄。

一、建立docker網絡，用於zookeeper和kafka互聯

docker network create --driver bridge --subnet 172.168.0.0/16 --gateway 172.168.0.1  zk_network

二、經過docker-compose部署zookeeper集羣

在zk目錄下建立zookeeper.yml，編輯後保存，內容以下：

version: '3.1'

services:
  zoo1:
    image: zookeeper:3.4.11
    restart: always
    hostname: zoo1
    container_name: zoo1
    ports:
      - 2181:2181
    volumes:
    - "./zoo1/data:/data"
    - "./zoo1/datalog:/datalog"
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      zk_default:
        ipv4_address: 172.168.0.2
        
  zoo2:
    image: zookeeper:3.4.11
    restart: always
    hostname: zoo2
    container_name: zoo2
    ports:
      - 2182:2181
    volumes:
    - "./zoo2/data:/data"
    - "./zoo2/datalog:/datalog"
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      zk_default:
        ipv4_address: 172.168.0.3

  zoo3:
    image: zookeeper:3.4.11
    restart: always
    hostname: zoo3
    container_name: zoo3
    ports:
      - 2183:2181
    volumes:
    - "./zoo3/data:/data"
    - "./zoo3/datalog:/datalog"
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    networks:
      zk_default:
        ipv4_address: 172.168.0.4
        
networks:
  zk_default:
    external:
      name: zk_network         

注意，zookeeper使用的是官方3.4.11版本。保存後，執行docker-compose命令完成部署，f參數用於指定yml文件。

docker-compose -f zookeeper.yml up -d

爲了驗證zookeeper集羣是否成功部署，須要進入各容器進行驗證。

docker exec -it zoo3 /bin/sh

進入容器SSH後，執行zkServer.sh status命令查看狀態，重點查看Mode模式。

/zookeeper-3.4.11 # zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
Mode: leader

我部署後，三個狀態分別是：follower，follower，leader。

三、經過docker-compose部署kafka集羣

在zk目錄下建立kafka.yml，編輯後保存，內容以下：

version: '2'

services:
  kafka1:
    image: wurstmeister/kafka
    restart: always
    hostname: kafka1
    container_name: kafka1
    ports:
    - "9092:9092"
    - "9992:9992"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 192.168.1.16
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      JMX_PORT: 9992 
    volumes:
    - ./kafka1/logs:/kafka
    external_links:
    - zoo1
    - zoo2
    - zoo3
    networks:
      zk_default:
        ipv4_address: 172.23.0.7

  kafka2:
    image: wurstmeister/kafka
    restart: always
    hostname: kafka2
    container_name: kafka2
    ports:
    - "9093:9092"
    - "9993:9993"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 192.168.1.16
      KAFKA_ADVERTISED_PORT: 9093
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      JMX_PORT: 9993
    volumes:
    - ./kafka2/logs:/kafka
    external_links:
    - zoo1
    - zoo2
    - zoo3
    networks:
      zk_default:
        ipv4_address: 172.23.0.8

  kafka3:
    image: wurstmeister/kafka
    restart: always
    hostname: kafka3
    container_name: kafka3
    ports:
    - "9094:9092"
    - "9994:9994"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 192.168.1.16
      KAFKA_ADVERTISED_PORT: 9094
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      JMX_PORT: 9994
    volumes:
    - ./kafka3/logs:/kafka
    external_links:
    - zoo1
    - zoo2
    - zoo3
    networks:
      zk_default:
        ipv4_address: 172.23.0.9
 
networks:
  zk_default:
    external:
      name: zk_network

 保存後，執行docker-compose命令完成部署，f參數用於指定yml文件。

docker-compose -f kafka.yml up -d

四、