CentOS7 部署 Kafka 集羣

環境

主機名	IP	操做系統	kafka 版本
kafka224	192.168.1.224	CentOS7.5	2.12
kafka225	192.168.1.225	CentOS7.5	2.12
kafka226	192.168.1.226	CentOS7.5	2.12

• 下載 kafka_2.12-2.1.0.tgz
• zookeeper 鏈接: 192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181

---

各節點初始配置

• 關閉 selinux、防火牆
• 部署 java 運行環境
• 建立數據目錄

  mkdir -p /var/lib/kafka

• 建立日誌目錄

  mkdir -p /var/log/kafka

---

部署 kafka

• 登錄 kafka224，下載 kafka，解壓至 /opt/ 下
• 修改日誌目錄

  cd /opt/kafka/
  rm -rf logs && ln -s /var/log/kafka logs

• 修改 /opt/kafka/config/server.properties

  broker.id=224
  listeners=PLAINTEXT://192.168.1.224:9092
  log.dirs=/var/lib/kafka
  zookeeper.connect=192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181

• 啓動腳本(/opt/kafka/bin/kafka-server-start.sh)太繁瑣，我精簡了下

  #!/bin/bash
  base_dir=$(dirname $0)
  if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
      export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
  fi
  if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
      # jvm 堆內存根據實際狀況修改
      export KAFKA_HEAP_OPTS="-Xmx2G -Xms2G"
      # 開啓 jmx
      export JMX_PORT=1099
  fi
  EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}
  EXTRA_ARGS="-daemon "$EXTRA_ARGS
  exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$base_dir/../config/server.properties"

• 打包 kafka 目錄，複製到 kafka225 和 kafka226 上，並修改 server.properties

  # kafka225
  broker.id=225
  listeners=PLAINTEXT://192.168.1.225:9092
  # kafka226
  broker.id=226
  listeners=PLAINTEXT://192.168.1.226:9092

---

啓動集羣(兩種啓動方式)

• 直接啓動二進制
  • 在每一個節點上啓動 kafka 服務

    /opt/kafka/bin/kafka-server-start.sh

• systemd 啓動
  • 建立 /usr/lib/systemd/system/kafka.service

    [Unit]
    Description=Kafka
    Requires=network.service
    After=network.service
    [Service]
    Environment=JAVA_HOME=/opt/jre
    ExecStart=/opt/kafka/bin/kafka-server-start.sh
    ExecStop=/opt/kafka/bin/kafka-server-stop.sh
    Type=forking
    KillMode=none
    [Install]
    WantedBy=multi-user.target

  • 啓動 kafka 服務

    systemctl daemon-reload
    systemctl start kafka