CentOS7 部署 Kafka 集羣

環境

主機名 IP 操做系統 kafka 版本
kafka224 192.168.1.224 CentOS7.5 2.12
kafka225 192.168.1.225 CentOS7.5 2.12
kafka226 192.168.1.226 CentOS7.5 2.12
  • 下載 kafka_2.12-2.1.0.tgz
  • zookeeper 鏈接: 192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181

各節點初始配置

  • 關閉 selinux、防火牆
  • 部署 java 運行環境
  • 建立數據目錄
    mkdir -p /var/lib/kafka
  • 建立日誌目錄
    mkdir -p /var/log/kafka

部署 kafka

  • 登錄 kafka224,下載 kafka,解壓至 /opt/ 下
  • 修改日誌目錄
    cd /opt/kafka/
    rm -rf logs && ln -s /var/log/kafka logs
  • 修改 /opt/kafka/config/server.properties
    broker.id=224
    listeners=PLAINTEXT://192.168.1.224:9092
    log.dirs=/var/lib/kafka
    zookeeper.connect=192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181
  • 啓動腳本(/opt/kafka/bin/kafka-server-start.sh)太繁瑣,我精簡了下
    #!/bin/bash
    base_dir=$(dirname $0)
    if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
        export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
    fi
    if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
        # jvm 堆內存根據實際狀況修改
        export KAFKA_HEAP_OPTS="-Xmx2G -Xms2G"
        # 開啓 jmx
        export JMX_PORT=1099
    fi
    EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}
    EXTRA_ARGS="-daemon "$EXTRA_ARGS
    exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$base_dir/../config/server.properties"
  • 打包 kafka 目錄,複製到 kafka225 和 kafka226 上,並修改 server.properties
    # kafka225
    broker.id=225
    listeners=PLAINTEXT://192.168.1.225:9092
    # kafka226
    broker.id=226
    listeners=PLAINTEXT://192.168.1.226:9092

啓動集羣(兩種啓動方式)

  • 直接啓動二進制
    • 在每一個節點上啓動 kafka 服務
      /opt/kafka/bin/kafka-server-start.sh
  • systemd 啓動
    • 建立 /usr/lib/systemd/system/kafka.service
      [Unit]
      Description=Kafka
      Requires=network.service
      After=network.service
      [Service]
      Environment=JAVA_HOME=/opt/jre
      ExecStart=/opt/kafka/bin/kafka-server-start.sh
      ExecStop=/opt/kafka/bin/kafka-server-stop.sh
      Type=forking
      KillMode=none
      [Install]
      WantedBy=multi-user.target
    • 啓動 kafka 服務
      systemctl daemon-reload
      systemctl start kafka
相關文章
相關標籤/搜索