k8s 內部的kafka要映射到外網,直接把 kafka 經過 expose 把pod 映射成服務,使用nodeport 鏈接,出現超時問題,node
解決思路:bootstrap
1. 查看zk中,kafka的註冊信息,PLAINTEXT://kafka-0.kafka-hs.sy-platform-demo.svc.cluster.local.:9093", 而應用代碼中,必須使用這個域名+端口進行鏈接,因爲k8s使用30000-32000,端口對外映射,因此9093端口必須修改api
2. kafka.yaml文件修改端口session
apiVersion: v1 kind: Service metadata: name: kafka-hs namespace: sy-platform-demo labels: app: kafka spec: ports: - port: 31902 name: server clusterIP: None selector: app: kafka --- apiVersion: v1 kind: Service metadata: namespace: sy-platform-demo name: kafka-cs labels: app: kafka spec: ports: - port: 31902 name: client selector: app: kafka --- apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: namespace: sy-platform-demo name: kafka-pdb spec: selector: matchLabels: app: kafka maxUnavailable: 1 --- apiVersion: apps/v1beta1 kind: StatefulSet metadata: namespace: sy-platform-demo name: kafka spec: serviceName: kafka-hs replicas: 3 podManagementPolicy: Parallel updateStrategy: type: RollingUpdate template: metadata: labels: app: kafka spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "app" operator: In values: - kafka topologyKey: "kubernetes.io/hostname" podAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 podAffinityTerm: labelSelector: matchExpressions: - key: "app" operator: In values: - zk topologyKey: "kubernetes.io/hostname" terminationGracePeriodSeconds: 300 containers: - name: k8skafka imagePullPolicy: Always image: 192.168.200.10/fengjian/kafka_2.11-0.10.1.1:20190326 resources: requests: memory: "3Gi" cpu: 2 ports: - containerPort: 31902 name: server command: - sh - -c - "exec /data/kafka/bin/kafka-server-start.sh /data/kafka/config/server.properties --override broker.id=${HOSTNAME##*-} \ --override listeners=PLAINTEXT://:31902 \ --override zookeeper.connect=zk-cs.sy-platform-demo.svc.cluster.local:2181 \ --override request.timeout.ms=60000 \ --override session.timeout.ms=150000 \ --override heartbeat.interval.ms=50000" env: - name: KAFKA_HEAP_OPTS value : "-Xmx1G -Xms1G" - name: KAFKA_OPTS value: "-Dlogging.level=INFO" volumeMounts: - name: datadir mountPath: /export readinessProbe: tcpSocket: port: 31902 initialDelaySeconds: 30 periodSeconds: 10 volumeClaimTemplates: - metadata: name: datadir annotations: volume.beta.kubernetes.io/storage-class: "ceph-rbd-provisioner" spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 30Gi
3. 對pod打標籤,而後映射成服務app
kubectl label pod kafka-0 kafkaInst=0 -n sy-platform-demo kubectl expose pod kafka-0 --port=31902 --target-port=31902 --name=kafka-0 --selector=kafkaInst=0 --type=NodePort -n sy-platform-demo
4. 修改kafka-0 的nodeport 端口號dom
kubectl edit service kafka-0 -n sy-platform-demo
5. 外部測試機修改host,寫入對應關係tcp
[root@node2 logs]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.20.171 master1 etcd1 192.168.20.172 master2 etcd2 192.168.20.173 master3 etcd3 192.168.20.174 node1 192.168.20.175 node2 192.168.20.176 node3 192.168.20.177 node4 192.168.20.178 node5 192.168.20.174 kafka-0.kafka-hs.senyint.svc.cluster.local. 192.168.20.174 zk-cs
6. 修改代碼文件,寫入域名ide
abcd.queue.internal.notify.kafka.producer.bootstrap.servers=kafka-0.kafka-hs.senyint.svc.cluster.local.:30010 abcd..queue.zkAddress=zk-cs:31097
7. 遺留問題測試
目前只映射了一個kafka pod, 若是所有映射,那麼須要修改端口後,保持kafka-0 ---kafka-2 的端口號不一樣ui