【Kafka】Apache Kafka ～ Web UI / Kafdrop編 ～

Kafka/Confluent

■ はじめに

Apache Kafka の Web UI である Kafdrop を学んだので
メモしておく。

目次

【０】KafkaのWeb UI
【１】Kafdrop
【２】できること
【３】サンプル
　１）動作確認
【４】おまけ：Kafka Topics UI
　１）サンプル
　２）動作確認

【０】KafkaのWeb UI

[1] Kafdrop << ★今回のテーマ
[2] Kafka Topics UI << 「【４】おまけ：Kafka Topics UI」参照
[3] kafka-ui << 以下の関連記事を参照
etc...

Apache Kafka ～ Web UI / kafka-ui編 ～
https://dk521123.hatenablog.com/entry/2023/07/31/000152

【１】Kafdrop

*  Kafka 用の Web UI
 => デモ画面は、以下の公式Githubを参照。

https://github.com/obsidiandynamics/kafdrop#-kafdrop--kafka-web-ui--

【２】できること

https://github.com/obsidiandynamics/kafdrop#features

[1] Kafka broker表示
[2] Topic一覧表示
[3] メッセージ表示
[4] Consumer グループ表示
[5] Topic作成
[6] ACL一覧表示
etc

【３】サンプル

compose.yml 

version: "3"

services:
  zookeeper:
    image: docker.io/confluentinc/cp-zookeeper:latest
    hostname: zookeeper
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: docker.io/confluentinc/cp-kafka:latest
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_LISTENERS: CLIENT://:9092,EXTERNAL://:29092
      KAFKA_ADVERTISED_LISTENERS: CLIENT://broker:9092,EXTERNAL://localhost:29092
      KAFKA_INTER_BROKER_LISTENER_NAME: CLIENT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

  cli:
    image: docker.io/confluentinc/cp-kafka:latest
    hostname: cli
    container_name: cli
    depends_on:
      - broker
    entrypoint: /bin/bash
    tty: true

  web_ui:
    image: docker.io/obsidiandynamics/kafdrop:latest
    hostname: web_ui
    container_name: web_ui
    depends_on:
      - broker
    ports:
      - "9000:9000"
    environment:
      KAFKA_BROKERCONNECT: "broker:9092"
      JVM_OPTS: "-Xms32M -Xmx64M"
      SERVER_SERVLET_CONTEXTPATH: "/"

１）動作確認

[1] 起動 

sudo docker compose up -d

[2] ブラウザで表示 

* 以下のURLを参照

http://localhost:9000/

[3] 後片付け 

sudo docker compose down -v

【４】おまけ：Kafka Topics UI

* 以下のサイトで紹介されている「Kafka Topics UI」についても
　試してみた。

https://kazuhira-r.hatenablog.com/entry/20180108/1515413171
結果 

* Topic作成がないので、開発で使うのは、いまいちかも。

１）サンプル

version: "3"

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:4.0.0
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    networks:
      - kafka-cluster-network
  kafka:
    image: confluentinc/cp-kafka:4.0.0
    depends_on:
      - zookeeper
    container_name: kafka
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka:9092"
    networks:
      - kafka-cluster-network
  kafka-rest:
    image: confluentinc/cp-kafka-rest:4.0.0
    depends_on:
      - kafka
    container_name: kafkarest
    environment:
      KAFKA_REST_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_REST_HOST_NAME: "kafkarest"
      KAFKA_REST_LISTENERS: "http://kafkarest:8082"
    networks:
      - kafka-cluster-network
  kafka-topic-ui:
    image: landoop/kafka-topics-ui:0.9.3
    depends_on:
        - kafka-rest
    environment:
      KAFKA_REST_PROXY_URL: "http://kafkarest:8082"
      PROXY: "true"
    ports:
      - "8000:8000"
    networks:
      - kafka-cluster-network

networks:
  kafka-cluster-network:

２）動作確認

[1] 起動 

sudo docker compose up -d

[2] ブラウザで表示 

* 以下のURLを参照

http://localhost:8000/

[3] 後片付け 

sudo docker compose down -v

参考文献

https://gitlab.com/ponzmild/ird-starting-kafka-sample/-/blob/main/setup/connect-cluster/docker-compose.yaml?ref_type=heads
https://dev.classmethod.jp/articles/connect-to-kafka-with-kafdrop/

関連記事

Apache Kafka ～ 基礎知識編 ～
https://dk521123.hatenablog.com/entry/2023/04/26/103421
Apache Kafka ～ 入門編 ～
https://dk521123.hatenablog.com/entry/2023/05/01/000000
Apache Kafka ～ 環境構築 / Docker compose編 ～
https://dk521123.hatenablog.com/entry/2023/04/24/153846
Apache Kafka ～ Web UI / kafka-ui編 ～
https://dk521123.hatenablog.com/entry/2023/07/31/000152