【Kafka】Apache Kafka ～ Web UI / kafka-ui編 ～

Kafka/Confluent

■ はじめに

https://dk521123.hatenablog.com/entry/2023/07/30/174239

の続き。

Apache Kafka の Web UI である kafka-ui ってのも学んだので
メモしておく。

目次

【１】Kafdrop
【２】できること
【３】サンプル
　１）動作確認

【１】Kafdrop

*  Kafka 用の Web UI
 => デモ画面は、以下の公式Githubを参照。

https://github.com/provectus/kafka-ui/blob/master/README.md#-ui-for-apache-kafka

【２】できること

https://github.com/provectus/kafka-ui/blob/master/README.md#features

[1] マルチクラスタ管理
[2] パフォーマンスモニタリングするためのメトリックスダッシュボード
[3] Kafka broker表示
[4] Topic一覧表示
[5] Consumer グループ表示
[6] メッセージ表示
[7] 動的Topic設定（Topic作成）
[8] 認証設定
[9] カスタムプラグイン
[10] アクセスコントロール
[11] データマスク

【３】サンプル

compose.yml 

version: "3"

services:
  zookeeper:
    image: docker.io/confluentinc/cp-zookeeper:latest
    hostname: zookeeper
    container_name: zookeeper
    environment:
      TZ: Asia/Tokyo
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
  broker:
    image: docker.io/confluentinc/cp-kafka:latest
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - 19092:19092
    environment:
      TZ: Asia/Tokyo
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:19092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9997
      KAFKA_JMX_HOSTNAME: broker
  cli:
    image: docker.io/confluentinc/cp-kafka:latest
    hostname: cli
    container_name: cli
    depends_on:
      - broker
    entrypoint: /bin/bash
    tty: true
  connect:
    image: docker.io/confluentinc/cp-kafka-connect-base:latest
    hostname: connect
    container_name: connect
    depends_on:
      - broker
    ports:
      - 8083:8083
    environment:
      CONNECT_BOOTSTRAP_SERVERS: broker:9092
      CONNECT_ZOOKEEPER_CONNECT: zookeeper:2181
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: "connect-group"
      CONNECT_CONFIG_STORAGE_TOPIC: "connect-config"
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 2
      CONNECT_OFFSET_STORAGE_TOPIC: "connect-offsets"
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 2
      CONNECT_STATUS_STORAGE_TOPIC: "connect-status"
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 2
      CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.storage.StringConverter"
      CONNECT_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_REST_ADVERTISED_HOST_NAME: "localhost"
  kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - 3000:8080
    depends_on:
      - broker
    restart: always
    environment:
      TZ: Asia/Tokyo
      KAFKA_CLUSTERS_0_NAME: broker-1
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: broker:19092
      KAFKA_CLUSTERS_0_METRICS_PORT: 9997

１）動作確認

[1] 起動 

sudo docker compose up -d

[2] ブラウザで表示 

* 以下のURLを参照

http://localhost:3000/

[3] 後片付け 

sudo docker compose down -v

参考文献

https://developer.mamezou-tech.com/blogs/2023/01/05/kafka-ui/

関連記事

Apache Kafka ～ 基礎知識編 ～
https://dk521123.hatenablog.com/entry/2023/04/26/103421
Apache Kafka ～ 入門編 ～
https://dk521123.hatenablog.com/entry/2023/05/01/000000
Apache Kafka ～ 環境構築 / Docker compose編 ～
https://dk521123.hatenablog.com/entry/2023/04/24/153846
Apache Kafka ～ Web UI / Kafdrop編 ～
https://dk521123.hatenablog.com/entry/2023/07/30/174239