【Kafka】Apache Kafka ～ 環境構築 / Docker compose編 ～

■ はじめに

Apache Kafka ～ 環境構築編 ～
https://dk521123.hatenablog.com/entry/2023/04/23/235534

で、Docker compose 版をいつか書こうと思ってたけど
意外と簡単だったので、
今回、Apache KafkaのDocker composeでの構築手順をまとめる

目次

【１】構築手順 - Docker compose版
　０）前提条件
　１）compose.yaml を書く
　２）「docker compose up -d」実行
【２】Quick Start
　１）Topicの作成
　２）イベントの書き込み
　３）イベントの読み込み
【３】環境破棄

【１】構築手順 - Docker compose版

* 以下を参考

https://developer.confluent.io/quickstart/kafka-docker/

０）前提条件

* Docker / Docker compose の環境が構築されていること

WSL ～ Docker-CE + Genie / 環境構築編 ～
https://dk521123.hatenablog.com/entry/2021/11/03/003836
Docker compose ～ Version 2 ～
https://dk521123.hatenablog.com/entry/2023/01/02/000000

１）compose.yaml を書く

compose.yaml

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.3.2
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-kafka:7.3.2
    container_name: broker
    ports:
    # To learn about configuring Kafka for access across networks see
    # https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/
      - "9092:9092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1

２）「docker compose up -d」実行

docker compose up -d

【２】Quick Start

１）Topicの作成

docker exec broker \
kafka-topics --bootstrap-server broker:9092 \
             --create \
             --topic quickstart

２）イベントの書き込み

docker exec --interactive --tty broker \
kafka-console-producer --bootstrap-server broker:9092 \
                       --topic quickstart

> This is my first kafka message...
> Hello world!
> This is my third kafka message. I’m on a roll :-D < Ctrl-C>

３）イベントの読み込み

docker exec --interactive --tty broker \
kafka-console-consumer --bootstrap-server broker:9092 \
                       --topic quickstart \
                       --from-beginning
<↓が表示されるはず>
This is my first kafka message...
Hello world!
This is my third kafka message. I’m on a roll :-D

【３】環境破棄

docker compose down

参考文献

https://amateur-engineer-blog.com/getting-started-kafka-with-docker-compose/

関連記事

Apache Kafka ～ 環境構築編 ～
https://dk521123.hatenablog.com/entry/2023/04/23/235534
Apache Kafka ～ 基礎知識編 ～
https://dk521123.hatenablog.com/entry/2023/04/26/103421
Apache Kafka ～ 入門編 ～
https://dk521123.hatenablog.com/entry/2023/05/01/000000
Apache Kafka ～ Kafkaコマンド ～
https://dk521123.hatenablog.com/entry/2023/05/16/000000
Apache Kafka ～ 設定値 ～
https://dk521123.hatenablog.com/entry/2023/05/28/151212
Kafka Connect ～ 基礎知識編 ～
https://dk521123.hatenablog.com/entry/2023/04/29/185133
Kafka Connect ～ 基本編 / PostgreSQL ～
https://dk521123.hatenablog.com/entry/2023/05/02/233806
WSL ～ Docker-CE + Genie / 環境構築編 ～
https://dk521123.hatenablog.com/entry/2021/11/03/003836
Docker compose ～ Version 2 ～
https://dk521123.hatenablog.com/entry/2023/01/02/000000
Amazon MSK ～ 入門編 ～
https://dk521123.hatenablog.com/entry/2023/04/21/000000
Amazon MSK ～ 基本編 / Connector ～
https://dk521123.hatenablog.com/entry/2023/05/20/003516
Scala ～ Apache Kafka / Producer ～
https://dk521123.hatenablog.com/entry/2023/04/27/235703
Scala ～ Apache Kafka / Consumer ～
https://dk521123.hatenablog.com/entry/2023/04/28/014737