Scribd
Upload
4 views

Kafka_Installation_Documentation

This document provides a step-by-step guide for installing Kafka using Docker on an Ubuntu server. It includes prerequisites, creating a Docker network, setting up a Docker Compose file, starting services, checking their status, and accessing the Kafka UI. Additionally, it offers optional steps to verify Kafka functionality by producing and consuming messages.

Uploaded by

Tasin
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
4 views

Kafka_Installation_Documentation

This document provides a step-by-step guide for installing Kafka using Docker on an Ubuntu server. It includes prerequisites, creating a Docker network, setting up a Docker Compose file, starting services, checking their status, and accessing the Kafka UI. Additionally, it offers optional steps to verify Kafka functionality by producing and consuming messages.

Uploaded by

Tasin
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Kafka Installation Documentation

Prerequisites
- **Docker**: Ensure Docker is already installed on your Ubuntu server.
- **Docker Compose**: Ensure Docker Compose is installed on your Ubuntu server.

Step 1: Create Docker Network


Create an external Docker network for Kafka and Zookeeper:

```bash
docker network create kong_mlajan-network
```

Step 2: Create Docker Compose File


1. **Create a directory for your Kafka deployment**:
```bash
mkdir kafka-deployment
cd kafka-deployment
```

2. **Create a `docker-compose.yml` file**:


```bash
nano docker-compose.yml
```

3. **Paste the following configuration into the `docker-compose.yml` file**:

version: '3.8'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
container_name: cp-zookeeper
networks:
- kong_mlajan-network
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000

kafka:
image: confluentinc/cp-kafka:latest
hostname: kafka
container_name: cp-kafka
depends_on:
- zookeeper
networks:
- kong_mlajan-network
ports:
- 9093:9093
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS:
DOCKER_NET://kafka:9092,HOST_NET://18.219.109.108:9093
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:
DOCKER_NET:PLAINTEXT,HOST_NET:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: DOCKER_NET
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_CONFLUENT_SUPPORT_METRICS_ENABLE: 0

kafka_ui:
image: provectuslabs/kafka-ui:latest
depends_on:
- kafka
networks:
- kong_mlajan-network
ports:
- '8080:8080'
environment:
- KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:2181
- KAFKA_CLUSTERS_0_NAME=local
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092

networks:
kong_mlajan-network:
external: true

Step 3: Start Kafka and Zookeeper Services


Run the following command to start the services in detached mode:

```bash
docker-compose up -d
```

Step 4: Check Status of Services


Verify that the containers are running:
```bash
docker-compose ps
```

Step 5: Access Kafka UI


You can access the Kafka UI at:
`http://18.219.109.108:8080`

Step 6: Verify Kafka Functionality (Optional)


To ensure Kafka is functioning, you can produce and consume messages:

1. **Open a shell in the Kafka container**:


```bash
docker exec -it cp-kafka /bin/bash
```

2. **Create a topic**:
```bash
kafka-topics --create --topic test-topic --bootstrap-server kafka:9092 --partitions 1 --
replication-factor 1
```

3. **Produce messages**:
```bash
kafka-console-producer --topic test-topic --bootstrap-server kafka:9092
```
Type your messages and press `Enter`.

4. **Consume messages**:
Open another terminal and execute:
```bash
kafka-console-consumer --topic test-topic --from-beginning --bootstrap-server kafka:9092
```

```

You might also like