Spring Boot – Pass JSON Object into Kafka Topic

Last Updated : 28 Feb, 2022

Spring Boot is one of the most popular and most used frameworks of Java Programming Language. It is a microservice-based framework and to make a production-ready application using Spring Boot takes very less time. Spring Boot makes it easy to create stand-alone, production-grade Spring-based Applications that you can “just run“. So some of the main features of Spring boot are listed below.

Create stand-alone Spring applications
Embed Tomcat, Jetty, or Undertow directly.
Provide ‘starter’ dependencies to simplify the build configuration.
Configure Spring and 3rd party libraries Automatically whenever possible.
Provide production-ready features like health checks, metrics, and externalized configuration.
Almost no code generation and no requirement for XML configuration.

Apache Kafka is a publish-subscribe messaging system. A messaging system lets you send messages between processes, applications, and servers. Broadly Speaking, Apache Kafka is software where topics (A topic might be a category) can be defined and further processed. Applications may connect to this system and transfer a message onto the topic. A message can include any kind of information, from any event on your Personal blog or can be a very simple text message that would trigger any other event. Read more about Kafka prior as already in the article, Spring Boot Kafka Producer Example we have discussed how we can publish messages to Kafka topics with Spring Boot. But in a complex program, we need to pass JSON objects into Kafka topics. So let’s see how can we do it in this article.

Prerequisite: Make sure you have installed Apache Kafka in your local machine for which do go through how to Install and Run Apache Kafka on Windows?

Implementation:

Step 1: Go to this link https://start.spring.io/ and create a Spring Boot project. Add the following dependencies to your Spring Boot project.

Spring Web
Spring for Apache Kafka

Step 2: Create a simple POJO class named Book.

Java

// Java Program to Illustrate Book Class 
  
package com.amiya.kafka.apachekafkaproducer; 
  
// Class 
public class Book { 
  
    // Class data members 
    private String bookName; 
    private String isbn; 
  
    // Constructor 1 
    public Book() {} 
  
    // Constructor 2 
    public Book(String bookName, String isbn) 
    { 
        this.bookName = bookName; 
        this.isbn = isbn; 
    } 
  
    // Getter 
    public String getBookName() { return bookName; } 
  
    // Setter 
    public void setBookName(String bookName) 
    { 
        this.bookName = bookName; 
    } 
  
    // Getter 
    public String getIsbn() { return isbn; } 
  
    // Setter 
    public void setIsbn(String isbn) { this.isbn = isbn; } 
}

Step 3: Create a Configuration file named KafkaConfig

Java

// Java Program Illustrating Kafka Configurations  
  
package com.amiya.kafka.apachekafkaproducer; 
  
// Importing required classes  
import java.util.HashMap; 
import java.util.Map; 
import org.apache.kafka.clients.producer.ProducerConfig; 
import org.apache.kafka.common.serialization.StringSerializer; 
import org.springframework.context.annotation.Bean; 
import org.springframework.context.annotation.Configuration; 
import org.springframework.kafka.core.DefaultKafkaProducerFactory; 
import org.springframework.kafka.core.KafkaTemplate; 
import org.springframework.kafka.core.ProducerFactory; 
import org.springframework.kafka.support.serializer.JsonSerializer; 
  
// Annotation 
@Configuration
  
// Class 
public class KafkaConfig { 
  
    // Annotation 
    @Bean
  
    // Method 
    public ProducerFactory<String, Book> producerFactory() 
    { 
  
        // Creating a Map 
        Map<String, Object> config = new HashMap<>(); 
  
        // Adding Configuration 
  
        // 127.0.0.1:9092 is the default port number for 
        // kafka 
        config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, 
                   "127.0.0.1:9092"); 
        config.put( 
            ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, 
            StringSerializer.class); 
        config.put( 
            ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, 
            JsonSerializer.class); 
  
        return new DefaultKafkaProducerFactory<>(config); 
    } 
  
    // Annotation 
    @Bean
  
    // Method 
    public KafkaTemplate kafkaTemplate() 
    { 
  
        return new KafkaTemplate<>(producerFactory()); 
    } 
}

Step 4: Now let’s create a controller class named DemoController.

Java

// Java Program to Illustrate DemoController Class 
  
package com.amiya.kafka.apachekafkaproducer; 
  
import org.springframework.beans.factory.annotation.Autowired; 
import org.springframework.kafka.core.KafkaTemplate; 
import org.springframework.web.bind.annotation.PostMapping; 
import org.springframework.web.bind.annotation.RequestBody; 
import org.springframework.web.bind.annotation.RestController; 
  
// Annotation 
@RestController
  
// Class 
public class DemoController { 
  
    // Autowiring Kafka Template 
    @Autowired KafkaTemplate<String, Book> kafkaTemplate; 
  
    private static final String TOPIC = "NewTopic"; 
  
    // Annotation 
    @PostMapping("/publish") 
    public String publishMessage(@RequestBody Book book) 
    { 
        // Sending the message 
        kafkaTemplate.send(TOPIC, book); 
  
        return "Published Successfully"; 
    } 
}

Step 5: Now we have to do the following things in order to publish JSON Object to Kafka topics with Spring Boot

Run the Apache Zookeeper server
Run the Apache Kafka server
Listen to the JSON Object coming from the new topics

Run your Apache Zookeeper server by using this command

C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

Similarly, run your Apache Kafka server by using this command

C:\kafka>.\bin\windows\kafka-server-start.bat .\config\server.properties

Run the following command to listen to the JSON Object coming from the new topics

C:\kafka>.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic NewTopic --from-beginning

Step 6: Now run your spring boot application. Make sure you have changed the port number in the application.properties file

server.port=8081

Let’s run the Spring boot application inside the ApacheKafkaProducerApplication file

Step 7: Let’s test this URL in our Postman. Hit the following URL and in the request, body adds your data in JSON format as seen in the below image. And in the response, you can see the “Published Successfully” message has been returned.

http://localhost:8081/publish

And in real-time you can see the message has been published on the server also. The streaming of the message is in real-time.

Spring Boot | How to consume JSON messages using Apache Kafka

AmiyaRanjanRout

Improve

Article Tags :

Practice Tags :

Java

Spring Boot – Pass JSON Object into Kafka Topic

Java

Java

Java

Similar Reads

Thank You!

What kind of Experience do you want to share?