Spring Boot - Pass JSON Object into Kafka Topic
Last Updated :
28 Feb, 2022
Spring Boot is one of the most popular and most used frameworks of Java Programming Language. It is a microservice-based framework and to make a production-ready application using Spring Boot takes very less time. Spring Boot makes it easy to create stand-alone, production-grade Spring-based Applications that you can "just run". So some of the main features of Spring boot are listed below.
- Create stand-alone Spring applications
- Embed Tomcat, Jetty, or Undertow directly.
- Provide 'starter' dependencies to simplify the build configuration.
- Configure Spring and 3rd party libraries Automatically whenever possible.
- Provide production-ready features like health checks, metrics, and externalized configuration.
- Almost no code generation and no requirement for XML configuration.
Apache Kafka is a publish-subscribe messaging system. A messaging system lets you send messages between processes, applications, and servers. Broadly Speaking, Apache Kafka is software where topics (A topic might be a category) can be defined and further processed. Applications may connect to this system and transfer a message onto the topic. A message can include any kind of information, from any event on your Personal blog or can be a very simple text message that would trigger any other event. Read more about Kafka prior as already in the article, Spring Boot Kafka Producer Example we have discussed how we can publish messages to Kafka topics with Spring Boot. But in a complex program, we need to pass JSON objects into Kafka topics. So let's see how can we do it in this article.
Prerequisite: Make sure you have installed Apache Kafka in your local machine for which do go through how to Install and Run Apache Kafka on Windows?
Implementation:
Step 1: Go to this link https://start.spring.io/ and create a Spring Boot project. Add the following dependencies to your Spring Boot project.
- Spring Web
- Spring for Apache Kafka
Step 2: Create a simple POJO class named Book.
Java
// Java Program to Illustrate Book Class
package com.amiya.kafka.apachekafkaproducer;
// Class
public class Book {
// Class data members
private String bookName;
private String isbn;
// Constructor 1
public Book() {}
// Constructor 2
public Book(String bookName, String isbn)
{
this.bookName = bookName;
this.isbn = isbn;
}
// Getter
public String getBookName() { return bookName; }
// Setter
public void setBookName(String bookName)
{
this.bookName = bookName;
}
// Getter
public String getIsbn() { return isbn; }
// Setter
public void setIsbn(String isbn) { this.isbn = isbn; }
}
Step 3: Create a Configuration file named KafkaConfig
Java
// Java Program Illustrating Kafka Configurations
package com.amiya.kafka.apachekafkaproducer;
// Importing required classes
import java.util.HashMap;
import java.util.Map;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.support.serializer.JsonSerializer;
// Annotation
@Configuration
// Class
public class KafkaConfig {
// Annotation
@Bean
// Method
public ProducerFactory<String, Book> producerFactory()
{
// Creating a Map
Map<String, Object> config = new HashMap<>();
// Adding Configuration
// 127.0.0.1:9092 is the default port number for
// kafka
config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
"127.0.0.1:9092");
config.put(
ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
StringSerializer.class);
config.put(
ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
JsonSerializer.class);
return new DefaultKafkaProducerFactory<>(config);
}
// Annotation
@Bean
// Method
public KafkaTemplate kafkaTemplate()
{
return new KafkaTemplate<>(producerFactory());
}
}
Step 4: Now let's create a controller class named DemoController.
Java
// Java Program to Illustrate DemoController Class
package com.amiya.kafka.apachekafkaproducer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
// Annotation
@RestController
// Class
public class DemoController {
// Autowiring Kafka Template
@Autowired KafkaTemplate<String, Book> kafkaTemplate;
private static final String TOPIC = "NewTopic";
// Annotation
@PostMapping("/publish")
public String publishMessage(@RequestBody Book book)
{
// Sending the message
kafkaTemplate.send(TOPIC, book);
return "Published Successfully";
}
}
Step 5: Now we have to do the following things in order to publish JSON Object to Kafka topics with Spring Boot
- Run the Apache Zookeeper server
- Run the Apache Kafka server
- Listen to the JSON Object coming from the new topics
Run your Apache Zookeeper server by using this command
C:\kafka>.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
Similarly, run your Apache Kafka server by using this command
C:\kafka>.\bin\windows\kafka-server-start.bat .\config\server.properties
Run the following command to listen to the JSON Object coming from the new topics
C:\kafka>.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic NewTopic --from-beginning
Step 6: Now run your spring boot application. Make sure you have changed the port number in the application.properties file
server.port=8081
Let's run the Spring boot application inside the ApacheKafkaProducerApplication file
Step 7: Let's test this URL in our Postman. Hit the following URL and in the request, body adds your data in JSON format as seen in the below image. And in the response, you can see the "Published Successfully" message has been returned.
http://localhost:8081/publish

And in real-time you can see the message has been published on the server also. The streaming of the message is in real-time.
Similar Reads
Spring Boot - Consume JSON Object From Kafka Topics
Apache Kafka is a publish-subscribe messaging system. A messaging system lets someone is sending messages between processes, applications, and servers. Broadly Speaking, Apache Kafka is software where topics (A topic might be a category) can be defined and further processed. Applications may connect
4 min read
YAML to List of Objects in Spring Boot
In this article, let us dive into Converting YAML files to a List of Objects in Spring boot. To understand this we need to have some knowledge of the following pre-requisites. YAML: YAML represents a key-value structure that is human-readable data serializable format. The features of YAML includingR
7 min read
Spring Boot - Customize the Jackson ObjectMapper
When using JSON format, Spring Boot will use an ObjectMapper instance to serialize responses and deserialize requests. In this article, we will take a look at the most common ways to configure the serialization and deserialization options. Let us do go through the default configuration. So by defaul
4 min read
Spring Boot | How to publish JSON messages on Apache Kafka
Apache Kafka is a publish-subscribe messaging system. A messaging queue lets you send messages between processes, applications, and servers. In this article, we will see how to send JSON messages to Apache Kafka in a spring boot application. In order to learn how to create a spring boot project, ref
4 min read
Spring Boot | How to consume JSON messages using Apache Kafka
Apache Kafka is a stream processing system that lets you send messages between processes, applications, and servers. In this article, we will see how to publish JSON messages on the console of a Spring boot application using Apache Kafka. In order to learn how to create a Spring Boot project, refer
3 min read
How to Make Put Request in Spring Boot?
Java language is one of the most popular languages among all programming languages. There are several advantages of using the Java programming language, whether for security purposes or building large distribution projects. One of the advantages of using JAVA is that Java tries to connect every conc
3 min read
Convert Json String to Java Object Using GSON
Pre-requisite: Convert Java Object to Json String Using GSONJSON Stand for JavaScript Object Notation. It's a standard text-based format which shows structured data based on JavaScript object syntax. It is commonly used for transmitting data in web applications. JSON is highly recommended to transmi
3 min read
Convert Java Object to Json String using Jackson API
JSON stands for JavaScript Object Notation. It's a standard text-based format that shows structured data based on JavaScript object syntax. It is commonly used for transmitting data in web applications. JSON is highly recommended to transmit data between a server and web application. In order to con
3 min read
Spring Boot - Spring Data JPA
Spring Data JPA or JPA stands for Java Persistence API, so before looking into that, we must know about ORM (Object Relation Mapping). So Object relation mapping is simply the process of persisting any java object directly into a database table. Usually, the name of the object being persisted become
6 min read
Spring Boot - Introduction to RESTful Web Services
RESTful Web Services REST stands for REpresentational State Transfer. It was developed by Roy Thomas Fielding, one of the principal authors of the web protocol HTTP. Consequently, REST was an architectural approach designed to make the optimum use of the HTTP protocol. It uses the concepts and verbs
5 min read