Spring Boot Kafka Producer Example

Kafka Producer Example - Spring Boot 

Apache Kafka is a distributed streaming platform that is widely used for building real-time data pipelines and streaming applications. In this blog post, we will explore how to create a Kafka producer using Spring Boot, which simplifies the process of sending messages to Kafka topics. 

how to create a Kafka producer using Spring Boot

Overview of Kafka Producer

A Kafka producer is responsible for sending records to a Kafka topic. Each record consists of a key, a value, and optional metadata. The producer can send messages synchronously or asynchronously, depending on the requirements of the application.

Setting Up the Spring Boot Project

To get started, you need to set up a Spring Boot project. You can use Spring Initializr (https://start.spring.io/) to generate a new project with the following dependencies:

• Spring Web
• Spring for Apache Kafka

Once you have your project set up, you can add the necessary configurations.

Configuration

In your application.properties file, you need to configure the Kafka producer settings:

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

Creating the Kafka Producer

Next, create a service class that will handle the message sending logic. Here’s an example of a simple Kafka producer:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

@Service
public class MessageProducer {

    private final KafkaTemplate<String, String> kafkaTemplate;

    @Autowired
    public MessageProducer(KafkaTemplate<String, String> kafkaTemplate) {
        this.kafkaTemplate = kafkaTemplate;
    }

    public void sendMessage(String topic, String message) {
        kafkaTemplate.send(topic, message);
    }
}

Creating a REST Controller

To expose an endpoint for sending messages, create a REST controller:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class MessageController {

    private final MessageProducer messageProducer;

    @Autowired
    public MessageController(MessageProducer messageProducer) {
        this.messageProducer = messageProducer;
    }

    @PostMapping("/send")
    public String sendMessage(@RequestBody String message) {
        messageProducer.sendMessage("my-topic", message);
        return "Message sent to Kafka topic";
    }
}

Running the Application

1. Start your Kafka server and Zookeeper.
2. Run your Spring Boot application.
3. Use a tool like Postman to send a POST request to http://localhost:8080/send with a message in the body.

Text-Based Diagrams

Here are some text-based diagrams to illustrate the architecture:

+-------------------+
|   Spring Boot     |
|   Application     |
|                   |
| +---------------+ |
| | Message       | |
| | Controller    | |
| +---------------+ |
|        |          |
|        v          |
| +---------------+ |
| | Message       | |
| | Producer      | |
| +---------------+ |
|        |          |
|        v          |
| +---------------+ |
| | Kafka         | |
| | Template      | |
| +---------------+ |
+-------------------+
        |
        v
+--------------------+
|   Kafka Cluster    |
|                    |
| +---------------+  |
| | Topic         |  |
| | my-topic      |  |
| +---------------+  |
+--------------------+

Conclusion

In this blog post, we have covered the basics of creating a Kafka producer using Spring Boot. This setup allows you to send messages to Kafka topics easily, leveraging the power of Spring's Kafka integration. You can extend this example by adding error handling, message acknowledgment, and more complex message structures as needed.