Apache Kafka Pub/Sub Pattern with Mule4 Part1

<BasicsOfKafka><BroadcastingPattern><CompetingPattern>

Overview of Pub/Sub

Pub/Sub is an asynchronous messaging service. Pub/Sub decouples services that produce events from services that process events. You can use Pub/Sub as messaging-oriented middleware or event ingestion and delivery for streaming analytics pipelines. In both scenarios, a publisher application creates and sends messages to a topic. Subscriber applications create a subscription to a topic to receive messages from it. A subscription is a named entity that represents an interest in receiving messages on a particular topic.

Demo To create Free Kafka with Confluent Cloud

please follow the steps in Video if you don’t have Kafka in handy for demo, where you can create Kafka topic with Confluent. Based on this link explanation I have used bootstrap server, username and password for my publish and consume connectors.

Uses Case for Demo (Video) Simple Pattern

Simple Pub/Sub Flow for Demo

Producer produces message where it is published to Kafka Topic, and message listener continuously listens to topic and consumes whenever message is available.

Anypoint Studio Flow

Anypoint Studio Flow for Pub/Sub

(1) Let's configure message to publish to kafka

Message to Publish

(2) Configuration to Publish Message to Kafka

Eg: bootstrap.server = pkc-7ojv2.us-west4.gcp.confluent.cloud:9082

Publish Kafka Configuration

(3) Once Message is published Successfully, we can Consume the Message as below.

Remember Consume Message from the topic published in previous step (topic_0)

Consume Message Configuration

(4) After Listening from Message Listener, we can consume as below.

Consumed Message.

Response After consumed

INFO 2022–10–16 17:09:39,741 [[MuleRuntime].uber.11: [demo-kafka].demo-kafkaFlow1.CPU_INTENSIVE @5282d748] [processor: demo-kafkaFlow1/processors/1; event: 38139e61–4d47–11ed-a016-cc6b1e57123d] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {

“done”: “success”,

“hi”: “{\n \”hi\”: \”rachana\”,\n \”time\”: \”2022–10–16T17:09:36.813+05:30\”\n}”

}

Youtube Links:

Demo on Apache Kafka Pub/Sub pattern with Mule4

Demo To create Free Kafka with Confluent Cloud

For the above example please refer my git Repo

git repo: racs07/demo-kafka (github.com)

git-clone: https://github.com/racs07/demo-kafka.git