Kafka

Creating and exposing custom Kafka Consumer Streaming metrics in Apache Spark using PrometheusServlet — In this blog post, I will describe how to create and enhance current Spark Structured Streaming metrics with Kafka consumer metrics and expose them using the Spark 3 PrometheusServlet that can be directly targeted by Prometheus. In previous Spark versions, one must set up either a JmxSink/JmxExporter, GraphiteSink/GraphiteExporter, or a…

In less than 5 minutes or your time back ;) Apache Kafka is an open-source distributed event streaming platform. It focuses on producer/consumer architecture. It delivers messages with latencies as low as 2ms. Highly scalable to the point of being able to manage trillions of messages per day. It has…

Kafka and Realia

When doing event-driven applications using Kafka, on the consumer side, after receiving the Kafka message, your application needs to do something with it. For this blog post, let’s call this part “Processing the message”. What is Kafka again? But before we dive into that, let’s make sure we’re all on the same page and…

Are you using windows machine for development and looking for simplest way to run Apache Kafka ? Follow these steps Step 1: Go to the Kafka Downloads page and download any Binary version as shown in below image.

by Pawash Ahuja, Senior Software Engineer at Cisco ThousandEyes Prerequisites: You have a fair understanding of Kafka Streams. You have some knowledge of Kafka Windowing. If not, we advise you to read about Apache Kafka’s Windowing. Abstract: ThousandEyes has tens of thousands of agents deployed across the globe to sense the operation…

No, this post is not sponsored by Redpanda (yet) — Last year we were building an ambitious “cryptocurrency arbitrage platform”, which, simply put, is basically buying crypto apples for $5 from some market, then selling them for $5 worth of crypto bananas from another market which then sold for $10 worth of crypto apples in yet another market. It sounds…

To set up a Kafka bus, you need to install and run a Kafka broker, and then connect producers and consumers to it. Here are the steps to set up a Kafka bus on a single machine for development or testing purposes. Install Kafka You can download the latest version of Kafka…

Hi there! Recently, one of the most common interview questions is the difference between Kafka and RabbitMq. This article will deeply dive into messaging systems and compare them by scalability, performance, and lifetime. As a result of this article, you can compare the features and choose the one that suits…