Confluent Kafka

In this application, we are going to create a Kafka application with the help of Confluent Cloud and Azure Function App and its step-by-step implementation using .NET Core 7 Agenda Overview of Confluent Cloud Setup of Confluent Cloud Step-by-step implementation using .NET Core 7 Application If you don’t have any…

If you’re using the enterprise version of the Confluent platform, you have the option to enable schema validation on the broker side. It can be disastrous if the consumer is unable to deserialize messages, which is why this feature is a great way to protect your topics. But how does…

Introduction Video streaming is a crucial aspect of modern technology, with applications ranging from entertainment to security and surveillance. In this article, we’ll be discussing how to create a basic video streaming app using Confluent Kafka and Python. Prerequisites Before getting started, you’ll need to have the following installed: Python 3.x Confluent…

Security is a primary consideration for any system design. For a Kafka deployment to be production-ready, it needs to be configured with security features such as authentication, authorization, encryption, etc. This article will helps you enable RBAC authorization feature for Confluent Kafka using Metadata Service. TL;DR: Example Configuration Files Example Variables’ Values # Connection to LDAP Service LDAP_HOSTNAME="openldap.example.com" LDAP_PORT=389 LDAP_BIND_USERNAME="exampleUser" LDAP_BIND_DN="cn=${LDAP_BIND_USERNAME},ou=KafkaUsers,dc=example,dc=com" LDAP_BIND_PASSWORD="examplePassword" USERNAME_ATTRIBUTE="cn" USER_OBJECT_CLASS="person" USER_SEARCH_BASE="ou=KafkaUsers,dc=example,dc=com" PASSWORD_ATTRIBUTE="userPassword" #…

Introduction Don’t you hate it when you lose a needle in a haystack? I do, and it keeps happening to me. But before we jump to a solution lets convert it into a problem statement. Problem — You want to get access to a struct (needle) that’s nested within an array…

Using Confluent KSQL in Kafka involves configuring and running the Confluent Platform, which includes Kafka, KSQL, and other components. Setup Follow this tutorial to get Confluent Platform running on you machine (preferably the Docker approach). You should end up with http://localhost:9021/clusters running and this screen displayed in your browser. Running KSQL Now you can navigate to ksqlDB and run your KSQL queries.

Provision Confluent Cloud resources (topics, acls, connectors) using YAML Driven Configuration In the last article I provided a high level walk through on how to automate Confluent Cloud Infrastructure provisioning using terraform. …

Before we jump into the topic of Schema Registry let’s understand how data are streamed between producer and consumer. In the above diagram, we see a producer application sending data to Kafka cluster but before it sends data to…

Tutorial: Basic: How to deploy Self Managed Confluent Connect Platform Confluent Platform components Confluent Control Center Clients Connect REST Proxy ksqlDB We can connect our existing Confluent Platform components to Confluent Cloud. Confluent for Kubernetes (CFK) provides a cloud-native and automated way to deploy Confluent Platform components connected to Confluent Cloud. Connect Kafka Connect to Confluent Cloud