Lydtech Consulting
Collaborate, Innovate, Excel. Successfully delivering projects across Government and Finance.
Kafka & Spring Boot Udemy Course
Lydtech’s Udemy course covers everything from the core concepts of messaging and Kafka through to step by step code walkthroughs to build a Spring Boot application that integrates with Kafka
Kafka Connect: CDC with Debezium and MongoDB
Kafka Connect: CDC with Debezium and MongoDB
When data is created, updated, or deleted on a database, a common design with event driven architectures is to emit events to Kafka topics…
Kafka Consumer Group Rebalance — The Next-Gen Protocol With Kafka 4.0
Kafka Consumer Group Rebalance — The Next-Gen Protocol With Kafka 4.0
A new generation of the consumer group protocol has been designed that is due for release in the Kafka 4.0 release, which is likely to…
Application Logging & Alerting With Graylog (2 of 2): Spring Boot Demo
Application Logging & Alerting With Graylog (2 of 2): Spring Boot Demo
Graylog is a logging system that ingests logs, provides search and filter options, and the ability to configure dashboards and alerts…
Application Logging & Alerting With Graylog (1 of 2): Introduction
Application Logging & Alerting With Graylog (1 of 2): Introduction
Having a reliable log management tool that enables querying, filtering, alerting, and visualising logs is an essential part of a system…
KafkaJS: Integration Testing With Testcontainers
KafkaJS: Integration Testing With Testcontainers
KafkaJS is a popular NodeJS Kafka client library used by applications to integrate with the Kafka messaging broker. It provides an API for…
Monitoring & Alerting: Prometheus, Grafana & Alertmanager — Part 3: Alerting Demo
Monitoring & Alerting: Prometheus, Grafana & Alertmanager — Part 3: Alerting Demo
The Prometheus Alertmanager component is the third tool in the monitoring and alerting setup that this article covers, alongside…
Monitoring & Alerting: Prometheus, Grafana & Alertmanager — Part 2: Monitoring Demo
Monitoring & Alerting: Prometheus, Grafana & Alertmanager — Part 2: Monitoring Demo
In this second part of the series on monitoring and alerting with Prometheus, Grafana and Alertmanager, the configuration required for…
Monitoring & Alerting: Prometheus, Grafana & Alertmanager — Part 1: Introduction
Monitoring & Alerting: Prometheus, Grafana & Alertmanager — Part 1: Introduction
Monitoring and alerting are an essential part of any system deployment. It ensures that the system is running smoothly, and if not that…
Three Key Principles For A Successful IT Project
Three Key Principles For A Successful IT Project
Engaging in any IT project always brings with it a myriad of challenges. Every project is different, but whether starting on a new…
Kafka Streams Windowing — Sliding Windows
Kafka Streams Windowing — Sliding Windows
A closer look at Sliding Windows
Kafka Streams Windowing — Spin it up in Kubernetes
Kafka Streams Windowing — Spin it up in Kubernetes
See the Streaming Windows running in the wild
Kafka Streams Windowing — Hopping Windows
Kafka Streams Windowing — Hopping Windows
A closer look at Hopping Windows
Kafka Streams Windowing — Tumbling Windows
Kafka Streams Windowing — Tumbling Windows
A closer look at Tumbling Windows
Kafka Streams Windowing — An Overview
Kafka Streams Windowing — An Overview
Kafka Streams builds on established concepts and claims a low barrier to entry, this may be correct, but some of those concepts are…
Kafka SASL Authentication: Spring Boot Demo
Kafka SASL Authentication: Spring Boot Demo
This article provides an overview of SASL authentication with Kafka. An accompanying Spring Boot application demonstrates the configuration…
Kafka Message Compression (2 of 2): Spring Boot Demo
Kafka Message Compression (2 of 2): Spring Boot Demo
In the first part of this two part series the reasons to apply message compression and the trade-offs in doing so were considered. In this…
Kafka Message Compression (1 of 2)
Kafka Message Compression (1 of 2)
Producers are able to compress batches of messages that are sent to the Kafka broker, which can then store them on disk in compressed…
Kafka Chaos Testing With Conduktor Gateway
Kafka Chaos Testing With Conduktor Gateway
Chaos testing validates that applications behave correctly under unexpected error conditions. Being able to simulate these error conditions…
“Simple! We’ll just dead-letter it.”
“Simple! We’ll just dead-letter it.”
In this article we investigate the use of Dead Letter Topics (DLT) in the context of Apache Kafka, although the points discussed could…
Kafka on Kubernetes — Part 3: Spring Boot Demo
Kafka on Kubernetes — Part 3: Spring Boot Demo
This is the final part in the three part series exploring deploying and integrating with Kafka on Kubernetes. In the first part an…
Kafka on Kubernetes — Part 2: Deploying Kafka
Kafka on Kubernetes — Part 2: Deploying Kafka
In the first part of this series the key components in a Kubernetes deployment were described. This second part builds on that learning to…
Kafka on Kubernetes - Part 1: Introduction to Kubernetes
Kafka on Kubernetes - Part 1: Introduction to Kubernetes
Kubernetes is a popular container orchestration tool used for managing deployments across different environments. This series of articles…
Kafka Keys, Partitions and Message Ordering
Kafka Keys, Partitions and Message Ordering
In order to guarantee ordered message delivery with the Kafka messaging broker, messages can be produced with a key. Messages with the same…
