Building a trading bot with Deep Reinforcement Learning(DRL)

Quantitative trading involves the use of computer…

Kafka to Spark Structured Streaming, with Exactly-Once Semantics

Spark and Airflow with Kotlin

Recently, I was thinking about something new I could learn, and I ended up with two options. The first was…

Partitioning and Bucketing in Hive: Which and when?

Introducing KSQLDB!!! Database on streams

What is Ksql DB?

Data Quality Dashboards: Is Your Data Doing Ok?

Everyone loves data dashboards, right? Fancy visualisations which provide key insights into otherwise opaque data? Of course you do.

While such dashboards are becoming an increasingly commonplace tool for business…

Building end-to-end MLOps pipelines with Spark ML, MLFlow, k8s/Helm and CI/CD tools

Kafka StateStore walkthrough

In this blog, we will discuss a brief introduction to state stores in Kafka streams.

Replicating on-premise data pipeline on Azure Data Factory