Homepage
Open in app
Sign inGet started

Data Engineering Blog

  • About
  • www.bakdata.com
  • Scalable Machine Learning with Kafka Streams and KServe

    Scalable Machine Learning with Kafka Streams and KServe

    In this blog post, we demonstrate how to combine state-of-the-art stream data processing with modern ML on Kubernetes.
    Go to the profile of Jakob Edding
    Jakob Edding
    Jun 3
    Optimizing Kafka Streams Apps on Kubernetes by Splitting Topologies

    Optimizing Kafka Streams Apps on Kubernetes by Splitting Topologies

    Understanding Kafka Streams processor topologies can be essential to reduce costs and improve complex applications’ manageability
    Go to the profile of Victor Künstler
    Victor Künstler
    Sep 10, 2021
    Conversational Search in Knowledge Bases using NLP/NLU and Chatbots

    Conversational Search in Knowledge Bases using NLP/NLU and Chatbots

    Conversational AI becomes increasingly popular in the form of chatbots, especially as a means of quick information access.
    Go to the profile of Emanuel Metzenthin
    Emanuel Metzenthin
    Mar 19, 2021
    Exploring Data Pipelines in Apache Kafka with Streams Explorer

    Exploring Data Pipelines in Apache Kafka with Streams Explorer

    When working with large-scale streaming data, it is crucial to monitor your pipelines and to explore their individual parts.
    Go to the profile of Salomon Popp
    Salomon Popp
    Feb 15, 2021
    Scaling Requests to Queryable Kafka Topics with nginx

    Scaling Requests to Queryable Kafka Topics with nginx

    In this blog post, we implement custom routing logic in nginx to efficiently scale requests to queryable Kafka topics.
    Go to the profile of Torben Meyer
    Torben Meyer
    Dec 1, 2020
    Solving my weird Kafka Rebalancing Problems

    Solving my weird Kafka Rebalancing Problems

    Imagine you are working on your Kafka Streams application. You deploy it to Kubernetes, wait a few hours, and suddenly … What’s happening?
    Go to the profile of Benjamin Feldmann
    Benjamin Feldmann
    Sep 22, 2020
    Continuous NLP Pipelines with Python, Java, and Apache Kafka

    Continuous NLP Pipelines with Python, Java, and Apache Kafka

    Advancements in machine learning, data analytics, and IoT, and the business strategic shift towards real-time data-driven decision making…
    Go to the profile of Victor Künstler
    Victor Künstler
    Jul 6, 2020
    Processing Large Messages with Kafka Streams

    Processing Large Messages with Kafka Streams

    Kafka Streams is a DSL that allows easy processing of stream data stored in Apache Kafka. It abstracts from the low-level producer and…
    Go to the profile of Philipp Schirmer
    Philipp Schirmer
    Feb 20, 2020
    Implementing a Queryable User Profile Store using Kafka Streams

    Implementing a Queryable User Profile Store using Kafka Streams

    In this blog post, we introduce how Kafka Streams can be used to process real time data and expose queryable user profiles.
    Go to the profile of Torben Meyer
    Torben Meyer
    Oct 1, 2019
    Data Warehousing Made Easy with Google BigQuery and Apache Airflow

    Data Warehousing Made Easy with Google BigQuery and Apache Airflow

    In this blog post, we share how we take on BI, ELT, and DWH projects using BigQuery and Apache Airflow on Google Cloud Platform (GCP).
    Go to the profile of Marcus Baitz
    Marcus Baitz
    Jun 18, 2019
    Transparent Schema Registry for Kafka Streams

    Transparent Schema Registry for Kafka Streams

    Painlessly test Kafka Streams with Avro
    Go to the profile of Lawrence Benson
    Lawrence Benson
    Feb 22, 2019
    Fluent Kafka Streams Tests

    Fluent Kafka Streams Tests

    A Java test DSL for Kafka Streams
    Go to the profile of Arvid Heise
    Arvid Heise
    Feb 22, 2019
    Running R on AWS Lambda

    Running R on AWS Lambda

    R is still one of the most popular programming languages for data scientists. However, it is fairly hard to integrate R with modern micro…
    Go to the profile of Philipp Schirmer
    Philipp Schirmer
    Dec 5, 2018
    Queryable Kafka Topics with Kafka Streams

    Queryable Kafka Topics with Kafka Streams

    In today’s data processing architectures, Apache Kafka is often used at the ingress stage. Usually, this step is used to enrich and filter…
    Go to the profile of Robert Schmid
    Robert Schmid
    Nov 30, 2018
    First Insights into Using Amazon Neptune

    First Insights into Using Amazon Neptune

    In many real-world use cases that we as bakdata see at our customer’s sites, relevant information is hidden in the connections between…
    Go to the profile of Sven Lehmann
    Sven Lehmann
    Oct 16, 2018
    About bakdataLatest StoriesArchiveAbout MediumTermsPrivacy