Niels ClaeysindatamindedbeThe building blocks of successful Data TeamsBased on my experience I will elaborate on key criteria for building successful data teamsMay 34May 34
Niels ClaeysindatamindedbeYou can use a supercomputer to send an email but should you?Discover the next evolution in data processing with DuckDB and PolarsMar 121Mar 121
Niels ClaeysindatamindedbeMy key takeaways for building a data engineering platformHaving been a member of a product team for two years, I aim to share three valuable insights that I have gained.Feb 152Feb 152
Niels ClaeysindatamindedbeQuacking Queries in the Azure Cloud with DuckDBThis post describes 2 Duckdb extensions that enable you to read data from Azure blob storage. It also shows code for both Python and dbt.Jan 101Jan 101
Niels ClaeysindatamindedbeHow we reduced our docker build times by 40%This post describes two ways to speed up building your Docker images: caching build info remotely, using the link option when copying filesOct 4, 202315Oct 4, 202315
Niels ClaeysindatamindedbeHead-to-head comparison of dbt SQL enginesCompare usage and performance of dbt against 3 popular open-source SQL engines, namely: Spark, Trino and DuckdbSep 8, 20235Sep 8, 20235
Niels ClaeysindatamindedbeUse dbt and Duckdb instead of Spark in data pipelinesDbt has become very popular for transformation on top of your data warehouse. We see potential to use dbt with Duckdb on top of a data…Apr 12, 202316Apr 12, 202316
Niels ClaeysindatamindedbeWhy data engineers should be more like software engineersData engineers are better when using a product mindset as well as software best practices: cicd pipelines, test code, develop iteratively.Jan 24, 2023Jan 24, 2023
Niels ClaeysindatamindedbeThe rise of remote development environmentsGitpod and Codespaces are the first remote development environments that we would use ourselves and may also be useful for you.Nov 16, 2022Nov 16, 2022
Niels ClaeysindatamindedbeMake Spark resilient against spot interruptions on kubernetesBased on our experience of running spark in production at our customers, we discuss 3 ways to improve the resilience of spark on kubernetesJul 25, 2022Jul 25, 2022
