I recently started to use BigQuery and I must admit I fell in love with the DB… This article is my attempt to explain the technology behind it, which is a requirement to efficiently utilise the DB in terms of cost and performance. BigQuery is the public implementation of Dremel…

In every startup comes the time when you have to design a large-scale ingestion of analytics events. The main idea is that you have a backend logging events and you want to have analytics insights about them. This is pretty much one of the classic data engineering tasks. The requirements…

This post is based on a talk I recently gave to my colleagues about Airflow. In particular, the focus of the talk was: what’s Airflow, what can you do with it and how it differs from Luigi. Why do you need a WMS It’s really common in a company to have to move and transform data. …

THIS IS A TRUE STORY. The events depicted in this blog post took place in London in 2018. At the request of the survivors, the function names have been changed. Out of respect for the dead code, the rest has been told exactly as it occurred. Background Today I stumbled on…

The aim of this post is to show you how amazing are empty structs in GO and I will do that showing you some examples. What’s an empty struct? An empty struct is a struct type without fields struct{}. The cool thing about an empty structure is that it occupies zero bytes of storage. …

This article has been inspired by this great talk by Dave Chaney, my personal frustration on a day at work and some other references on the net. Few lines of code to live by I do not know how many times I’ve written this code in Python. The purpose it’s easy: we have…

I stole this line from a friend of a friend working in the army…that’s his company motto and that should become the motto of any software engineer out there. An easy rephrasing would be: don’t be lazy. Hundreds of times a day, we face a choice: be lazy or do…

I love GO. I love how it is a programming language written by engineers for engineers. I love how it is focused on functionalities and forces you to think in terms of them when you engineer your code. I started using GO few months ago, when I moved from building…

Since I started to improve my software engineer skills, one thing that always puzzled me is: how to write a good function. The usual comment you hear about writing good functions is: A function should do one thing and one thing only (possibly it should do it right). I have…