I heard about a recent conference called “Modern Data Stack 2021” which seemed interesting. I watched some of their talks and learnt quite a bit. My notes below. Fantastic data products and how to build them (video link) by engineers from 4Mile Analytics and Betterhelp A data-product will enable data…

This is actually two books masquerading as one. The first is full of great advice on how to identify a genuine customer need and validate it before creating a company around it. In my opinion, you should absolutely read this part, ideally 1–2 years before setting up shop. The second…

After attending a conference called The Makers Summit a couple of weeks back where I saw two founders talk about “10x” products, I figured there must be experts who have looked for patterns in how they emerge and written about them? And sure enough, as I started reading, I discovered…

The AWS Firecracker paper is fascinating because Amazon has implemented a new Virtual Machine Monitor (VMM) which closely matches the requirements of their AWS Lambda service. While doing so, they carefully navigated the space of existing virtualization options, reused the best parts where they could, and implemented the rest. They…

Given that database transactions have been around for decades, I was surprised to find that some concepts and implementations still don’t have generally agreed-upon terminology, especially across vendors. And even the ANSI SQL standard’s description of Isolation Levels has been found to be lacking in places. I made some notes…

TiDB is an interesting new database, and PingCAP is the company driving most of its development. Recently I read the paper written about it, titled TiDB: A Raft based HTAP database, partly due to my interest in its Rust-based TiKV component. I found the paper to be comprehensive and well…

If you are familiar with Java or C/C++, understanding the assignment operator (=) in Rust will demystify a lot of its memory semantics. Assignment in Rust can have a range of outcomes depending on what you are assigning from and to, and on whether you are using the reference operator…

This post describes the features k8s provides for Automated Scaling of applications. This is Part 3 of a series. Part 1 is an Introduction to K8S and Part 2 talks about Kubernetes Internals. k8s can automate horizontal scaling of pods and has experimental support for vertical scaling. Scaling can even…

This post summarizes the internal architecture of Kubernetes (k8s). For my previous post on user-facing aspects, see Part 1 of the series. An architectural style that is followed throughout the control plane of k8s is of components communicating with each other using a data store (etcd) as the intermediary. …