Database sharding and managing CAP theorem tradeoffs with distributed databases

Image for post
Image for post
Photo by Taylor Vick on Unsplash

When the amount of data gets too large for our databases, it’s natural that we consider scaling them to cope with the increased load. However, depending on whether it is a SQL or NoSQL database, the techniques and principles behind scaling are different. We’ll be exploring the main ideas behind scalability for both databases. Since this article assumes some fundamental knowledge about databases and their properties, it would be helpful to review SQL/NoSQL databases.

Content Overview

  • Vertical v.s. Horizontal Scaling
  • Database Sharding
  • Scaling SQL Database
  • Scaling NoSQL Database

Vertical v.s. Horizontal Scaling

Before we jump in, let’s understand the differences between the two types of scaling.

A high-level overview of SQL/NoSQL, ACID vs. BASE consistency models, and CAP Theorem

Image for post
Image for post
Photo by fabio on Unsplash

Content Overview

  1. Overview of SQL vs NoSQL Databases
  2. ACID vs BASE Consistency Models
  3. CAP Theorem
  4. Advantages vs Disadvantages of NoSQL

SQL Databases

SQL stands for Structured Query Language. When people refer to SQL Databases, they’re actually referring to Relational Database Management System (RDBMS).

Data is stored in the form of tables. Each row is uniquely identified by a primary key and each row can also have a foreign key, which is the primary key of another table. Through the foreign key, a relation is formed with another table — hence the notion of relational databases.

Examples of SQL databases include MySQL, PostgreSQL and Oracle…

Load balancers and consistent hashing in six minutes

Image for post
Image for post
Photo by Chaitanya Tvs on Unsplash

Vertical vs. Horizontal Scaling

In a monolithic architecture, clients typically make requests to one single server. As the number of requests starts to scale, the single server does not have sufficient capacity to serve all the incoming requests.

Vertical scaling could be an option, where more CPU/RAM is added to the servers. This option could work for only so long before the hardware limitations are encountered.

In most cases, horizontal scaling, in which more servers are added, is usually a more scalable alternative.

A high-level understanding of JSON Web Tokens (JWT) and session IDs

Image for post
Image for post
Photo by ZSun Fu on Unsplash

Recently, I came across a usage of authentication with JWT in the codebase at work. I was curious to figure out the mechanism difference between JWT and the standard session token/IDs, especially since JWT has been increasingly popular in recent times.

Why Do We Need Authentication?

HTTP is a stateless protocol, which simply means the current client request is independent of any previous requests, and the server doesn’t have any knowledge of any prior requests.

For example, if you’re simple fetching static data, such as going to, the servers don’t really need to know who the client is since they’re returning a log-in page…

Dynamically loading components without directives

Image for post
Image for post
Photo by Émile Perron on Unsplash


Often times, we need to render a child view dynamically. For instance, we might have a parent view that has a tab menu, and we’d like to render the views in accordance with whichever tab is being selected.

Being new to Angular, this is something I experience, and it took me a fair amount of time to get the implementation working despite following the official Angular guide. Reading Maxim Koretskyi’s article on DOM manipulation definitely helped my understanding, so I’ll recommend you give it a read, too, if you want more in-depth explanations.

In this piece, I’ll highlight and summarize…

Seven higher-order functions to improve code readability

Image for post
Image for post
Photo by Priscilla Du Preez on Unsplash

As junior engineers, most of us have an inherent desire to build things from scratch and take more control over our code. When working with arrays and objects in JavaScript, some of us might prefer to use a for loop to iterate through the elements or keys while performing some form of logic. Well, doing so is not necessarily wrong — control loops were one of the first few things we learnt in CS classes. …

How to access nested objects and write safer code

Image for post
Image for post
Photo by Blake Connally on Unsplash

If you’re working with Javascript, chances are you might have encountered a situation where you have had to access a deeply nested object. If everything goes well, you get your data without any problems. That’s great!

However, in reality, we are not always guaranteed that the data exists, but mistakenly assume that it does. I have been in that position multiple times and I often find myself thinking: “How could that value not be populated?”

Thankfully, all we need to prevent these pesky errors is an additional check for undefined values.

Example of the Error

For those folks who are fortunate to not have…

Image for post
Image for post
Photo: Apple

This article is a product of a final project from a course NM3239 (Retrieving, Exploring, and Analysing Data) in the National University of Singapore. Since this is a Communications and New Media class, we hope our readers extend some grace when it comes to the data science process. All analyses and plots were generated in R Studio. You can check out my Github repo here.

The Power of Apple

On 12th September 2017, the Apple Event caused the share price of Energous, a wireless charging company, to drop as much as 21%. This decline in share price was induced by Apple’s latest announcement that…

Zeng Hou Lim

Excited about living my best life and becoming a better engineer. I like taking complex ideas and breaking them down.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store