Scaling is a good problem? Programming for an exponential Growth

Chhotastock — Engineering for Scale

Background

How do you grow your money? Crypto? Digital Gold? Fixed Deposits? Mutual Funds? With less than 1% of the Indian population having had an opportunity, rather willing to be part of India’s $3.21Tn Equity market, India is preparing for a larger financial inclusion. Chhotatock is here to address the Gen-Y/Z, experience the potential of Equity investments and still feel investing is fun!! Yes you heard it right.

Context

Recently, Chhotastock launched its D2C initiative for Equity Investments in tier-2 and tier-3 cities of India. Having served more than 2L users in the last 3 months, this gave us an opportunity to re-evaluate the technologies Chhotastock has been reliant on so far, and their performance at scale.

These are the major considerations to undertake -

1. Framework / technology choices
2. Coding paradigms, patterns

With each of the above legs, a re-evaluation was required to check how scalable the systems inherently are. This exercise also helped us in figuring out the limitations of our systems, and how better do they scale.

Assumptions

1. The audience is aware about micro-service architecture, and the in-general benefits it provides over monolithic code.
2. The audience is working primarily with any one of the Functional Programming Languages (in this article — Golang)

Pre-requisite Questions

Along with the functional requirements, the following evaluations need to happen.
These cover both a) Team Evaluation, and b) Project Evaluation -

1. What is the core expertise/knowledge base of the team? Which language/framework is the team familiar with?
2. Do we have enough people, with varied skills and expertise, to avoid an echo chamber?
3. How much time will it take for the team to ramp up on new frameworks/technologies?
4. What is the scale for which the system is being designed?
5. What is the timeline that product is envisioning? What are the rough timelines from the engineering POV [including tech evaluations, POCs and considerations]?

Considering these questions before-hand will make sure that the project does not turn out to be a tug-of-war.

Business vs Market vs Product vs In

Framework/technology choices

Rome was not built in a day.

Here, we are also proposing two alternatives, and also using them in various services.

Comparison

Why Golang? 🤔

Over the last 7 years I have written production web softwares mainly in Golang, Elixir, Erlang, Java, Python & R-on-R. Why I feel Golang beats every other programming language — Next Generation Programming Language. Go is statically typed, but a number of features make it “appear” to be (at least somewhat) dynamically typed.

Type safety is actually a different “axis” of the type system. For example, Java is not type-safe because it allows a very powerful way of organising the type-space at run-time (through user-extensible class loaders ). This power can be utilised to write a program that exposes some flawed design pattern decisions in the JVM.

Google Go is also type-safe in the sense that you can’t mess around with types and crash the program (no direct access to pointers).

And the “USP” of Golang — Concurrency

Lambda vs Kubernetes vs Fargate

AWS Lambda lets your code to be more resilient when under load and in sub-optimal execution conditions.

The main advantage of using K8s is more cost predictability. You pay for resources to the cluster, that can be predicted better than in the serverless approach.

AWS Lambda are more stateless & indempotent and there is a need to shift to Fargate with the increased load & traffic. Moreover, its important to shift to Fargate if your application is latency sensitive & when you don’t need complete control of your compute instances but need control of the compute environment — CPU and memory configs

Schema Migration [TBD]

MongoDB is a document DB, and doesn’t require schema migrations [there is no fixed schema that a collection adheres to]

In case of indexes and creation of new collections, Mongock[link] is an option. Alternatively, we are also thinking of a lambda push for any ad-hoc changes.

OpenAPI Generator

OpenAPI generator saves the team from writing a lot of boiler plate code for the clients/models required for the service. I have to still to use OAS 2.0 in Go today. Because I think it’s stable and supported SDKs to generate it.

Looked at go-swagger and Swag but they’re both stuck at Swagger 2.0. Is there another package I am missing?

Ideally, I would add comments and struct tags to document our REST API and the OpenAPI spec file would then be generated from these alongside perhaps some reflection.

Mysql vs Postgres vs DocumentDB

Start with SQL and add NoSQL only when/if needed. (unless you don’t need anything beyond very simple queries)

Mysql & Postgres is not a naturally shard-able database. It provides partitioning, which helps in keeping indexes smaller, and makes searches faster.

However, with infinite scale, come infinite problems. When the data bloats up to an extent where a single server can’t process it, even partitioning is difficult to handle.

NoSQL helps in horizontal scaling by making the tables naturally shard-able, by just defining a shard key. If the access patterns of an application are not too complex [i.e., it doesn’t require too many joins, or doesn’t depend on database triggers + transactions], document DBs provide faster key-value access, and are virtually infinitely scalable.

Other considered options: MongoAtlas, ScyllaDB, Cassandra

Cache

Redis has primarily served as our time-tested caching layer, serving any subsequent and duplicate user queries from the cache layer itself. With AWS Mem-cached Redis allowing to run on-demand larger nodes with multiple cores & threads

With Mongo, we are planning to expand the use case to also act as our data store for some fast access data especially real-time stock information, and therefore, making the system less reliant on indexes.

For instance, alongside stock’s real-time data, we will be storing all user transaction IDs sorted by their timestamp in redis, such that slicing and dicing can happen on the cache data itself, and the DB need not be queried for each user request.

The DB will primarily serve one purpose — given a record ID, fetch the record details [be it for users, transactions, orders, etc]

Coding Paradigms & Patterns

There are two things that can be primarily focused on -

1. Algorithms
2. Access Patterns

There are a lot of resources available for this part of the problem, and therefore I will just mention the key points to consider.

1. Adhere to SOLID DESIGN Principles [link]
2. Avoid re-computing/re-fetching things that are already known. If data isn’t changing often, it can be cached at multiple places [client-side, CDN, server-side, DB fast access layer etc]
3. Avoid premature optimisation [link]
4. Keep the over-all UX in mind

TL;DR

1. Project the user traffic on your systems, first. Analyse, that over the next 2 years, what is the expected traffic on your application, and try and provision for at least 10x of the same. [This can be a simple projection of the current traffic].
2. Based on the user traffic, try and figure out the bottle-necks. Is there a query which is taking too long to run, or using a lot of database compute? Can the query be optimised? Can the queryable parameters be indexed, if not already?
3. Understand the inherent limitations of your system. Is the code monolithic, or has any other single point of failure? Does the architecture have inherent limitations, like too many dependencies and blocking calls? How can this be re-architectured with minimal changes?
4. DO NOT refactor code unless absolutely required. Do not over-optimise, and avoid making the life of other engineers much more complicated in the project’s future.