Forget your Microservices! The Unparalleled Benefits of Pool Architecture.

When monoliths are the solution to your scaling challenges.

Microservices are praised because they allow each feature to grow independently. While they shine in the B2C world, where traffic is uniform per user, B2B often grapples with unique traffic disparities among their users. Behind this click-bait title, I’ll present you a simpler solution, called pool architecture designed to tailor-fit scaling for each client!

A tank shared by coral reef fish and a whale shark (Georgia Aquarium)

Let’s take the example of this aquarium. There are a lot of small fish sharing their tank with a whale shark. Don’t worry, despite their name whale sharks feed exclusively on plankton. They cohabit peacefully with coral reef fish. However, there is still a problem with food distribution.

Service as a fish tank

Imagine your service is this tank. Each client is a fish, and they all share the same infrastructure (the water and the food). You can not feed each fish individually. The only option is to pour food into the tank. If you don’t put enough, the whale shark could eat everything before smaller fishes and let them starve. If you put too much food, each species will eat what they need, but some remaining food will go to waste. That’s exactly what’s happening with the CPU. If a client uses your service intensively he may DDoS the other clients (eat all the food). If you have too many instances you are just wasting money.

For instance, AWS resources are shared across clients. It means you may use the same CPU within the same datacenter as Netflix. When they release a new show, you may end up competing for resources. A service where infrastructure is shared would look like this.

Every client shares the same infrastructure.

One fish, one tank

The idea behind cell architecture is to split your system into cells: “ a collection of components, grouped from design and implementation into deployment. A cell is independently deployable, manageable, and observable” (see WSO2 website).

Domain Driven Development (DDD) splits cells based on the domain. Microservices spilt cells per function. Finding the correct boundaries, or the right size for a cell is always challenging. First iterations of cells-architecture are usually slow and don’t allow you to scale quickly.

With Pool architecture, we stretch the concept of cells. We define a simple rule for boundary: One cell per Client. In other words, each client runs its dedicated monolith, with reserved infrastructure, allowing you to scale depending on their needs.

Pool Architecture.

If I keep the metaphor, each client can now have a different water temperature, dangerous species can have their own small tank, and whales can have a huge one. Food is not a problem anymore as you can choose which tank to fill.

Using the classic 3-tier architecture of service:

3-tier architecture in pools.

👍 Benefits

• Spend money only where it’s needed, scale per client.
• If a client churns, just safely remove its infrastructure.
• Limits the radius blast of an incident. A crash only impacts one pool.
• Adds a layer of security/privacy. Client data is never shared with others.
• Easier to deploy than microservices, just deploy the entire monolith (copy/paste your terraform).
• You can deploy changes based on the client's timezone to avoid any downtime.

What is a whale? 🐳

A whale is a large client that requires a dedicated Pool. It could be technical criteria:

• A Client that runs a given threshold of requests
• A Client that performs slow or expensive requests. (upload vs download, read vs writes)

It could also be a business criteria

• A Client that pays a premium fee
• A Client with compliance requirements

Migration

Most early-stage startups choose to optimize development speed and engineering cost. That’s why so many have a monolith. But as they grow, some clients scale and become heavy users, impacting the shared infrastructure.

Moving a user from a shared to a dedicated infrastructure is not an easy task. One example of migration:

• Step 1: migrate the network layer: give the user a dedicated DNS pointing to the old infra.
• Step 2: migrate the compute layer: make them run on dedicated instances but keep the shared database.
• Step 3: migrate the data layer: move all data from the shared DB to their own.

The last step is, by experience, the most complex part. You want to extract all the data for a given user and replicate that to another DB. It usually requires complex Queries to load only what you need. It is even more complex with indexes, or composite keys.

step 0. Everyone on the same infra.

step 1 & 2. set up the new compute layer, still pointing on the old DB.

step 3. stop the traffic, copy your database and switch traffic.

In some aspects, it looks like splitting a monolith. In his book, Sam Newman mentions techniques used to migrate monoliths to microservices. They all require some code modification and complex business logic changes. Pool Architecture migrations tend to be cheaper as they mostly rely on writing Infrastructure as Code.

👎 Pitfalls

I talked about the benefits of the pool architecture, but this article wouldn’t be complete without mentioning some of the pitfalls.

1. Dedicated infrastructure is not customized code
   All your clients should share the same code. Be careful pool-architecture is NOT a way to customize code per client. When you make a deployment of a fix, deploy all clients. In the long term, it will make your life difficult if you try to maintain multiple codebases per client. As the code remains the same, you can use DB or env variables for specific needs, like the DNS name. Refrain from putting CSS in the DB to have a different look and feel. You are not building a CMS.
2. Harder to maintain
   Logs and monitoring are not centralised anymore. You need multiple accounts to connect to each client environment and find out what’s wrong. You don’t have analytics across clients anymore.
3. Move to early
   As tempting as it could be, pool architecture takes a lot of work to put in place. You need tooling. For instance, Terraform scripts to make sure Infrastructure is maintained as code. So when you make a change, every client can be easily deployed.
4. More expensive at first
   The cost of setting up a specific infrastructure per client will be important at first. Having two small databases is more expensive than a medium one at first.

Conclusion

While the concept of pool architecture presents a compelling alternative to the omnipresent microservices approach, it is imperative to acknowledge that there is no one-size-fits-all solution. Developers and architects must exercise caution before leaping onto the latest trend.

Just like in our aquarium analogy, the ecosystem we build for our services must be carefully balanced. What benefits a B2C platform may not yield the same results for a B2B service. We must consider the individual needs of our clients.

Therefore approach any new architectural pattern, with a mix of enthusiasm for innovation and a healthy dose of scepticism. Ask the right questions: Will it save resources? Is it the right time? What’s the cost of the migration?

For more pragmatic insights, be sure to follow. Stay informed, stay sceptical, and keep building responsibly.