Tidying Up with Wealthsimple

Our journey to reduce clutter, spark joy, and onboard engineers faster began with a single step toward a modular monolith architecture.

Artwork by Sammy Yi

Wealthsimple clients love how easy our ecosystem is to use: even the least tech-savvy user can move money and assets into, out of and within our system with just a few clicks. Until quite recently, processes such as depositing funds from a bank account and moving assets between Wealthsimple accounts were implemented via one monolithic Ruby on Rails service. This system was developed piecemeal over several years with multiple major features added in close succession. For our rapidly growing number of clients, everything was running smoothly. Under the hood, however, we sensed things were starting to get a little complicated.

The system was making it harder and slower for us to onboard new engineers. So, we decided to untangle this “big ball of mud” architecture that constantly required more planning and testing for unintended side-effects, as development of new features continued.

That’s why we’ve been working on migrating to a new and improved system, aiming to decrease complexity and increase development efficiency. But as Marie Kondo notes, decluttering is just the beginning, the hard work comes with organizing a space thoroughly and completely. In this post, we’ll walk you through our journey toward developing in a modular monolith architecture.

A Fresh Look

The responsibilities of this system had grown considerably over time. In an effort to reduce some of this complexity, we decided to look at it through a domain-driven design lens so that we could properly model and define the boundaries between different business functions within the system. This would allow engineers to have deep knowledge only on the bounded contexts their team was responsible for, or the current project they’re working on.

Knowing we wanted to split responsibilities by bounded contexts, the next goal was to decide how to actually implement this idea. To this end, we looked into several approaches to ultimately build a better model — and learned a lot along the way, including what not to do.

Microservices

One possible approach was to move to a full microservices architecture. This made intuitive sense because much of the larger Wealthsimple ecosystem is already built using microservices. That said, this would be our first attempt at splitting this service up around bounded contexts, and we couldn’t break existing functionality. This meant that, in addition to the increase in infrastructure and CI setup required to handle new microservices, we wanted to have more flexibility.

We needed a system that would allow us to iterate on ideas quickly and make changes to our core functionality with minimal impact and effort — for example, moving logic between components. Since the microservices architecture is already broken into items with distinct deployment artifacts, this would require coordinating across multiple services. We didn’t think this was the right way to go.

That’s not to say that we’ve stopped looking at microservices as a potential solution. We may look to split our system into microservices in the future if the need arises. At this point, though, we wanted to find a simpler solution to enforce more well-defined boundaries. So, onward and upward to a new idea.

Rails Engines

The next place we looked for a solution was Rails Engines. Engines act like small Rails applications that can be included and leveraged within a larger “parent” application. As Ruby on Rails makes note of, a Rails application is actually just a ‘supercharged’ engine itself, inheriting a lot of behaviour from the Rails::Engine.

Because engines share a lot of concepts and a general structure similar to Rails applications, it provides a level of "familiarity" when working with them. Having this baked into Rails itself and running within the same root application would have alleviated some of the potential concerns of moving to a full microservices architecture, reducing complexity.

However, we quickly realized that using Engines would create additional complexity in other ways. We were concerned, for instance, about how we would deal with dependencies. Things like sharing gems that are common between different engines and handling lockfile version differences between them, or working with private gems, add complexity. Similarly, we recognized that this approach would also require changes to our testing setup. There are some solutions to these issues, such as Engems, but in the interest of simplicity we didn’t want to rely on additional external dependencies that engineers would have to learn — further increasing the time needed to get comfortable working in the codebase.

A New and Improved Solution

Ultimately, we decided to start building individual “components” within the rails application itself. With some relatively minor tweaks and taking advantage of some wonderful libraries, our current project structure is similar to the following:

Within this (simplified) structure, we also made some changes to the way that the main application functions, to help support us in clarifying the boundaries between components. For instance:

• We’re enforcing boundary constraints using packwerk. This allows us to define each component’s dependencies and test to ensure boundaries are being adhered to. Put simply, this allows us to treat components within the application as individual services, in the sense that they expose a well-defined API the consumer must go through to take advantage of the underlying business logic or process.
• We’ve also built a shared events component which provides a lightweight wrapper around wisper, allowing components to communicate more efficiently through a publish/subscribe pattern.
• Combined with Sidekiq and wisper-sidekiq, this communication between components can even happen asynchronously. This allows us to shift more work away from the web containers towards background job processes, making our solutions more scalable.

Striking a Balance

We’ve found the best part of this new modular monolith approach is it provides a good middle ground between monolith and microservice. It also means we don’t have to take on very much additional complexity around configuration, but can still benefit from cleanly separating the various business processes we are trying to model.

And so far, results have been strong. Our team has been growing quickly, and developers have been able to pick up context and make changes to the codebase efficiently. Developers have also enjoyed the experience of working within a component, as it lowers the amount of context switching and knowledge needed to work on one area, while boosting confidence in predicting side effects.

Looking to the Future

This, of course, is not the end of the journey, because our system continues to develop and change. Our new modular monolith provides a stable platform for future growth and development. Initially, this growth will focus on improving our testing setup to make things easier and our CI builds more efficient.

Once we feel comfortable with the boundaries we have, we’ll likely look to start splitting components out into proper microservices to bring in some of the added benefits — such as independent deployability and scalability, and narrowing team focus and ownership. To realize those, we’ll need to determine how to cleanly split components out into their own proper microservices.

For now, however, our modular architecture gives us a firm reset point for development, and it’s already proving much more efficient to develop in than the system it replaced.

We’re always looking for opportunities for future growth, so consider checking out some of the open roles on our Engineering team today.

Written by Nahla Davies, in collaboration with Mike Errington, Staff Engineer at Wealthsimple. Edited by Mark Adams.

Wealthsimple is a new kind of financial company. Invest, trade, save, spend, and even do your taxes in a better, simpler way. “Maker Stories” is an inside look at how we get things done. Interested in joining our team? Visit our “Work With Us” page to learn more and view open roles.

The content on this site is produced by Wealthsimple Technologies Inc. and is for informational purposes only. The content is not intended to be investment advice or any other kind of professional advice. Before taking any action based on this content you should consult a professional. We do not endorse any third parties referenced on this site. When you invest, your money is at risk and it is possible that you may lose some or all of your investment. Past performance is not a guarantee of future results. Historical returns, hypothetical returns, expected returns and images included in this content are for illustrative purposes only. Copyright © 2021 Wealthsimple Technologies Inc.