Serverless at scale
A year ago, I wrote an introductory blog post about why we’d decided to use a full serverless architecture at Freetrade.
I highlighted the main benefits and reasons behind that decision at the time:
- Faster deployment and shipping
- It costs less
As we were racing to get our first version of the iOS app out to the public, and with limited engineering resources, going full serverless made sense, as it gave us a fast and unobstructed path towards the only real thing that mattered: shipping a good product, and doing it quickly.
But even though serverless proved invaluable in meeting our initial goals, how does it fare now?
Our product, the company, engineering team, and most of all our customer base have grown so much.
Do the initial promises of a serverless stack still hold at scale?
In this article, I wanted to look back and reflect on our year-old journey, see what went (and is still going) great, and also list some of the challenges that we encountered along the way.
What’s been great 😍
Still shipping fast ⏩
The minimum maintenance required to deploy code in production still constitutes a huge advantage when it comes to shipping new features, or the odd bug fix, as fast as possible. Besides the obvious fact that we don’t have to worry about running and orchestrating a fleet of containers ourselves, another great benefit we’ve seen is the ability to organically strip our code down to (almost) nothing but our domain logic.
This makes it so much easier to navigate and focus on our core product, while also minimising the number of places it can fail.
Scaling up with our user base 🙏
From our 10 first beta testers to tens of thousands of clients, and our growth accelerating, we’ve never had to care about stacking up some new users, their impact on load, or whether or not our infrastructure could support them. And although limited computing power is far from being the only restricting factor when it comes to scaling up, not having that concern hanging over your head remains a very enjoyable feeling.
Scaling up our engineering team 👩💻
Another great advantage we’ve been experiencing is how easy it’s been for new joiners to make their first contribution to the product. Because our code base is solely (mostly) domain focused, and our architecture mainly concentrates around Google Cloud Functions and Cloud Composer, it means we don’t have as many moving parts as, for instance, a fully-fledged microservice architecture.
This makes it very simple for new engineers to get their heads around how things work as a whole, start digging, and make impactful changes with the confidence that they understand all the ramifications.
Hanging out with the cool kids 😎
The Serverless space keeps moving at an incredibly fast space, maturing a little bit more with every iteration. Feeling like you’re somehow being part of that movement is very exciting for any software engineer passionate about keeping up with the latest technologies.
The advantages of a serverless architecture should be considered more broadly than just ‘Function as a Service.’ Google Cloud Platform, for instance, allows you to run a multitude of services almost instantly and with little to no configuration.
For example, we used to manage most of our scheduled workflows from inside functions. But these eventually got quite painful to manage.
The orchestration logic had become too intertwined with the domain one, which made each of them hard to understand, and therefore prone to errors and regressions.
Although we knew it was time to move those jobs out to a different system, we didn’t want to leave the elegance and flexibility of our serverless stack to plunge into the dark world of self-hosted services. A world where endless configuration files reign as kings, and computing power seems to always run dry…
Luckily, it was also the time that Google chose to release their first version of Cloud Composer: AKA exactly what we needed. The setup was (almost) painless, and after only a few days, we were sold on the product and ready to start transferring most of our complex workflows. The timing was perfect.
Cloud Composer hasn’t been the only addition to our stack, we’re constantly monitoring new Serverless, Cloud, or hosted services that come out to understand if, and how, they could be beneficial to us. Firestore, Cloud SQL, Cloud VPN, Cloud Run, are other examples of services that we’ve either started using, or are looking to introduce as our platform matures and new requirements arise.
Basically, while serverless popularises with major tech businesses, service providers are driving to scale up the variety and complexity of what you can do serverlessly.
What we’re still learning 🤔
Perfecting our development process 🏆
Because we wanted our functions to run on the same real-time database that our mobile clients connect to, we’ve been leveraging the Firebase tools to handle their deployment and configuration. Although this made it really easy for us to get started, it also requires that all your functions be deployed from the same folder inside your project, effectively mapping one codebase to one Firebase project. Eventually, it meant for us that our development and deployment lifecycle ended up looking a lot like the ones of a monolithic application, bringing about well known pitfalls that need to be paid special attention to, such as:
- Increasing build and deployment times as project grows
- An extensive and dependent codebase that can get harder to maintain
- Preventing merge conflicts
In order to deal with these issues, we’ve started splitting our functions into multiple Firebase projects. We’re also looking to get rid of the extra Firebase layer and use Google Cloud Functions directly for some parts that require greater flexibility.
As our product, codebase, and teams grow, we constantly need to come up with innovative solutions to make sure that the technology stack we’ve chosen does not become a blocker to the company expanding, but instead enables its growth. This is not always simple, and we sometimes make mistakes, but it’s also a very exciting and rewarding challenge to take on as a software engineer.
Testing and debugging in the Cloud 👩🏻🔬
Due to the level of abstraction that serverless applications are built on, it can be difficult to properly run your functions, or other cloud components, on your local machine, which eventually can make your own testing dependent on how long it takes to build and deploy the whole project to a dedicated environment in the cloud. Google and Firebase do have an emulator to run your functions locally, and even though it’s not perfect yet, it’s tremendously improved recently and now works great for testing and debugging your functions in isolation.
Because you don’t have as many frameworks and guidelines for testing serverless functions as you do with other, more traditional ways of deploying code (i.e. Java/Go Microservices), a lot of hard work has been put into making sure that our integration tests work consistently, are easy to write, and therefore protect us as much as possible against the introduction of new bugs and regressions.
Let’s be honest, we’re not entirely there yet, our code is well tested and well protected, but our pipelines can still be a bit capricious, as they currently have to rely on many external components to play nice with each other in order to work properly.
As serverless frameworks, tools, and good practices, keep maturing these issues will eventually be met with well defined and “less painful” solutions. In the meantime, just have fun trying to make things work. Who knows? You might just end up paving the way.
Autoscaling is a wild beast that sometimes needs taming 🐺
Autoscaling of your serverless functions is fantastic, but you need to remember that not every part of your system will scale with such ease. Blindly proxying requests at a massive rate to slower components, or external providers with rate limiting in place for instance, can massively hurt your application as a whole.
Cold Starts… ❄️
Everyone remotely interested in serverless technologies will have heard of cold starts, often depicted as one of the main (if not the main) problems with serverless functions running in the cloud. Cold starts are the result of your functions going dormant when they’re not getting any traffic, which is great for saving up on costs, as only running instances will be billed to you, but it also means that these functions need to be woken up later on when more traffic comes back. That waking up period can grow significantly as more and more code and dependencies need to be loaded before your functions can start receiving requests.
Cold starts are indeed real, and even though you can mitigate them, you should make sure you understand their impact on the most time sensitive parts of your application, if you want to avoid any unpleasant surprises further down the line…
Looking to the future 🗓
A lot of our most complex and exciting features — fractionals, SIPPs — will be built on serverless technology. Come to our next community meet-up if you want to hear more about that.
And if you’re already intrigued and want to help build a serverless stockbroker, head to our careers page for open engineering roles.
~Mathias, Senior Software Engineer
Freetrade does not provide investment advice and individual investors should make their own decisions or seek independent advice. The value of investments can go up as well as down and you may receive back less than your original investment. Tax laws are subject to change and may vary in how they apply depending on the circumstances.
Freetrade is a trading name of Freetrade Limited, which is a member firm of the London Stock Exchange and is authorised and regulated by the Financial Conduct Authority. Registered in England and Wales (no. 09797821).
