Testing at Casumo
Why do we test our products? How do we do it? What works for us?
In this post, I go through some of our history at Casumo to explain what led us to such a (relatively speaking) unique approach from the eyes of a software tester.
Why do we test?
Our goal at Casumo is to deliver the best possible experience to our users. There are many ways we can improve on that experience, such as providing new and exciting games weekly, 24h player support and by modernizing the look of our apps, to name a few. Writing automated tests is no exception.
When written alongside new features, tests can help determine if the code we wrote works the way we intended it to. Once in place, they will continue to serve as a safety layer for all future changes on the area being tested. They become proof that our services, components, scripts, or even the entire site work the way we designed them to, and so we are able to deploy dozens of changes daily knowing they will not negatively impact our players. If that wasn't reason enough, they also work as documentation support for when we need to better understand how an old piece of code was meant to work.
In short, tests allow us to make sure the cool things we build are in line with our own expectations; now and later.
With the why out of the way, let's take a deeper look into the different automated tests we write at Casumo and how we test.
How do we test?
We didn’t always know how our testing structure would look like, but even during the very early development stages we knew it had to follow some very basic principles:
- Tests have to be deterministic — they either always pass or always fail.
- Tests have to be integrated into the development workflow.
- Tests have to be fast!
Before I go any further, it is important to note that Casumo has never had a dedicated QA or Test team. From the moment it was founded until today (or, more accurately, up until I joined 😄), we’ve been relying on developers to write and maintain tests. While not inherently a good or bad thing, it definitely helped shape our development culture.
Testing is part of the Casumo culture
First, we embrace testing as part of development. As a tester, I don’t have to be the one to point out we need to write tests. We’ve been doing it for years, long before I joined! Developers write tests and incentivize others to do the same.
Because testing has always been a part of the daily job, it also made sense to invested time into enabling it. We’ve created the tooling and set up the infrastructure that makes testing easier. Developers will, at times, change their code to make it more testable. Today, we can even spin up entire dynamic environments on demand to help test our products.
The second effect of this organic attention to testing, and perhaps the most important, is that all developers share the pains of bad tests.
We’ve set on having tests fully integrated into our pipelines, which is great! So if a flaky test finds its way into our code base it will eventually start to cause whole builds to fail. Someone will need to spend time investigating why and eventually the flakiness will be addressed. This process usually comes with lessons learned and, as a result, we get better at writing tests.
So how do you actually test, you might ask. Well, that will depend heavily on what is being tested, but here is my take on it:
In later posts, I will go into the details of each testing technique and expand on how they've helped us catch those pesky bugs!
