Sit back and REST for a while [Chapter One: API Testing Project]
Imagine you’re working in one of these fast-growing and probably very successful startups. Harsh business, strong competition on the market, rough times — it’s very challenging, but also certainly exciting and rewarding. You and your teammates deliver top-notch features every sprint, iterate on the technologies and processes to improve the performance, research new tools, stay up to date with latest technologies (which are not mainstream — obviously!) or even go a step further and create your own libraries and packages to boost your productivity, — you’re very effective as a team. Working from the office? It’s great, speeds up the communication and the view is really nice. Switching the whole company to work remotely basically in a single day? No problem at all![check this out: Full Remote Work (Chapter One: Collaboration Tools)]. Then comes the continuous integration, delivery, tests automation — you want it all, you want to be the next unicorn 🦄.
In many of these areas at Fresha, we already feel very confident, however, there is always room for improvement.
It isn’t any different as it comes to the E2E tests automation — I guess that most of us have been there before — the land of painful maintenance efforts, unexpected flakiness behind every corner and scalability nightmares 👻. But… does it always have to be like that? Not until you do tons of research, countless proof of concepts and finally find the right toolset for your organization to utilize. That one’s checked (you can find out more on ‘how-we-roll’ at Fresha and why we chose Cypress for our E2E testing here).
As we continued to grow, we’ve seen more awesome folks joining newly created feature teams, our tests were going green, we could effortlessly merge the code we’ve been working hard on — it all did work, we felt satisfied 😌. More and more regression test cases were being automated with a comprehensive E2E framework — it translated to less manual work and extra time to focus on actually valuable things, it’s cool, isn’t it?
However, in the distance, we started to notice a bottleneck, a set of old rusty rails that might slow down our train going full throttle right now — and we definitely didn’t want to slow down, in fact, the only way to achieve all the goals we set was to push a bit harder and go even faster. As we approached the problem it was clear to us that more E2E test coverage brought more confidence to the release, but it also meant that your CI tools needed more time to execute and evaluate them.
The good news was that we’re not the first company that encountered such issues, we didn’t have to reinvent the wheel (or maybe we actually did?), we just had to try to adjust it and fit into your organization — easy. Or… maybe not really? There’s plenty of solutions to problems like that, but what combines all of them is that you need time and resources to do so. Say, test parallelization — running a couple of CI instance processes with your testing suite simultaneously, it can save a lot of time but will require smart orchestrator mechanisms to queue test cases and avoid potential bottlenecks, but it costs money to throw better CPUs and more RAM to your AWS ecosystem — basically, just find whatever works best for you. It’s a positive change, but it does not solve all the problems, however... Here comes the all-mighty testing pyramid you’ve seen so many times on literally every testing conference, QA meetup, video course, etc. — you name it.
And yes — it may be trivial, but who cares if it actually works and if it could show you the direction that was sufficient for most IT companies? I know that its theory barely scratches the surface of our problem here, but it is good enough as a starting point for further research conducted with more organizational context. So, let’s dive deeper into situation analysis.
After a quick chat around the office with our engineers, I realized that there are no API tests so far in your stack. I had to find out more information about the architecture of your services, endpoints, specific modules and generally how things work under the hood to have a bigger picture. Confluence is the place to start, then documentation and … here we go again — another potential blocker — API documentation seemed to be residual and not very helpful. Once again, I had to talk with living human beings (seriously) — architects and tech leads 🤓. After one more iteration, I had plenty of materials to start outlining the plan to introduce that whole new level of testing.
Things to consider ✅
There are a couple of things one needs to take into account before starting any kind of Proof of Concept. Here are some points you can highlight and answer together with the team:
- What value this new level of testing will bring to my organization and development process?
- Who will be responsible for developing and maintaining these tests?
- What tools should we use (out of the box, custom, paid, free, combined)?
- How will we track the performance of these tests (benchmarks, KPIs)?
It’s just a couple of questions that will ensure you and other teammates that you understand the problem thoroughly and any decision made from this point onward will be well judged and justified both — business and technology-wise.
After countless meetings and discussions at Fresha we have come to the following conclusion:
“By starting this proof of concept we want to achieve the knowledge of whether API testing will fit into our organization and challenges ahead of us. We want it to be a complement for our current test and tech stack. We might want to replace some E2E tests with REST API tests. API tests must share as many utils, seeders and overall technology with our current solutions. They need to be scalable, easy to maintain, fast to develop and reliable”
So, that is how it all started. If you want to see how we managed the problems along the way, check the next chapter in which I will reveal more details of our first API testing attempt! 📯
