Implementing Quality in the Restaurant
How to make QA a integrated part of our transformation journey
When I took my role as QA Team Lead at Immoweb, Belgium’s number one Real Estate website, I faced few challenges:
- Setting up a team,
- Dealing with new technology, as the company was in the middle of a transformation journey,
- Revising a working development process by implementing a new testing methodology.
I would not only be diving into new learnings. I would also have to lead a change of process and minds.
Internet is full of articles related to what testing should be about: test automation, shifting left, continuous testing… But rarely do articles talk about the reality behind it:
How do we reach these goals and implanting new concepts in the minds of people who couldn't care less about testing?
Our Mission
When my CTO explains our technology transformation to non-technical stakeholders, he compares our company to a restaurant that needs to evolve. We first renovated the dining area (improving the design and the technology of our website). We are now focusing on serving better food (bringing qualitative classifieds and better targeted to our users).
From a QA perspective, few questions come to mind.
What is the place of QA in this restaurant metaphore?
QA ensures that the client gets what he ordered.
How?
By verifying the whole chain:
- The menu he’s reading is correct,
- The waiter understands the order and transmits it correctly to the kitchen,
- The cook picks the required ingredients, does his kitchen magic and gives the result back to the waiter,
- The waiter comes back to the client with the dish he ordered.
When?
The obvious answer would be: “Every time there’s a change in the chain”.
The more appropriate answer is: “Before there’s a change.” And that is precisely the biggest challenge I faced: being able to anticipate.
How do we do our magic?
Immoweb’s QA team is really a small one. There were only three of us when I joined the company at the end of 2018. We became four a year later, and we are now five in 2021.
We had quite a huge task ahead of us for such a team. The company was preparing to release its new website and support APIs. And it needed to be regularly and thoroughly tested.
We conducted an inventory of existing assets. Unfortunately after some attempts, we realized that we couldn’t reuse any of them. They were too complex and dramatically lacking documentation. So, even though we only had a short timeframe, we chose to develop a new test automation framework from scratch.
We selected a programming language (Python) and toolset that would enable us to connect all the layers of our infrastructure, and implement end to end testing. We chose our framework’s structure by looking at some important aspects:
- Maintainability
- Extensibility
- Flexibility
- Learning curve
- Ability to connect API, Web frontend, database
Today, a short twenty-four months later, we own a growing regression set of 700+ tests (API + Web), described using a Gherkin syntax, and automatically executed from a Jenkins CI server. Full test result logs are accessible in a html format and stored in a S3 bucket. Metrics summary is automatically reported in CloudWatch and Datadog, our company’s selection of reporting tools.
But this only covered new developments.
So we also developed automated health check tests to survey critical aspects of our production environment, so that we could automatically raise alarms in case of inconsistency.
Wait, the restaurant doesn’t serve only one customer at a time.
What happens when the restaurant welcomes a lot of hungry customers at the same time? Do we have enough tables, seats, menus? What is our real capacity? When will the waiters and kitchen start to be overloaded? When should we hire extra help?
If hundreds of people are ordering food simultaneously, will the restaurant continue to operate properly? Will our customers get the dish they ordered in an appropriate amount of time?
With the upcoming new website sitting on fully new architecture, we needed to answer these crucial questions and handle any risky situation.
So, in parallel of the automation framework, the team also developed a load test set that helps identifying performance issues and fine tuning environmental parameters.
Remember me?
While tackling technical questions, we still had to push our vision through, into the existing development workflow with a clear statement:
Let’s anticipate what’s going to be released.
We soon realized that the only way to reach our goal would be for us to be included in the discussion whenever the mere idea of a change or a new feature would rise.
How could we certify the accuracy of our system if we weren’t aware of a change on the menu? How could we be prepared if we didn’t know a new delivery service would be available for our customers?
I clearly remember the first time I attended a meeting with developers and POs. I listened to them discussing releases and deployments of new versions and features. Everyone was happy and proud of the great work delivered: Dev was ready, infra was ready, POs were happy… So everyone agreed it was a Go for production.
Do you spot the problem? You guessed it:
Where the h**l is QA?
This frame of mind was what I precisely needed to change. QA should not be an afterthought or the final bottleneck anymore. Some trust had to be established.
A long way to go
The journey started at making sure the other teams were aware that QA existed in the first place and how it could be used. That phase involved a lot of communication concerning the future QA process and its added value. I sometimes had to voice a No Go just because I had never heard of the new feature before.
Once we caught their attention, we had to bring proof of our worth. We shared the first test results, identified defects with the newly created automation framework, pointed the lack of performance of the new architecture.
Our investment was soon rewarded. The whole Immoweb team saw the benefits of our intervention and eventually changed its habits. Slowly, but surely, things shifted.
Today, things are not always perfect, but who could claim perfection?
After almost 2 years, QA is now included in early grooming. On several projects, we have prepared automated tests while the development was ongoing and we were able to give first feedbacks at the early stage of the coding. We are now only a few steps away from real TDD!
But at least, I know now that Quality is on everyone’s mind.
