Software Quality & Craftsmanship at ManoMano
Almost two years ago, we published an article presenting the Quality Assurance strategy we were putting in place at the time. While most of the concepts we presented still stand to this day, the philosophy and role of our team have evolved so much that we thought it would be a good idea to share an update.
ManoMano continues its rapid growth as the leading European online marketplace for DIY, Home Improvement and Gardening. Delivering high quality software while keeping the pace has become an obligation. We’ve been struggling to recruit the right QA engineers. We faced the limits of a model where the quality gates put in place by a team of dedicated testers were immediately defeated by an uninterrupted stream of changes. We realised — the hard way — that testing alone doesn’t improve quality. All these factors contributed to a drastic shift of paradigm in our QA strategy.
A holistic approach
Collaborating on the design, development and operation of a software platform is exciting and rewarding, but it’s also very challenging due to the immense complexity of the interactions involved.
Traditional approaches of software development are mostly variations of the intuitive yet outdated waterfall model, where things happen in a well-established, sequential order. Starting from the 80’s, the idea of developing and delivering software by small increments started to emerge. It’s really with the Manifesto for Agile Software Development in 2001 that things began to accelerate, and during the last decades we have seen a wide adoption of Scrum as the go-to methodology for a vast majority of software development teams across the world.
While frameworks like Scrum can be a very effective reference for optimising the delivery flow of a small development team, they don’t provide any guidance when it comes to quality. As a consequence, most teams take what they have learnt in the past (Waterfall, V-Cycle…) and try to superimpose it on Scrum. They usually end up with a mini-waterfall process for everything related to quality. And in that context, “quality” is often dumbed-down to a set of black box checks designed and executed by dedicated persons (the testers).
Been there, done that
This model quickly finds its limits. It focuses on detecting superficial issues in a supposedly finished product, which is a highly ineffective approach. When bugs are eventually found (ironically meaning the testers did a good job), developers have to come back to and fix what was considered done already. The release is delayed, everyone is frustrated. Management wants an explanation. Business asks for a revised ETA. Upcoming developments are postponed. Or much worse, the team decides to ship anyway in order to meet deadlines, even though it harms the quality of the product delivered to customers.
A team shall not have to choose between quality and speed. A team wants both. Quality must be built in.
Quality x Craft = QRAFT
In September 2020, we redefined the mission of our QA team with the strong conviction that we’d get better results helping other teams improve the way they work, rather than acting as a bottleneck down the delivery stream.
To be in capacity for undertaking that mission, we renamed to QRAFT (credits to Aurélien LAJOIE for that name) and widened our scope by defining four complementary areas:
Software Craftsmanship is about bringing professionalism, technical excellence and ethics to Software Engineering. Our internal Craftsmanship coaches help developers better their working habits and sharpen their techniques. Coaches are neither instructors nor consultants, their job is to help developers reveal their full potential.
Quality Assistance: we moved away from the traditional concept of Quality Assurance to highlight the fact that we are here to support teams with their quality, not to sign off their software releases. Making teams responsible and autonomous on ensuring the quality of their productions is beneficial to everyone. The concept has been promoted and well documented by Atlassian. This model is also much more scalable, which is a must-have due to our business context.
Quality Engineering: our QEs are software development engineers specialised in quality-related topics. Their focus — at the time of writing — is on improving DX (Developer eXperience). If you’re not familiar with the concept, you can think of it as UX (User eXperience) but for Developers. The idea is to provide developers with better development tools, infrastructure, training and support.
Quality Management / Governance: this is the “control tower” of quality. Here we define the overall quality improvement strategy, align with leadership on a global vision and enable their decision-making based on meaningful quality metrics.
Our impact on quality is indirect
Even though combining those forces allows for a holistic approach, it is important — and somewhat counterintuitive — to understand that nothing we do has a direct benefit on the quality perceived by our users. While we are here to encourage and empower product development teams to implement continuous improvement, they are the ones that concretely take the quality of ManoMano to the next level.
Results
We’ve been monitoring the number of known defects in Production before and after adopting this new strategy. This not an ideal metric because it doesn’t give a direct indication of level of quality, but it did the job.
The first few months of adopting Quality Assistance have been brutally demotivating: there was no visible improvement, known defects were piling up in backlogs. Leading the change is very hard when there’s zero visibility and no guarantee of success.
But after 6 months, it started to pay off. Several teams started to put a strong focus on getting rid of defects, they embraced the idea that test automation is a developer thing and that quality was a whole team’s matter. Thanks to their tremendous commitment, the improvement has been spectacular.
Your milage may vary
It’s a long journey. We’re barely halfway through the scale described in the Quality Culture Transition Guide created by Alan Page. Our team will definitely keep on challenging and refining itself.
I hope you’ll get inspiration from our experience, but you should probably not copy and paste our approach because each organisation is unique. Instead, you would certainly benefit from defining a set of values and principles, try new things and see what works for you and what doesn’t. The real key in our case has been communication. Know your colleagues, understand what their pain points are, draw together strategies that match your specific business, organisational and technical context.
We ❤️ learning and sharing
Feel free to post your feedback and (constructive) criticism below or reach out to me on LinkedIn. Whether you had a similar or totally different experience, I’d love to hear about it.
Oh, and by the way: if you recognise yourself in this approach, we are hiring in France and Spain.
