Our model to develop a strong engineering culture at Qonto: Kaizen Spirit

This post is part of a series of articles about the model we are building to scale our tech team and create a strong engineering culture at Qonto.

Every tech person who joins a fast growing team experiences it: scaling a tech team while keeping on shipping quality fast is a real challenge. Today, mainstream thinking sees Agile methodologies as the only way to achieve this. However, after collaborating with several companies using Agile, I became strongly convinced that this creates strong negative side-effects at scale:

1. Scaling agile methodologies often ends up creating a lot of bureaucracy, sustained by an army of agile coaches.
2. Engineers apply a set of granted rules without questioning their purpose. For instance, why do we need to split tasks? Why do we estimate using points? Why are we following a Fibonacci sequence in our estimations? It eventually creates frustration and triggers the feeling of being an executant.

As I faced those issues as a CTO, I tried to find a smarter way to scale tech teams. I began this journey by asking myself a simple question: where do agile methodologies come from? They obviously did not appear by spontaneous generation. So what’s behind it?

While digging into the origin of agile methodologies, I discovered their roots in a framework called the TPS (Toyota Production System) — more broadly known today as “Lean”. Beware, we are not talking about Lean Startup. We are talking about real Lean manufacturing — a powerful organizational framework invented during the ’60s in Japan — that enabled Toyota to become one of the most successful car manufacturers in the world in less than 30 years.

This series of articles is about our journey to define the model on how our tech team operates as we scale, relying on the TPS, and how we build something different from what already exists in the tech ecosystem. In this first blog post, we will focus on one of the cornerstones of our culture: the Kaizen (改善) spirit. In other words: How do we deal with problem-solving at Qonto and to which extent does it help us create a strong engineering culture?

Traditional problem solving

First, how is problem-solving handled in most tech teams (and used to be at Qonto!)? From what I saw in multiple tech companies trying to solve problems, here is the pattern that came out:

• Developers meet once every two weeks in a meeting room during a “retrospective” to talk about what went wrong. Without a scientific analysis framework, they propose large and cost-intensive solutions to large and unclearly defined problems. “We need to redefine all our devs standards,” “we need to use a new framework,” “we need a new organization!”
• Management does not challenge the solutions because it will not allocate the time to implement them anyway… “We need to ship!” ;-)
• It creates frustration and soon devs get tired of proposing solutions that will never be implemented.

Bottom line, devs stop being real engineers… They only ship code through a top-down management style.

Kaizen Spirit: how we do solve problems at Qonto?

At Qonto, in order to escape that vicious circle, we use a concept called “Kaizen,” literally “improve for better” in Japanese. The main principles behind Kaizen are:

• Involve team members into clearly defining and solving a problem.
• Trust their abilities to propose the best solutions to the problems they face.
• Give them time and support to test their ideas.

Ok, but how does it work?

A few months ago, following our migration to Kubernetes, we started to have stability issues on our platform, generating a large amount of failed builds when devs were trying to push code in staging. Since this problem was slowing down everyone’s work and created a lot of frustrations, we decided to launch a Kaizen to tackle it.

Here are the key points we tried to follow:

• Clearly define the problem: we first need to clearly define the problem we are trying to solve — then find a way we will measure our success. In our case, we decided to track the number of builds failed per week. We realized we had around 30 builds failed a week. It even peaked to 43!
• Invest time: we can’t solve complex problems if we don’t invest enough time. We decided to meet for at least 30 minutes every day to analyze the failed builds that occurred and proposed countermeasures. The following day, we would implement them no matter what (not postponing them forever).
• Have an engineering mindset: Taiichi-Ohno, father of the TPS, once said, “we are doomed to failure without a daily destruction of our preconceptions.” A big part of Kaizen is about finding facts and analyzing what really happened instead of rushing into implementing solutions we already have in mind. We analyzed every build failed one by one to make sure we really understood what happened in each case. It represents hundreds of cases that led us to 40 different root causes!

In a matter of weeks, we managed to eliminate 90% of root causes. Here’s the synthesis of our work.

We put together an “A3” at the end of a Kaizen to make sure that learnings are available to all.

What’s the purpose of a Kaizen?

Solving the stability of our platform was obviously a great win. But this is just the tip of the iceberg of what a Kaizen is really about. We see three main underlying reasons for running Kaizens:

• Solve tough problems and thus help the team work more calmly and more efficiently.
• Create teamwork. It makes people collaboratively think about the problems they are facing and about the solutions they are willing to implement.
• But most importantly, it creates strong engineers and leaders. And this is what the Kaizen is all about — training teammates to solve complex problems.

Salim, Vincent and Amine during a Kaizen session

At Qonto, we strongly believe that we will only become better if we invest time into creating a team of strong engineers able to solve complex problems, not a team of devs solely following rules. Kaizen is a strong tool for us to achieve that goal.

During our Kaizen, our teammates not only deeply increased their knowledge of Kubernetes (they learned how to size a cluster, what the key metrics are to ensure clusters stability, how to properly handle maintenance…) but they also became quicker and stronger at solving problems they face. Basically, they became better engineers.

What’s next for Qonto?

Of course, everything is not as easy and we face a lot of problems running Kaizen at Qonto. How do we choose the right problems to invest time on? How do we fight against the “production urge” and ensure that everybody has time to solve problems? How do we teach Qontoers to train fellow teammates to run a Kaizen properly?

We aim to involve more and more people into Kaizen and create a culture in which everybody can spend time on effective problem-solving. As of today, we run between five and eight Kaizen at the same time in the tech team — and around 12 in total at Qonto. This is of course just the beginning.

Are you interested in helping us build our engineering culture at Qonto? Join us! Careers at Qonto

Useful link: How can I make sure my teams do Kaizen the right way? By Michael Ballé