Akeneo PIM 3.0, let’s tidy up!

Photo by Samuel Zeller on Unsplash

Since its creation in 2013, Akeneo has been continuously growing and evolving. The product, the team, the offices, the code, everything has changed. Every single time, changes were never introduced just for the sake of it. It has rather been a necessity, an adaptation aimed to try to solve a new problem, to catch a dawning opportunity or to better answer a problem we better fathom.

This post will be about the latest “big” change of the code base for the upcoming PIM 3.0: its reorganization by business concerns. As usual, we’ll provide the tools to handle this change smoothly in your projects. The public API didn’t change, we just tidied up all the classes in different namespaces. Actually, that change is not that “big” for you, even it’s crucial for us.

Akeneo PIM 2.3 on the left vs Akeneo PIM 3.0 on the right

We’ve been growing…

What do you think the following figures relate to: 5 in 2013, 9 in 2014, 13 in 2015, 23 in 2016, 39 in 2017 and 50 in 2018? Our number of customers? Our number of partners? No, actually, it’s the headcount of people working in the product team. At the time of this post being written, more than 30 of them work on our main product: the PIM.

Now let’s take a look at the number of lines of PHP in the PIM per version: 90k in 1.0, 190k in 1.4, 390k in 1.7, 460k in 2.0 and 510k in 2.3. Yes, it’s constantly growing. Of course it comes to the fact that for each release, we ship new features.

In a matter of 5 years, code base is 5 times larger and headcount has been multiplied by 10.

Building a product is an extremely pleasant and exciting journey. It’s a never ending trip, with more and more cities to visit (aka features or products to build), luggage to carry (aka code base to maintain) and friends to meet (aka teammates to onboard). However, an enterprise seminar gathering 150 people does not require the same logistics and organization than a trip with 3 friends. It goes exactly the same for our product and our product team.

How do we work at Akeneo?

Before digging into the problems we are currently facing, and the reasons why we believe a code reorganization will be the stepping-stone to solve them, we have to understand how we built the PIM so far, from an organizational, business and technical point of view.

As you may know, our product team is divided into squads. Each squad is an independent cross functional small group which is made of developers, a product owner, a UX designer and a devops. They have to take care of a specific subject and lead their mission successfully, whichever way they deem appropriate. We are happy with this way of working and organization since we introduced it, mid 2015. It answered a lot of problems we encountered in the past.

Understanding what hundreds of customers need and imagining which features could serve them best is already a tough exercice. But if you also take into account the cost of delay, the schedule of the squads, the risks, the financial aspects, the sales pipeline, the Community and Enterprise editions, the on-premise and SAAS versions, the support and maintenance, the improvements, the new features and the market targets, it becomes a headache. This strategic management is complex by nature. There is no one-size-fits-all type of answer that would match all criteria. There is no one-size-fits-all type of answer that would please all the persons impacted by those decisions. That’s why, sometimes, in the past, we had to leave some existing features aside to be able to ship a brand new product or component. For instance, a squad could give birth to a feature and then move on to something else. Once shipped, the feature becomes the responsibility of all the different squads. This kind of project management is not a problem per se. With a small team owning the whole product, it’s totally viable. But it becomes harder with a larger organization and a far bigger product.

Most of the bundles that exist in the PIM 2.3 were already there at the release of the 1.0. To be more precise, 11 bundles out of the 20 existing in PIM 2.3 Community Edition originally comes from the version 1.0. Some internal parts have been totally revamped, the storage has evolved twice, the business code has been decoupled from the infrastructure, but still, from a purely macro perspective, the code organization has not changed that much over the last 5 years. The only noteworthy change on that side appeared on the Enterprise Edition; each squad created its own folder dedicated to the feature they were in charge of and tried to isolate the most possible its code from the rest of the code base.

All of that worked well. Well, that works well.

So, everything sounds good, why such a change?

Apart from this idealistic depiction, the squads working on the PIM face mostly two important challenges:

1. We feel it’s getting more and more difficult to ship a new feature promptly (or to improve an existing one). There is no reason to panic now, but the sooner we act, the easier and cheaper it will be.
2. We feel it’s getting more and more difficult to onboard a new teammate in the PIM. The application is so huge that it’s now impossible to understand and master all the functional and technical aspects.

Both difficulties are rooted in the technical debt as well as in the current squad organization and the project portfolio management. On a daily basis, this translates into:

• Some parts of the code are too tightly coupled: changing a class here can cause a piece there to break.
• Some features are not open-closed proof: from a functional point of view, some features modify an existing behavior, instead of extending it.
• Builds are too long: regarding our tests, we chose quantity over quality by a lack of experience.
• Maintenance is sometimes complex: as each developer joins the maintenance squad once in a while, on occasions, you may have to fix something you never heard about before.
• Collective code ownership is hard to maintain: when a group become too numerous it’s a natural tendency that “the responsibility of everybody” turns slowly into “the responsibility of nobody”.

Absolutely none of these problems is serious. Some may take time, some may require work, but none of them is insurmountable or out of control. We just try to pay close attention to early warning signs.

This is not just a code reorganization

We believe we can solve the problems listed above by giving squads more autonomy. We believe that with guidance, nurturing and autonomy, squads will deliver more value more frequently to our customers. We believe we would be more efficient if each squad had a smaller and well defined perimeter. How to achieve that? More or less, the squad α would officially be responsible of the features B, D and E while the squad β would officially take care of A and C. In that case, a squad could:

• Capitalize on its functional knowledge.
• Make strong choices that impact them, and only them.
• Regularly deliver functional improvements on their expertise scope.
• Prioritize their own functional debt.
• Prioritize their own technical debt.
• Take care of their own maintenance.
• Take care of their own tests.

To achieve that, we need three different and complementary things:

1. Reorganize the code base so that we effectively see and understand what are the features A, B, C, D and E. As of today, the code base does not reflect the existing functional parts of our business.
2. Minimise the impacts between features so that the squad α can’t easily break the work of the squad β (and vice versa).
3. Carefully map squads to features. This is for sure the most complex part, as it may eventually impact the way we prioritize the functional roadmap. For instance, it’s easy to prioritize D over B and E, but it becomes more difficult to prioritize A over B, D and E.

We are confident it’s the right path to follow. During the last months, two squads worked this way on two different upcoming Enterprise features. Their code base is isolated, their tests and builds are independent from the rest of the PIM. They are more autonomous, happy and efficient. They feel more productive and the feedback loop between the squad and the stakeholders is shorter. They feel more confident about their code base, to maintain it and to make it evolve and change.

This blog post comes to an end. It focused mainly on the reasons why we needed to reorganize the code base. The next one will detail how we achieved this reorganization and what are the impacts on a project. It will be more technical and concrete. Stay tuned!