How we rolled out our Kubernetes platform in Adevinta Spain

By Pau Gay, Head of Enablers

Will moving to Kubernetes actually get us in a better place? I’d like to tell you about the results of one of our recent projects, which brought together engineers from several different teams in Adevinta. The goal was to take a set of Adevinta Spain microservices, running on EC2, and migrate them onto the Common Platform, our internal Platform-as-a-Service product based on Kubernetes.

The Common Platform is run by our Central Platform team, and it’s where software engineers from across Adevinta can build, deploy and operate their microservices. Once we migrated this initial set of microservices, we ran comparisons to check our deployment speed and cost efficiency hypothesis.

Since completing the project, the Common Platform has become the default microservices platform in Adevinta Spain, among many other marketplaces.

🚀 The next step is to expand the platform out to the whole of Adevinta!

Business case for the collaboration

Behind the project was the Central Enablers team, which belongs to the Strategy & Enablement tribe and works to build bridges between our Central Teams and marketplaces.

🎯 The objective of the collaboration was to migrate some microservices from AWS EC2 into the Common Platform.

Our hypothesis at that time was that the Common Platform would bring a quicker speed of delivery (code reaches production much faster) and that with increased adoption in the Common Platform and Kubernetes as a container scheduler we would be able to optimise the use of resources (time of CPU and use of Memory) and spend less money in our underlying infrastructure.

🤔 Quicker speed of delivery? What does it mean?

By changing the runtime of our applications from EC2 to Kubernetes we transitioned from AMI (Amazon Machine Images) to Docker containers. The speed at which we can create Docker containers is typically two to four times faster than the time it takes to bake an AMI.

To demonstrate this is the case, we’re going to look into the time it took to deploy a particular service before and after the migration. To review actual figures, we’ve selected an internal service from Adevinta Spain called:

ms-ma—-search-url-processor

This is a screenshot of the Jenkins pipeline before the migration:

Screenshoot from the Jenkins pipeline of the service, before migration

Below are the different pipeline steps and times:

1. [4:00 min] to build the AMI
2. [0:02 min] to deploy the dependencies in PRE
3. [4:00 min] to deploy in PRE
4. [0:15 min] to share the AMI
5. [0:02 min] to deploy the dependencies in PRO
6. [4:00 min] to deploy in PRO

The total time to deploy is around 12 minutes.

This is a screenshot of the Spinnaker pipeline after migration:

Screenshoot from the Spinnaker pipeline of the service, after migration

Pipeline steps and times:

1. [0:30 min] to deploy the dependencies in PRE
2. [2:30 min] to deploy in PRE
3. [0:30 min] to deploy the dependencies in PRO
4. [3:00 min] to deploy in PRO
5. [0:30 min] to deploy the dependencies in PRO

Total time to deploy is around 7 minutes.

✍️ Summary

• Before migration: 12 minutes
• After migration: 7 minutes
• 41% improvement ⭐

🧐 And … are we really saving money?

When we were using Amazon EC2, we were paying for a whole VM, where most of the time the CPU/MEM use was not well optimised. This meant that sometimes there were services running in EC2 with CPU use at a very low level (<10%). We were therefore paying for more than we used.

In Kubernetes we could have a similar pattern where a service requests far more resources than it needs. But by running all the Adevinta Spain services in the same cluster, it’s easier to detect and identify any cost saving opportunities.

We are going to show some real numbers based on a set of microservices from Milanuncios, which is a popular marketplace from Adevinta Spain.

In AWS EC2, the cost of running those services was approximately 10.400 € per year.

When we migrated those services into our Kubernetes cluster, we looked at the monthly cost of the namespace owned by Milanuncios (called es-microma-pro) and it was 450 $ per month.

Output of the “kubectl” command that shows all the applications from the namespace

The monthly cost of the namespace in Kubernetes is: 450 $ / month = 5.400 $ / year = 4.500 € / year

✍️ Summary

• Before migration: 10.400 € / year
• After migration: 4.500 € / year
• 55% improvement ⭐

How did we achieve the migration?

To optimise our way of working, we formed a “squad”, which included engineers from Adevinta Spain, the Central Platform Services team and the Central Enablers team, and worked together for a whole quarter.

We discussed how we could achieve our goals, starting with the migration of 10 services into the Common Platform.

The Central Enablers team pays a lot of attention to how to manage these types of projects that we refer to internally as “collaborations”, because they always involve people from different teams where collaboration is required.

There are some 🔑 key aspects we consider important and always address before starting a collaboration, such as defining the goals, resolving how we’re going to measure the progress, defining success, etc.

🌅 We did all the necessary work to achieve alignment and prepare the ground in order to enjoy the amazing technical challenge we were about to face.

How did the migration go in the end?

Our objective was to migrate 10 services from Adevinta Spain to the Common Platform and by the end of the collaboration, we had moved 21!

This was 100% more than what we initially expected!

Today’s scenario in Adevinta Spain is that from ~ 300 microservices, more than 275 have already been migrated. So, 90% of the services today in Adevinta Spain run on the Common Platform, managed by the Central Platform team.

Image from the Adevinta Common Platform landing page

This was possible due to the high level of engagement from everyone involved in the project. In Adevinta Spain, the project involved the whole Platform team, as well as the Infojobs core team, Cross Components and some other individuals who sponsored the transition to Kubernetes. In the Central Teams, the project involved the three teams from the Common Platform group (🚚 Delivery, 🏗️ Common Platform Runtime and 🤖 Engineering Productivity) as well as the Enablers, who coordinated the project.

In Adevinta we live by our values and this is a perfect example of one of our favourites: We win and lose together!

Closing retrospective

With the help of Adevinta’s Agile Coaches we ran a collaboration retrospective with a group of stakeholders. We extracted a lot of valuable feedback:

• Working closely as a squad was fundamental, as it enabled quick and easy access to information, close support and interesting conversations around the coffee machine!
• The Common Platform still has some rough edges but it’s maturing at a good pace.
• Migrating services from Adevinta Spain has become a deterministic action. It now takes less than 1 day of work (including some hours dedicated to training).
• The earlier you test in production, the quicker you see any potential problems. For example, we had a few challenges when routing production traffic into a service that would never have been discovered in DEV or PRE environment. See a related post: Kubernetes made my latency 10x higher.

All of these learnings have helped us improve and be ready for future collaborations.

One of our Agile Coaches (Natalia) with a member of the Platform team in Adevinta Spain (Ramon)

Closing words

The collaboration with Adevinta Spain has helped us better understand our stakeholders’ challenges, pain points and their points of view. This has been proven as critical to achieve successes.

Thanks to all of these projects and opportunities, the Enablers team has become a clear strategic partner for product and tech collaborations across Adevinta on a global scale.