The Technology Portal of Trendyol aka Pandora & Developer Productivity Highlights from 2023
At Trendyol, we have 7 data halls in 3 data centers, 464K CPU in use, 7000+ microservices, 600+ deployments per day, 52% of service maturity points earned, several KBTs, 2000+ members, and one big team. Do you want to learn more about how this is possible? Here we start.
Throughout the years I have been in Trendyol’s product team, Trendyol has invested substantially in developer experience and productivity. In this article, I will mention the last couple of releases we made to improve the developer experience and track developer productivity to have actionable results.
These releases cover DORA metrics: deployment frequency, cycle time, change failure rate, mean time to restore; maturity of our services by several categories such as observability, security, maintainability, DORA, ownership, and reliability, and how we developed the experience around these products with a technology portal and self-service developer platform for its enthusiasts.
How the story starts
Once upon a time, it all started with the idea of a developer portal in the tech organization that scales up 100% each year. Like all the largest technology companies, Trendyol needed to focus on making the development lifecycle easier, more efficient, and more productive for its employees.
Firstly, to avoid discovering the world from scratch, we deployed backstage.io, the developer platform project of Spotify, to enable all developers in the company to manage their services on a catalog, create documents, and use needed tooling services through one user interface. Meanwhile, we were trying to understand the user needs and define Trendyol Tech’s strategy for the portal, which can be provided with more capabilities by considering the needs of the company and a good performance on the systems.
Although the backstage.io project was saving us time thanks to all the improvements from the community, over the years, we scaled up to 7000+ microservices in our overall structure and 2000+ team members inside the technology team. Therefore, our need for customization has increased to deliver a product developed in-house.
After researching for the technology team, we saw the need to develop a portal for all technology members who contribute to the development lifecycle, from developers, engineers, product managers, QAs, UI/UX responsible, etc.
There, we present Pandora!
Pandora’s Vision is to be the Technology Portal of Trendyol by presenting the first entry point for any tech member to find all knowledge, take actionable outcomes from engineering insights, and manage processes in the end-to-end development journey that they are looking for.
A Plus: Self-Service Developer Platform
“A compound of development (Dev) and operations (Ops), DevOps is the union of people, processes, and technology to continually provide value to customers. Organizations that adopt DevOps culture, practices, and tools become high-performing, building better products faster for greater customer satisfaction.” as mentioned here, we also believe in this definition.
In addition to Pandora, we serve a self-service developer platform, aka Trendyol Builder Platform, to our users that visions to speed up and simplify developers’ application deployment process, making the development workflow more efficient by offering a scalable and customizable platform that enhances productivity and accelerates development cycles. TBP embraces an app-centric approach which lets developers focus only on their application and its lifecycle rather than managing the dependencies required for an application. All dependencies and prerequisites that an application developer needs can be bound with a given application without a hussle such as deployment methods & strategies, security configurations, configs & secrets, databases, and so on.
Our motto is; “You Build it, You Run it!” within the practices and policies set by the whole community of engineers in Trendyol.
A start for the series of Developer Productivity Highlights in Trendyol
Developer Productivity is about removing frictions in the continuous delivery cycle, achieving faster lead times with reduced cognitive load. Further, the time it takes for a new engineer to implement a straightforward service, including dependencies such as DBs, Messaging, etc., to deploy to production via CI/CD with the everything-as-code approach is a good metric for us. This is no simple task and depends on many factors, so we will relentlessly continue to improve engineers’ experience with a product mindset.
With that being said, here is the improved developer experience thanks to feedbacks from our users and developers who built upon these feedbacks.
Users First!
When Pandora was built up, as it was built from the backstage.io project and its plugins, all the user experience and user interface were precisely the same as the needs the backstage community had collected.
Thanks to the easy implementation of all plugins, we deployed several new features quickly to collect our users’ feedback from Hotjar surveys and regular user interviews, user engagement metrics from Google Analytics, and Hotjar’s heatmap recordings.
With this, the vision of Pandora has been set, and our developers started the development of an in-house product by considering the needs of users and the company.
Over the years, the main parts of the framework around these products:
- Service Catalog is for showing the overall picture of services, managing configurations/secrets, scoring maturity levels and adaptation of services, and enabling plugins such as GitLab and Swagger for easy access to the information of that application.
- Documentations for presenting Trendyol Tech’s primary Documentation Platform to serve one user interface for combining all kinds of usage like user-friendly knowledge workspace(wiki) and code variented documenting(GitLab) where users can choose within the several options.
- Teams for introducing all teams and tribes in the technology team, and the overall situation of the technology team by sharing the overview of the teams like the team’s members, which languages are used in the team, docs of the team, etc., while including the engineering metrics such as team’s maturity and adaptation, DORA metrics, observability metrics, etc. that the teams want to check regularly to see their progress, and what they can do more to grow in the organization and for the organization.
Besides these main products, we offer several other products to make the lives of our users easier. These will be topics of upcoming medium articles! Let’s continue with the Teams plugin and how we turn it into an engaging product inside the organization by considering the feedback we received.
Teams Plugin Release: Overview, Engineering Metrics, and more
Team pages are created to introduce all teams’ information within the organization both to the executive level and to teams internally. Every team has its team page with the respective information regarding its members, health on the development processes, maturity of their services, documents, observability data, etc. With this plugin, we can see the overall picture of any team or organization as the backstage.io already introduced.
When I started working on Pandora, the biggest target was to turn Pandora from a backstage project to an in-house product. Yet, there were already 100+ items in the backlog from customer requests and technical debts. Some of them were waiting for development directly, and some of them needed analysis first. Even though we had a handover for all roadmap and backlog items, I started to have user interviews with the people motivated to share the improvement areas they see on the product.
A tip for product managers: Talk with your users, or hear from them via metrics, and repeat this cycle regularly.
As a result of the user interviews, most feedback has been gathered on the engineering metric topics. DORA metrics, service maturity, and observability metrics added to the KPIs of the teams were the ones most questioned by the teams.
Therefore, we prioritize the improvements for the Teams Plugin that includes engineering metrics, to see the big picture and give actionable insights to the organization.
DORA Metrics aka 4Key Metrics
For a little bit of introduction
- For software development organizations, 4Key Metrics provides specific data to measure their organization’s development performance and recommend improvements.
- It enables development teams to understand their current performance and deliver better software. Also, it helps development teams determine whether they are meeting customer requirements.
- These metrics collectively measure the speed and efficiency of the software delivery pipeline, helping organizations enhance their development processes, reduce time to market, and improve overall system reliability by efficiently addressing and resolving incidents.
With this knowledge, we present Deployment Frequency, Cycle Time, Mean Time To Restore, and Change Failure Rate to give insights into any service that teams have.
Improved User Experience
After the feedback that we gathered, prioritized user stories are listed as the following:
- As a user, I want to see the DORA metrics of any team on a user-friendly graphic while comparing my metrics to Technology’s average value,
- As a user, I want to add KPI for my teams’ metrics for a specified timeline on the team pages while being able to check the KPI value and success rate,
- As a user, I want to see the details of all metrics to take actionable insights by issue details, commit details, observability metrics, etc.,
So that I can understand our current performance and take action to deliver better software that meets customer requirements.
Metrics Calculated as
- Deployment Frequency: Calculated by dividing the number of deployments by the number of days in the specified day range
- Cycle Time: Calculated as the time of tasks added into an active sprint until completion.
- MTTR //Mean Time To Restore: Differences between the first moment when the error with a success rate of over 95% is restored and the moment when the success rate drops below 95%
- Change Failure Rate: Deployment count in which failure occurred divided by the total deployment count
Brief Summary of Other Actionable Metrics
All the sections below will be detailed in separate articles soon! However, here is a quick recap of other engineering metrics that we track;
Service Maturity
- In the 7K+ microservices world, it is hard to track the maturity of all services without a centralized way,
- Therefore, we created a system with six categories to measure the maturity of our services,
- While sharing the possible actions with our users to improve their services’ health in maintainability, observability, DORA, security, reliability, and ownership categories.
In the 2024 roadmap, we plan to improve our product by providing hints for the rules to make the development by teams easier, and present a better user experience by considering the feedbacks that we gather.
Observability Platform
- The development of an internal observability platform has been an aim over the years.
- Our vision is to become the first choice Observability Platform for our users, providing full visibility, seamless integration, flawless user experience and data intelligence.
- With this release, we provided the Defined Alerts and KBT based SLO with a more customizable and user-friendly interface to our users.
- Defined Alerts enable development teams to easily view relevant alert information with alert details such as severity, query or threshold configurations.
- KBTbased SLO reporting allows users to track Service Level Objectives for Key Business Transactions in detail to understand the problem.
- SLO definitions for KBT is expected to minimize need of defining alerts for specific services as it will work with intelligent alerting where dependent services contributing to drop in KBT SLO are identified and alerted.
This is an Epic that will be continued throughout the year with several milestones.
Conclusion
In Trendyol, developer productivity is the highest topic that we prioritize the most. Several teams work to create a smooth development cycle for everyone in the technology teams. Thanks to all of my teammates who contribute with their creativity, passion, and motivation to all the problems that are being raised on a daily base.
Upcoming releases: Change Lead Time for first commit to deployment calculation, Cycle Time Breakdown charts for better sprint evaluation to understand where we waste our time the most, Maturity Hints for suggested actions of the services, smoothest Application Deployment processes through TBP, and many more.
This article was the start of a series, so from now on, you will hear more about what we develop for our users in Trendyol. Hope it helped, and please stay tuned :)
Useful Links & References
- For a better understanding of Developer Productivity, I would recommend you to follow the newspaper of Abi Noda here
- DevOps Explanation here
Want to work in this team?
Be a part of something great! Trendyol is currently hiring. Visit the pages below for more information and to apply.
