Two years of Backstage — behind the scenes of our developer portal: A turnkey MVP
At Peaksys, we have been using Backstage as our developer portal for two years now.
In this series of four articles, I’ll be telling you about our journey with Backstage, focusing on the following points:
· Why we chose Backstage and how it has transformed our developers’ experience (article 1).
· How Backstage has revolutionised the way we manage our resources and design our features (article 2).
· How Backstage has led to greater alignment between the Platforms and Developers teams thanks to Jobs To Be Done, a little UX and a lot of patience (article 3).
· How we went about designing this Tech product, paying close attention to the specific requirements of UX developers (article 4).
TL;DR version
This first article explains how Peaksys chose to use Backstage as its developer portal, after discovering the limitations of developing its own tool in-house, which proved too time-consuming, expensive and complex. It describes the advantages of Backstage, including how easy it is to set up, its application catalogue, its system for creating components, and the way it integrates with Keycloak for authentication.
The need
At Peaksys, we recently carried out a survey to assess our developer Net Promoter Score (NPS), using the SPACE framework. This survey revealed that our developers had one major need. They needed to be able to easily view and manage the configuration of their application components within a product, on our on-premise platform (code/deployment/authentication/databases/…). Although they had created dedicated areas for their products on Azure DevOps, they lacked an overview of all the information relevant to an application and its associated components.
The background
We had one main tool available to us:
- Odin, a platform for rapid application creation in just three clicks, which simplifies the process of deploying to our Kubernetes infrastructure (K8S). With the automatic creation of a Git repository and CI/CD pipeline, developers can focus on code development rather than on configuration. However, this tool turned out to be too expensive and time-consuming to implement and maintain. This Proof of Concept (POC), deployed in production without a clear strategy, quickly revealed its limitations. It was also too dependent on the other products managed by the team responsible for these. Here are one example [KB1] of a tight coupling with features placed in the wrong product:
- The creation of the CI/CD pipeline uses an in-house API developed by the team for its tools, in a development language that only one of the team’s developers has expertise in. Any request to change the Odin process requires a change to this API. Requests made through the tool are processed in two other tools, sometimes in Odin and sometimes in Jira, making use of the tool rather inefficient.
Faced with this situation, we had two options:
- Invest in improving Odin to meet our current and future needs, reduce the technical debt and redesign all the workflows to consolidate them into a single user experience.
- Explore solutions available on the market to find a suitable product.
Lessons learned from an unsuccessful implementation and the emergence of Backstage
Our attempt at developing our own portal may have been unsuccessful, but it did provide us with some important lessons for understanding what we needed to do to make life easier for our developers:
- Asynchronous architecture: We realised that using an entirely synchronous architecture for our portal was not viable. Implementing asynchronous task management with visual tracking was a no-brainer.
- Logical structure: A logically organised tree structure for applications within teams turned out to be crucial for intuitive navigation.
- Collaborative design: Designing features cannot be done in isolation within one platform team. The user experience (UX) must be the result of collaborative effort, avoiding disparities arising from the influence of dominant technical voices.
- Customisation: Adapting product behaviour to the user’s role and team has become a prerequisite for a consistent user experience.
- Simplicity: We reiterated our commitment to the “Keep It Simple, Stupid” (KISS) principle. Interactions must be kept simple and repeatable, avoiding excessive complexity. For example, it proved too restrictive to build validation via Jira into the core of a process.
- Development costs: The cost of developing a fully customised system soared, forcing us to reconsider our approach.
Given these considerations, Backstage stood out as a promising solution. With its native application catalogue that supported our tree structure and its “scaffolder” — an asynchronous component creation system with centralised tracking, built on bricks defined in YAML and coded in Node.js, similar to a YAML CI/CD pipeline — Backstage seemed to meet our needs.
Curious to find out more about this potential solution, we launched a Proof of Concept (POC) to explore Backstage’s capabilities and promises.
The POC
Setting up Backstage locally turned out to be a breeze, taking no more than 10 minutes once we had cloned the GitHub repository. The tool’s comprehensive, well-structured documentation is perfectly suited to integrators, making it quick and intuitive to learn.
Working with the Backstage Catalog was remarkably easy. With just ten or so lines of code, you can design a “processor” to generate components within Backstage. Its internal asynchronous processing system makes it easy to push hundreds of objects into the catalogue in a flash.
To populate the catalogue, we developed a processor using the La Carto API. This is our official component repository, updated by extracting and cross-referencing data from our platforms. Although accurate in terms of teams and components, this repository features an impractical, static web interface, which makes getting an overall view difficult. What’s more, only 20% of the information is directly relevant to developers, the rest being aimed at architects or technical managers.
A simple call to this API provides us with the full list of applications in our repository. A few lines of Node.js are all we need to analyse this data, map the fields in our repository to those in Backstage, and watch the catalogue automatically populate with all our applications.
Half a day was enough to achieve this.
We then had the component AND the owning team, which meant we could filter our searches.
However, as we hadn’t created the teams directly in Backstage, we couldn’t click on them to get a “team” view.
We therefore created a second processor that retrieved the data it needed from an API that returned our organisation structure (used by internal invoicing systems linked to our SAP system).
In just a few minutes, our teams began appearing in Backstage. Here we see the Component Team, which is responsible for the repository.
The API we used for the Backstage POC also included information on team members. This meant that we were able to add them as users to Backstage in record time. In just one day, we set up an entire catalogue, associating all the applications in our repository with their respective teams and members.
Backstage’s ease and simplicity of use was evident once again when it came to implementing authentication. When a user’s technical ID is mapped to a property associated with the logged-in user, team members are automatically added to their Backstage environment. The interface displays their applications, co-workers and team, making the user experience seamless and personalised.
At Peaksys, we use Keycloak for authentication. For the POC, we configured a Keycloak server using their Docker images and created a dummy user with a developer login. Backstage’s native support for Keycloak authentication via OAuth2 made the process very easy. It took just a few lines of configuration, guided by the documentation, to enable authentication effortlessly. The result was a Backstage environment giving developers instant access to their applications and systems.
Automating the creation of applications was a key step in our Backstage POC. We reviewed and adapted the code from our old portal to reproduce this functionality. Designing templates in Backstage, using the YAML format, proved to be a straightforward and hassle-free process, making it easy to configure components intuitively.
We needed to add two user input fields for our specific requirements: a “Team project” field for Azure DevOps and a “namespace” field for our on-premise Kubernetes (K8s) clusters. This information, already available in La Carto, was added to our catalogue processor. This generated Team Project and Namespace “resources” in Backstage. With a few additional properties in the YAML form, we were able to create dynamic drop-down lists, eliminating input errors and ensuring that the lists were always up-to-date and self-explanatory.
Migrating the code from our old portal to Backstage marked a clear improvement in how we manage our development processes. Unlike the old system, which used manual Git commands, Backstage provides a set of predefined actions, making integration and automation much easier.
The modular design of Backstage templates is a major advantage. After creating a form, all we have to do is associate a sequence of actions with it, either selecting from those available or developing our own. This flexibility has saved us a great deal of time by eliminating the need to rewrite code for Git and Azure DevOps.
All that was left to do was to modify the source code from the old portal to generate the code and set up a CI/CD pipeline. In less than a week, our POC was not only completed, but also fully operational. With Backstage, we were able to create an application on our Azure DevOps according to our model and deploy it effortlessly on our Kubernetes (K8S) cluster.
The revelation: Backstage adopted by Peaksys
When we presented what we’d achieved with Backstage to our community of developers, the enthusiasm was immediate and palpable. Backstage’s efficiency has dispelled the frustrations inherent in the old portal, and optimism is high. We put this down to:
- A development-friendly architecture: Backstage’s structure makes development intuitive and user-friendly.
- Reduced development costs: The efficiency of the tool minimises development costs.
- A streamlined usability philosophy: A catalogue to browse and a scaffolder to build, making it simple to use.
Based on these findings, the decision was taken to replace the old portal with Backstage for good, transforming it into an independent product that will lead the way in automating processes on our platform.
Conclusion
Backstage is an easy-to-implement solution. Reusing an existing repository accelerated the process, allowing us to concentrate on adding value for our users. In the next article, we’ll tell you about the implementation of our first Backstage feature — the ability to visualise and create Kafka topics completely independently. This was also an opportunity to explore community plugins and helped us lay the foundations of our mindset in designing Backstage.
