Building a Developer Portal with Backstage (Part 1)
We wanted a service catalog; but got a developer portal.
Dominik Henneke with Oliver Sand
Hello, we are the SDA SE. We are an InsurTech based in Hamburg, Germany and provide a technical platform to support established companies in their digital transformation. We offer a unique architecture for modern cloud-native applications and established an ecosystem of ready-to use solutions that are built to transform business.
We build an Ecosystem! But how to navigate it?
I would like to make my idea a reality using composable off-the-shelf modules, third-party integrations, and only few homegrown extensions. What do you offer? Where can I find your APIs and other docs that help me understand the building blocks?
— Some customer
An ecosystem is only helpful if you have the means to navigate and expand it. When you shift from a single customer to an ecosystem approach, information discoverability becomes the barrier to success. And that barrier is represented by a scattered knowledge library of outdated wiki pages, internal slide decks, and incoherent service descriptions. We struggled with our goal of making our technical ecosystem tangible and understandable to all. Business design documents were stored without much structure. API documentation was—though available—only accessible once a service was deployed. That’s not self-service from a customer perspective. So, we started to draft a solution for this shortcoming.
Do I need a software catalog or a developer portal?
Rollback to fall 2019. Our priorities were to catalog APIs and software and making these available to our customers and in our ecosystem. We looked at many tools in the market but didn’t find the perfect solution. We already had some experience with homegrown solutions from a related company and considered building our own solution.
But we struggled with this decision because we encountered some issues in creating a software catalog:
- It’s a software catalog — that’s documentation and it’s hard to write documentation and keep it up to date.
- It’s disconnected from the developers. They are not the ones that use the catalog daily. But they are the authority for the most valuable assets.
- We have few experience in modeling such a catalog and it will tie up a lot of development and design resources.
In March 2020, Spotify announced the Open Source project Backstage “An open platform for building developer portals”, providing the tool we’ve been missing all along.
But this is a “developer portal”; we were looking for a service catalog?! Or do we? It turns out that Spotify solved many of the problems we faced by creating a tool that mainly targets developers, but also addresses the needs of the sales and management disciplines. It solves the motivation issues by encouraging teams to keep their catalog entries up to date to make the most of Backstage. And it’s the perfect source to derive a deep technical overview of everything we offer.
What is Backstage?
Backstage is a CNCF Sandbox project founded by Spotify. But unlike many other CNCF projects like Open Policy Agent or Prometheus, Backstage is not a one-size-fits-all product that can be used out-of-the-box. There’s no Docker container or helm chart to install and get started — at least not today. Instead, it can be more compared to a project such as “Create React App”: It creates a ready-to-use application from a template and uses a foundation of purposefully built dependencies. From there it can be customized to meet your organization’s needs. You can either create your own functionalities internally or integrate existing ones from the plugin marketplace — a collection of Open Source plugins.
Create — Manage — Explore; The DNA of Backstage
Backstage is not just a software catalog that you can explore; it is meant to be the central place for all development-related activities. But why is that important?
Powered by a centralized software catalog, Backstage restores order to your infrastructure and enables your product teams to ship high-quality code quickly — without compromising autonomy.
Modern application development is all about autonomous teams, end-user-driven design, and DevOps. Especially in large scale organizations, autonomy can be both a blessing and a curse. Autonomy promotes decentralization and innovation—lack of autonomy leads to centralization and stagnation. But too much autonomy can also lead to stagnation: teams must make technical decisions all over again and inevitably drift apart technologically. Thoughtworks calls this the “Negative Developer Effectiveness Flywheel”. In the insurance sector, standardization, regulatory requirements, compliance, and security are high values, but these shouldn’t be a barrier for autonomy and a reason for poor developer experience. After all, it’s not a law of nature that compliance must be intimidating, complex, and boring. Or that modern application development and operations must be an unpleasant experience.
Backstage wants to break out of this wheel with two main ideas: to be an attractive developer platform and to offer the opportunity to rethink organizational processes. Let’s see some examples:
- Create — You are using a microservice framework to build new services? Use the Backstage Software Templates to scaffold new services quickly — following your guidelines and best practices. Your organization requires non-functional review processes before going into production? Streamline them, increase visibility, and automate as much possible or rational to keep teams feeling productive, not lost.
- Manage —You have existing tools that gather insights about your running services, such as quality checks, security checks, monitoring, or on-call? Integrate them into Backstage to make existing data and tools easy to find. Your supervisor needs a list of all your team’s responsibilities? Send a link to Backstage where they can find and export these always up-to-date information.
- Explore —You want to find the API endpoint that provides the data you need, or want to figure out who to call if another service sends unexpected responses? Check out the software catalog of all the services and APIs in your organization and contact service owner directly. You are a new hire and need information on a specific topic? Use search to find the correct documentation page in the knowledge base or browse the organization’s onboarding guide.
As you can see, Backstage has a lot to offer!
Backstage adapts to your organization — Not the other way around
Using Backstage won’t transform or improve your development processes just by installing it. It’s not just another tool that you add to your tool chain, it wants to be a trigger to see the world like a developer and value developer experience. It doesn’t force you to replace any tool or cloud service, nor does it force you to commit to new processes or throw away things that work well for you. It provides a unique opportunity to improve and get inspired by other Backstage adopters.
Everything that is provided is aimed towards customizability. For example, the way you populate your catalog: You can choose to create catalog entries in the standard Backstage descriptor files. But if you already have a homegrown file with metadata about your services or an existing service catalog, you can simply import it into the Backstage Software Catalog and make your services compatible with all the Backstage plugins. The same pattern is present in the Backstage core and all plugins. It doesn’t constrain you or try to block you. And if something is missing in the Backstage UI, it will always provide you with a link to jump to the original data source of the information.
What does that mean for us?
This way of thinking about the topic was astounding—not because it came from Spotify, but because this was the tool we were looking for all along. We get a software catalog that is focuses on developer effectiveness. It’s fun and encouraging to use. And it is driven by a flexible and sound data modeling. So, we decided to invest into the project and select it as the foundation for our ecosystem initiative. But spoiler alert: At that time, Backstage was far from finished and didn’t offer most of the features we originally wanted. But that’s a great motivation for Open Source contributions! Our platform is built on Open Source software, even though we often missed our goals when it comes to public contributions — this is our change to give something back to the community.
What did we do to adopt Backstage? How do we take part in the Open Source development? And what can you learn from our experiences? Let’s talk about that in part two of this series.
Did you find this post interesting? We are hiring.
