We’ve gone Backstage — This is how we use it on our Digital Platform
Hi — I am Rob Hornby (Product Lead) for the John Lewis & Partners Digital Platform. Today I am giving a overview of our new Backstage implementation on the John Lewis Digital Platform.
We’ve just gone Backstage to improve developer experience on our John Lewis Digital Platform, we’ve built a number of cool features which we believe are useful for our developer and delivery community
- A list of teams (40), logical services (100+), and the components that make up those services, microservices (100s), frontends (10s) and jobs (10s)
- Created a set of technical health measures to assess cost, security, delivery, reliability and feature adoption, along with leaderboards for those metrics
- Used a range of plugins to create announcements linked to Slack; Pagerduty for incident response and Gitlab for deployments
- Built out techdocs with a range of documentation for Platform and onboarding
- Templates from which to build documentation sites or a microservice with one click
- We’ve looked at providing API documentation for the services we offer for better discoverability
With all these we are creating small slices of potential value to engage with our engineering communities then building out further where we find useful value.
Identify the teams, the services they own and who has permissions to that service
We wanted to make it easy to find a team, the services they owned and who had access. We linked information held within Google Groups, Slack and within configuration for each service to display ownership. The key here was work with the Backstage data model to express a team, service and then service components. This relationship provides the backbone to everything then built.
Evidence the detail of the services owned by a team
We can show the different types of workload within that logical service. This allows us then to capture much more granular information around a given service and make is observable to other teams and a much better record of what is running on our platform.
Key metrics and leaderboards around the health of services
We provide teams with data on the health of each logical service on the Platform. The data is built using jobs which aggregate data from key applications such as Gitlab, Pagerduty, JIRA, Kubecost and Google Cloud and serves that data via Big Query to specific Backstage panels.
- This allows us to evidence active services based on deployment frequency
- Whether those services are in production or building readiness towards
- Their availability against SLO target
- A view of key deployment lead time/frequency vs reliability/quality using DORA metrics
- Whether the service passes security policy based on security scans run and aggregated through our various scanning tools
- Cost efficiency of the service based on the CPU, memory efficiency of the service and overall spend
- Key service management metrics for incident, problem and change
- Technical health which measures feature adoption of Platform changes where required
We aggregate some of this data in to leaderboards that helps guide teams and provide reference points to teams on how well they are doing in relation to others.
Templates to help teams get started quickly with a new microservice or documentation
This is a relatively new feature for us. The Platform provides the base template for any language to which our specialists can then build upon. These provide the relevant frameworks and conventions for anything they want to template.
A few clicks will get you a basic Microservice configured and ready to be deployed from Gitlab. We’ve also created one to support documentation also. We expect to see more over time.
Documentation via techdocs
Our documentation is scattered across a number of sources making it difficult to discover, especially API documentation. We’ve pulled our Platform documentation in to Backstages Techdocs. We’ve also provided templates from which any team can create their own dedicated space, even when not directly related to a service on the Platform. We can also link to existing documentation in other locations such as Confluence.
Added some plugins for interesting features
We don’t want to build all our own custom modules in to Backstage, luckily Backstage has a range of plugins available both commercially and open source to add and configure. We’ve integrated a number of those to provide interesting insight on a service
- Deployment data through Gitlab
- Incident on-call rotas using Pagerduty
- Announcements, integrating with Slack to keep record of communications and feature releases
- Kubecost insights
- Google Analytics
As backstage is adopted by more companies we are seeing more and more potentially useful plugins
Summary
So what are our thoughts on the implementation and how it is working for us?
The Good,
- It’s been great to have a framework in Backstage on which to build. It’s range of inbuilt features has given us a great starting point, the ability to use the catalogue, templating, plugins and docs to quick start the portal
- We’ve gone with thin slices to understand how developers might use the features we are offering. Using feedback to help inform the next direction
- We’ve been able to make it our own through changing the look to be more in line with our organisation
- Search is a huge plus, being able to search across our catalogue, teams, docs and templates in one place is a real win for us and discoverability. It helps new starters, team to team communication and knowledge management
- We’ve seen a uplift in usage (we track via Google analytics) and we track CSAT with our internal developer community which has grown as we publish more features and respond to feedback
Where we’ve needed to think
- The data model needed some thought, mainly because our own opinions didn’t align with Backstage’s default data model, we’ve spent time to work out how to use that structure for our own needs representing a team, logical services owned and then the components within each service
- It’s React/Typescript and different to building in Python, we’ve got some developers in our team with experience but otherwise it is a learning journey for our team.
- We already had available data based on our older catalogue implementation, on which to build features within Backstage. That gave us a advantage in building out features and capability. It would have been slower to get where we have
- The other Platforms within John Lewis are adopting Backstage and ensuring alignment (we’ve decided on different instances based on Platforms maturity) will be a challenge and also fun to see the different levels of adoption and use of Backstage
