Evolution of design system KPIs
The evolution of KPIs for the NewsKit Design System
Introduction
Roughly two years old now, the NewsKit Design System is maturing into an industry leader, with adoption increasing sprint by sprint and the system itself being put to the test, increasing the robustness of NewsKit daily.
If you’d like to read more about the journey we’ve been on, you can do so here.
In this article I’d like to talk you through how the KPIs for the NewsKit design system have evolved over time and, crucially, how what we may have found useful two years ago may no longer be of significance when it comes to tracking our progress. It seems to be an age-old problem for design system teams and, whilst I’m not going to sit here and tell you we’ve cracked it, what I share with you may go some way to helping you demonstrate value to your stakeholders.
Why do we need KPIs?
“Product key performance indicators (KPIs) are metrics that measure your product’s performance. They help you understand if the product is meeting its business goals and if the product strategy is working. Without KPIs, you end up guessing how your product is performing” — Klipfolio.
In product development, you need to have a robust set of KPIs available in order to be able to measure the success of what you’re doing — otherwise, you could end up wasting vast amounts of resource moving in the wrong direction. For many technology teams, this is how you demonstrate value to the business and is vital to the success and longevity of your product.
Setting KPIs and demonstrating business value for a customer-facing product is one thing. However, things get a lot more complicated when you’re attempting to do this for a design system. Your consumers are engineers and designers, and KPIs for the pace and/or efficiency of product development is often much harder to extract than bottom-line revenue or growth metrics
(if you’re not familiar with design systems, you can get up to speed here).
Within NewsKit, we have adapted our KPIs over time depending on where we are in the product development lifecycle and hope to be capable of demonstrating the value of our design system more accurately as time passes.
The beginning
Number of components
It’s July 2019. A huge technology team restructure at News UK has just taken place and the NewsKit design system team is born. In the early months of NewsKit, we were aware that no one would be picking up any of our components or our theming system whilst still in their infancy, and it’s for that reason we decided to focus our energy on designing and developing a foundational set of the core components needed to build a typical news website.
So, to set viable targets for our team, we decided to set the number of components as our primary KPI. A simple measure for the number of supported components made available for product teams to consume and use. But did this demonstrate the true value of our systems? Probably not.
Year 2
NPS
July 2020. The world is wrestling with Covid-19. Everyone has been ordered to work from home. Our team has now grown and the first product to be built using the NewsKit design system in Times Radio has been launched.
Once we actually had users for our design system (a big milestone in itself), we needed to understand more about their experience of working with it — warts and all. As an industry-wide measure of customer satisfaction, we decided to use Net Promoter Score (NPS). We began by sending out quarterly surveys, aimed at capturing a whole host of feedback on client perception of the design system i.e. the reality of using NewsKit — with the aim to then action that feedback.
As part of the survey, we ask the user how likely they would be to recommend NewsKit to a colleague. Depending on what score they answer, this will determine whether they are a promoter, detractor, or a passive user.
Averaging these responses gives us the NPS score for NewsKit.
For the first time we were able to track the users’ perception of NewsKit and analyse how this went up or down each quarter, mapping our NewsKit releases to an uptick or downtick in NPS.
Overall, we have found NPS to be a really valuable KPI to track as we’re now transparently surfacing our users’ sentiment towards the product we are building. As more products adopt NewsKit, this KPI becomes an even more accurate and important tool for demonstrating business value. It helps us cut through the noise all over the place — from prioritising within our tech debt stream to defining our client support strategy
Year 2 KPIs:
- Number of components
- NPS
This year
Pace of development
July 2021 brought us into a new (Australian) financial year. Wireless, News UK’s audio arm, launches a brand new Virgin Radio website on their new Wireless Web Platform, built entirely using the NewsKit design system and its sister products, NewsKit API and NewsKit Render.
This was a big moment for the team and the company as a whole, and represents a further step closer to a future-facing vision of News UK’s architecture, if and when all brands have migrated over to this new technology stack. The core principle is that the Wireless Web Platform will underpin all future radio/tv station website builds, significantly increasing the pace of product development by utilising NewsKit’s innovative theming system to skin the sites to their respective brand colours and styles.
(To learn more about theming, see this blog by our principal engineer, James Spencer)
Time and money spent on product development will no doubt become a critical KPI for us in order to demonstrate the true business value of the capabilities we’re delivering. The challenge here lies in our ability to set realistic targets.
Telling the story post product delivery is achievable, presuming the products in question are broadly in the same ballpark in terms of complexity and/or capability:
- Product A took X period of time to build and cost X
- Product B took X% less time to build and saved X cost when compared with Product A
Setting ourselves viable targets, though, for any given financial-year increment, is very tough. It presumes that we can always compare apples with apples i.e. that all products are similar enough to extrapolate an annual cost/efficiency saving. We can’t.
It also presumes that entire products will always be built using NewsKit. They won’t. As mentioned further through this article, some products will only adopt NewsKit incrementally, which further dilutes our ability to measure true value and/or set robust targets.
That said, the more products that are delivered using NewsKit — either in their entirety, or piecemeal — the more ammunition we’ll have to inform realistic annual objectives.
Contribution
We believe a design system should never be owned solely by the team developing it. Whilst the NewsKit product team are the ‘governors’ of the design system, we aim to become a project that every user, across both design and engineering, has a stake in — and encourage contribution at every opportunity.
In July we began to capture the number of items contributed to the design system. Component contribution in code is the north star but this can be a lengthy and challenging process, so we include other forms of contribution too, such as;
- Patterns
- Component specifications in design
- Component contribution in Figma
- Bug fixes in code
These incremental calls for contribution are helping to establish ownership (and enthusiasm) amongst engineers and designers across the wider technology team.
Performance
Some brands using NewsKit are incrementally adopting, rather than taking on wholesale platform rebuilds, and replacing key components on their pages only when there’s demonstrable business value on the table — with a high-level goal to have a fully NewsKit-supported site over time.
Cristiano Rastelli (formerly of Badoo) wrote a brilliant piece on measuring the impact of a design system. You can find it here.
Taking what we learned from Cristiano, we are looking into how we can track the adoption of our components over time, in each of our brands’ codebases. News UK is a large organisation with multiple brands. Each brand has its own (or sometimes multiple) codebases, which makes extracting an accurate figure as a percentage quite challenging. However, only tracking the adoption of the design system still does not prove out any business value. We need to dig a little deeper for this…
All of our NewsKit components and subsystems are designed and built to be as performant, accessible, and flexible as possible. From the live products that have used NewsKit and the various proof of concepts carried out using our components, we know that Google Core Web Vitals metrics are improved when our components are used over other, legacy components.
We are working towards capturing a KPI that will allow us to correlate the adoption of NewsKit as a % against the Core Web Vitals scores. We would expect to see an improvement in the CWV scores as the adoption of NewsKit increases, however, we are only at the beginning of the journey to get to the bottom of this metric, so expect to see another blog post in the not too distant future.
Summary
Whilst I don’t have a definitive answer to the age-old question of which KPIs are the right ones to track, I hope I’ve highlighted some of the areas that could demonstrate the value of a design system. We will no doubt update these KPIs again in the coming months, but that’s ok. As the system evolves, so does our means of tracking our progress. What’s most important is ensuring our focus is always on providing value to our consumers.
KPIs moving forward:
- Number of components — Useful to see the growth of a design system
- The pace of development (£ saved) — A real diamond for proving out business value. This is what your CFO will care about
- NPS — How likely are our consumers to recommend the design system to a peer?
- Contributions — A good way to measure the buy-in to the design system from other product, design, or engineering teams
- Adoption % in code — Nice to see how well adopted the system is
- Adoption vs Core Web Vital score — A useful way to prove the value of the system through tangible numbers
Thanks to Joe Cooke & Nick Dorman for their collaboration on this piece.
