How we built BBC Sounds on the web
In late 2017 engineering teams that had previously worked on iPlayer Radio and BBC Music began work on a new product. Launched 16th October 2018, BBC Sounds is an audio product from the BBC that brings together live and on-demand radio, music and podcasts into a single personalised experience.
In this blog post we’ll talk about the technologies and engineering practices that helped us to build the BBC Sounds web application.
We designed BBC Sounds as a completely new web application. It’s built upon experience from developing and maintaining iPlayer Radio and BBC Music. It allowed us to re-evaluate architecture and take advantage of new web technologies.
Technology
Cloud infrastructure and performance
The web version of BBC Sounds was released to the existing iPlayer Radio audience as well as absorbing traffic from other parts of the BBC web estate. It is a cloud hosted application built to scale to handle a large number of requests, with the business logic encapsulated into the API layer and shared amongst clients.
As our applications migrate into the cloud from on premise infrastructure our engineering teams assume DevOps responsibilities. We often use Gatling for load testing, along with mock servers that provide upstream responses for specific scenarios. For BBC Sounds we developed new metrics and defined actionable alerting. This improved application availability making it more resilient and scalable than ever before.
To find out how BBC serves its web pages to the public, check out the post “How we deliver BBC pages to the internet”.
New stack
Building a new product provides an opportunity to start using new technology. BBC Sounds has been built using Node.js and React with server-side rendering.
“BBC Sounds was built from the ground up, using Node.js, React, Redux and Express. The migration to Node.js wasn’t completely alien to us, as by 2017 we were already using it for client-side pipeline, such as Gulp, Sass, minifying and linting tasks. What was new was transitioning to server JavaScript using Express as web framework for Node.js. We chose it for its simplicity and minimal footprint, allowing us to get application setup quickly, but also providing engineers with freedom to define development style.”
— Jason Williams, Principal Software Engineer, BBC Sounds
We began by thinking about the UI components, and how they would be laid out. Our existing Radio network home pages (Radio 1, Radio 2 etc) were also built from UI components, but they were completely independent packages separately fetched to build the page. The problem with this approach was that modifying the components in the browser remained difficult, as they were composed server-side (using Twig templates). We needed to change our approach.
Building Components with Atomic Design
By using React to build components we can render sections of the page server-side and then continue to modify them in the browser without duplication of code. Some of these components need to be reusable, so we needed a versatile, but consistent approach.
We used Atomic Design methodology to break components up into distinct levels: atoms, molecules and organisms.
Here is an example using Radio 1’s Listen Live roundel on the homepage:
The process starts with the design of a component from our UX team. As you can see, they want to show the user what’s playing live on our radio networks.
We start to break this down by identifying the smallest discrete units of the design e.g. the ProgressBar, the live Label, a ResponsiveImage etc. These are atoms which we can compose to represent a single network, lets call it a NetworkItem molecule. Finally we can represent our networks in a scrollable list — a Carousel organism.
Each component we then build and test separately, with API simplicity and reusability in mind. All that’s left is to hydrate the components with the data from our API and voila, it renders! The more reusable components we build, the faster we can test different configurations with users, and deliver new sections of Sounds.
Rendering on the server and isomorphism
We run server-side JavaScript alongside a JavaScript front-end framework. It helps to avoid having disparate languages for rendering and interacting with the UI. The application renders the full markup of its initial state in the server context. Then, we deliver both the state and a copy of the transpiled application, to pick up where we left off client-side.
When a user visits a page in Sounds, the application responds with the static HTML for the page requested . Users don’t have to wait for JavaScript to load. Then, as the React app mounts atop the existing HTML, components become more interactive; buttons appear on items to fetch more episodes from a podcast, or allowing users to subscribe.
The benefits of this approach are multiple: ready-to-render initial responses are great for SEO and accessibility where good network connectivity and JavaScript is not always a given. But for engineers it’s also powerful to remove the context switch between client/server languages and tools, to code one language everywhere.
— Stephen Roberts, Senior Software Engineer, BBC Sounds
React
Choosing a front-end framework from the plethora offered up by the JavaScript ecosystem is no small feat for a new product. Our decision to use React was ultimately guided by some core features we knew Sounds would need to be future-proof and flexible to changing requirements, but also scale horizontally with our development capability.
React is among the frameworks which implement a virtual DOM (VDOM). The VDOM is a layer of abstraction which represents the UI in memory and employs a “diffing algorithm” to apply updates to the DOM efficiently given a change in application state. This frees us up from writing boilerplate, data-binding / DOM manipulation code, to build the audience-facing components which make BBC Sounds.
React is a mature, open-source framework from Facebook, with a large community of developers and learning resources. Many large-scale applications run on React in production externally to the BBC, but also internally — product teams have been reaping the benefits of using React and React-like frameworks. Choosing React therefore, leaves the door open for inter-product code and component sharing.
Redux
Delivering richer experiences around our content on the web translates to added complexity in frontend logic — so it becomes increasingly important for engineers to be able to: reason about the internal state of our JS, communicate state changes to UI components efficiently, and debug when things go wrong.
In Sounds, we use Redux as a predictable state container to provide our React app with a central, read-only store for its state. The state store can only be updated via a set of pure functions called reducers. These functions take the current app state plus a plain JavaScript object describing a state-change (an action), and update the store accordingly. React-redux then connects the data from the store to React components, creating an easy to understand pattern for unidirectional data flow.
In this way, application state is always predictable; any sequence of actions can be replayed on state with the same result. And, since the reducer functions that handle the changes are pure, they’re very straightforward to test.
Just as we changed the way we build applications, we also changed the way engineering team works and adopted new practices.
Engineering team practices
Each software engineering team comprises of a project manager, product team (e.g. product owner, business analyst), embedded UX designers and QA testers, team lead and software engineers, with streamlined processes and agile methodology in place, similar to other BBC teams.
We use a variety of tools to speed up our development process and facilitate communication, e.g. GitHub, custom Slack integrations etc. There’re also various practices that engineering team has been following — some of them are listed below.
Cross-discipline collaboration
Developers often pair with UX designers, especially for branding related tasks and fine-tuning interactive experiences. Developing the component library has also been a joint effort from both engineering and UX teams to ensure consistency for users.
Running regular sessions with engineers working on the service layer and collaborating with other client teams ensures consistency in requirements and reduces complexity for the clients. You can read more about developing Sounds in the API.
Open source and inner source
It has been an ongoing aspiration within the BBC to open source more as well as use inner source tools and libraries.
Here’ are examples of open-source BBC libraries that have been used for Sounds development on the web:
- bbc-a11y — BBC Accessibility Guidelines Checker
- express-simple-timing — Express middleware that sets Server Timing API headers and optionally sends timers to stats systems.
- BBC Grandstand — collection of common CSS abstractions and utility helper classes
- speculate — automatically generates an RPM Spec file for a Node.js project
Architecture decision records (ADRs)
Following API-led product development, clients of the API contribute to architecture decisions. Depending on context and complexity, these decisions could be discussed in the form of pull requests, white-boarding sessions, and stubs.
Architecture design decisions are short documents which enable our teams to have a current understanding of the context and decisions the team is making. As decision is made, it gets documented as an ADR (Architecture Decision Record) and added to version control with the code.
“The hope is these can act as a unit of knowledge being passed across team boundaries to allow us, much like a distributed system made of people, to achieve an eventually consistent base of knowledge of our technical estate. They also form an historical record for a team, to help them bring people up to speed and remind themselves of their own technical foundations.”
— Paul Caporn, Lead Technical Architect BBC TV&Radio
OKR methodology
In Sounds each team is responsible for setting its own Objectives and Key Results — a methodology which is described in detail in the book “Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs” by J. Doerr. OKRs are meant to set strategy and goals over a specified amount of time for teams and individuals. At the end of that time period, OKRs provide a reference to evaluate whether these objectives have been met.
A/B testing
While developing Sounds we started extensively using A/B testing tools and success measures to ensure that the changes we’re making are the right ones for our audience. Every visit of an audience member is an opportunity to improve the product. Our experimentation toolkit allows teams to quickly test variations against an existing experience to determine what the optimal approach is in terms of a set of metrics. You can find out more about experimentation in the BBC in this blog post on optimising iPlayer experience.
10% time
Sounds have been making use of 10% time initiative adopted by the BBC software engineering teams that enabled developers to research and implement innovative solutions. Typical sprints in Sounds teams lasts two weeks, which means that one day per sprint is dedicated to 10% initiatives. This ensures 10% doesn’t skew delivery estimates for project managers, allowing flat distribution of allocated time per sprint. 10% time has been highly beneficial for engineers, giving space to grow, learn and think outside the box.
What’s next?
In 2019 we’re looking to personalise BBC Sounds even more, and follow our data-driven approach to implement meaningful improvements. By knowing our audience we aim to deliver a better experience of sounds, innovate on content, challenge our users, and provide an equal opportunity to access the best content for everyone.
If you are interested in working on projects like BBC Sounds, solving exciting, complex and large scale problems, why not come and join us? See latest jobs in D&E TV & Radio.
