The Tech Stack that powers BBC iPlayer
A snapshot of our overall technical estate
Background
BBC iPlayer launched 10 years ago in 2007. It was the same year Apple launched the iPhone which changed how people interact with their mobile devices and transformed the mobile industry. I would like to think BBC iPlayer has had a similar impact on how people watch TV in the UK and how the video on demand industry has grown and evolved.
The goal of BBC iPlayer is to provide UK Licence Fee Payers with BBC TV whenever, wherever, and however they want.
10 years on, our audience can watch over 3,600 hours of on-demand BBC TV content on over 10,000 different devices, from computers, smart phones and tablets to a wide range of connected TV devices. On top of that, we have over 170 hours of live streams every day from all the BBC channels plus extensive live coverage of special events such as Glastonbury and Wimbledon.
Usage of BBC iPlayer has continued to grow. Last year alone saw almost 3 billion TV programme requests.
The Challenges
The scale of BBC iPlayer has translated into many technical challenges. Our API has to consistently handle over 140 million requests every day. Our video distribution platform has to tirelessly encode and distribute all the on-demand and live content while balancing quality, performance and cost.
There is also the constant challenge of delivering the best user experience to our audience across all the supported devices and platforms. We have to keep up with the opportunities that new platforms and technologies are providing us. 10 years ago mobile usage was minimal, let alone connected TVs, which are now our fastest growing platforms.
In this post, we will outline some of the technology and infrastructure that powers BBC iPlayer and the challenges each part faces. It will be a snapshot and very high-level view of our technical real estate.
So let’s start with the user facing products on the 3 key platforms we support.
Web + Mobile + TV
BBC iPlayer started life on the web and has had four major versions along with innumerable minor iterations. The most recent major revision of the web offering was a responsive web app launched in 2014 which supports everything from low-end smartphones through tablets to a wide variety of desktop browsers.
We want to make sure everything is presented to our users quickly irrespective of the type of devices they are on, so we deliver most of the UI elements from the server. Our web apps are built predominantly on AWS using Node.js.
As smartphone usage grew, we took the opportunity to cater for the increasing number of mobile users and built native apps for the two dominant platforms iOS and Android — since launch, our apps have been downloaded over 40 million times. These mobile eco-systems bring a particular challenge to maintain parity of features as they use different technology stacks (Objective C/Swift for IOS and Java for Android) but from around 2012 we took the decision to always develop any key features on both platforms simultaneously.
BBC iPlayer on connected TV is the most recent addition to our multiscreen world, introduced in 2011 and now the fastest growing platform. The wide range and rapid evolution of Connected TV devices means that it is the most fragmented platform for us to develop on. An important goal for the BBC is accessibility for all so we aim to be available on the widest possible range of devices. This presented a challenge which lead to the birth of our TV Application Platfrom (TAP), a single page, data driven application used to surface all of the BBC’s content offerings, not only iPlayer but News, Sport and large audience events too. TAP provides our product abstraction and it’s built upon TAL, a BBC open sourced abstraction layer which provides a uniform device surface for writing TV applications. It has helped move the industry forward and bring some coherence to a fragmented platform.
Data - treating our product as an API
All of these iPlayer platforms use a shared API, the iPlayer Business Layer (iBL). This API gathers information from a variety of internal sources of metadata, transforms that data and applies the iPlayer product logic, allowing us to serve a consistent product to all of our clients (Web, Mobile & TV). The Web and TV platforms apply further platform specific transformations before the user client receives it but, logically, the iBL encapsulates the iPlayer product definition.
This API needs to be resilient, fast and flexible. On an average evening, it handles over 3K requests per second with that number only increasing over time. The service is built using a microservices architecture of over 70 different components with the majority built using Node.js. This suite of services handle functions such as fetching and transforming the latest programme data in preparation for serving, exposing personalised content selections or promoting the curated editorial programme selections on the homepage.
Media
Media processing, distribution and playback is a centralised capability that is used by multiple BBC products. Having high quality media available for our audience is the most important aspect of iPlayer. All media travels through this pipeline whether it be event streams supporting live music and sport, long form programmes on BBC iPlayer and iPlayer Radio or the short form clips used on BBC News.
There is a standard media player (SMP) optimising the playback experience for browser-based playback on our Web estate. It provides functionality such as subtitles, seeking, volume controls and selecting the correct encoding and streaming formats for the device. It’s also responsible for adapting the bitrate consumed for the smoothest playback experience given the network conditions. The HTML5 player uses DASH to provide playback to a growing share of our audience, as major browsers deprecate support for Flash.
Programmes come from a variety of sources supporting both linear broadcast (simulcast), IP broadcast (webcast) and pre-delivered sources. A set of AWS Java services, collectively called Video Factory, encode the media and push it to the distribution edge nodes. Our media is distributed via a mixture of content delivery network (CDN) edges, including BBC’s very own BIDI service, to provide resilience and cost effective delivery across the estate.
Personalisation
Work is continuing to make BBC iPlayer a more personal experience and identifying the user is key to making it work. BBCiD is a user identification service shared across multiple products and platforms. It follows the OAuth 2.0 specification to make integration for clients as standardised as possible.
On our native mobile platforms we use shared libraries to achieve this integration while the Web platform has a standard component to call upon. This usage of componentisation ensures the user has an identical experience irrespective of which BBC product or service they are making use of.
Once the user has signed-in, we can start understanding what the user is doing across devices. The User Activity Service (UAS) allows us to push user behaviour into a single data store. From there, we can offer users a more personalised experience, from features like cross device resume points to personalised recommendations. We are just at the start of this evolution of iPlayer and it brings the challenges of designing for serving at scale into focus as more requests at the point of consumption become harder to cachable.
Tools
The development pipeline begins with source control and continuous integration, for which we make use of GitHub as a code repository and Jenkins for CI. The artefacts that are generated then need to be deployed to our AWS infrastructure.
The BBC is fully embracing the possibilities of the cloud and our central team has created a thin layer of abstraction sitting on top of AWS called Cosmos. Cosmos streamlines the process of building services at scale and is structured for a continuous delivery workflow. Definition, deployment and support of the AWS services are the responsibility of the development teams working in a Continuous Delivery, DevOps style.
What’s next?
There is so much to do. We still need to make BBC iPlayer as resilient and reliable as traditional TV. So we’re focusing a lot on Quality of Service. We are continuously iterating to improve the user experience and we believe a more personalised BBC iPlayer is key. This is a big challenge for us from ‘how do we measure the impact of our changes?’ to ‘how do we scale our services technically to remain performant whilst also being cost efficient?’.
As the product continues to iterate and deliver a better service for the public, areas of learning and experience will be added to this blog. The intention is to highlight interesting technical problems we face and the thinking behind our approach and solutions.
Join us
If you’re interested and excited by the technical challenges we’re facing, we’re always looking for driven engineers to join us. We’re not prescriptive about what languages you know since we think a good engineer can learn quickly. That said, experience with iOS, Android, JavaScript or Java would definitely give you a head start.
You can see all our job listings for BBC iPlayer here by searching for iPlayer.
