BBC Online — 2020 in review

2020 has clearly been a very unusual and oftentimes painful year in many ways for just about everyone on the planet. Like many organisations, we’ve had to shift the majority of our staff to working from home in a very short timeframe — this was easier for some teams and people than others.

It’s well documented that in troubling times, the world comes to the BBC. Nowadays, that means we see big increases in demand for our online services. That extra demand and adaptation to new working patterns has been coupled with enormous changes in our output — far fewer TV programmes being made, much more frequent breaking news events, a significant bump in the volume of online education we provide and regular live press briefings, to name but a few.

With all this going on, it’s easy to forget about the fantastic work which has been going on, so we wanted to follow on from last year’s “BBC Online — 2019 in review” and document 2020. Lots of our teams have contributed so you’ll be able to see some of the highlights of their achievements.

BBC Children’s and Education

The Children’s and Education team are responsible for building and maintaining 4 flagship CBeebies app propositions, 1 CBBC app proposition, 350+ interactive games and content items, the CBBC, CBeebies and Own It websites along with the Bitesize website and app.

• Jacob Clark has written a blog post “Shipping Progressive Web Apps everywhere”, which outlines in detail the journey we’ve been on these past 12 months transforming how we build our 5 flagship Children’s apps. Shifting away from proprietary native app frameworks to building on modern, open, web standards, developing universally distributable web apps on a new platform.
• We’ve been working hard over the course of the past year ensuring children and parents have access to resources to Daily Lessons to continue their education as well as developing new, personalised and adaptive learning experiences.
• Across our games estate, we’ve been working on ways to provide users with onward journeys into more games they might be interested in through our cross platform “Exit Tile” whilst developing features such as Shops within our core game engine technology Genie.

BBC Account

The BBC Account team are responsible for enabling sign in or registration for a BBC Account. This helps to enable a personalised service for our audience members across TVs, the BBC website and mobile applications.

In 2020 we’ve brought lots of great improvements to BBC account:

• Children’s profiles: We’ve worked closely with the iPlayer team to enable our younger viewers to see the content that’s more relevant to them using our new children’s profiles on televisions.
• Password breach checker: Building on an idea from our engineering 10% innovation time, we aim to help our audience to create safer passwords by checking if they have been used in a previous data breach. Read more here.
• Showcasing Account Memories: To remind users of the amazing content they have enjoyed and to associate happy memories with the BBC, we’ve started to play back some of the user activities within the Account space.
• Performance improvements: We’ve iterated and optimised our applications to enable better resource utilisation and autoscaling. This has allowed us to cope with the increased demand for our accounts during lockdown.
• Rolling refresh tokens: To improve the experience for our TV users, we’re keeping regular users signed in for longer.
• iOS and Android: We’re making better use of the capabilities of these platforms to make it easier for our users to sign-on to multiple BBC apps and websites.

BBC World Service News

The World Service News teams are responsible for building and maintaining an open source platform called Simorgh which is used for the BBC’s 41 language World Service News websites. We’re made up of 13 Engineers, 2 Business Analysts, 2 Product Managers, 2 Delivery Managers, 3 Testers and 7 User Experience Designers.

This year we’ve been working on:

• Simorgh: Over the past 12 months we’ve migrated our pages which are spread across 41 discrete sites from a legacy PHP monolith to a new React based application. This application is called Simorgh, an open source, isomorphic single page application developed by the World Service languages team.
• Performance improvements: A large proportion of the BBC World Service audience are on slower 2G and 3G networks, they use lower end budget-friendly android handsets or feature phones. Blocking JavaScript requests dropped by 100% from 9 to 0, JavaScript requests dropped by 79% and total page weight is now 60% smaller than before. We have made other efforts to improve our web page performance, meaning our worldwide audiences can access stories quicker than before.
• New reading experiences: We have started work on a project to deliver a new article experience to our users, which will allow our editorial teams to more easily create content for our audiences.
• Improved media experiences: We have launched new ways for our worldwide audiences to access audio and video content on the World Service News websites.

BBC Website (including News, Sport and Home)

This year has been a huge year for the BBC website, which has completed its rebuild and move to the cloud. Tens of millions of people now consume articles, video clips, and other content on a rebuilt site that’s faster and more accessible. Multiple pages have been rebuilt, including the BBC Home Page, topic pages, articles, clips, and search. And the technology used has changed, from PHP to Node.JS/React. It’s also partly hosted on serverless technology, and we believe we’re one of the largest sites in the world to do so.

This recreation of our website, ready for the next decade, has been a huge programme of work. It’s tackled duplication and cross-team challenges, ensuring new site is as efficient and high quality as possible. Matthew Clark has written a separate post on the approach taken to make such a large change happen.

Radio & Music Services (BBC Sounds Back End)

Radio & Music Services are a team made up of around 12 software engineers and 5 software engineers in test, along with product and business analysts. We’re responsible for building the API that encapsulate the product functionality of BBC Sounds. This API is used across web, apps and other clients to provide the Sounds Experience, drawing on content from around the BBC.

This year we’ve been working on:

• Audience: Rolling out Sounds to International users.
• Tooling: Migrating our build system to Code Pipeline.

BBC Voice+AI

The Voice+AI team are responsible for building & maintaining several Voice-based and AI-supported services in the BBC. We’re made up of 45 engineers, 9 Product Managers, 5 Delivery Managers and a team of UX designers and editorial.

This year we’ve been working on:

• Beeb, the voice assistant from the BBC: We’ve built, from the ground up, a fully-functioning voice assistant and released a Beta demonstrator on Windows 10. Built in partnership with Microsoft, we’re using several new technologies, including DirectLine Speech and BotFramework, a custom Natural Language Understanding model, and a stellar bespoke neural voice that we developed for the BBC. Beeb for Windows 10 is available today on the Microsoft Store.
• BBC Corona Bot: Launched to the BBC News Facebook pages’ 50 million followers. Built using Azure’s QnA Maker Cognitive service, the model for the chatbot was trained using editorially curated question and answer pairs on the latest information from lockdown rules, to relevant medical information with support from the NHS.
• Songbird: In partnership with BBC World News we launched a ‘listen to this article’ feature. An entirely new content format for the BBC the feature provides text-to-speech rendered articles using our bespoke neural voice. Try it out on The Life Project.
• Local news bulletins: At the beginning of lockdown in March, we launched an ambitious project in partnership with Nations and Regions to make available news updates from all 40 local stations in England, Scotland, NI and Wales. In the midst of a pandemic, local information is more important than ever.

BBC Online Technology Group (OTG)

The Online Technology Group is responsible for the provision and management of the BBC’s core online services such as networking, DNS, CDNs, audience-facing TLS, load balancing, caching, routing and high-availability facilities to content origins, our on-premise hosting platforms, and media analytics.

Here’s what some of our teams have been working on in 2020:

Website traffic management

• DNS: Migrating our core DNS platform to a new supplier and building a robust, fast, low-cost, automated deployment and testing CI/CD pipeline which is driven from source control.
• Statistics: Adapted our Reporting API collector endpoint to also receive Web Vitals data from our client-side code.
• Website traffic management: Removed our old traffic managers and caching layer from the majority of traffic on www.bbc.co.uk and www.bbc.com.
• Quality & efficiency: Made our www edge services more consistent & efficient (cache offload) across CDN & in-house traffic managers (GTM).
• Traffic peaks: Like many organisations, the events of 2020 have resulted in regularly/constantly elevated traffic to our websites. We’ve seen around 40% more than we did in 2019 — a big bump on the usual year-on-year increases.

Media Distribution

BBC Internet Distribution Infrastructure (BIDI) is an in-house Content Delivery Network (CDN) designed for high volume distribution of iPlayer video and audio streams.

• Traffic peaks: BIDI’s traffic peak reached 1.2 Terabits per second in 2020, serving nearly 250,000 concurrent iPlayer clients.
• SmartShard: A new playback routing algorithm called SmartShard has been developed, greatly improving cache efficiency and automating capacity management around the network.
• Capacity: BIDI estate size has doubled in 2020. BIDI now consists of 102 caching servers, across 24 datacentres, using over 1,200 Solid State Drives. Deployments are run in both BBC sites and within 3 UK Internet Service Provider networks.
• Resilience: Automatic failure detection is now active, reducing service disruption when either the hardware or underlying network fails.

For a look at BIDI growth so far and plans for the future, see: virtualUKNOF November 2020 — Building the BBC’s media CDN.

iBL (iPlayer API)

We are the iBL team — the iPlayer Business Layer, and we are responsible for the data that powers all BBC iPlayer apps — mobile, web, and smart TVs. From promotions on the homepage, categories, channels, to personalisation, user data, schedules and recommendations. We are a team of 5 engineers, a product manager, and a delivery manager.

In the past twelve months we:

• Improved promotions and relevancy: Creating “hero” imagery to showcase high profile content, and using audiences’ watching history to highlight content they are interested in rather than treating everyone the same. This includes messaging why we recommend certain programmes, and tailoring homepage promotions for younger audiences.
• Safeguarding younger users: There is a breadth of content on iPlayer, and not all of it is suitable for all ages, so we are taking steps to allow users make better informed decisions.
• Technical Reliability: We’ve worked on our resiliency a lot — we have improved capacity and redundancy across about a dozen of our database-backed services this year. We’ve also moved some of our error handling to a CDN so we can more reliably serve peaks of demand.
• Developer workflows: We’ve already had our “infrastructure in code”, using CloudFormation. This year we have also moved our CI pipelines into code, and include infrastructure changes in our CI/CD workflows to make sure the code is always up to date with what is actually deployed. We are also iterating on our internal documentation, using OpenAPI schemas to ensure the data we exchange internally is what we expect it to be.

BBC iPlayer — Off-Product Discovery

We are the people that share the iPlayer catalogue and availability with partners and link up voice interactions from non-BBC platforms. It’s what allows you to choose to watch a programme on iPlayer from somewhere which isn’t inside iPlayer! Next time you say ‘watch his dark materials on BBC iPlayer’ to your fireTV stick or Chromecast with GoogleTV think of us!

Big deliverables in 2020 have been:

• Amazon catalogue and Voice integration: Following on from many months work, we are now delivering our standard catalogue and integrating with the Amazon video platform. We completed the work to allow you to find content and then control the playback of this content in iPlayer using voice interactions as well as from the remote. You can now see that Dr Who and Killing Eve have the newest episodes on iPlayer, available for no additional charge.
• GoogleTV catalogue and Voice integration: A very different technical solution completed quickly by following on from the lessons learned working with other partners and having a solid understanding of the user journeys that voice control opens up. Again, we have developed and delivered the code to link up google’s infrastructure with the BBCs and allow you to watch great content in iPlayer via voice control alone.
• LG Catalogue and Search: Having a strong, well-defined catalogue format and a means of integrating this with partners allowed the team to complete a VoD catalogue and promotions integration with the team from LG in a very compressed time period. This is what powers the row of links to content above the iPlayer logo on LG TVs and allows the LG voice search to find content in the iPlayer catalogue.

Digital Publishing

The Digital Publishing team work for the creators: the journalists, editors, curators, programme-makers, podcasters and more who make BBC online. This year we’ve been working on how they do what they do, moving away from online’s late-1990s roots to ways that get the content they make to the right audiences, at the right time (and more cost-effectively, too).

We’ve built:

• Passports, which rethinks how we describe the content we make. That means better recommendations, better curator tools, and a focus on how the audience discovers content. Looking for relaxing music in Sounds or good news stories? Passports gets it to you.
• Curation tools to allow editorial to showcase content. Alongside pan-BBC Topics, this lets us builds collections that span the BBC’s services, and will soon be used in Homepage.
• “Optimo”, a tool for writers that moves beyond the classic 500-word story and supports the articles audiences are asking for. It’s starting in BBC World Service, and will be a big focus in 2021.
• Signal Box, which brings together production data (things like ‘how much content did we make?’) with content data (‘what was it about?’) and audience data (‘how many people consumed it?’) in one place. That answers the questions our content teams need to ask.

Technology Strategy & Architecture (TS&A)

Technology Strategy & Architecture is a community of technologists and architects who help ensure the BBC makes technology decisions that provide the best value and most effective technology systems to deliver world class services to its audiences. It’s 8 different teams cover areas such as security, connectivity and digital products and represent disciplines from data science to broadcast engineering to architects who oversee the infrastructure of the systems we all use. Below is a snapshot of what some of the TS&A teams have been working on in 2020:

BBC Blue Room

The Blue Room is the BBC’s consumer technology research lab. The team offers access, insight and guidance on consumer technologies, new experiences and changing behaviours, through its Blue Room demo spaces across the UK and ‘on the road’ at public and industry events. This is what the Blue Room team has been up to in 2020:

• Launched ‘Blue Room in Zoom’ and delivered 400+ Blue Room tech insight sessions remotely to over 3,500 BBC colleagues across the world.
• Demonstrated at on-line external events including BBC Digital Cities and Sounds Amazing 2020.
• Presented ‘Synthetic Celebrities’ at the BBC’s first ever virtual Creative Diversity Festival (CDX), giving audiences a glimpse of the future, with demos of the latest in synthetic media.
• Opened the BBC’s Innovation Showcase to bring together some of the best of the BBC’s emerging tech, including exciting innovations in augmented reality, voice and immersive listening.

BBC Datalab

BBC Datalab is responsible for using machine learning to provide great recommendations across the BBC’s product portfolio.

This is what BBC Datalab have been up to in 2020:

• Significantly progressed development of the BBC’s in-house recommender systems and successfully launched a personalised recommender for BBC Sounds, as well as recommenders for several World Service language sites and the new BBC News app.
• Facilitated another pan-BBC effort to build on the BBC’s Machine Learning Engine Principles, a framework for responsible development of technology.

BBC Digital Ecosystem

BBC Digital Ecosystem is a small team of architects working mainly with the BBC Platform team to create foundational software capabilities for digital transformation. In 2020 the team have been focussing on:

• Building a suite of advanced software services to make BBC content media and metadata simple to access across the BBC and for our Partners, allowing us to get the best value from our content for our audience.
• Defining the shape and technology for the BBC’s next generation data platform to help us understand how to serve our audience better.
• Project Origin — a collaboration between the BBC, Canadian Broadcasting Corporation/Radio Canada, Microsoft and The New York Times to focus on one aspect of the disinformation challenge — content source verification. The goal of Project Origin is to apply clear signals to videos and pictures from publishers so you know the content is coming from where it says it is and has not been manipulated.

BBC Product Architecture

BBC Product Architecture helps BBC teams to develop and deliver audience experiences across all surfaces and modalities. In 2020 the team has:

• Provided architecture support and guidance to the BBC Voice and AI team across client facing products, such as Beeb and the BBC’s presence on Alexa, as well as efforts such as Songbird, which brings Text to Speech capabilities to bbc.com’s The Life Project.
• Successfully introduced the C4 Architecture Model to D+E teams for architecture visualisation and started to roll out Structurizr to evaluate the suitability of a tool in the production environment.

BBC News and Weather Apps Team

We are responsible for building and maintaining the News and Weather apps, across Android and iOS, for UK and international audiences.

• BBC News app beta: We are reinventing the News app from the inside out to give it a fresh feel. This includes recommendations, new ways to showcase the top stories, short video clips and an article overview to present articles in a short, digestible way (pictured). The first version is being rolled out to both Android and iOS before the end of the year.
• ABL: The Apps Business Layer is a backend for frontend technology that will produce app-specific data at its endpoint. This will make a faster, cleaner and more flexible app experience on the device with more work being done on the backend.
• Weather app: We started work on an updated weather app with split screen, animated icons and a weather widget for iOS 14

BBC Research & Development

This year BBC Research & Development is 90 years old. We have played a huge part in broadcast technology innovations from Digital Television to Object Based Media. Whilst the technology evolves, our purpose hasn’t. We look forward 3, 5, 7 years. All the time asking how we use technology to keep public services relevant.

Here are some things we’ve been working on in 2020:

• AI: Autumnwatch used R&D’s AI system — YOLO (You Only Look Once) to identify different animals captured by multiple cameras saving hours of production time.
• 5G: BBC R&D, including IBC’s Accelerator programme, were integral to an early stage proof of concept experiment exploring cutting edge 5G remote production; and 5G Records a major EU project looking at professional production workflows across audio, video and immersive.
• Sustainability: Our Sustainability team released research exploring the energy used to deliver and consume television and radio.
• Streaming: We published the technical specification for Dynamic Adaptive Streaming over IP Multicast, which would enable the distribution of live television at scale over the Internet.
• Sounds Amazing: Something that was better in Lockdown! We brought together producers, presenters, reporters and engineers to talk about what will be next and later for audio. Over 3000 experts attended this virtual conference from around the world.

And lots more — https://www.bbc.co.uk/rd