The Live Summer of Sport
As we enter the concluding stages of the European Championships and the dust begins to settle on a phenomenal summer of sport, this post explores some of the approaches BBC Sport and BBC Live use to serve rich digital experiences at scale to our audience as the action unfolds.
This summer tens of millions of us found ourselves entranced by the beautiful game, action from the courts of Wimbledon, and the best in Europe competing across a variety of sports in Glasgow and Berlin. For many this meant a trip to the local pub, fan festival, or just the living room sofa. However for those with immovable commitments BBC Live was vital for staying in the loop while the boss was looking away or before the soon-to-be-exchanged vows.
BBC Live launched in 2014 as part of BBC Sport’s digital offering for the Sochi Winter Olympics and Brazil World Cup. It has since been used across the BBC to provide coverage on everything from the results of the EU Referendum to Glastonbury. This widespread adoption meant developing BBC Live to have suitable flexibility and reusability to cover the range of use cases for the brands using Live.
BBC Live is built on top of Morph, the BBC’s serverless platform for servicing data and html content. Morph leverages Node.js for backend services and React.js for frontend HTML views. Crucially Morph is built to utilise a modular approach, building templates that can import and call other templates, serviced by an interconnected series of small, stateless nanoservice components (for more details, see Jacob Clark’s post regarding Newsround’s use of the platform).
The widely varied nature of Live is a brilliant fit for Morph’s modular approach. There are modules specific to sports, such as particular score displays for football or cricket, event-specific modules such as tables for Olympic medals and the World Cup groups, and more standard, multi-purpose modules such as the component used to provide the live-updating stream of text and media posts that is a fundamental feature of all live pages.
Maintaining separate versions of the page for individual sports, tournaments, or even TV shows such as Springwatch would be a burden in development terms as well as in terms of serving these pages reliably at scale. Being able to assemble pages flexibly through the use of appropriate modules served as and when provides the requisite adaptability without compromising on reliability or performance.
Morph Templates
These modular components, Morph ‘templates’, can either serve data or a view. View templates assemble React.js views, collating data returned from data templates, child view templates, and standard React.js components. These view templates can be served to the audience to be rendered or instead be consumed by higher-order view templates to produce more complex views.
Data templates will transform data from one or more sources (such as scores, possession statistics, live standings, and so on), either from other data templates or external suppliers, for consumption by view templates (but are never consumed directly by the audience). These data templates are able to consume multiple other data templates in order to transform data for specific tasks or features.
These templates are cached individually, from a top-level template that is served to the audience, to some template serving data to a parent template. Crucially, for a dynamically updating page this caching reduces the amount of data surfacing through the network, isolating the fetch of data to only that which is fresh, and rapidly serving a cached version of a page at the point at which someone navigates to the page.
With our live pages peaking at over 4.4m unique browsers for the Colombia vs England knockout match, Morph’s highly performant and scalable nature was imperative to our capacity to deliver a successful World Cup. With these browsers listening for updates every thirty seconds this compounded approach to caching ensures an efficient and effective way of delivering timely updates to a massive audience.
New and Improved Live Reporting
In the opening stages of the World Cup we completely re-engineered the crucial component of our live page that is responsible for serving the stream of text and multimedia posts, rebuilt from the ground-up to enhance performance and improve reliability. The old endless stream of posts (fetched via the ‘show more’ button) gave way to a new ‘paginated’ text stream (where groups of posts are separated across pages within the main page). Paginating the stream of posts allows more comprehensive caching and more exhaustive search engine optimisation. Most crucially, it supports a more efficient data flow both internally between templates as well as a reduction in volume of the live push data consumed by clients. This reduction in data volume means reduced operational costs and, for our audience, decreased data usage. With millions of browsers using Live pages, every bit of superfluous data served translates to unnecessary cost and load so improvements in this area are of significant benefit.
Many of these major events present challenges of scale, but one slightly different challenge of scale was surfacing live video streams of up to 16 courts from Wimbledon to our audience elegantly and effectively. UX improvements to our promotional components that better surfaced this range of live content were developed and shipped during the World Cup in time for Wimbledon to open its gates. Another UX improvement was the expedited release of an upgrade to our ‘reactions’ feature. In normal circumstances reactions allow our audience to like and dislike individual posts on a page, and the upgrade tweaked this to allow the audience to react in agreement to football ‘coming home’ or eventually to lament the fact that England would have to wait four more years for its return.
Player Rater
Introducing the Player Rater for the World Cup provided a new, live, in-play service that enabled the audience to award every player in a particular match a rating between one and ten. Traditionally, content for Live pages is built for consumption by audiences. The Player Rater takes an alternative approach with its primary purpose being for the audience to interact and participate, as much as the resulting ratings are for consumption during and after matches. Despite being a fundamentally different product to Live pages, however, the Player Rater has a very familiar structure to our Live pages, being built from Morph templates and even reusing many of the same components and backend services.
The Player Rater exists primarily as two Morph templates; a view template (the front-end component that is embedded into articles on the BBC Sport website) and a data template. The data template takes detailed data from BBC Sport’s data partners and raw ratings from the backend store tracking these ratings. This data is then reduced and transformed to provide details of the players that play (and substitutes), ratings, player images, and other metadata such as whether the match is ongoing.
The view template then calls this data template and presents the result as a React.js view to be rendered as part of a higher level template (an article) before being served to the audience. Again, Morph caches these templates separately to reduce internal traffic and to optimise response times for clients.
With the knowledge that the Player Rater would be promotedon programmes that would draw audiences in the region of eight figures, the backend solutions responsible for handling this traffic had to be able to count submitted ratings reliably and in near real-time, while scaling to accommodate sudden dramatic traffic spikes. Teams across BBC Design + Engineering have, for some time, already been tracking counts in near real-time, such as for reactions on BBC Live and video play counts on BBC Ideas. These counts have been powered by the Counting Service.
The Counting Service
From the inception of the Counting Service it was built to be highly scalable. The Counting Service has offered suitable performance and reliability to accurately track and return, in a timely manner, reactions to a volume of concurrent commentary posts across multiple live text streams. When viewing Live pages, there are typically 20 different posts to react to. Scaling the range of interactions to 22–30 players was a straightforward task allowing the confidence to reuse the Counting Service, as an already well-proven system.
As with Morph, the Counting Service deliberately avoids a monolithic architecture. The approach in this instance is a pipeline of serverless and stateless components pointing to a store of JSON documents. Or rather two separate pipelines, one for reading and one for writing.
Reading of the Counting Service is relatively low demand. A data template within Morph makes requests to the Counting Service at intervals. This data template then provides cached responses to parent templates/components. These requests from Morph are met by the Counting Service using a serverless compute function to read the appropriate JSON document. The response of read function is surfaced through the network back to the template in Morph. Morph’s caching means that no matter the demand for the ratings for a match, the read requests to the counting services are reasonably consistent, with Morph absorbing the bulk of the load for read requests.
Submitted counts, however, are entirely variable. During a football match, events in play such as goals, fouls, and full time can cause huge spikes in submissions. Calls-to-action promoting the Player Rater as part of our television coverage also compound the volume of submissions. With more than 2 million ratings submitted during England’s opening match against Tunisia, these spikes were a substantial challenge.
The Counting Service was able to scale to meet this demand by using an event-driven serverless approach. This approach ensures that components that could present bottlenecks if monolithic, particularly the write functions, can easily and rapidly be scaled out to reactively process submissions in parallel.
Data is queried from a firehose (a massive non-stop stream of data) of counts from BBC iD (specifically the User Activity Service, which records a variety of user interactions across the BBC). This queried data is then streamed to a serverless compute function, the ‘paralleliser’. The ‘paralleliser’ distributes the streaming data to several more parallel serverless compute functions. These parallel functions in turn update corresponding JSON documents in storage to reflect the updating count. It’s this highly parallel approach that enables millions of ratings to serve a sizeable barometer for our audience’s opinions on the player’s performance during the tournament.
Conclusion
Both Wimbledon and the World Cup draw substantial national interest, even before England’s best men’s football performance in 28 years. The football hysteria that swept England may have been exceptional, but the scope of the BBC and of Live means that our products are engineered from the ground up to be able to scale and support exceptional demand, whether that be from a major sporting event or a major breaking news event, which can rarely, if ever, be meticulously planned for.
While such a number of eyes are on our pages it can never truly be ‘business as usual’, but a tried and tested product means that our audience are able to be as close to the action of unpredictable major events as they are to any other.
