General election 2017 at the Times: what we built and how
In April this year, British newsrooms were given just six weeks to prepare their coverage of the snap election. A big event like this usually requires months of planning and careful cross-departmental execution, so it was a rush to pull everything together and give our readers an election experience worth paying for.
We’re very proud of what we managed to produce at such short notice, and just in case you missed it in the scramble, we’ve put together a short summary of what we published and how.
What we built
Several projects deserve a mention for our pre-election coverage.
We brought readers a daily briefing from the Times and Sunday Times political teams, published every day at 5pm, for the five weeks leading up to the election. An important complement to Matt Chorley’s morning Red Box political newsletter, this evening briefing was a good spot to promote other articles from the paper and capture a different audience — those on the commute home rather than the commute in. It also featured a bespoke “poll of polls”, as well as a constituency of the day highlighting key data (such as the Leave vote share and demographic indicators).
Chatbot
As an experiment, we also created a chatbot based on Facebook Messenger. Pure R&D projects are rare at The Times — our focus is on satisfying our subscribers on a daily basis — but the snap election was an excellent opportunity to flesh out a template and to try out interesting editorial approaches.
In this instance, we wanted to help readers burst their ‘filter bubble’ and send them content they would not necessarily see outside of their immediate social circle. We sent a list of articles to readers each day based on declared vote record and voting intention for June 8. From the readers’ postcodes, we sent them server-side rendered images of our daily updated poll of polls, as well as some data about their constituency (MP, 2015 vote, Brexit vote, some demographic indicators).
The prized element of our coverage on the night was of course our results page. It’s the moment interactive and web teams from across the country look at each other to see what the competition has prepared.
Our results page included everything readers would have needed, whether they came to us in the morning or followed the results coming in through the night. Seamlessly redrawing and updating itself, the page provided near real-time information about the latest declared constituencies, as well as how the national results looked at any given moment.
The top of the page boasted a bar and dots chart, which we saw as an innovative way to display the exit poll and the results — both in terms of vote share (bar chart) and of seats (dots in the bar).
The Data Journalism team assisted this work by putting together key datasets to be used by the apps. They compiled and analysed constituency data, including demographics from multiple sources including the ONS and Parliamentary Petitions. In addition to this, they wrote two analysis pieces: one explaining the different approaches being taken by polling firms, the other one, which proved to be insightful, outlining how the mobilisation of young voters could narrow the Conservative lead, among other factors.
Our post-election coverage and analysis was led by the Data Journalism team, making good use of the results data collated overnight. They plotted the results against demographics data, using an innovative ternary chart illustrating how Britain was more polarised than ever.
We produced a series of gifs and videos based on screen recordings annotated by the Social and Multimedia teams.
The technology behind the election
Our results page and our 5pm briefing were built in React thanks to our fantastic generator that scaffolds a custom app in minutes — and makes getting going very easy. This generator pulls in the basics of a React app, as well as all the furniture from thetimes.co.uk so we can inherit styles and work in an environment mirroring production.
All the features of these pages are built as separate components for maximum compartmentalisation. Practically, it also allowed the developers to work on separate components in a manner much similar to git-flow, based on feature branches of the code periodically merged into a master one ready for release.
We’re particularly happy to have invested into building things this way. We were able to pull out components as needed, relatively effortlessly. For the chatbot, for example, we needed to generate and render constituency results as an image. We forked the constituency component from the 5-for-5 page to recreate, server-side, 650 versions of it (one per constituency) to screenshot with the newly released Headless Chrome.
In terms of workflow, both the 5-for-5 and results page were built based on Sketch designs and specifications drawn by digital designer Jake Cox, which served as reference. Matt Taylor, leading the Development team, scoped these designs in detail to order features by priority, ranking some of them as part of the MVP, and some others as enhancements, given limited timescales and resource.
The charts and plots were built prior to being included in the final designs, in order to ensure it was feasible for the team to produce them up to standard. The results chart at the top was a particular cause for worry, and the prototypes studied both the need for SVG or Canvas due to potentially thousands of DOM elements.
A set of helper functions running in the background of the React app maintained our results page up to date. Server-side services were in charge of the polling for new data from the Press Association feed, while the front-end app loaded the resulting data every 30 seconds, feeding those down the page for seamless redraws. Refreshing the page is so 2011.
Finally, feedback on the pages was limited to a handful of people this time, as to limit the number of last-minute interventions from all corners of the newsroom. While clearly being an anti-pattern to what a feedback process should look like, it put more pressure on a core group of people and allowed us to work through the changes quicker.
We put a couple of days work into the chatbot. The quickest way off the mark for Callum Christie, our developer, was to crack on with familiar PHP and cron tasks. The bulk of Callum’s time was spent getting the right format to talk to Facebook’s API, and to fit the dialogue script.
In the future, we’d love to have a crack at Serverless patterns.
Stats, analytics, subs
From our results page, a number of people subscribed to the Times. The page gathered well over 100,000 views and our readers spent a good deal of time with us (around nine minutes on average). There were a few interactive elements on the page, such as the menus, constituency search, and constituency carousels — which 60 per cent of our readers interacted with. Our readers weren’t skim-reading, they were highly engaged.
The results are similar for our 5-for-5 page, except that we noticed a very high conversion rate from non-subscribers to Registered Access (whereby a reader registers their email address to get two complimentary articles a week).
A manifesto quiz collected massive registrations as well, and had significant traffic — all thanks to the Social team’s efforts to boost up the SEO.
Our chatbot, launched on May 22, attracted 400 users, 93 per cent of whom stayed with us until polling day.
“We chose to keep these pages behind our paywall because we believe this is content worth paying for. The number of readers who registered for access and subscribed to The Times and The Sunday Times proved us right,” said Alan Hunter, head of digital.
Performance analysis
Chris Hutchinson, senior developer in the Development team, worked on accessibility features for our web apps.
The 5-for-5 page was the first time we implemented full static rendering, while journalists were still given their familiar CMS to change and update the copy. This gave us a huge improvement in time to first meaningful paint, and time to interactive.
We also focused on ensuring we implemented appropriate keyboard navigation, among other accessibility features. We also optimised our resultant Webpack bundle to ensure we served a minimal amount of Javascript to the readers. Oh, we also use Preact to help reduce the bundle size even further.
Lessons learned
- The collaboration between Design, Development, and Data teams, led by Sam Joiner of the Interactives team, was a big success.
- Tasks and areas of responsibility need to be clearly defined so teams can work in parallel. In this case, the Data team was able to work on collating datasets based on a structure defined by the Dev team, while the devs were able to build with predictable, mock data.
- Inverting the feedback process to keep it under control is sometimes necessary and helped us to keep it manageable under a tight deadline.
- Precise scoping of features and hierarchisation manages everyone’s expectations.
- Build re-usables. And when the project is done, invest more time in abstracting those and keeping it ready for next time.
- If your own results page performs so well that the top editorial team uses it to get their news, you’re winning.
EDITORIAL AND PRODUCTION: SAM JOINER AND MATT TAYLOR
DEVELOPMENT: BASILE SIMON, CALLUM CHRISTIE AND CHRIS HUTCHINSON
DATA: TOM WILLS, LOUIS GODDARD AND STEFANO CECCON
DESIGN: JAKE COX
