Inside the BBC’s “innovation incubator”
What does a typical day in BBC News Labs look like?
We often describe News Labs as the BBC’s “innovation incubator”. It’s a good description of our role in the corporation, but it doesn’t give potential applicants — or friends and inquisitive family members, for that matter — a clear idea of what our team does every day in the BBC’s Broadcasting House.
So we decided to take a snapshot of a typical day in the office, as seen from the desks of seven of our developers, engineers and producers. Only a few of us specialise in one subject area, so each view is representative of the work that could land on any Labber’s desk during one of our project sprints. At the end of a two-month project cycle, we reshuffle our teams and start prototyping our next set of ideas. These generally fall into two categories: new tools to help journalists produce better content, or new news formats for BBC audiences.
And speaking of potential job applicants — we’re hiring! If this sounds like your dream day in the office, check out our ad for a front-end developer.
10:00 — Manish, Principal Software Engineer
I’m currently working with my colleague James to help transform the output logs from one of our messenger bots into a form that we can query, to gain insights on reach and usage.
We’re using some of the technologies and services provided by AWS to store and transform our log data, giving us flexibility to search it easily. The logs are currently going into AWS CloudWatch Logs streams and we’re looking to inject these into an S3 bucket using Amazon Kinesis Firehose. This should give us the flexibility to run queries directly against the log data using Amazon Athena without too much overhead. It also frees us from having to build and maintain our own database.
“I’m getting to learn about both new technologies and the editorial process.”
I frequently find myself working together with my teammates on similar tasks and this is exactly what I find great about my role looking after our infrastructure. Earlier this week, I was helping Grant to improve one of our Jenkins deployment jobs. Next week, I might be supporting another team who are helping migrate some of our internal hardware between datacentres, I might also be supporting our various projects to make the deployment of prototypes easier, or generally advising on good development practices. All the while, I’m getting to learn about both new technologies and the editorial process, from my teammates.
11:00 — Remi, Junior Software Engineer
I’m finishing up tweaks to the search on Window on the Newsroom (WON) 2, a six-week project that explores how we can help journalists enrich their articles with additional content relating to their story. WON pools data feeds from around the BBC into one accessible and searchable interface.
I just fixed a pesky bug and now users can sort their search results. Now that it’s working, I want to clean up and refactor the code before I create a Git pull request.
As well as using familiar tools like Git, React, AWS and Docker, at Labs we’re encouraged to get involved with fun cutting edge technology such as bots, facial recognition and Alexa — and I’ve had the opportunity to work on all three since joining last autumn. The other Labbers are so supportive, and I can’t believe how much I’ve learnt.
I’ve also had the opportunity to get involved in outreach work. I’ve shown off our work at the BBC’s Design and Engineering Town Halls, written blog posts for our website and also gotten stuck into state-of-the-art tech at MozFest.
12:00 — Grant, Bot Development Producer
I’m working on bots. They’re part of the BBC’s drive to make news accessible to every audience, not just the politically engaged. They’re great because they make it possible for a reader to join a complicated story “halfway through” — but they’re also tricky for journalists, because it’s harder to write an interactive, non-linear story than the traditional 400 or 2000 words, or three-minute TV slot.
So right now, we’re asking questions like “How can journalists get a chatbot online in 15 minutes to help flesh out a popular story?” Or, “If a story is presented as a Q&A, will that stop light readers giving up on complex issues?” To test this, I coded a bot-builder in Rails, whose output you can see in the in-article bot here. It’s a team project: the front end was coded by Tom Maslen, an ex-Labs staffer, and the BBC Visual Journalism Team. And James and Luke from Labs have been working on making bot output available in Facebook Messenger.
And the future? This afternoon, I met with BBC R&D to discuss how we could centralise the BBC on a single “bot story” data format. Then it should be easy to support bots across all content, all channels and right across the Corporation. But that conversation immediately turned into what, to me, is a much more interesting discussion:
“Can we use bots to make it a very normal, standard thing for an audience to talk back to news organisations? And if we had real two–way conversation with audiences, how could that change news gathering?”
13:00 — Team Lunch 🍪
On sunny days, you might find us in Cavendish Square. On rainy days, we might be in the BBC’s Media Cafe overlooking the newsroom. And regardless of the weather, you’ll almost always find a Labber restocking our biscuit tin or making a round of afternoon teas or coffees for the team.
14:00 — Tamsin, Software Engineer
I’m working on a pilot project experimenting with the new AMP Story format. For the past few weeks we’ve been working closely with journalists from the Russian and domestic services and we’re about to publish our first three stories, after making a few final adjustments.
My colleague Pietro has improvised a simple CMS which converts the data to AMP Story format using some open source software created by Mic News. I’m uploading some final images to a S3 bucket and checking that the videos work for another story we’ll be publishing tomorrow. The project has involved a lot of collaboration with teams in BBC R&D, the World Service, Operations, Media Services, Marketing and Visual Journalism, and everyone’s been very helpful in making it happen, especially with a certain event in Windsor keeping everyone busy.
My colleague Lei has received some final analytics tracking labels and added them to the project so we can judge the success of the stories, which was our final blocker — we can now go live! You can read our first three stories here, here and here.
I email a few stakeholders to let them know the project is now live and the response is very positive. I bump into Nathalie Malinarich, Mobile & New Formats Editor on the stairs and we go through the stories together. She notices that the onward journey links aren’t working, so I head back upstairs to put a fix in place.
15:00 — James, Software Engineer
I’m currently working on taking some of our in-article chatbot formats and exporting them to Messenger, with a focus on engaging underserved audiences. By using Facebook’s Messenger platform, we can also provide daily and weekly updates on the news local to them, and explore new ways of delivering stories through conversational formats.
My colleague Grant has made a separate web app that helps journalists create conversational journeys for our in-article bots. By taking the data output from this app, we can port the responses into a format for Messenger and optimise the look and feel for the platform.
“There’s also a few other AWS services we’ve utilised that are brand new to me — such as Kinesis Firehose and Athena, so it’s a great learning experience.”
We’ve gone server-less for this Messenger Bot runner — using AWS Lambda fronted by API Gateway to talk with Facebook. There’s also a few other AWS services we’ve utilised that are brand new to me — such as Kinesis Firehose and Athena, so it’s a great learning experience.
Currently we’re looking how we can push the daily top stories to our subscribers at scale. Facebook’s Broadcast API takes away the complexity of trying to reach hundreds of thousands of users simultaneously. It’s time to integrate this into our Lambda and test to make sure that we can reach the subscribers.
16:00 — Alvin, Front-End Developer
I’m working on improving the user interface of one of our prototypes that is nearing its release date.
Our current prototype is about addressing a series of little pain points that journalists and content producers face when they’re searching for videos and related content in our internal systems.
Behind the scenes: this is a web prototype that consists of a React UI that communicates with a node.js server that acts as an interface to our backend, which is powered by ElasticSearch.
17:00 — Alli, Outreach Producer
I’m currently working on cleaning up a few GitHub repositories from a recent project that we’re now looking to open-source.
I think this current task really encapsulates the things I love most about my job: the opportunity to use so many different skills, from social media production to news reporting to front-end development, on a day-to-day basis. Everyone in Labs has been extremely supportive of my development as a coder and a journalist, and I often feel that the biggest challenge is focusing on just one thing to do during a project sprint.