A reality check about augmented reality in journalism
We talked to Francesco Marconi, Dan Archer, and John Keefe to discuss the many opportunities and challenges brought about by AR in the newsroom.
Augmented reality ‘seems simultaneously futuristic and obsolete’, writes Adi Robertson in The Verge.
The hugely popular and gimmicky Pokémon Go thrust AR centre stage in the summer of 2016, only to fizzle out once players had caught all the little beasts. Jeff Koons’ AR Balloon Dog Sculpture, born out of a collaboration with Snapchat, has just been vandalised in a ‘stance against an imminent AR corporate invasion’.
While AR is becoming a part of ‘everyday life’ through apps like Ikea Place, which allows customers to visualise what ‘their furniture might look like in their living room’ and L’Oréal’s AR beauty App, which lets potential customers try out make-up via their phones, you might be left wondering where AR comes in useful for more serious news reporting.
We spoke to Francesco Marconi, strategy manager and AI co-lead at the Associated Press, John Keefe, bot developer and product manager at Quartz, and Dan Archer, founder of Empathetic Media, about the added value of AR in news reporting, the diverse challenges, and why you should invest in immersive technologies today.
Opportunities
AR and immersive media are a method of satisfying the ‘craving of news consumers to get closer to the story’, says Marconi.
Archer sees AR as causing the ‘same seismic shift in interface evolution that was made when smartphones introduced the touchscreen’.
He believes that in the near future 3D scenes or experiences will be viewable from any angle of our living rooms and users will be able to move around — and interact — inside them.
‘I don’t think this will render 2D screens obsolete, given how accustomed we are to passive, editorially curated viewing experiences (particularly when it comes to narrative), but I do envisage this being touted as the new step in narrative. A wholly new experience that is much better suited to a multi-user situation.’
3D Objects
3D objects are one part of the Augmented Reality experience.
With the September launch of ARKit, a new framework that enables AR experiences on iPhone and iPad, AR barriers to entry were lowered overnight. Quartz quickly jumped at the opportunity to incorporate this tech trend into their news app.
When asked about whether the introduction of AR had a positive impact on engagement, Keefe says that they saw ‘10,000 new downloads’ within the first week. He also says that in the app, two thirds of the people ‘who are presented with the possibility of seeing something in AR will open the AR viewer and see it’, suggesting that there is a real demand to get ‘closer to the story’ through immersive tech.
‘With AR, we are not creating a new world. The world is your world and we’re just adding to it’, says Keefe.
‘3D objects are ephemeral, says Keefe. While they don’t ‘have to consume your day’, they allow you to explore certain news objects more thoroughly than you would be able to with a photograph or animated gif.
Quartz’s AR coverage of the Cassini spacecraft shows that a 3D object can make a news story a lot less abstract: the user is able to place the spacecraft model into their room, walk around it, and get a feel of its actual size.
3D objects are however incredibly difficult to create from scratch, according to Keefe, and are therefore normally acquired from an online marketplace.
‘NASA has a lot of 3D objects of their spaceships and even departments of the government are building 3D objects… so you normally do some kind of acquisition’, says Keefe.
The process of preparing a 3D object and having it ready for display in a phone however can take as many as one or two days, says Keefe, making it unsuitable for breaking news.
3D objects are therefore used primarily to provide ‘additional context to a [news] story told through text’, says Marconi.
AR-Walkthroughs
Another great AR case study, says Marconi, is the coverage of the Freddie Gray trial by the Washington Post in collaboration with Empathetic Media. Freddie Gray was a 25-year old Afro-American man from Baltimore who sustained serious injuries, fell into a coma, and died while in police custody. The AR walk-through shows Gray’s interaction with the police that led to his death.
‘With 3D imagery, audio, maps and text based on court documents and witness testimony, users can better understand the complexities of the case and the differences in what the prosecutors and defence say about when and how Gray was hurt’, writes the Washington Post, showing that one of the most obvious perks of AR is to tell a story from multiple perspectives.
‘Simply explaining Gray’s interaction with police through text can’t achieve the same effect as when a user has the ability to choose from which angle and from which point in the altercation they want to experience this encounter’, says Marconi.
Keefe mentions that Quartz has something similar in mind, by creating AR experiences of things that would normally be put in an interactive, such as a terrain or map.
‘If there is a city in Iraq in the news, we could have an actual terrain of that, which you could put on the floor in front of you and explore’.
Snapchat for journalism
More light hearted AR apps such as Snapchat can also have ‘serious applications for journalism’, according to Yusuf Omar, mobile editor of the Hindustan Times in an article on journalism.co.uk.
Omar used Snapchat filters — which transform users into a dog, cat or fire spitting dragon — to film interviews with underage rape survivors during India’s Climb against sexual abuse, an event which hopes to remove the stigma around talking about sexual violence.
Snapchat filters allow viewers to see the eyes and expressions of the interviewees without giving away their identities, making the stories and the people behind them a lot more relatable.
Photo archives
The authors of the Times’ innovation report in 2014 wrote that they needed to make better use of its archives, ‘we can be both a daily newsletter and a library — offering news every day, as well as providing context, relevance and timeless works of journalism’.
‘You are here’, an immersive journalism prototype about the history of New York aims to do just that and more with an added layer of AR.
Sandeep Junnarkar and Jere Hester of the CUNY Graduate School of Journalism want to pick out old photos and overlay them in order to ‘create a virtual window to the past’ on people’s phones as they walk around the rapidly changing city of New York. These photos will be put into context through little explainers, which will appear as pop-ups.
Their first test case is Times Square, for which they have gathered photos, shot a 360-video landscape, and built an AR experience with help from Barbara Gray, Jeff Roth, John Smock, and Ian Burns.
Challenges — ‘AR: an acclimatisation process that we’ve only just begun’
Archer says that the biggest hurdle at present is that a large part of the population is used to getting their information from text. This affords text greater legitimacy and authenticity than other media, especially those that are predominantly visual. This suggests that people might be initially very sceptical of AR in news reporting, particularly because it is a medium they might have previously only come across in gaming.
False information
When working in immersive media, says Marconi, the editorial standards of journalism shouldn’t be re-written, but instead adapted to fit the needs of the medium.
As a hypothetical example, Marconi imagines what it would be like to cover a protest in the near future using 3D scanning tools. While the stationary objects, such as the buildings, trees, and the street the protest is taking place on can be easily captured, the people marching down the street can become easily blurred and distorted.
‘How should the news professional handle this natural distortion from an ethical perspective?’
The solution he has come up with is to have ‘journalists present their editorial options to the public, without making assumptions’.
He says that immersive media needs a heightened level of transparency from news organisations on how they are collecting, processing, and disseminating facts to the public.
Diversity
In an article published on Wareable, Catherine Allen, who specialises in AR and VR at the BBC, draws attention to the fact the mixed realities worlds are built by Silicon Valley companies, where a lot of decisions are primarily made by young or middle-aged white men.
‘But as AR tech is so immersive, as well as focused on touching, communicating, learning and seeing, do we really want that all governed by such a small portion of the population?’
Their biases (conscious and unconscious), she says are likely to seep into any ‘structures and systems of any world that is created’.
The danger is that mixed reality worlds can become ‘inaccessible and exclusive spaces’.
Keefe says that gender and racial diversity is something that needs to be considered.
‘We have huge digital platforms that we all live in every day that have been built by a small subset of the population. I think those issues are real and it’s about time those issues are addressed’.
How they will be addressed is unclear.
Strategy
The hesitation to get into immersive media is understandable due to the technological costs, new skills and workflows, says Marconi, but this doesn’t mean news organisations should shut themselves off from exploring this new form of journalism.
According to the AP report — ‘The future of augmented journalism: a guide for newsrooms in a world of smart machines’, the best use of AR is to add context to a larger story package, which is also a way of introducing immersive media to audiences more generally. A journalist wishing to incorporate AR has three main things to consider:
- What aspect of this story is the most explorable?
- How can I give my audience a perspective that other mediums can’t?
- How is my audience going to play with this AR extension?
Tips from Dan Archer
Start small
- Build out smaller, experimental projects quickly and consider immersive elements as the story comes together in the newsroom.
- Experiment with lower levels of immersion that require less investment, such as 360 video.
- Iterate test versions of stories for colleagues in the newsroom to try out. This is the best way to measure and estimate future audience reactions, such as how far readers might be prepared to walk within a location-based AR story (à la Pokémon Go).
Work together
- Collaboration is often key to the successful integration of new practices into existing workflows. In 2016, the New York Times, for example, acquired Fake Love, an agency specialising in AR and VR, in order to fully dive into the world of immersive tech.
Get the skills
- 3D skills such as Blender/Maya or Unity/Unreal engine experience are a huge plus.
- Strong UX or UI skills are also a real advantage in building an experience that those less tech-savvy can easily negotiate.
Focus on the future
- Editors need to devise a strategy that incorporates new AR and VR workflows into their news products both in the present and future, rather than focus all their efforts on one-off experiments that lack a context or follow-up.
Invest now
‘The territory that divides informative news media from entertainment media in readers’ minds is being annexed by the day, and companies such as Snap, Facebook and Apple are leading the charge.
‘News outlets need to dedicate time and staff resources to investigating this new tech so they can understand how it will change the way they present their stories to audiences in the future, in the same way that many now have staff solely in charge of social media.
If they don’t, readers will naturally gravitate more towards the immersive worlds and immersive VR/AR experiences are offering, which will only result in a more homogenised, less diverse news ecosystem’, concludes Archer.
