Journalism on the edge

The internet of things relies on processing at the edge of networks not at the centre. Maybe journalism could learn from that…

Over the last few years I’ve spent a lot of time with people from the open data and open government data fields. You don’t spend much time in that crowd without hearing the phrase ‘smart city’ and if that gets mentioned then ‘the internet of things’ or IoT, is usually not far behind. One concept that seems core to a lot of IoT (and so smart city) thinking, and crops up more and more in the chats I hear, is the transition from cloud to fog to edge.

Stick with me…

Doing stuff ‘in the cloud’ is a phrase we’ve become familiar with over the years. Data gets collected up, moved and then processed ‘in the cloud’; great things come back. The benefit being that all the heavy lifting and infrastructure isn’t your problem. It’s in the cloud! Of course all the data needs to get there and the more you rely on the cloud to do the number crunching, the more data you need to send. That means there’s a lot of pressure on communication links. Great if you’re at the end of a big fat communication pipe, not so great if you’re in the middle of nowhere with an intermittent signal. That’s where fog and edge come in.

Fog and edge computing do some of the processing before it goes up to the cloud, reducing the amount of data that needs to be sent. Here’s a good example from a CapGemini report

For example, a Boeing 787 generates 40 TB per hour of flight, but just half a TB of this is ultimately transmitted to a data centre for analysis and storage. Similarly, a large retail store might collect approximately 10 GB of data per hour, but just 1 GB of that is transmitted to a data centre. As it is not sensible, nor possible to install a full data centre either on a plane or within a store, edge or fog computing steps in to validate and pre-process this data either within a local network (fog) or a gateway device (edge).

40TB of data per hour of flight! Who knew!

Perversely though, all this reading around edge and fog has made me think less about the IoT (and jets). It got me thinking more about journalism.

Replace cloud, fog and edge with newsrooms and reporters and the efficiency in good local coverage is clear.

The key thing with fog and edge is not the ‘network’ per se. The key thing is the level of processing and autonomy allocated to the edges of the network and the interaction with the fog. They can do what processing they need without a burden on the cloud. The cloud gets what it needs and, importantly, the whole thing doesn’t collapse for want of a good connection. That got me thinking about the ‘cloud’ that journalism has become.

Creating more fog.

Increasingly, media orgnaisations optimise for central processing. This is especially true of analytics; data about the way people consume content. But in some its also the way social media, UGC content and other forms of content are hoovered-up and processed en masse. At the same time, the ‘fog’, the myriad of regional and local newsrooms that existed to do some of the pre-processing are disappearing. As are the nodes at the ‘edge’; the reporters and journalists with the context and local knowledge to know what to ‘process’ and send on and what to ignore. The gateways to local data.

Seeing journalism as a network, its easy to believe a level of technological and structural connectedness mitigates against the disruption media orgs face. Why not clear some of the fog? Connect the edge to the cloud and benefit from that extra processing power? But seen as effective mechanism for processing and moving information around, extracting value and saving resources at each step, things look different. Seen as a Internet of Storytellers, the fog becomes really important. Its the extra level of value.

Invest in the edge

Investing in edge computing is about capacity. It’s about taking advantage of all the smart gadgets and gizmos to fill in the gaps about the world around us without the network collapsing from the shear weight of data they create. In the same way, investing in the edge journalism — local, in the field journalism — could help fill the gaps and make better use of the connections.

There are already some good moves to do that. Initiatives like the Bureau Local help put some of the capacity back into the ‘fog’ part of the network. Likewise, the BBC local democracy scheme contributes to the fog but is also looking to add more nodes to the edge of the network. That scheme in particular recognises that there a myriad of community and hyperlocal media outlets already operate on the edge. It’s an attempt to get them ‘connected’.

But I think we could go further. Perhaps one way for the centre to encourage and connect with the edge is in analytics. Mainstream media orgs could pay hyperlocals and community media orgs to have the same analytics tracking code that they use — put Omniture and Chartbeat code on hyperlocal sites. Give them access to the data. That way they can process the data at the edge and make decisions about their content and audience. At the same time, useful data comes back into the cloud, already processed.

Recognising the value of the edge is something that the IoT community has learned fast. Its the only way it can deal with the massive growth in devices and the demands on processing. But the value of the edge is something journalism has maybe lost sight of in favour of the economics of the network rather than the processing it can do.