From Big Data to Big Value

Data as a product

At this point all big companies have collected tons of data about customers, markets, competitors etc., and many are hoping that big data analytics will 1)help them become more competitive and 2)even provide a new business model based on selling data.

The problem is that most organizations that stand on large quantities of data are old and traditionally bureaucratic— telecom companies, banks, consumer goods producers, freight shipping companies, airlines. They are very passionate speakers about the potential value of big data, but they have equally big issues in drawing value from it. Some of their challenges include:

• difficulty to consolidate data — they are heterogeneous, frequently formed of separate entities forcefully integrated in their expansion frenzy
• dependency on old technology — old warehouses where data stored in siloes, important transactions still made on mainframes etc.
• lack of skills — most big data technologies are in their initial phases of development and you need really skilled people that know how to juggle with all the complexity

However, consolidated digital data combined with huge processing power is a dream worth investing in, both from a competitive advantage point of view, but also from a commercial one. In Copenhagen there’s been some buzz about an experimental project of Hitachi that’s trying to build a big data platform — a space for organizations to sell & buy data — that is aimed at creating the premises for a smart city.

“We believe that the combination of public and private data in a city-wide Big Data digital infrastructure will transform urban planning and provide benefits to businesses, improve the safety, wellbeing and comfort of citizens, and enhance the efficiency of the City of Copenhagen”, says Hans Lindeman, Hitachi Consulting, Senior Vice President of Social Innovation Business.

However, they’re facing the same issues even if most parties see a lot of potential — it’s difficult to find companies that have clean data, and they don’t know yet what it’s possible to sell, nor how much it’s worth.

Do we have privacy anymore?

All this data is generated by regular individuals whose behavior is tracked to gain insights and this attracted a lot debate around the topic. Our personal data cannot be sold without our consent, only aggregated anonymized data can, which isn’t supposed to affect us. However, the level of details about an individual even if it only appears as an ID in a database can sometimes be enough to derive sensitive information about them.

One very good example about how things can go awry is that of Janet Vertesi, a sociology professor at Princeton who made an experiment to try and hide her pregnancy from the web. As pregnant women make important long term buying choices before the birth of their kids, internet advertisers put a lot of bets in gaining their attention.

She used only cash or Amazon gift cards acquired with cash to buy things for her future new born, called directly to relatives to discuss her pregnancy and didn’t post anything on social media. In the process she managed to avoid advertisers, but also to get flagged for criminal activity — because if you try to pay too much in cash nowadays in the US apparently it’s a sign that you are hiding something.

So, are do we have real privacy anymore? Yes, to a large extent we choose what we want to share. But we’re also so much more exposed than before and that’s a tradeoff we’ve made. All individuals that are on social media, that use search engines, make online purchases and populate the internet with their articles, opinions, and comments, have chosen to give away some of their privacy. We’ve deliberately offered our data for others to use in exchange of the “free” internet.

Where are we now?

According to Gartner’s emerging technologies hype cycle, in a peak of inflated expectations, just before the big fall.

There are a lot of promises and hopes around advanced big data analytics, and we’re still trying to figure out how to extract value out of it and what levels of access to impose in order not to abuse its power. Both technologies and markets are immature yet, so there’s space of discussion about what we can/should build with this.

I’m on the positive side.