Solving The Data/Privacy Conundrum

Big data and privacy seem to be at odds. Providing data to companies creates modern day conveniences, but it also allows businesses to see too much into our everyday consumer lives.

Big Data to generate massive potential to organizations

Big data, IT industry analysts tell us, will enable both commercial companies and the public sector to gain ever richer insights into their operations and their customers, transform the efficiency of their organizations and enable them to deliver many more compelling, personalized services. From smart cities to citizen science, omnichannel retailing to monitoring people’s health or the spread of disease, the potential value that can be generated from big data is vast.

Privacy: Is our data anonymous?

But there’s a problem. Many applications and services will rely on people sharing what is, in some cases, quite sensitive, personal information. For example, a smart city system to monitor people’s patterns of movement and improve transport and traffic flows will rely on people agreeing to share their location data with the system. Similarly, a smart health monitoring system might need people to agree to have, say, their vital signs continually monitored and sent wirelessly over the cloud. And a retailer hoping to provide its customers with the right offer in the right place at the right time will rely on customers agreeing to have their browsing and buying habits observed and recorded.

Even where data is anonymous and collected for the purpose of detecting broad patterns rather than for targeting an individual, customers will need to be persuaded that the data they are providing is truly anonymous and secure once it leaves their hands.

While it’s true that to date most people unquestioningly agree to share their data in exchange for access to compelling free services like social networks and productivity applications, we might see a new trend approaching. Many people today check the ‘accept our terms and conditions’ box or the ‘allow the use of cookies’ box as a matter of course, generally without ever reading the small print. However, attitudes and awareness to data privacy are changing fast.

Is our data safe? High profile data breach is on the rise

Hardly a month goes by without a new high-profile data breach hitting the headlines, affecting everyone from retailers to banks to government departments. What message does this send to people? It tells them that if they share their data with a third-party organization — even a big, well-known one — they can’t trust that organization to keep their data safe from prying eyes.

People are also increasingly irritated by overly intrusive advertising (targeted or otherwise) and the past few years has seen a big rise in the use of ad blockers and anonymizing software. And the EU is currently pushing through new rules on data protection which will make the process of data collection much more transparent for citizens and consumers, requiring explicit consent for the use of sensitive personal data and giving them far more oversight and control of their data.

Security and anonymizing technologies to ensure people’s consent

Without people’s consent to collect, store and process their information, many of the promised benefits of big data will never be realized. As a baseline, organizations need to be sure they have demonstrably rigorous end-to-end security when handling sensitive data. And where they employ data anonymizing technologies and techniques, they must be sure those techniques are robust, given that there have been a number of cases where hackers have been able to reconstitute personal details from supposedly anonymized datasets.

Benefits of sharing data

But security alone is not enough. Above all, organizations must convince people they will see compelling benefits from sharing their data, whether that’s dramatic reductions in their energy use, commuting time or other innovative new services that make their lives significantly more convenient or cheaper.

A panel of CIOs and industry experts discussing the subject at last month’s IoT Tech Expo in London reached the same conclusion, with broad agreement that organizations handling user data need to create a ‘virtuous circle’. As one panelist noted: “The more data from the more sources you get, the better the chance you have of being able to generate really beneficial outcomes for customers. And the more benefit people see from providing their data, the more data they’ll be prepared to give you.”

Never miss another post. Sign up for the weekly 100TB newsletter.

Originally published at

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.