Do we have to choose between AI and Privacy?
Our monthly analysis on machine learning trends
--
We just launched a newsletter program about trends in machine learning and artificial intelligence. If you’d like these analyses delivered directly to your inbox, subscribe here!
Our newsletter does more than filter news: it helps you get smart about AI. We explain why new AI research matters, give practical tips on doing AI in the enterprise, and challenge you to think deeply about ethics and society. Newsletters are curated into series to help you go deep on the topics that matter most in the field.
Yesterday’s dispatch focused on privacy. We discussed how best intentions in privacy often fall short, how companies like Apple and Google are adapting privacy techniques, and how this all fits into the bigger puzzle of the roles that trust and responsibility play in artificial intelligence. Along the way, we took a closer look at a technical approach called differential privacy, which is reshaping how companies analyze and share user data.
You can read excerpts below!
appropriated Facebook photos without permission
Anonymization is Harder than it Looks
Many privacy programs focus on personally identifiable information (PII), but over the last few years it’s become clear that removing PII from a data set is not enough to protect privacy. Even more rigorous attempts to conceal PII, such as the cryptographic (hash) functions Facebook uses to protect user IDs in their Custom Audience advertising products, have their limitations: they attend to the security of the data without attending to its underlying statistical features. In an age increasingly dominated by machine learning, this limitation can have serious repercussions.
There’s a famous anecdote that shows how best intentions in data practices end up deeply — even comically — inadequate. In 1996, William Weld authorized the release of Massachusetts state employee medical records for research. Explicit identifiers (name, address, phone number, etc.) were removed, but zip code and date of birth remained. Within a few days of the data being released, Latanya Sweeney re-identified the governor’s own records and sent him a copy. (American politics junkies may be interested to know that Weld has recently tapped into the marijuana market.)
The recent Cambridge Analytica fallout has brought privacy concerns into the spotlight. People need more education on how data is used: a recent survey showed that roughly two-thirds of Twitter users were unaware that tweets are sometimes incorporated into research studies.
But the bigger question still looms: how do companies ensure that any research done internally or externally on their users’ data remains truly, incontrovertibly anonymized?
Differential Privacy and the Quantification of Risk
Differential privacy is a new technique to ensure privacy while harnessing the power of machine learning. Basically, it entails adding noise when querying a data set, allowing most of the data set’s statistical features to be preserved while limiting the risk that any individual’s data will be identified. It uses different mathematics than standard de-identification techniques. In ideal implementations, this risk remains close to zero, guaranteeing that whether or not an individual’s data is included in a data set will have virtually no adverse effect on them from an informational standpoint.
Until recently, differential privacy was only used in practice at companies like Apple, Google, and Uber. Apple incorporates differential privacy into user analytics, enabling improvements invarious products. Google uses differential privacy locally on Androids, applying Federated Learning to share learnings from users without disclosing details. Uber is also investing engineering resources to build out more rigorous privacy algorithms, making its code open source.
Differential privacy will really hit its stride as it becomes easier to apply in startups. Georgian Partners’ new Epsilon product, cleverly named after the mathematical bound of a privacy guarantee, indicates this is on the horizon. Bluecore is using the technique to enable companies to benefit from shared data without compromising competitive insights: the goal is to help solve the infamous cold start problem in recommendation engines.
At the same time, research in differential privacy advances. Uber is focused on ensuring the privacy of SQL database queries with Elastic Sensitivity. Bolt-on differential privacy is designed for stochastic gradient descent (SGD) (a standard optimization technique). Two issues can complicate making SGD differentially private: reduced model accuracy (from the added noise) and reduced runtime efficiency. The bolt-on approach solves these obstacles by only perturbing the output of the model, essentially treating the training process as a black-box. The result is an SGD-ready privacy algorithm that both maintains accuracy and keeps runtime reasonably low.
Differential privacy will soon be a privacy best practice. Still, it’s only one piece of a larger puzzle. If you’re ensuring user data remains private but failing to treat users fairly and consistently, you haven’t really done much to build trust. As machine learning models becoming increasingly ubiquitous, so do the threats of bias, misallocation, and misunderstanding.
In next month’s newsletter, we’ll take a closer look at fairness in machine learning, ponder the ethics of AI, and add more clarity to what it means to build responsible products. Subscribe to have this analysis delivered directly to your inbox!