Security & Privacy Risks of Machine Learning Models

Steve Weis
Apr 5 · 5 min read

Background on Machine Learning Models

The training phase takes a set of input training data, applies a learning process, and outputs a model.
The prediction phase applies the model to real data in order to make a prediction.
A machine learning model could be trained to classify whether images are chihuahuas or blueberry muffins.

Risks against Machine Learning Models

Poisoning Attacks

An adversary able to alter training data can control how a model behaves.
Microsoft learns the perils of machine learning poisoning attacks.

Evasion Attacks

A demonstration of fast adversarial example generation. Image from Explaining and Harnessing Adversarial Examples by Goodfellow et al.

Unintentional Memorization

This illustrates how an overfit model may encode specific input instances that may later be extracted.

Steve Weis

Written by

Working in security and cryptography. Opinions are entirely my own.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade