Differential Privacy for Deep Learning
Using Differential Privacy to classify MNIST Digits and perform PATE Analysis on the model
Introduction
By now we all know the benefits of Differential Privacy, how it can protect individual privacy while still providing accurate query results over a large dataset. This post will illustrate how to apply Differential Privacy to the MNIST Digit classification problem and analyze it using a technique called Private Aggregation of Teacher Ensembles (PATE).
Methodology
First, we will divide the private data into N number of sets (in this case, 100) and train a classifier on each of the N datasets. These are called Teacher classifiers. We will then use the teacher classifiers to predict the labels for our public data. For each image in the public dataset, the most predicted label by the N classifiers will be considered as the true label for that image.
Now, using the predictions of the Teacher classifiers as true labels for our public data, we will train a Student classifier which can then be used to classify new unseen images.
The MNIST Train data will be considered as private data on which we will train our Teacher models. The Student model obtained by combining the predictions of the teacher models will then be trained on the MNIST Test data (90% of test data will be used to train the model and the remaining 10% will be used to test its accuracy)
Okay, but where does the privacy part come into play?
Deep Learning models have a tendency to overfit the training data. Instead of learning general features, neural networks can learn features of an individual which then can be exploited by an adversary to obtain the individual’s private information.
By not training the Student model directly on the private data, we prevent it from directly learning key individual features of a single person from the dataset. Instead, the generalized features and trends learned by the Teacher models are used to train the Student.
However, there is one small caveat. If the label of an image can be changed by removing the prediction of a single teacher, an adversary can narrow down the search to that model.
To avoid this we add random Laplacian Noise to the predictions of the teacher models before selecting the most predicted label as the true label for the public data. In this way, we add a bit of randomness and skew the final result so that the true label doesn’t easily change by dropping just one teacher.
Implementing Differential Privacy using PyTorch
Conclusion
Using the Student-Teacher architecture guided by the PATE analysis method is a great way to introduce Differential Privacy to your deep learning models. However, Differential Privacy is still in its early stage and as more research in the space occurs more sophisticated methods will be developed to reduce the privacy-accuracy tradeoff and the downside that differential privacy only really performs well on large datasets.
References
[1] Dwork, C. and Roth, A. The algorithmic foundations of differential privacy (2014), Foundations and Trends® in Theoretical Computer Science, 9(3–4), pp.211–407.
[2] Abadi, Martin, et al, Deep learning with differential privacy (2016), Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2016.
[3] Figure 1, Unsplash (2016)
[4] Figure 2, Nicolas Papernot, et al, Scalable Private Learning with PATE (2018), Published as a conference paper at ICLR 2018
