Encrypted Deep Learning Training and Predictions with TF Encrypted Keras

Yann Dupis
· 6 min read

TF Encrypted aims to make encrypted deep learning accessible. For this reason, we are pleased to share with the community that TF Encrypted now offers a high level API, TF Encrypted Keras, which matches a subset of Keras’s features. This is an important first step to supporting complex neural network architectures and use cases.

François Chollet, the creator of Keras, aimed to provide the best user experience possible when designing the Keras API. As he explained in this blog post, good UX reduces cognitive load (e.g., remembering how things work). Therefore, users focus their energy on solving their actual problems. That’s what we love about Keras: the user is at the center.

Encryption adds an entirely new dimension, but with TF Encrypted Keras we hope to still allow the users to focus on solving their problems. Our goal with TF Encrypted Keras is to enable:

  • Data Scientists to iterate quickly to build, for example, the best skin cancer detection model or fraud detection model on encrypted data.
  • Cryptographers to focus on creating the best encryption scheme without having to rebuild higher-level machine learning concepts.
  • Machine Learning researchers to run quick experiments to discover optimized architectures for encrypted deep learning.

In the rest of the blog post we will show how you can train models and provide predictions on encrypted data. We will also share how TF Encrypted Keras can integrate with other TensorFlow libraries such as TF Privacy specialized in differential privacy.

Encrypted Training

As a data scientist or machine learning engineer, one of the biggest challenges is getting access to training data, especially when the data is sensitive. The data owner doesn’t want to be exposed to liability risks for sharing private, sensitive data. For example, a hospital may want to work with an AI startup to train a skin cancer detection model, but (understandably) refuses to share private patient data. These difficulties can also occur internally at a company. For example, two different bank entities might not be able to share their data to train a fraud detection model for legal reasons.

Imagine, instead, that the raw data is never shared with the data scientists. Only its encrypted form is shared. As we described in an earlier blog post, even though the data is encrypted with multi-party computation, it’s still possible to perform computations on this encrypted data. This means that data scientists can train, predict, and validate his model without revealing the original data.

Encrypting this data prevents liability risks, which enables data scientists to get access to this sensitive data to train their models. That AI startup may be able to create that skin cancer detection model after all, and enable large scale access to medical expertise.

Using TFE Keras, Data Scientists can train models on encrypted data

With TFE Keras, it’s very easy to train your deep learning model. The code snippet below shows how you can train a simple logistic regression model.

Encrypted training with logistic regression

You can define the model using the Sequential API. If you have used Keras before then, this will look very familiar. Once you compile the model with the right optimizer and loss, you can start training the model with the fit method. To experiment, you can run this notebook.

Encrypted Predictions with Public Training

Encrypted deep learning also preserves user privacy when using machine learning services. For example, the user could take a photo of his skin lesions with his cell phone. The image will be sent in an encrypted form to the TF Encrypted model. Finally, TF Encrypted will perform a private prediction without having to decrypt the image during computation. We could imagine a similar approach to get a loan pre-approval from a bank.

Using TFE Keras, users only share encrypted data to get the prediction

In the code snippet below, we define a convolutional model with TF Keras to perform an image classification.

Define your model with TF Keras and convert it into TFE Model with clone_model

With the functions tfe.keras.models.clone_model you can easily transform a normal TF Keras model into a private TFE Keras model. Your model is now ready to serve private predictions. This is very useful in a context where the data scientist is able to get access to the data, but wants to serve private predictions to preserve the privacy of his users.

If you prefer, you can also manually define the TFE Keras model and set the weights from a numpy file.

Define your model with TFE Keras and set weights with set_weights method

As you can see TFE Keras matches a subset of TF Keras API exactly.

Here you can find a series of notebooks, showing in three steps how you can train a model, serve private predictions, and query the private model.

Encrypted Predictions After Public Training with Differential Privacy

Unfortunately, encrypted deep learning can’t always prevent privacy leaks. When the model is trained on raw data or encrypted data, it’s possible that the model memorized some sensitive information contained in the original training dataset. Once the model is deployed, adversaries could reveal this sensitive information by querying the model or inspecting the model weights. Two common attacks are membership inference and model inversion.

Differential Privacy prevents the model from memorizing sensitive data

The good news is that we can solve these problems with the library TF Privacy, which is specialized in differential privacy. The library is extremely easy to use. To train the model, you just have to call their differential privacy optimizer (DP-SGD), which will prevent the model from memorizing sensitive data with specific privacy guarantee (epsilon). We will focus a blog post or two on differential privacy in the future, but in the meantime, you can read these two excellent blog posts to learn more about differential privacy and how to use TF Privacy:

Once your model is trained with differential privacy, it’s extremely easy to transfer the weights to the TFE Keras model to start serving private predictions (since TFE Keras matches the TF Keras API). Here you will find a series of notebooks demonstrating how you can train your model with TF Privacy, then serve private predictions with TFE Keras.

Next Steps

We hope these examples give you an idea of how you can apply encrypted deep learning, and demonstrate that you don’t have to be an expert in cryptography to start training and serving predictions on encrypted data. You can experiment with these new features in the version 0.5.8 of TF Encrypted.

The next steps for Encrypted Keras are:

  • Expand the TFE Keras API to train and serve more complex models and tackle new use cases such as NLP.
  • Adjust some implementation details to improve user experience.
  • Give the option to train the model with differential privacy on encrypted data.
  • Provide documentation and tutorials to run TFE Keras in the cloud.

As mentioned recently by Morten Dahl, “the motivation behind @tf_encrypted is simple enough to fit in a tweet”:

Please join the TF Encrypted community on Slack to contribute to democratizing encrypted deep learning!

About Dropout Labs

We’re a team of machine learning engineers, software engineers, and cryptographers spread across the United States, France, and Canada. We’re working on secure computation to enable training, validation, and prediction over encrypted data. We see a near future where individuals and organizations will maintain control over their data, while still benefiting from cloud-based machine intelligence.

Visit our website or blog or TF Encrypted repository for more information, or follow us on Twitter for up-to-date announcements.

If you’re passionate about data privacy and AI, we’d love to hear from you.

Dropout Labs

Bringing Cryptography and Machine Learning Together

Thanks to Morten Dahl

Yann Dupis

Written by

Machine Learning Engineer / Privacy Researcher at Dropout Labs

Dropout Labs

Bringing Cryptography and Machine Learning Together

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade