Demystify the TensorFlow APIs

Are you confused about which TensorFlow APIs to use? In this blog post I will give you an overview of the various model building APIs in TensorFlow, and how they fit together. I will cover some best practices and address a few commonly asked questions.

The TensorFlow team is spending lots of efforts on developing easier-to-use APIs and TensorFlow is evolving rapidly. This blog post covers both the current state (as of TensorFlow 1.12) and a glimpse of what’s coming for TensorFlow 2.0.

Intro

TensorFlow, open-sourced in late 2015, is the most popular deep learning framework. It is for both researchers and developers.

TensorFlow is flexible:

• write TensorFlow code in high level APIs such as tf.Keras (recommended), Estimators, or in low level APIs if you need greater control.
• write TensorFlow code in different languages: Python, JavaScript or SWIFT.
• train on CPU, GPU or TPU and deploy to mobile (Android / iOS), Web, Android Things, Raspberry Pi, and AIY kits etc.

The flexibility of TensorFlow also brings complexity to its architecture. I often hear questions (from both beginners and expert TensorFlow users) like these “What’s the difference between tf.Keras and the other TensorFlow high level APIs? Which one do I choose?” So I wrote this blog post to help clarify some of the commonly asked questions:

• tf.Keras or Keras?
• tf.Keras or Estimators?
• Low level TensorFlow APIs or high level APIs?
• How does eager execution fit into all of this?

TensorFlow (Model Building) APIs

First let’s start with the various options of the TensorFlow model building APIs, from low level to high level:

Note: the tf.layers API is going away with TensorFlow 2.0. It is recommended that you use tf.Keras instead of tf.layers API going forward.

tf.keras or Keras?

If you are new to deep learning and just getting started, start with tf.Keras which is Keras integrated as part of the TensorFlow core API. Keras continues as an independent open source project.

• No more questions of “do I use Keras or TensorFlow?” because Keras is now part of TensorFlow.
• No need for “Keras with TensorFlow as the backend” because Keras is now part of TensorFlow. You can continue to use Theano and CNTK etc. as Keras backend if you wish.

So instead of importing keras as a separate package:

$ pip install keras
import keras as keras

Use keras as part of the TensorFlow core API:

$ pip install --upgrade tensorflow
import tensorflow as tf
from tensorflow import keras

FAQ: “Why should I use tf.Keras?” Use tf.Keras instead of Keras for better integration with other TensorFlow APIs such as eager execution and tf.data etc.

There are three ways of creating a model in tf.Keras:

1. Sequential API — with the Sequential API you are able to define and train a image classification model using ~10 lines of code. Check out this tutorial I wrote on how to classify Fashion-MNIST with tf.Keras as an example.
2. Functional API — a good tutorial on this is Predicting the price of wine with the Keras Functional API and TensorFlow.
3. Model subclassing — see the TensorFlow Keras Guide on Tensorflow.org for examples on this. There are also lots other great tf.keras tutorials and sample code there.

Protip: Deep Learning with Python by François Chollet is a great book for learning Keras.

tf.Keras or Estimators?

Estimators is another high level TensorFlow API that simplifies the ML process by encapsulating the steps of training, evaluation, prediction, and export for serving.

There are two types of estimators: 1) premade Estimators and 2) custom Estimators. According to the official TensorFlow documentation, the difference between the premade and custom Estimator is on the model function:

• Premade Estimator: the model function is already written for you. These are models in a box for you when you don’t want to train your own model.
• Custom Estimator: you need to write your own model function.

It is recommended that you explore a premade Estimator to see whether that solves your problem, before writing your own custom Estimator.

Premade Estimators

Premade Estimators are subclasses of the tf.estimator.Estimator base class. Examples of premade Estimators include DNNClassifier, LinearClassifier, LinearRegressor etc.

Custom Estimators

As of today, there are 3 possibilities for defining the model for creating a custom Estimator:

1. Define your model with Keras and then turn it into an estimator. (link)
2. Use a module from the TensorFlow Hub, a library for reusable machine learning modules. [Feedback for the TensorFlow team: please provide us with some good examples of how TensorFlow Hub works with tf.Keras.]
3. Write the function with the layers’s API. See example here. Note: tf.layers is going away with TensorFlow 2.0 but you can use tf.keras.layers instead for this purpose.

FAQ — “Do I need to use the Estimators API?” In my opinion, go with tf.Keras unless you want to make use of the premade estimators. Note: with the recent TensorFlow 1.12 release, Keras models can now be directly exported to the SavedModel format(tf.contrib.saved_model.save_keras_model()) and used with Tensorflow Serving.

TensorFlow low level APIs

With the low level APIs you will be working with Tensors, Constant, Placeholders, Graphs, and session etc. directly. A few learning resources:

• TensorFlow official documentation on low level TensorFlow
• Blogpost TensorFlow: The Confusing Parts, written by Jacob Buckman with details on this.

FAQ: “When do I need to write low level TensorFlow code?” You only need to work at this low level if you are a student studying deep learning or researcher that needs greater control for your experiments; otherwise stick with the high level APIs.

Eager Execution

Eager execution was announced in late 2017 in the blog post Eager Execution: An imperative, define-by-run interface to TensorFlow. It moved out of contrib since TensorFlow 1.7.

This is an important feature for TensorFlow that enables you to debug TensorFlow without creating a graph — calling session.run(). Eager execution makes TensorFlow much more intuitive and pythonic. You just need one line of code to enable Eager Execution:

import tensorflow as tf
import tensorflow.contrib.eager as tfe
# Enable eager execution
tfe.enable_eager_execution()

Here is a great example of how debugging is made easier with eager mode enabled.

Protip: use defun to boost performance when in eager mode.

Eager execution can be slower than executing the equivalent graph as it can’t benefit from whole-program optimizations on the graph, and also incurs overheads of interpreting Python code. You can use tf.contrib.eager.defun to create graph functions and get a performance boost. For example call defun on your training step like this:

# define train_step function
...
# use defun on train_step
train_step = tf.contrib.eager.defun(train_step)

Protip: Use Eager execution for research, and for production you should still use Graph execution.

There is a new TensorFlow feature called Autograph that will make the conversion between the eager and graph mode seamlessly — read the blog post by the TensorFlow team here. Also with TensorFlow 2.0, eager mode will become the default.

In Summary, enable eager execution and write your TensorFlow code in tf.Keras. The TensorFlow team is working hard to make the process for us as simple as possible. They also provided an official Guidance on High-level APIs in TensorFlow 2.0 with more details on the topics I discussed above. I’m looking forward to TensorFlow 2.0 with simpler APIs, better documentation and more sample code.