Getting started with TensorBoard

TensorBoard is an open-source service launched by Google packaged with TensorFlow, first introduced in 2015. Since then, it has had many commits (around 4000) and people from the open-source community keep the code base alive by creating issues and making pull requests for new features and/or bug removals or documentation updates. TensorBoard’s back end is written in Python and the front end in TypeScript. To get the full story of TensorBoard, you can simply visit the https://www.tensorflow.org/tensorboard website under the TensorFlow website.

In recent times, there have been multiple services added like Hyperparameter Tuning, What if Tool, Fairness indicators. This proves that this community has come a long way and is still going strong.

Hopefully, by the end of this blog, the reader will be able to make pull requests and contribute to the community of TensorBoard. By the end of this blog, you will be able to understand the key concepts behind TensorBoard and its work. And you will be able to incorporate TensorBoard in your Machine Learning workflows and also make these experiments easier to manage using tools like tensorboard.dev.

We are going to see how we have laid out this Blog:

• Introducing TensorBoard
• Understanding TensorBoard Services
• Installing and Setting up TensorBoard
• Integrating TensorBoard with TF and Keras

Introducing TensorBoard

TensorBoard provides a visualization toolkit for your optimizations and/or your metrics, defined in the code as a real-time solution. This is a game-changer because it provides real-time loss values and evaluation metrics as defined by the user. You can also see the model architecture using TensorBoard and view your weights as histograms. You can also display your data, like images and Dataframes.

The following image shows you the main page of TensorBoard.

If we look at visualization toolkits, there are multiple options to choose from in Python, like Visdom by Facebook; Neptune, which is an experiment management and collaboration tool; Guild AI; Sacred; Weights & Biases; and Comet.ml. All these toolkits offer some services, but TensorFlow is the only toolkit that is an all-in-one bundle with ease of use and perfect compatibility across different platforms and different versions of the framework. Some key points that TensorBoard provides which are not easily found in other frameworks include:

• Sharing results with the team(s).
• Logging the experiments into checkpoints so they can be used as a history of the model.
• Making reports for clients and or your colleagues for a presentation.
• Integrated with almost all major Deep Learning Frameworks in Python.

The most unique thing about TensorBoard is that it is made for all types of users, the beginners, intermediates, and advanced. For beginners, adding simple checkpoints and metrics is a piece of cake; whereas an intermediate user might be interested in maintaining the history of its checkpoints., while an advanced user might be using TensorBoard with Hyperparameter Tuning to find the optimal set of hyperparameters.

Now that we have a basic idea of what TensorBoard is for, let’s get an overview of the various services it offers.

Understanding TensorBoard Services

As mentioned earlier, we have a bunch of services provided by TensorBoard that make it unique, and more powerful than any other visualization tool. It provides a set of services that make our job a lot easier as scientists and engineers. We’re going to get a brief overview of each service in this blog:

• Scalars and Metrics: Logging metrics for training is very important in Machine Learning. For this purpose, there is a real-time visualization of these metrics. This is particularly useful when we are finding out whether the model is overfitting or underfitting.
• Image Data: When dealing with an Image Processing or Computer Vision problem we need to display images in order to understand their details and sample images from the training or validation set. This can be used to visualize weights or some generated results during the process of training.
• Model Graphs: With TensorBoard model graphs method, you can easily view a beautified version of your model architecture and that is too responsive in nature that will show you the details at each node. This is useful to understand and or debug the model when it’s too complicated to be understood by mere reading the code.
• Hyperparameter Tuning: Hyperparameter tuning is a way to find the best hyperparameters from a set. TensorBoard lets you find the best set of hyperparameters based on the given metric.
• Embedding Projector: TensorBoard can also be used to represent higher dimensionality embeddings to understand embedding layers by visualizing them.
• What if tool: Understanding the Machine Learning model is a necessity. For this purpose, we have a What if Tool (WIT) to help us better understand black-box machine learning.
• Fairness indicators: This service is used for easy computation of commonly identified fairness metrics for binary and multi-class classifiers. You can easily visualize fairness metrics for your runs and compare them across multiple groups
• Profiling tool: To check how your model performs on CPU and GPU and profile and to see if you are running the most optimized version of your model.
• TensorBoard Data as DataFrames: To access data logs of TensorBoard, we have TensorBoard Data that lets you use them as data frames.
• Debugger: With TensorBoard you can now debug weird problems in your model where you might end up facing issues like Nans.

Next, let’s walk through the installation and setup of TensorBoard.

Installing and Setting up TensorBoard

In order to complete this installation guide, you will need to understand how python works and how packages are installed in Python. A better understanding of pip and operating system commands is useful. So, let’s check how we can install it in each of the following operating systems:

• Linux (Ubuntu)
• Windows
• MacOS

Linux (Ubuntu)

Essentials of installing TensorBoard on ubuntu requires python to be installed on the system. And along with Python we need to have pip installed with which we can install TensorBoard.

Ubuntu16.04+ comes shipped with Python3+. You can check your Python version by:

1. Opening Terminal.
2. Write python.

This will display the version of the Python installed. Additionally, you need to install Python package manager called pip.

1. You can install pip with the following command:

sudo apt-get install python-pip

sudo apt-get install python-dev

These steps can be skipped if you have installed anaconda python on your system. Anaconda ships pip with some basic packages already installed.

1. We are going to install TensorBoard now. The command to install TensorBoard is:

pip install tensorboard

1. You can also specify one of the latest versions you want to install from the list (2.3.0, 2.4.0, 2.4.1) by using the following command:

pip install tensorboard==[VERSION]

Next, let’s see how to install it on UNIX.

UNIX MacOS

Installing TensorBoard on MacOS requires python being installed on the system. And along with Python we need to have pip installed with which we can install TensorBoard.

MacOS11.1 comes shipped with Python3+. You can check your Python version by:

1. Opening Terminal.
2. Write python.

This will display the version of the Python installed. Additionally, you need to install a Python package manager called pip.

1. You can install pip with this command:

sudo apt-get install python-pip

sudo apt-get install python-dev

1. pip uses PyPI as the default source for packages and their dependencies. So, whenever you type:

pip install [package_name]

pip will look for that package on PyPI and, if found, it will download and install the package on your local system.

1. These steps can be skipped if you have installed anaconda python on your system. Anaconda ships pip with some basic packages already installed.

But nonetheless, we are going to install TensorBoard now. The command to install TensorBoard is:

pip install tensorboard

You are also able to specify one of the latest versions you want to install from the list (2.3.0, 2.4.0, 2.4.1).

Let’s move on to the Windows OS.

Windows

Installing TensorBoard on Windows requires python to be installed on the system. Along with Python, we need to have pip installed, with which we can install TensorBoard.

1. In order to install python, there is a multitude of installation methods: download official Python distributions from Python.org, install from a package manager, and even install specialized distributions for scientific computing, Internet of Things, and embedded systems. The better method is to install python via an installer, available with 32-bit and 64-bit versions.
2. Selecting Customize installation will allow you to select the features to install, the installation location and other options or post-install actions. Once the python installation is complete, you can check your Python version by Opening Terminal.
3. Write python. This will display the version of the Python installed. Additionally, you need to install a Python package manager called pip.
4. Securely Download Pip from the link: https://bootstrap.pypa.io/get-pip.py.
5. Run python get-pip.py. This will install or upgrade pip. Additionally, it will install setup tools and wheels if they’re not installed already.

pip install tensorboard

You will be able to specify which of the latest versions you want to install from the list (2.3.0, 2.4.0, 2.4.1)

Integrating TensorBoard with TF and Keras

Let’s see a simple example for now, and try to integrate TensorBoard with TensorFlow/Keras. We are going to see a simple example of classifying MNIST Dataset; let’s put together some code in TensorFlow using the higher API of Keras to integrate with TensorBoard. This piece of code assumes that you have already installed TensorBoard and TensorFlow.

Let’s dive into the code.

In this code cell, we are importing a TF package into our system along with a default library for datetime. Then we download the dataset from tf.keras.mnist:

import tensorflow as tf
import datetime
mnist = tf.keras.datasets.mnist

Next, we split the data into training and testing sets:

(x_train, y_train),(x_test, y_test) = mnist.load_data() # Splitting dataset into train and test
x_train, x_test = x_train / 255.0, x_test / 255.0 # Normalizing the images in 0–1 format

In this function we are creating a simple FCN model with two Dense Layers and Dropout:

def create_model():
  return tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(512, activation=’relu’),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation=’softmax’)
  ])

So far, we’ve imported the dataset and created a simple model to train on our dataset. Now we are going to train our dataset and add TensorBoard as we do it:

model = create_model()

While compiling the model graph, we have added accuracy as our metric:

model.compile(optimizer=’adam’, loss=’sparse_categorical_crossentropy’, metrics=[‘accuracy’])

Here we are selecting a directory to store the logs of TensorBoard:

log_dir = “logs/fit/” + datetime.datetime.now().strftime(“%Y%m%d-%H%M%S”)

In this line, we initiated a callback (we’ll study later) for TensorBoard and provided the log directory for it to store logs in. Additionally, we have provided a histogram_freq=1 which will identify to save the logs after # every 1 epoch:

tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)

In the following line, we are training our model, by defining the training and testing dataset, the number of epochs. Additionally, the more important part is the callback where we have assigned the tensorboard callback we have defined:

model.fit(x=x_train, y=y_train, epochs=5, validation_data=(x_test, y_test), callbacks=[tensorboard_callback])

Meanwhile, the model is training, open another window and write:

tensorboard — logdir logs

This will display the address you can copy on your browser to open:

TensorBoard 1.15.0 at http://imran:6006/ (Press CTRL+C to quit)

When you open this address, this window pops up:

We can see the validation accuracy increasing and the loss value decreasing, which further establishes that our experiment was successful.

Summary

Basically, in this blog, we have learned the use cases for TensorBoard, why it’s used and when it’s used. This is fairly important because we have seen that there are many other visualization toolkits available. We have established why TensorBoard is the best option we have and has the biggest community handled by Google itself. Moreover, we have different TensorBoard services available and different use cases where TensorBoard is the best option. We have also gone through the definition of each service and when each could be used. We have also gone through how to set up the TensorBoard on different platforms like Ubuntu (linux), MacOS (Unix), and Windows.

References

https://www.tensorflow.org/tensorboard