Getting Started with TensorFlow
The Basics of Tensorflow in Python
About TensorFlow
TensorFlow is an open-source library for graph-based numerical computation. Google developed it as a machine learning system based on deep learning neural networks. This platform allows the use of different levels of APIs to build machine learning models that rely heavily on linear algebra calculations.
- The low-level APIs give the tools for building network graphs from the ground-up using mathematical operations. This level provides the highest level of customization for the user.
- In mid-level APIs, TensorFlow provides a set of reusable packages for simplifying the process involved in creating Computational Graphs.
- The High-level APIs provide a simplified API calls that encapsulate lots of the details that are typically involved in creating a deep learning TensorFlow model. This level is the quickest way to deploy an end-to-end model. However, it restricts the amount of customization.
For this blog post, I will be focusing on the low-level python frontend to explain what tensors are, how to perform simple calculations, and provide references on how to start building models using higher-level APIs, with Keras, for example. Below is a graphical representation of the three API levels in TensorFlow.
Before getting started, let’s get TensorFlow installed if you do not already have it in your environment.
pip install --ignore-installed --upgrade tensorflow==2.1What is a Tensor?
In short, a tensor is a TensorFlow object. It is a generalization of vectors and matrices of a specific shape. It could be a scalar, a vector, or a matrix. Let’s start by creating a simple 3x3 array in numpy.
import numpy as np
sample_matrix = np.random.randint(5, size=(3, 3))
sample_matrix
--------------------------------
array([[1, 3, 1],
[4, 2, 4],
[0, 1, 1]])If we want to transform this numpy array to a tensor, it is as easy as:
import tensorflow as tf
tensor_sample=tf.constant(sample_matrix)
tensor_sample
----------------------------------------
<tf.Tensor: shape=(3, 3), dtype=int64, numpy=
array([[1, 3, 1],
[4, 2, 4],
[0, 1, 1]])>Notice that, when we call on the numpy variable, all we get displayed is the numpy array. Meanwhile, a tensor will display its shape, datatype, and the array itself. The tf.constant() is the simplest tensor category. It can have any dimension and is not trainable. It can receive a python list or a numpy array. When training a model, your features and labels will need to be parsed as a tf.constant() object to work on TensorFlow.
sample_list=[1,2,3,4,5,6]
tensor_list = tf.constant(sample_list,shape=(2,3),dtype=tf.int32)
tensor_list
------------------------------------------
<tf.Tensor: shape=(2, 3), dtype=int32, numpy=
array([[1, 2, 3],
[4, 5, 6]], dtype=int32)>To continue, tf.Variable() is another tensor category. Although similar to tf.constant(), the difference is that the tf.variable is trainable. Machine learning models with use tf.Variable() tensors as a trainable model parameter. The instantiation process of a tf.Variable() tensor is identical to that of the tf.constant(). This post does not include the two other tensor categories: tf.placeholder(), and tf.SparseTensor().
Basic Operations
In TensorFlow, you can perform all kinds of mathematical computations among tensors. The rule of thumb is that element-wise calculations should have tensors with identical shapes. At the same time, in matrix multiplication (dot product), the number of columns of the first matrix must equal the number of rows of the second matrix. There are two ways to multiply tensors or matrices in general. To perform element-wise multiplication, you should use the tf.multiply() method. To perform matrix multiplication, you should use the tf.matmul() method. Let’s see how they differ by multiplying two 2x2 tensors.
tensor_x = tf.constant([1,2,3,4],shape=(2,2),dtype=tf.int64)
tensor_y = tf.constant([5,6,7,8],shape=(2,2),dtype=tf.int64)
print(tensor_x)
print(tensor_y)
----------------------------
tf.Tensor(
[[1 2]
[3 4]], shape=(2, 2), dtype=int64)
tf.Tensor(
[[5 6]
[7 8]], shape=(2, 2), dtype=int64)Now that we have our 2x2 tensors, let’s compare the results from both multiplication methods:
print('Using multiply: ',tf.multiply(tensor_x,tensor_y))
print('Using matmul: ',tf.matmul(tensor_x,tensor_y))
------------------------------
Using multiply: tf.Tensor(
[[ 5 12]
[21 32]], shape=(2, 2), dtype=int64)
Using matmul: tf.Tensor(
[[19 22]
[43 50]], shape=(2, 2), dtype=int64)We can notice that in the tensor resulting from element-wise multiplication, each of the elements is paired to its corresponding element in the other matrix and multiplied. The first element using the tf.multiply() method is the result of 1x5 =5. That is why tensors should be identical in shape. On the other hand, the first element in the tf.matmul() resulting tensor is obtained by (1x5)+(2x7)=19, this is also called the dot product. For this, to work, the second matrix should have the same number of rows as the first matrix has columns. The resulting matrix will have the number of rows of the first matrix and the number of columns of the second matrix after performing the dot product or tf.matmul(). But, what if your matrices do not follow the shape requirements? Reshape your tensor! Assuming you have a 3-dimensional tensor for a color picture, you can reshape it to a 2-dimensional tensor.
photo = tf.random.uniform([2, 2, 3], maxval=255, dtype='int32')
photo = tf.reshape(color, [2*2, 3])
photo
------------------------------------
<tf.Tensor: shape=(4, 3), dtype=int32, numpy=
array([[ 97, 33, 99],
[177, 7, 100],
[ 11, 192, 46],
[ 94, 112, 9]], dtype=int32)>There you go. Now you have a 4x3 matrix. In training neural networks, you will often need to parse the image as a vector. You would yous use the same function and multiply all the axis to have a shape that fits all of the values.
photo = tf.reshape(color, [2*2*3])
photo
---------------------------------
<tf.Tensor: shape=(12,), dtype=int32, numpy=
array([ 97, 33, 99, 177, 7, 100, 11, 192, 46, 94, 112, 9],
dtype=int32)>Higher-level APIs
I have covered some of the most frequent basic operations that are happening under the hood of a machine learning model built on TensorFlow. When getting ready to start a project, you can input your data into your jupyter notebook, using pandas, for example. And then, transform your features and your label as tensors using higher-level APIs. Remember that the higher the API level, the less customization. However, it is an excellent way to start and work your way down to lower levels. One of these high-level APIs is Estimators, which is a model-level abstraction. It encapsulates, training, evaluation, and prediction of your model in a few lines of code. For a quick tutorial, follow this link:
Keras, in my opinion, is the most popular of the APIs. The new TensorFlow release was optimized to play nicer with Keras making it an integral part of its framework. It is used for building and training deep learning models. It’s used for fast prototyping, state-of-the-art research, and production, with three key advantages:
- User-friendly
Keras has a simple, consistent interface optimized for common use cases. It provides clear and actionable feedback for user errors. - Modular and composable
Keras models are made by connecting configurable building blocks, with few restrictions. - Easy to extend
Write custom building blocks to express new ideas for research. Create new layers, metrics, loss functions, and develop state-of-the-art models.
For a quick tutorial on how to get running on Keras follow this link:
Conclusion
In my opinion, there is no way better to get started on TensorFlow than using High-level APIs such as Keras and Estimators. These APIs provide a user-friendly environment to build a model in a few guided steps. As you become more familiar with your data and your model requirements, you can start working your way down to low-level APIs to gain more flexibility in customizing your model.
