Introduction to Pytorch and Tensors

I’ve taken up the 6-week course by Jovian.ml on “Deep Learning with PyTorch: Zero to GANs” taught by Aakash N S. In this article (and the following ones), I will share what I’ve learned during the course.

Intro to PyTorch

Pytorch is an open-source machine learning and deep learning framework developed by Facebook. It is created to process large-scale image analysis, including object detection, recognition and classification and several other tasks. It is written in Python and C++ language. It can be used with other frameworks like Keras etc., to implement complex algorithms.

Tensors

PyTorch is built on tensors. A PyTorch tensor is an n-dimensional array, similar to NumPy arrays.

Installing PyTorch

Open the Anaconda terminal, type and run the following command:

conda install pytorch torchvision cudatoolkit=10.2 -c pytorch

Using Pytorch

Start by importing the PyTorch library into your Jupyter Notebook. I’m importing it as tr (too lazy to type the whole thing). You can just import it as torch if you wish.

To create a tensor, tensor() function is used. It takes a list of elements (can be of any number of dimensions) as input to create a tensor.

— randint( low, high, size)

Creating a tensor can be tedious if it has many dimensions. In such cases, the randint() function comes in handy. It takes three inputs.

1. The lowest integer to be drawn from the distribution (low)
2. The highest integer to be drawn from the distribution (high)
3. A tuple defining the shape of the output tensor.

The above tensor is two dimensional, having the lowest value as 0 and the highest value as 50.

Now let’s create a 3-dimensional tensor using randint( ) function.

— numel(tensor)

num-el stands for number of elements. If you want to know how many elements are in a tensor, all you need to do is to pass the tensor as an input to this function.

— torch.cat(tensors, dim=0)

The cat( ) function concatenates the given sequence of tensors along the given dimension. The argument tensors is a tuple which should contain a sequence of tensors that are to be concatenated. dim is for dimension. Its default value is 0 which means the tensors are concatenated along the x-axis by default.

If the dim is passed as 1, then the tensors are concatenated over columns.

The dimensions of all the tensors that are to be concatenated must be the same and would return a traceback otherwise.

— chunk(tensor, chunks, dim=0)

The function splits a tensor into a given number of chunks. Each chunk is also a tensor. The last chunk would be smaller if the tensor is not exactly divisible by the number of chunks. If dim is 0, then the tensor is split along the rows.

In the above example, the tensor is split into two chunks. The second one has lesser rows than the first one as the tensor cannot be exactly split into two.

If dim=1 is passed, the tensor is split along the columns.

— unbind(tensor, dim=0)

The function returns a tuple of tensors sliced along the given dimension dim.

By default, dim = 0 hence the tensor is sliced along its rows. If dim = 2, then it unbinds the tensor along its third dimension. In the below example, a tuple of three tensors are created, each of size 4*5.

— bitwise_and(t1, t2)

The function computes the bitwise AND operation of the tensors t1 and t2. The dtypes of the input tensors for this operation must be either integral or boolean.

If the dtype of the tensors is neither integral nor boolean then the function returns a traceback.

Other bitwise functions are:

• bitwise_or()
• bitwise_not()
• bitwise_xor()

— gather(input, dim, index)

The gather( ) function is a multi-index selection method. The first argument, input is the source tensor that we want to select elements from. The second one, dim is the dimension along which we want to gather the values. The argument index is a tensor containing indices to the input tensor.

Let’s see how this function operates:

For a 2-D Tensor,

out [i][j] = input[index [i][j]] [j]  #if dim == 0
out [i][j] = input[i] [index [i][j]]  #if dim == 1

Let’s see an example of how this function works.

If the dim = 1, then the function gathers values along the columns.

For 3-D tensors, let’s see how this function operates:

out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0
out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1
out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2

Commonly used PyTorch Tensor functions:

zeros( )

Returns a tensor filled with the value 0 with the shape defined by the arguments.

ones( )

Returns a tensor filled with the value 1 with the shape defined by the arguments.

eye( )

Returns a tensor with ones on the diagonal and zeros elsewhere.

Conclusion

In this article, we discussed about PyTorch, Tensors and some basic operations that can be performed on tensors which are creating, concatenating, slicing and performing bitwise operations on tensors.

References

Official documentation for Tensor: PyTorch Docs

I’ve also used Stack Overflow to understand more about these functions.

Also, check out my notebook on jovian.ml which contains all the code for the functions mentioned above.

I hope you’ve found this article helpful to you in gaining knowledge about PyTorch and Tensors.