Introduction to Pytorch with Tensor Functions

Before we dive into the topic I want to thank Mr. Aakash, Co-Founder of Jovian.ml and the team for their continuous support via the Data Science community.

What is Pytorch? The obvious question

PyTorch is an open source machine learning library. It is used for applications such as computer vision, natural language processing and Deep Learning. It was primarily developed by Facebook’s AI Research lab.

Pytorch models(Programs) are capable of running both on a CPU as well as GPU(for enhanced performance). To install Pytorch locally use this command in your programming environment. (Psst. Set ‘CUDA = None’ if you do not have a GPU)

Introducing Tensors

Tensors are special data-types in Pytorch. They can store multidimensional arrays (1D, 2D, 3D, 4D, …) which are of the same data-type. A Tensor can be created from python Data types and converted back with ease. Pytorch has numerous mathematical and special functions that can be performed on these tensors. These functions use very efficient algorithms boosting program performance.

Creating our first Tensor

Let us import the Pytorch module first…

torch.tensor() is used to declare and initialize a tensor.

Here [2., 3.] the ‘.’ represents float data type. We can also pass in sequential Python Data Types to convert them to a Tensor.

Lets now learn few Tensor functions…

Use this as a reference to my Jupyter Notebook.

Tensor Functions which will be discussed:

1. torch.reshape()
2. torch.tensor.apply_()
3. torch.matmul()

For more functions check this out…

The Reshape Tensor Function

Syntax — torch.reshape(input, shape) → Tensor

This function is used to change the dimension(shape/size) of a tensor(matrix). The function returns new tensor and does not modify tensor inplace.

In the above code tensor with dimensions [12](1 Dimensional) was reshaped to a 2 dimensional tensor tensor2. A tensor of 12 elements can be reshaped to any dimension (n x p x l x ...) such that the elements perfectly fit in the new shape with no extra space or less space.

Explanation: mat(3 * 6) has 18 elements. After reshaping mat(3 * 3 * 2) has 18 elements. Hence no space is extra or less in new matrix

The code breaks if the same 18 elements were tried to fit in a matrix with dimensions [3,3,3] which can hold upto 27 elements(creates extra null space for 9 elements).

The tensor.apply_() Function

Syntax — tensor.apply_(callable)

The .apply_() function of Pytorch is similar to the .apply() function from pandas. This function is used to perform an operation over all the elements of a tensor. It takes an argument as a callable function/lambda function which alters and returns modified values of tensor. .apply_() is an inplace function. In Pytorch the _ is a symbol for inplace functions.

Let us now declare a 2D Tensor with float data type

Let us try to increment the values of the tensor by 0.2using apply().

The lambda function declared returns x + 0.2 i.e. an incremented value of x by 0.2 where x is an element from tensor(matrix). The lambda function is then performed of every element.

We can also pass a function directly instead of using lambda.

Lets see how it works…

In the above example we have created a function check_three(x) that simply checks the element 3 and replaces it with 0.
the .apply_() takes the function and passes elements from the matrix tensor2 one by one. The function returns 0 if the element is 3 else returns the element itself.

psst.. Always remember to make the function return a value. the .apply_() method expects a return value for every value it passes in a function (callable). apply_() is used to modify values in tensor and if no value is returned it throws error.

The Matrix Multiplication Function

Recommended Reading — What is Matrix Multiplication? , What is BroadCasting?

Syntax — torch.matmul(input, other, out=None) → Tensor

.matmul() is a matrix multiplication function for two tensors.

The behavior depends on the dimensionality of the tensors:

• If any one of the values in matrix multiplication are scalar then dot product is calculated.
• If the dimensions of both tensors are same then normal Matrix Multiplication is performed.
• If one argument is >=1D and other is >=2D then a batch matrix multiplication is performed where the lower dimension matrix is brought to a dimension equal to the other matrix by prepending a 1 in dimension. Later the prepended dimension is removed.

Example:
Assume the dimensions of 2 matrix as — [j, 1, n, p] @ [k, n, p]
The lower dim [k, n, p] is brought to higher dimension by prepending dim(1) - [j, 1, n, p] [1, k, n, p] (broadcasting step)
Result:[j, k, n, p]
Note: the above matrix multiplication is valid as the number of columns p in mat[j, 1, n, p] is equal to number of rows p in mat[k, n, p]
(Recommended reading Matrix Multiplication)

Let’s take another example. Matrix multiplication mat[2,3,4] @ mat[4,5].

mat[2,3,4] @ mat[4,5]. Here Number of columns 4 is equal to number of rows 4 but the dimensions are not same.

Resolving Dimension Breakdown.
mat[2,3,4] @ mat[4,5]
mat[2,3,4] @ mat[1,4,5](broadcasting step)
Resultant - mat[2,3,5]

psst.. @ is a left associated operator introduced to substitute .matmul() function and make matrix multiplication easier to read and perform.

Refer this for better understanding…

Conclusion

This article was my first Tech-Blog Post. It was a part of my Learning and experience writing technical blog. If at all this helped you do share and give feedbacks.

You can Connect with me here. I’d be happy to help. Happy Learning : )

Reference Links

• Official documentation for torch.Tensor: https://pytorch.org/docs/stable/tensors.html
• The Jupyter Notebook from where this whole article was originally produced: blog-tensor-operation/tensor-operation-tutorial.ipynb