Day 2: PyTorch Tensors: The Building Blocks | Deep learning Series

PyTorch, a popular deep learning framework, relies on a fundamental data structure called a tensor. Understanding tensors is essential for anyone venturing into the world of PyTorch. This blog post serves as a comprehensive guide, introducing tensors, exploring basic operations, and delving into their creation and properties.

What are Tensors?

Think of a tensor as a multi-dimensional array that holds numerical data. Similar to spreadsheets, tensors can have rows and columns, but they can extend to even more dimensions. This allows them to efficiently represent complex data structures like images (3D) or even video sequences (4D).

Properties of Tensors

Tensors in PyTorch hold several crucial properties:

• Data Type: Tensors store data in specific data types, like floats, integers, or booleans. You can define the data type during creation using the dtype argument.

tensor = torch.tensor([1, 2, 3])

print(tensor.dtype) #  torch.int64

• Device: Tensors can reside on either the CPU or the GPU (if available). PyTorch allows you to transfer tensors between devices using the .to method.

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tensor = torch.tensor([1, 2, 3]).to(device)
print(tensor.device)

• Shape: The shape of a tensor defines the number of elements along each dimension. You can access the shape using the .shape attribute.

tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])

print(tensor.shape)   # torch.Size([2, 3])

Creating Tensors in PyTorch

PyTorch offers multiple ways to create tensors. Here are two common methods:

1. Using the torch.tensor() Function:

This function is the most straightforward way to create a tensor. You simply pass a list of lists (or nested lists for higher dimensions) containing the data.

import torch

my_tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])
print(my_tensor)

2. Importing NumPy Arrays:

import numpy as np
import torch

numpy_array = np.array([[1, 2, 3], [4, 5, 6]])
my_tensor = torch.from_tensor(numpy_array)
print(my_tensor)

Basic Tensor Operations

1. Arithmetic Operations

a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])

add_result = a + b
print(add_result) # [5, 7, 9]

sub_result = a - b
print(sub_result) # [-3, -3, -3]

mul_result = a * b
print(mul_result) # [4, 10, 18]

div_result = a / b
print(div_result) # [0.2500, 0.4000, 0.5000]

2. Matrix Operations

matrix_a = torch.tensor([[1, 2], [3, 4]])
matrix_b = torch.tensor([[5, 6], [7, 8]])
# either use .matmul or @ for matric multiplication
matmul_result_1 = torch.matmul(matrix_a, matrix_b)
print(matmul_result_1)

matmul_result_2 = matrix_a @ matrix_b
print(matmul_result_2)

transpose_result = torch.transpose(matrix_a, 0, 1) #  swap first two dimension [A, B] --> [B, A]
print(transpose_result)

tensor_3d = torch.tensor([[[1, 2, 3, 4], 
                           [5, 6, 7, 8], 
                           [9, 10, 11, 12]],
                          [[13, 14, 15, 16], 
                           [17, 18, 19, 20], 
                           [21, 22, 23, 24]]])


transposed_tensor = tensor_3d.transpose(-2, -1) # swap last two dimension [A, B, C] --> [A, C, B]
print(transposed_tensor)
print("Shape of transposed tensor:", transposed_tensor.shape)

3. Stacking

# Creating tensors
tensor1 = torch.tensor([1, 2])
tensor2 = torch.tensor([3, 4])
tensor3 = torch.tensor([5, 6])

# Stacking tensors along a new dimension
stack_result = torch.stack((tensor1, tensor2, tensor3), dim=0)
print(stack_result)  
       # tensor([[1, 2],
       # [3, 4],
       # [5, 6]])

4. Slicing

# Creating a tensor
tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])

# Indexing a specific element
element = tensor[0, 1]
print(element)

# Slicing specific rows and columns
row = tensor[0, :]
column = tensor[:, 1]
print(row)
print(column)

5. Squeeze

tensor = torch.tensor([[[1], [4]]])

# Removing dimensions of size 1
squeezed_tensor = tensor.squeeze()
print(squeezed_tensor)  # torch.tensor([1,4])

PyTorch tensors are the building blocks of deep learning models, offering powerful capabilities for data manipulation and computation. In this blog post, we introduced you to tensors, covered their creation methods, explored basic tensor operations, and discussed essential properties. Understanding tensors is crucial for working with PyTorch and developing sophisticated neural networks. With this foundational knowledge, you’re now ready to dive deeper into the world of PyTorch and harness its full potential for your deep learning projects. Happy hacking!