Leap of Faith in PyTorch: Tensor Operations (Day 2)
We continue our attempts to fit a few countries in this complicated world, namely China, US, Canada, Russia, and India. Welcome to Day 2 of our 29-day PyTorch journey!
Today, we’ll dive deeper into tensor operations, the fundamental building blocks of any PyTorch model. We’ll explore various tensor operations and practice with examples combining financial, social, and political data from five countries: USA, China, India, Canada, and Russia.
Recap from Day 1
In our previous post we covered some basics of PyTorch and created our first tensors. Then, we combined some financial, social, and political data to calculate GDP per capita and adjusted governance scores for our selected countries.
Basic Tensor Operations
In a way, the tensors in PyTorch are similar to NumPy arrays, but they also come with additional capabilities for GPU acceleration. We will explore some below.
Creating Tensors
Let’s start by creating tensors for our financial, social, and political data.
import torch
# Financial data: GDP in trillion dollars (year 2023)
gdp = torch.tensor([21.43, 14.34, 5.08, 1.64, 1.48]) # USA, China, India, Canada, Russia
# Social data: Population in millions
population = torch.tensor([330, 1400, 1380, 38, 144]) # USA, China, India, Canada, Russia
# Political data: Governance score (hypothetical)
governance_score = torch.tensor([85, 70, 60, 80, 65]) # USA, China, India, Canada, RussiaBasic Operations
When starting to work with a tool, regardless of how complicated or fancy it is, we should first ask ourselves what basic things we can do with it. We should not start running before we learn to walk.
Such operations as addition, subtraction, multiplication, and (you guessed it) division have been considered as basic operations for thousands of years now. They still remain as such. We will start with these first.
Addition and Subtraction
We can perform element-wise addition and subtraction of tensors, as follows:
# Increasing GDP by 10%
gdp_increased = gdp + gdp * 0.1
print("Increased GDP:\n", gdp_increased)
# Reducing population by 5%
population_reduced = population - population * 0.05
print("Reduced Population:\n", population_reduced)Multiplication and Division
Element-wise multiplication and division are straightforward as well.
# Multiplying GDP by governance score
gdp_governance = gdp * governance_score
print("GDP multiplied by Governance Score:\n", gdp_governance)
# Dividing GDP by population to get GDP per capita in thousands of dollars
gdp_per_capita = (gdp * 1000) / population
print("GDP Per Capita (in thousands of dollars):\n", gdp_per_capita)Advanced Operations
Now that we have tested the waters, we can proceed further. While working with data in PyTorch, we will rarely need to perform only basic addition and subtraction.
Given that most of your data will come in tensors of different dimensions (called shapes), you will often need to play with these tensors to change their dimensions while not losing the data it is contained in (called reshaping), accessing specific entries only (called indexing), and much more.
Follow along with some more examples.
Reshaping Tensors
Reshaping tensors is a common operation when preparing data for models.
print("GDP Tensor before reshape:\n", gdp)
# Reshaping a 1D tensor into a 2D tensor
print("\nReshaped GDP to 2D tensor:\n", gdp.view(1, 5))
# Reshaping a 1D tensor into a 3D tensor
print("\nReshaped GDP to 3D tensor:\n", gdp.view(1, 1, 5))Expected output:
GDP Tensor before reshape:
tensor([21.4300, 14.3400, 5.0800, 1.6400, 1.4800])
Reshaped GDP to 2D tensor:
tensor([[21.4300, 14.3400, 5.0800, 1.6400, 1.4800]])
Reshaped GDP to 3D tensor:
tensor([[[21.4300, 14.3400, 5.0800, 1.6400, 1.4800]]])The view() function that we use is capable of more complex transformation of the original data, as well as of the transformation of the original data types from one type to another. For brievity, we demonstrated only one use-case, in which we take a 1-dimensional tensor, and make a 2-dimensional one, then a 3-dimensional, by nesting the original tensor deeper and deeper.
Indexing and Slicing
We can index (aka accessing a specific entry inside the tensor), and slice tensors (aka extracting only parts or segments of a tensor), to access specific elements or subsets.
# Accessing GDP of the first country (USA)
usa_gdp = gdp[0]
print("USA GDP:\n", usa_gdp)
# Slicing to get GDP of the first three countries
gdp_first_three = gdp[:3]
print("GDP of first three countries:\n", gdp_first_three)Expected output:
USA GDP:
tensor(21.4300)
GDP of first three countries:
tensor([21.4300, 14.3400, 5.0800])More Financial, Social, and Political Data
Let’s practice more of these operations by combining our financial, social, and political data to access and reshape some of the tensors containing data on the five countries we decided to analyze.
Reshaping
We have already seen examples of reshaping the gdp tensor, but let’s repeat these operations on the other data we have — the population and the governance scores.
# Financial data: GDP in trillion dollars (year 2023)
gdp = torch.tensor([21.43, 14.34, 5.08, 1.64, 1.48]) # USA, China, India, Canada, Russia
# Social data: Population in millions
population = torch.tensor([330, 1400, 1380, 38, 144]) # USA, China, India, Canada, Russia
# Political data: Governance score (hypothetical)
governance_score = torch.tensor([85, 70, 60, 80, 65]) # USA, China, India, Canada, Russia
print("Population Tensor before reshape:\n", population)
# Reshaping a 1D tensor into a 2D tensor
population_2d = population.view(1, 5)
print("\nReshaped Population to 2D tensor:\n", population_2d)
# Reshaping a 1D tensor into a 3D tensor
population_3d = population.view(1, 1, 5)
print("\nReshaped Population to 3D tensor:\n", population_3d)
# Reshaping population data to 2D tensor
governance_score_2d = governance_score.view(5, 1)
print("\nReshaped Governance Score to 2D tensor:\n", governance_score_2d)
gdp_float = gdp.view(torch.float32)
print("\nGDP Tensor with data converted to Float datatype:\n", gdp_float)Expected output:
Population Tensor before reshape:
tensor([ 330, 1400, 1380, 38, 144])
Reshaped Population to 2D tensor:
tensor([[ 330, 1400, 1380, 38, 144]])
Reshaped Population to 3D tensor:
tensor([[[ 330, 1400, 1380, 38, 144]]])
Reshaped Governance Score to 2D tensor:
tensor([[85],
[70],
[60],
[80],
[65]])
GDP Tensor with data converted to Float datatype:
tensor([21.4300, 14.3400, 5.0800, 1.6400, 1.4800])As we can see above, we didn’t only reshape the tensors to different dimensions. We actually managed to change the GDP data from integers to floats using that same view() function.
Isn’t that cool ?!
Indexing and Slicing
We can index (accessing a specific entry inside the tensor), and slice tensors (extracting only parts or segments of a tensor), to access specific elements or subsets.
For instance, below we’ll access the GDP value of USA, specifically. We’ll access India’s governance score as another example.
We will then use slicing to print out the GDP data for the first three countries in our tensor and the last two as well.
Finally, we’ll use a stack() function to demo to you one more advanced way of slicing we can do in pytorch .
# Accessing GDP of the first country (USA)
usa_gdp = gdp[0]
print("USA GDP:\n", usa_gdp)
# Accessing the governance score of the third country (India)
india_governance = governance_score[2]
print("\nIndia Governance Score:\n", india_governance)
# Slicing to get GDP of the first three countries
gdp_first_three = gdp[:3]
print("\nGDP of first three countries:\n", gdp_first_three)
# Slicing to get population of the last two countries
population_last_two = population[-2:]
print("\nPopulation of the last two countries:\n", population_last_two)
# Advanced slicing: Selecting GDP and population of China and India
china_india_gdp_population = torch.stack((gdp[1:3], population[1:3]))
print("\nGDP and Population of China and India:\n", china_india_gdp_population)
Expected output:
USA GDP:
tensor(21.4300)
India Governance Score:
tensor(60)
GDP of first three countries:
tensor([21.4300, 14.3400, 5.0800])
Population of the last two countries:
tensor([ 38, 144])
GDP and Population of China and India:
tensor([[ 14.3400, 5.0800],
[1400.0000, 1380.0000]])Conclusion
Today, we explored various tensor operations in PyTorch, including basic arithmetic, reshaping, indexing, and slicing. We also applied these operations to combine financial, social, and political data from five countries of interest — China, US, Canada, Russia, and India.
Find below a link to the Day 1 material if you want a refresher:
Stay tuned for Day 3, where we will dive into PyTorch’s autograd and computational graphs to understand how PyTorch handles automatic differentiation.
Feel free to run these examples in your Jupyter notebook and experiment with different data to enhance your understanding. Happy learning!
Follow me here on Medium, and on Tweeter, for more posts on Software Development and Data Science, with hands-on examples that are very much connected to our real world!
Your clicks on Follow button, your claps, and your comments are helping me in more ways than you think they are.
I greatly appreciate these gestures!
