A short Introduction to Pytorch using logic gates in Perceptron

A Perceptron can be thought of as an algorithm with an objective to classify the output into binary outcomes i.e. 1 or 0, True or False. It is a linear classifier, thus it uses a linear combination of weights, inputs and the bias input.

The above equation contains w0 x0which is called a `bias` or `intercept` and the equation can be further summarised in the following way.

The output y = 1 if the result of the dot product is ≥ 0 else if the dot product is < 0 then, y = 0

Pytorch is an open source deep learning platform that provides a seamless path from research prototyping to production deployment. Pytorch is an extension of numpy and allows us to use GPUs to solve compute-intensive problems in research and in business. We will implement the perceptron algorithm in `Pytorch` and use logic gates to better understand the effects that weights and bias have on the output.

Logic Gates

Imagine we have 4 observations with 1 and 0 as inputs and we want to predict 1 or 0 based on some function. In the below examples we will explore AND, OR and NOT function and we will use the 4 observations to predict the correct value for each of the 3 functions.

Please focus on the equation that we described above as we will use it to create the input tensor in Pytorch. Vector x1 and x2 are defined as 1 or 0 switching their places as we move down the observations, while the constant 1 is the bias i.e. x0 vector. We now have the inputs ready but we are still missing w0,w1, and w2, we need to define them for each logic function.

#importing pytorch library
import torch
#creating input tensor 
input_tensor = torch.tensor([(0, 0, 1), (0, 1, 1), (1, 0, 1), (1, 1, 1)],dtype = torch.double)
print(input_tensor)
#tensor([
#[0., 0., 1.],         
#[0., 1., 1.],         
#[1., 0., 1.],         
#[1., 1., 1.]], dtype=torch.float64)

1| AND

We will use the above input tensor and the linear combination of weights and their inputs to give us the right output for AND function.

As we described above we need to perform the dot product between the weights and the inputs, thus the most obvious question is what are the correct weights and how do we perform the dot product in Pytorch?

Throughout this post, I will define the weights that I know will work. However, I encourage you to test other weights that might work as a solution and give us the right answer. I am using torch.tensor to create a tensor object in python, it accepts dtype parameter, I will encourage you to read more about it here.

and_weights = torch.tensor([1, 1, -2],dtype = torch.double) #creating a tensor
print(and_weights.shape)
#torch.Size([3]) object shape

As you will notice the shape of and_weights is torch.Size([3]) and thus to perform a valid dot product we need to reshape it into rows and columns, we will do like so and_weights.resize_(1,3) The object is now resized to torch.Size([1, 3])

pay attention to the underscore(_) in .resize_ function

Any operation that mutates a tensor in-place is post-fixed with an _. For example: x.copy_(y), x.t_(), will change x.

We now use the weights and the inputs to perform the dot product, but the catch is .dot() will not work as we are used to innumpy. We must use mm i.e matrix multiplication for 2d size tensors or more.

input_tensor also needs to be transposed and thus we perform a valid matrix multiplication i.e. (1 x 3) (3 x 4) for weights and input tensor respectively.

and_matrix_mult = and_weights.mm(input_tensor.t())
print(and_matrix_mult)
#tensor([[-2., -1., -1., 0.]], dtype=torch.float64)

The matrix output gives us some negatives and positive numbers which when passed through the above formula gives us and_output.

and_output = [[1 if i>= 0 else 0 for i in x ]for x in and_matrix_mult]
print(and_output)
[[0, 0, 0, 1]]

That’s it for the perceptron on AND gate!

2| OR

The method to get OR function is similar to the AND function so I will race through the code and explain the changes. Once again, we use the same input tensor and use the linear combination of weights and inputs to give us the right OR function output. The following 4 lines code does that for us.

or_weights = torch.tensor([2,2,-1],dtype = torch.double).resize_(1,3)
or_matrix_mult = or_weights.mm(input_tensor.t())
or_output = [[1 if i>= 0 else 0 for i in x ]for x in or_matrix_mult]
print(or_output)
#[[0, 1, 1, 1]]

3| NOT

The NOT function is a bit different to other perceptrons, the NOT function only cares about one input. The operation returns a 0 if the input is 1 and a 1 if it’s a 0. The other inputs to the perceptron are ignored.

not_weights = torch.tensor([2,-4,1],dtype = torch.double).resize_(1,3)
not_matrix_mult = not_weights.mm(input_tensor.t())
not_output = [[1 if i>= 0 else 0 for i in x ]for x in not_matrix_mult]
print(not_output)
#[[1, 0, 1, 0]]

Summary

You have just explored the very basic application of Pytorch using perceptrons. The operations we have explored were AND, OR and NOT, however, there are many other operations like XOR, NAND etc. The other operations can be thought of as multi-layered i.e. once you have passed them an operation you will need them to go through another operation before you get the expected result. for e.g. NAND = NOT + AND.

I hope this post was helpful, Please leave some constructive feedback and definitely few claps it keeps me motivated to write more!