Exploring the Pytorch functions
This is my first blog. I am starting this journey with writing about pytorch. Currently I am learning pytroch on jovian.ml and freecodecamp. I am bit late for this blog but I was busy working with some other thing so I am writing this now.
Now we will see short introduction of pytorch.
Pytorch is a open source python library used for various machine learning applications such as Natural language processing, Image processing etc. Now days pytorch is most popular machine learning library of the python to create small scale model. As we all know pytorch is created by and strongly supported by facebook but as pytorch is open sourse it can be used by anyone with knowledge of the pytorch. Pytorch library is generally used for small working model of the algorithm whereas tensorflow is used for bigger model. But this is changing now days because there is big change in pytorch since its inception.
There are many best things about the pytorch. One of the best thing I saw till day is it take gradient of the function on its own and I think is best thing because before I know about that function I used to take gradient of function manually by finding derivation of each step and that was hectic part.
Now we we will see some useful functions of pytorch.
- tensor.index_fill()
tensor.index_fill(dim,index,value)
This function is used to fill the row or column of the tensor with specific element. This function don’t create tensor it fill element into the self tensor. Like last function it also takes three arguments; first argument is dim, dim is used to determine weather you want to fill element column wise or row wise. When dim=0 element is filled row wise and when dim=1 filled column wise. second argument is index, this argument is used to give the index of row or column into which element to be filled. Third argument is actual element to be filled.
Let us see some examples of the index_fill() method.
This is first working example of index_fill() function. In this example we fill elements row wise. We first create temp tensor of zeros with size 5x5, Then we decide we want to fill elements row wise by giving dim argument 0. Then we pass index of the temp tensor and then actual number to be filled into the tensor in this case.
This is second working example. In this example we fill elements column wise. We first create tensor of ones. Then we create index tensor. Then we call index_fill() method. We fill elements column wise by passing the value 0 to dim argument then we pass index tensor and then we pass actual value to be filled into the tensor.
This is breaking point of the index_fill() function.In this example we want to fill the value into the column 6 but size of the temp tensor is 5x5 and hence column 6 don’t exist and at that point function breaks.
This function is very useful when we want to fill the specific value to the whole row or whole column. Only point is this function fills only one element cant fill number of elements means it can fill only 0 dimensional tensor.If we want to copy many elements into specific row then we want to use different method of copy.
2. index_copy()
tensor.index_copy(dim,index,input_tensor)
index_copy() function fills the element of one tensor into another tensor index wise. This function takes three arguments; first one is dim, this argument is used to determine weather we want to copy element column wise or row wise. when dim=0 elements are copied row wise and when dim=1 elements are copied column wise. second argument is index, this argument gives index of tensor to the function in which we want to copy the elements. Generally index is another tensor which gives either column index or the row index. Third argument is actual tensor from which we want to copy aur elements or row or columns. This function takes row or column on the original tensor indexed by index tensor and copy that into new tensor. To work this function respective rows of both tensor should be same if we want to copy rows otherwise column need to be equal
In this example we copy content of the tensor k into tensor temp. index_copy() method cant create new tensor it copy elements into self tensor which means we need to provide the tensor to the method. In this example we copy first row of k into first row of the temp and then second row of k into fifth row of the temp tensor. Index of the temp tensor is given in the index tensor.
In this example we copy element of the k tensor into tensor temp column wise. First row of k is copied into the first column of the temp tensor. Properties of this are same as before just the change is we copy element column wise.
As I stated before if we want to copy column wise columns of both tensors must be same and if we are doing it using row wise the law remains same. But in this example we want to copy 3 columns into temp tensor but shape of temp column is 3x2 which is not valid.
This function is useful when we want to copy one whole tensor into another tensor but only at specific locations. This function lags behind other copy function because it copies complete tensor not the specific rows of the tensor. Another important drawback is it needs row or column of both tensors same because this function don’t use broadcasting ability of tensors.
3. torch.pow()
torch.pow(input,exponent,output)
This function is used to take power of tensor. This function takes three arguments; first argument is input, input is the tensor we want to take power of. Second argument is exponent, this argument is value of the exponent this value can be either single value or tensor. Third argument is output, this argument is optional. This argument decide what type of output we want.
In this example we take square of elements in the tensor. First create input tensor. then we call torch.pow() function. In that function we pass input tensor and value of exponent. As we see before output argument is optional.
In this example we pass value exponent value as tensor. It takes exponent of each row of the tensor.
This breaking point of the function. In this function only two values to the power tensor. There are three rows in the input tensor. So power tensor must have three values but we pass only two values which is not valid. and hence function gives error.
torch.pow() function is perfect to take power of the whole tensor or the specific tensor. As we see this function take power only rows so take power column wise we need to Transpose of the input tensor and after that operation reverse that by again using transpose operation. This function breaks when we don’t pass perfect value to the power tensor.
4. torch.max()
torch.max(input1,input2)
torch.max() functions is used to find the maximum values between two tensors. torch.max() function takes two arguments. Both the arguments can be either single tensor or the tensors. When we pass tensors to the function it find and returns element wise maximum between two elements. It checks each element with corresponding element with other tensor. This function also find maximum value stored in single tensor.
In this example we find maximum between two tensors. As we can see we pass two tensors to the function and then we chaecking element wise maximum.
In this example we are finding biggest value in the tensor. In this example we are not comparing two tensor. As we can see this function can find maximum element from the single tensor.
In this example we are trying to compare two tensors. But shape of input1 is 3x3 and shape of input2 is 3x2 which is not equal. To find maximum between two tensors the need to be of same shape.
This function is very useful in finding the maximum values between two tensors. It is also useful to compare two tensors. This is function is useful in RELU function. RELU funaction is stated as max(0,z). So to find maximum in relu function we use torch.max() function.
5. torch.bitwise_and()
torch.bitwise_and(input,other,output)
torch.bitwise_and() function is used to find bitwise and of two tensors. The values provided to the function must be in the form of integral or boolean. torch.bitwise_and() function takes three argument two compulsory and one optional argument. First argument this function takes is input. we provide tensor of either boolean or integral to find the bitwise and with another tensor. Second argument is other. In this argument we pass other tensor to find the and with first one. Third argument is optional which is output. Output means in which form you want your output.
In this example we are passing integrals to the function and finding and between those. To the function we pass two tensors of integrals and tried to find the bitwise and of those.
In this example we are finding the bitwise and of two boolean tensor. In this example we pass boolean values to the function and find the bitwise and of the boolean values.
Function breaks if we pass tensors of different size to the function. Broadcasting capability of python is not useful in this situation. In this example shape of input 1 is 1x3 and shape of input2 is 1x4 which is not equal hence the function fails to provide the output.
This function is very useful in finding the bitwise and of any value. Many times in cryptography we need to perform various logical operations at that time this function is very useful. Like any other function this function also have break point but it is ignorable.
Conclusion
Each function its own advantages and disadvantages. But one of the main disadvantage is the size of the tensors. In many functions when size of input tensors differes then the function always break. Python Broadcasting is very useful in many functions as the size difference can be avoided using that. Apart from this five functions there are many functions in the pytorch which are vary useful. So we can try all those functions or we can read about those functions in the official documentation and then use them in our project.
