PyTorch For Deep Learning — Binary Classification ( Logistic Regression )

This blog post is for how to create a classification neural network with PyTorch.
Note : The neural network in this post contains 2 layers with a lot of neurons. but, if the number of out features and number of layers are reduced to 1, this would just become an ordinary logistic regression

Having said that, let’s jump into the code

1. Importing the libraries

#importing the libraries
import torch
import numpy as np
import matplotlib.pyplot as plt

2. Dataset

For this post, we are going to be using sklearn’s famous breast_cancer dataset

#importing the dataset
from sklearn.datasets import load_breast_cancer
data = load_breast_cancer()
x = data['data']
y = data['target']
print("shape of x: {}\nshape of y: {}".format(x.shape,y.shape))
output:
shape of x: (569, 30) 
shape of y: (569,)

3. Feature Scaling

It is important to scale the features to a standard normal before sending it to the neural network.

basically, This subtracts the mean of the column and divides by the standard deviation of a column for each value in the column ( Independent Variable)

#feature scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
x = sc.fit_transform(x)

4. Dataset and DataLoader

Dataset class in pytorch basically covers the data in a tuple and enables us to access the index of each data. this is necessary to create dataloader class which can be used to shuffle, apply Mini-Batch Gradient Descent and more.

Check out the previous post for more examples on how this works

#defining dataset class
from torch.utils.data import Dataset, DataLoader
class dataset(Dataset):
  def __init__(self,x,y):
    self.x = torch.tensor(x,dtype=torch.float32)
    self.y = torch.tensor(y,dtype=torch.float32)
    self.length = self.x.shape[0]
 
  def __getitem__(self,idx):
    return self.x[idx],self.y[idx]
  def __len__(self):
    return self.length
trainset = dataset(x,y)
#DataLoader
trainloader = DataLoader(trainset,batch_size=64,shuffle=False)

5. Neural Network for Classsification

in Pytorch, neural networks are created by using Object Oriented Programming.The layers are defined in the init function and the forward pass is defined in the forward function , which is invoked automatically when the class is called.

These Functions are possible because of the class nn.Module from torch which was inherited.

The output of the neural network is between 0 and 1 as sigmoid function is applied to the output which makes the network suitable for binary classification.

#defining the network
from torch import nn
from torch.nn import functional as F
class Net(nn.Module):
  def __init__(self,input_shape):
    super(Net,self).__init__()
    self.fc1 = nn.Linear(input_shape,32)
    self.fc2 = nn.Linear(32,64)
    self.fc3 = nn.Linear(64,1)
  def forward(self,x):
    x = torch.relu(self.fc1(x))
    x = torch.relu(self.fc2(x))
    x = torch.sigmoid(self.fc3(x))
    return x

If simple logistic regression is enough , the layer fc2 and fc3 could be removed.

6. Some more Parameters

Selecting various parameters such as number of epochs , loss function , learning rate and more

#hyper parameters
learning_rate = 0.01
epochs = 700
# Model , Optimizer, Loss
model = Net(input_shape=x.shape[1])
optimizer = torch.optim.SGD(model.parameters(),lr=learning_rate)
loss_fn = nn.BCELoss()

BCELoss is a pytorch class for Binary Cross Entropy loss which is the standard loss function used for binary classification.

Training

The Gradients that are found from the loss function are used to change the values of the weights and the process is repeated several times.

This is done to minimize the loss function and increase the accuracy

Also , the Dataset is not split into training and test set because the amount of data is already low

#forward loop
losses = []
accur = []
for i in range(epochs):
  for j,(x_train,y_train) in enumerate(trainloader):
    
    #calculate output
    output = model(x_train)
 
    #calculate loss
    loss = loss_fn(output,y_train.reshape(-1,1))
 
    #accuracy
    predicted = model(torch.tensor(x,dtype=torch.float32))
    acc = (predicted.reshape(-1).detach().numpy().round() == y).mean()
    #backprop
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

  if i%50 == 0:
    losses.append(loss)
    accur.append(acc)
    print("epoch {}\tloss : {}\t accuracy : {}".format(i,loss,acc))
output:
epoch 0 loss : 0.6731628775596619  accuracy : 0.6274165202108963 epoch 50 loss : 0.2154722362756729  accuracy : 0.9507908611599297 epoch 100 loss : 0.1049698144197464  accuracy : 0.9718804920913884 epoch 150 loss : 0.08159173280000687  accuracy : 0.9824253075571178 epoch 200 loss : 0.06935682147741318  accuracy : 0.984182776801406 epoch 250 loss : 0.0607854388654232  accuracy : 0.984182776801406 epoch 300 loss : 0.053541962057352066  accuracy : 0.9859402460456942 epoch 350 loss : 0.047878947108983994  accuracy : 0.9876977152899824 epoch 400 loss : 0.043598778545856476  accuracy : 0.9894551845342706 epoch 450 loss : 0.039949361234903336  accuracy : 0.9912126537785588 epoch 500 loss : 0.036903925240039825  accuracy : 0.9929701230228472 epoch 550 loss : 0.03445163369178772  accuracy : 0.9929701230228472 epoch 600 loss : 0.032331496477127075  accuracy : 0.9929701230228472 epoch 650 loss : 0.030418962240219116  accuracy : 0.9929701230228472

detach() function removes the requires_grad from the tensor so that it can be converted to numpy and accuracy is a list that stores the accuracy at each epoch.
Except that, Everything here is self explanatory if all the previous posts have been read.

Analysis of the Model

plotting loss and accuracy over epochs to see how it changed over training

#plotting the loss
plt.plot(losses)
plt.title('Loss vs Epochs')
plt.xlabel('Epochs')
plt.ylabel('loss')

#printing the accuracy
plt.plot(accur)
plt.title('Accuracy vs Epochs')
plt.xlabel('Accuracy')
plt.ylabel('loss')

This loss and accuracy plot proves that our model has learnt well.

Thank You