The following blog will help the readers to understand how we can combine image and tabular data together in PyTorch using deep learning and generate predictions from the model . I will go step by step to understand the whole end to end scenario easily : →
#1# Create a custom dataset class →
I am going to use here OpenVaccine: COVID-19 mRNA Vaccine Degradation Prediction dataset . After cleaning and processing the tabular data I will create the dataset class for our data as follows……
As there are a lot of codes and syntaxes , I recommend reading the codes with respect to the comments given beside each syntax written . The steps involved are: →
- First we declared some of the specific libraries that we will need .
- we will define the constructor for the dataset class that we created.
- then we will define the __len__ function to calculate the length of the dataset
- then we will define the __getitem__ function extract the input and the targets that we will be feeding into our model that we will create.
#2# Create object for dataset class and dataloaders →
#3# Create model class →
I have commented the code syntaxes as much as possible , but reader if you still feel that you are unable to understand any part of the code please please please feel free to comment on this article , I will be very glad to answer your doubts .
Still there are 3 major syntaxes in this present in this at line no. 15, 21 & 56 where we combine our tabular and image data layers to generate predictions out of it .
15 self.image_dense_layer_2 = nn.Linear(512, 1) # in constructor 21 self.tabular_dense_layer_4 = nn.Linear(4, 1) # in constructor56 x = torch.cat((x, tab), dim=1) # in forward function
#4# Define loss function , optimizer and training loop for our model.
After model creation we will define the following things easily .
Please consider going through my notebook I have written on this topic once , if you find difficulty in implementing it or simply put your comments here.
I hope you have understood the logic behind how we can combine image data & tabular data in a deep learning model using pytorch . Let me know , if you have any Questions , Comments or Concerns regarding this in the comment section ; Until then enjoy LEARNING……………….
For more awesome data science related blog posts check out the top data science bloggers on medium.