Using PyTorch for Kaggle’s famous Dogs vs. Cats challenge Part 2 (inferencing and submitting)

This story is the second part of the series where I explain how to do Dogs vs. Cats challenge on kaggle using PyTorch. If you didn’t read the first part of the series, I recommend you read it first. It talks about data preprocessing, model training from a pretrained model, saving/loading the best model etc. In this second part, I will talk about how to make inferences on test data, writing it to csv file and submitting it to kaggle. The code shown below may need the imports, functions, variables from first part to run without errors.

So we trained our model in the previous part, now let’s first visualize how our model is doing on validation set.

def visualize_model(model, num_images=6):
    was_training = model.training
    model.eval()
    images_so_far = 0
    fig = plt.figure()
    with torch.no_grad():
        for i, (inputs, labels) in enumerate(dataloaders['val']):
            inputs = inputs.to(device)
            labels = labels.to(device)
    outputs = model(inputs)
            _, preds = torch.max(outputs, 1)
            for j in range(inputs.size()[0]):
                images_so_far += 1
                ax = plt.subplot(num_images//2, 2, images_so_far)
                ax.axis('off')
                ax.set_title('predicted: {}'.format(class_names[preds[j]]))
                imshow(inputs.cpu().data[j])
                if images_so_far == num_images:
                    model.train(mode=was_training)
                    return
        model.train(mode=was_training)
visualize_model(model_conv)
plt.ioff()
plt.show()

The above code will show you 6 (by default) images from validation set and show what our model think they are. The predictions should look quite correct at this point.

To load the images with PyTorch, we need to have subfolders in the dataset directory. test folder does not have any subfolders. One way to get around this is making a dummy folder as a subfolder and put all data in the subfolder and start loading it just like the train and val image_datasets. However this is not the way I chose to do. I chose the harder way which is not depending on PyTorch for loading data. I thought this might be helpful practice in understanding how it machine learning really works behind the APIs of machine learning libraries and frameworks.

Firstly, I define a function that takes Python Image Library (PIL) image and outputs a torch.Tensor applied with same transformation as validation data.

def apply_test_transforms(inp):
    out = transforms.functional.resize(inp, [224,224])
    out = transforms.functional.to_tensor(out)
    out = transforms.functional.normalize(out, [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    return out

To give the PIL image to this function we should first get the image as a PIL image.

from PIL import Image
test_data_dir = f'{data_dir}/test'
test_data_files = os.listdir(test_data_dir)
im = Image.open(f'{test_data_dir}/{test_data_files[0]}')
plt.imshow(im)

We see that im is a PIL image that we can pass to the apply_test_transforms.

im_as_tensor = apply_test_transforms(im)
print(im_as_tensor.size()) # torch.Size([3, 224, 224])
minibatch = torch.stack([im_as_tensor])
print(minibatch.size()) # torch.Size([1, 3, 224, 224])

apply_test_transforms returns us a tensor but to feed it to PyTorch for inference, we need to convert it to a batch because PyTorch models always expect batches. Using torch.stack it is easy enough to make a single tensor into a mini batch of size 1.

model_conv(minibatch) # tensor([[ 2.0083, -1.8386]], grad_fn=<ThAddmmBackward>)

Making inference is easy! just call the model with the batch as an argument. the prediction class in this case was a cat (index 0) because 2.0083 is greater than -1.8386. But wait, the kaggle submission is expecting a csv file with id and label header and we should provide a probability that an image is a dog image for the label header. Something like the following.

id,label
1,0.8
2,0.1
3,0.55
...

Applying the tensor to the Softmax function will convert the tensor to the probabilities which add up to 1.

softMax = nn.Softmax(dim = 1)
preds = softMax(model_conv(minibatch))
preds # tensor([[0.9791, 0.0209]], grad_fn=<SoftmaxBackward>)

We can see that our model is 97.9% confident that the image was a cat image. Let’s define a function where we can get the dog probability by passing model and a tensor.

def predict_dog_prob_of_single_instance(model, tensor):
    batch = torch.stack([tensor])
    softMax = nn.Softmax(dim = 1)
    preds = softMax(model(batch))
    return preds[0,1].item()

We also need a function to get PIL image from a file name.

def test_data_from_fname(fname):
    im = Image.open(f'{test_data_dir}/{fname}')
    return apply_test_transforms(im)

Another function we need is a function that extracts the numeric id part from file name.

import re
def extract_file_id(fname):
    print("Extracting id from " + fname)
    return int(re.search('\d+', fname).group())
extract_file_id("cat34432.jpg") # 34432

Notice that we are returning it as an int since we want to sort ids numerically. Sorting string ids will not give us the result we want (‘12000’ is earlier than ‘39’ when sorted as string for example).

Let’s put our model to evaluation mode and start inferencing the whole test data set (12500 images). We need to switch to evaluation mode because we want to set dropouts and batch normalizations to evaluation mode. Otherwise our predictions can be inconsistent. As I am using a Macbook without GPU this step takes a lot of time.

model_conv.eval()
id_to_dog_prob = {extract_file_id(fname): 
                  predict_dog_prob_of_single_instance(model_conv,
                                                      test_data_from_fname(fname))
                  for fname in test_data_files}

Pandas DataFrame has easy API to output to csv file so I am now going to turn id_to_dog_prob dictionary into a pandas DataFrame.

import pandas as pd
ds = pd.Series({id : label for (id, label) in zip(id_to_dog_prob.keys(), id_to_dog_prob.values())})
ds.head()df = pd.DataFrame(ds, columns = ['label']).sort_index()
df['id'] = df.index
df = df[['id', 'label']]
df.head()

Now that we see our dataframe has id and label column and everything looks as expected so let’s write to a csv file.

df.to_csv(SUBMISSION_FILE, index = False)

Examine the csv file and see if everything looks good. And then finally you can submit to dogs-vs-cats-redux-kernels-edition competition.

!kaggle competitions submit -c dogs-vs-cats-redux-kernels-edition -f submission.csv -m "Using PyTorch"

Even though I deleted a lot of images from training and validation datasets, still the loss was pretty low (0.07951). And it would put me at 248th position among 1314 entries in the Public Leaderboard if my solution had been submitted while the competition was still going on. Sadly, the board doesn’t seem to be updated anymore so I cannot see my kaggle handle in the board.

This wraps up the the series of 2 parts. Hopefully it was helpful to all the patient readers and feel free to tweak your approach to get even better results. Questions, claps and followings are welcome.

You can see the full code from my Github repo. Everything from this part is in catsanddogs.ipynb.