Google Summer of Code 2024: OpenVINO™ Adapters - PyTorch Adapter

Published in

OpenVINO-toolkit

7 min readAug 26, 2024

Hello everyone, I’m Luca. I decided to write this article to share my experience as a student in the Google Summer of Code (GSoC) 2024 program. My project (OpenVINO Adapters — Pytorch Adapter) involved providing a POC for creating an adapter for PyTorch using the OpenVINO API. Before diving into the project details, I’d like to share how I prepared to be selected by the OpenVINO team. OpenVINO™ toolkit is an open-source toolkit developed by Intel® for optimizing and deploying AI inference. If you’re wondering whether GSoC is difficult or requires extensive experience, I’ll share my journey to give you a clear idea.

Who am I?

I’m a 20-year-old guy from Rome, studying Computer Engineering. I’m in my second year of studies, with exams on track and a solid GPA. Besides studying, I enjoy working out and maintaining a balanced life between university, friends, family, and personal projects. Since I focused solely on exams during the summer of my first year, I promised myself that the next summer I’d do something more — hence why I decided to attempt GSoC.

How did I prepare?

Before even applying to GSoC, I immersed myself in the open-source world. This involved the usual steps: creating a GitHub account, watching videos that explain Git, GitHub, and how to contribute to open-source projects. Once I had laid the groundwork, I just needed to decide where to invest my time. I did this around December 2023, during the holidays. To choose the right repository for me, I did some brainstorming. I asked myself:

What do I want to learn?
Should I use languages I already know, or should I explore new ones?

Since I’m only in my second year, my skills in Machine Learning (ML) aren’t very advanced — in fact, they’re almost zero. But that didn’t bother me. I kept hearing about ML, Neural Networks, and AI, so I knew I needed to learn more. That’s how I answered the first question. As for the second question, it’s a delicate concept. I believe that with all the programming languages out there, it’s impossible to be proficient in all of them. Personally, I’m skilled in Java, C, and Python. That was a solid foundation, and learning C++ after knowing those three languages was relatively easy (I mention C++ because OpenVINO is mostly written in C++).

So, my research phase began. After exploring interesting repositories like PyTorch, TensorFlow, and others in the ML field, I stumbled upon OpenVINO. It had an excellent Repo, a Discord channel, and a strong community — everything I needed to get started with them.

The main challenges I faced at the beginning were:

Building the project (as a beginner, this wasn’t as straightforward as it might seem).
Finding which files to modify to fix bugs.
Finding time to dedicate to the project.

Looking back now, these things seem trivial, and I laugh at them (especially the first one), but back in January, they were not easy at all.

Starting to contribute to OpenVINO was very straightforward. They have a section in their project dedicated to open bugs labeled “Good First Issues” (GFI), which are incredibly helpful for beginners like me. All I had to do was focus on a GFI and make my first PR.

GSoC

After about a month of being active in the repo, making frequent PRs, and so on, I noticed that the OpenVINO team applied to be a mentorship organization for GSoC 2024. That’s when I realized this was my chance to do something meaningful this summer. I began looking up all the important dates, such as when to submit the application and when the participating organizations would be announced. Meanwhile, I continued contributing to the repository daily, ensuring I maintained the flexibility I had gained and kept getting noticed.

Once OpenVINO was selected (I believe it was February), I immediately checked the proposed projects from the mentors and quickly fell in love with the one I’ll discuss later. I reached out to my future mentor to clarify key aspects of the project and ask for a review of my application before submitting it.

Once I submitted the application (on March 18th), all I could do was wait for the result. Instead of sitting idle, I continued contributing to the repository, learning, and improving. I still remember when I received the email saying I had been selected — it was official: I was a GSoC student!

What do I recommend?

Be proactive: Start contributing to open-source projects as early as possible, especially to a repository that has already been chosen on the GSoC page if your intention is to participate.
Track Deadlines: Keep track of all the important dates to have enough time to write a strong application.
Communicate, communicate, communicate. You should reach out to your mentor as soon as possible about the project, whether it’s your own idea or one proposed by the organization. Talking to them will only increase your chances of being selected.

About my Project

Now, let’s talk a bit about my project: creating an adapter for PyTorch using the OpenVINO™ API. Obviously, recreating all of PyTorch would be impossible, so my goal was to provide a POC and demonstrate that the performance on x86 processors would be better using my adapter compared to PyTorch. From the user’s perspective, the difference would be switching from:

import torch

to:

import pytorch_adapter as torch

There wouldn’t be any changes to the rest of the code. Adapter works with just a single line change to the user’s script.

What was my focus?

My mentor and I used this script as an example (link: ResNet on PyTorch Hub), which we used as a reference script for benchmarking the PoC. We divided it into two parts:

PreProcessing
Inference

For the PreProcessing part, all the ResNet transformations, such as Resize, CenterCrop, Pad, (…), are performed by OpenVINO. In order to do that, the Torch API has been “overridden” underneath with calls to the OpenVINO PrePostProcessing module, which performs the transformations using Intel’s code. The implementation has been tested with numerical comparisons of OpenVINO results against Torch results (the relative tolerance has been kept under from 1e-02 to 1e-05 for many transformations)

A noteworthy technical difficulty was the difference in how OpenVINO and Torchvision handle preprocessing — in OpenVINO it's embedded into the model graph, whereas in Torch it’s a completely separate object. In order not to break Torch API compatibility it resulted in having to take an interesting approach, where a model with preprocessing is actually kept as two separate OpenVINO models. During inference, they are being run sequentially.

After completing this first part, we ran benchmarks to compare the transformations with PyTorch. The results for transformations like Normalize, Pad, and ConvertImageDtype showed excellent performance. However, Resize and CenterCrop appeared to be slower using the adapter. This performance gap is due to the fact that, when using the PrePostProcessing module, I don’t have full control, so the issue lies deeper within the OpenVINO core.

The model preparation and model inference were the final part of the working PoC. ResNet model downloaded using torch.hub.load() has first to be converted to an Intermediate Representation - a model which can be compiled by OpenVINO. In order to do that I used OpenVINO PyTorch Frontend, which serves as a translation layer between different standards.

OpenVINO brings performance benefits for models executed on Intel processors. Running benchmarks on the entire pipeline (PreProcessing and ResNet execution) resulted in an average inference time reduction of 64.3% for the input of shape (8, 3, 1000, 1000):

Below, you can see an example usage of my adapter:

import torch_adapter as torch
from torch_adapter import transforms
import urllib
from PIL import Image

# You can also choose to create random data
# import numpy
# input_image = numpy.random.randint(255, size=(1, 3, 270, 270), dtype=numpy.float32)

model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet18', pretrained=True)
# or any of these variants
# model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet34', pretrained=True)
# model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet50', pretrained=True)
# model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet101', pretrained=True)
# model = torch.hub.load('pytorch/vision:v0.10.0', 'resnet152', pretrained=True)
model.eval()

# Download an example image from the pytorch website
url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
try: urllib.URLopener().retrieve(url, filename)
except: urllib.request.urlretrieve(url, filename)

input_image = Image.open(filename)
preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)
output = model(input_tensor)
# Tensor of shape 1000
print(output[0])

Conclusion

Being part of GSoC has been by far the most important experience for me. Talking with my mentor daily and learning from him as if I were his student — I really can’t express how much this has meant to me. I want to sincerely thank Jiwaszki, my first mentor, who supported me before and during the development of the project, and p-wysocki, who guided me through the second half of the project and deepened my passion for ML even more than I already had.

I also want to sincerely thank the entire OpenVINO team for giving me this opportunity. It has truly been an incredible experience that has opened my mind in so many ways.

Now, I’m speaking directly to you, the reader of this article and aspiring GSoC participant. If you want to join, don’t sit there thinking: “I’m not good enough,” “What if I fail?”, “I don’t have time.” If you want to do it, DO IT! I can assure you that you won’t regret it. All your worries will disappear once you take the leap, otherwise, you’ll live with the regret of not trying.

Thank you for your attention, have a great day!

Notices & Disclaimers

Performance varies by use, configuration and other factors. Learn more on the Performance Index site.

Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure.

Your costs and results may vary.

Intel technologies may require enabled hardware, software or service activation.