A Taste of PyTorch C++ frontend API

Venkata Chintapalli
May 2, 2020 · 4 min read

Introduction

One major enhancement of the recently released PyTorch 1.5 is a stable C++ frontend API parity with Python¹. C++ frontend API works well with Low Latency Systems, Highly Multi-threaded Environments, Existing C++ code bases, you can check the motivation and use cases of C++ frontend here³. I want get a taste of the PyTorch C++ frontend API by creating a small example. So I took a simple two layer neural network example from Learning PyTorch with Examples². The rest of this post details the steps to convert the two layer neural network using Python frontend API example to work with the C++ frontend API. The complete code example with steps to run the code are detailed in the Github repo⁴.

Update: Performance measure plot added 05/04/2020.

There are three main components in PyTorch, Tensors, Automatic differentiation (autograd), and nn (neural network)module.

Tensors

Python is a dynamic language and it doesn’t need to declare data types for variables. Where as C++ is a statically typed language with type safety for creating compiled and optimized code which helps in creating fast and demanding applications. So the first step to do in converting the Python code is to start defining the data types. Modern C++ also defines the auto type specifier that helps the compiler to infer the type during compile time.

Let’s start by creating a random input and output tensor data for training the neural network to map x to y, i.e., to learn the function f that defines y = f(x).

Tensors in C++ uses scope resolution operator :: instead of dot notation.

Two Layer Neural Network Model

In Python, neural networks are derived from the reusable base module torch.nn.Module. Submodules are automatically detected and registered using wrappers when they are assigned as an attribute of a module. Here is a simple Two layer network using Python interface.

Neural network modules are implemented as structs in C++. There are two ways to define the modules using either value semantics or reference semantics. First, we will see an example of creating a module with value semantics.

To register submodules in C++, register_module() method defined in the constructors initializer list makes it possible to recursively access the module tree’s parameters.

Defining C++ modules using the reference semantics with std::shared_ptr modular holder API is recommended. TORCH_MODULE(TwoLayerNet) macro defines the TwoLayerNet class. This “generated” class is effectively a wrapper over a std::shared_ptr<TwoLayerNetImpl>. This simplifies the coding instead of writing std::make_shared<TwoLayerNetImpl>(1000, 10, 64) you can writeTwoLayerNet(1000, 10, 64). Reference semantics are the recommended way of defining modules with the C++ frontend API.

Optimizer

The optimizer code looks very similar in Python and C++.

In C++:

2x+ Performance Improvement

I have plotted the training time for both Python and C++ varying the number of epochs from 500 to 5000 on Ubuntu 18.04 LTS Intel IvyBridge computer. I see C++ model training is more 2 times faster compared to the Python model.

To conclude, PyTorch maintains parity between Python and C++ frontend interface. C++ frontend follows the design and ergonomics of Python frontend in most cases. Python and C++ code for this simple neural network example is provided in the Github repo.

References

  1. https://pytorch.org/blog/pytorch-1-dot-5-released-with-new-and-updated-apis/
  2. Justin Johnson. Learning PyTorch with Examples.
  3. Using the PyTorch C++ Frontend
  4. Code examples: https://github.com/venkatacrc/PyTorchCppFrontEnd

PyTorch

An open source machine learning framework that accelerates…

Venkata Chintapalli

Written by

Machine Learning, Hardware, and High-Performance infrastructure enthusiast. Holds MS in Machine Learning GeorgiaTech and MTech in Electronics Design from IISc.

PyTorch

PyTorch

An open source machine learning framework that accelerates the path from research prototyping to production deployment

Venkata Chintapalli

Written by

Machine Learning, Hardware, and High-Performance infrastructure enthusiast. Holds MS in Machine Learning GeorgiaTech and MTech in Electronics Design from IISc.

PyTorch

PyTorch

An open source machine learning framework that accelerates the path from research prototyping to production deployment

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store