Uber ATG Open-Sources Neuropod DL Inference Engine

Rideshare giant Uber continues to explore deep learning’s potential to provide safer and more reliable self-driving technologies. This week, the Uber Advanced Technologies Group (ATG) released Neuropod, an open-source library that provides a uniform interface for running deep learning (DL) models from multiple frameworks in C++ and Python.

“As our self-driving software evolves, we are always looking for new ways to improve our models and sometimes that means experimenting with different deep learning frameworks,” writes Uber ATG Senior Autonomy Engineer Vivek Panyam in a company blog.

Over the past few years, Uber ATG researchers have used DL frameworks such as Caffe2, TensorFlow, PyTorch, and TorchScript, and they are also looking to try out newly open-sourced frameworks such as JAX, Trax, Flax, Haiku, and RLax.

Although experimenting with new frameworks is fairly simple, actually adding production support for new DL frameworks across all systems and processes is very time-consuming, and Uber developed Neuropod to streamline that process. Neuropod functions as an abstraction layer on top of existing deep learning frameworks, providing a uniform interface for running different DL models.

Every Neuropod model implements a problem definition — a formal description of a problem for models to solve. As a result, any models that solve the same problem are interchangeable even if they use different frameworks. Existing models can be wrapped in a Neuropod package, which contains the original model along with metadata, test data, and custom ops if any.

Since its internal release in early 2019, hundred of Neuropod models have been deployed across Uber ATG, Uber AI, and the core Uber business — including models for demand forecasting, estimated time of arrival (ETA) prediction for rides, menu transcription for Uber Eats, and object detection models for self-driving vehicles.

Neuropod makes it easy for researchers to build models in a framework of their choosing while also simplifying product-ionization of these models, says the company. Neuropod currently supports TensorFlow, PyTorch, TorchScript, and Keras.

Journalist: Yuan Yuan | Editor: Micahel Sarazen

We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.

Thinking of contributing to Synced Review? Synced’s new column Share My Research welcomes scholars to share their own research breakthroughs with global AI enthusiasts.

Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!

2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.