The end-to-end PyTorch to TensorRT pipeline for YOLO models you must know about.
The challenge of serving deep learning models in production environments. A pipeline to optimize and serve TensorRT engines for YOLO Object Detection family of models using Triton Inference Server.
I’ve been working with Computer Vision systems for a while now and there were a few instances where I and my team nearly missed our release schedule due to wrongly approaching the “automation” aspect of the process we have to do.
For context, our solution had to run in real-time or as close to real-time as possible, and model optimizations to ONNX and TensorRT formats became trivial — one problem though, I was doing this conversion manually at first because I thought I could handle it and it’s no big deal — I was wrong.
Time and time again, I wasted hours finding out and fixing the correct version sets between CUDA, CUDNN, TensorRT, and ONNX to match the client’s hardware — a real pain.
In this article, I want to walk you through the implementation of a pipeline that handles the full optimization of PyTorch models to TensorRT targets and generates the Triton Inference Server format ready to be loaded and served in production.
