Object Detection using a ssd_mobilenet_coco model with OpenCV 3.3 & TensorFlow 1.4 in C++ and XCode

see https://github.com/moorage/OpenCVTensorflowExample

In our robotics project, we’ve been using deep learning models alongside OpenCV. While we’re pleased that OpenCV 3.3 comes with the Deep Neural Network module for research, we needed a deeper integration with Tensorflow in C++ for production.

Of course, as of Tensorflow 1.4.1, embedding Tensorflow’s sdk in your own C++ project — particularly on macOS — is not straightforward. Luckily, with a lot of caffeine and Googling, we got to a working solution. As much for our own recollection, we’re providing it here also to help others.

Here’s what the example project outputs:

Image courtesy @happyoliver

It works with OpenCV 3.3.1 and Tensorflow 1.4.1 on macOS High Sierra.

Please be sure to read the entire README, as you really do have to follow all the minutia of the instructions.

Why wasn’t this as straightforward as it should have been? There are a few fun reasons.

Here’s a quick summary, for those who might come across similar issues:

• The code requires the use of the C++ SDK, in tensorflow_framework.so
• However, initializing Tensorflow at runtime requires libtensorflow.so.
• The Tensorflow session.h, has a great API for programmers, but also a mess of un-distributed required header files.
• The required header files from session.h also include external library header files, some of which don’t compile with XCode’s LLVM without upgrading them.
• OpenCV uses BGR whereas Tensorflow uses RGB
• You ideally OpenCV and Tensorflow to share memory when possible, to avoid too many copies
• I never got a Tensorflow network-graph browser running on macOS, so knowing the input and output tensor names of the COCO graphs required research

We really hope you find this useful, and I welcome comments & pull requests on the example project!