HowTo Build Tensorflow Apps for RICOH THETA
Build Tensorflow deep learning apps that run directly inside the RICOH THETA camera using the internal camera OS
Makota Shohara recently published a tensorflow-theta code repository on GitHub for a workshop he ran in Tokyo. The code runs inside the RICOH THETA and provides features such as object identification.
Note: if you want to run TensorFlow in the THETA first prior to working on the code, read Running Tensorflow on RICOH THETA V
Four Demos
- Classify
- Detect
- Speech
- Stylize
Additional Features
Speech Recognition
Stylize
Classify
Detect
Android Examples
As the RICOH THETA V runs Android internally, the demo uses the android examples from the GitHub repository in /tensorflow/examples/android/.
Machine Learning Flow
Using a machine learning model, the THETA V can infer the correct answer with Tensorflow.
RICOH THETA Tensorflow Plug-in Architecture
Running TF Detect Overview of Steps
- Modify code to work with the RICOH THETA Camera API
- Import RICOH THETA pluginlibrary
- Detect objects and shoot picture. — Using the Plugin SDK. — Use camera API for inference -> stop inference -> Shoot with Web API -> resume inference with Camera API
- Fix PluginLibrary — Add functionality to upload to Google Photos
Follow the 4 Steps with GitHub Code Examples
Each of the steps has a corresponding GitHub branch.
Step 1 — Modify code to work with RICOH THETA Camera API
Step 2 — Import RICOH THETA pluginlibrary
Step 3 — Detect objects and shoot picture
Step 4 — Fix PluginLibrary
You can head straight to the code and work with the Tensorflow examples and modifications. Future articles will cover development tips for each of the steps.
Improving Equirectangular Detection
The current demo isn’t optimized for equirectangular images. Community member Fabien Benetou is looking at this problem and passed along a link to this paper.
If you have anything to add, please pass on the information.
Next Step
Step 1 — How to Set Up the THETA V for Tensorflow
