O-Nect: Open Source Interface for Motion Capture using RGB Camera
Introducing O-Nect : An open source Motion Capture Interface using tensorflow for game-motion,animation designers.
The use of neural networks for evaluation of Human Pose Estimation has been around for a long time in the field of entertainment , gaming , modeling using Motion Capture systems.
However , these systems require expensive hardware installations.As a result , not everyone is able to get access to this technology.
In this project , the aim is to eliminate dedicated hardware systems and attain efficient performance using minimal tools. As the project is Open Source, developers across the world can utilize the features from our project to create their own projects.
Our approach is capable of obtaining a 2D pose of a human from just a RGB camera. Estimating 2D pose from a single RGB camera is a challenging
problem . We provide an overview of our method to tackle this challenging problem.
Our network consists of two primary components. The first is a convolutional neural network (CNN) to regress 2D and 3D joint positions .
To analyze the runtime performance of our method, we used a blender(game engine) generated keypoint humanoid .
The final product is implemented on a laptop with a 4gb NVIDIA GeForce GTX-1050Ti GPU and a 720p Laptop camera. We used person detection and single-person CPM to generate a json file of keypoints which was later sent through a socket pipeline , retrieved in the game engine and utilized accordingly.
We were able to achieve around 15 fps on our mentioned specs and are working on improving the performance with the help of the open source community
We believe this approach can be used by the community to build many more projects such as game development, motion based exercise modules,controller free VR, entertainment ,etc. With the help of resources across the internet,we aim to solve queries and improve performance regularly.
