3D Athlete Tracking with AI
Authors: Nelson Leung (3DAT), Maajid N Khan, Sesh R Seshagiri, Yamini Nimmagadda, Akhila Vidiyala, Sachin Rastogi (OpenVINO Team)
The 3D athlete tracking team uses AI to recognize, track, and analyze over 1,000 biomechanics data points from a standard video. The team allows customers to create rich and powerful biomechanics-driven products, such as web and mobile applications with detailed performance data and three-dimensional visualizations.
In this blog, we will discuss how the 3DAT team collaborated with the OpenVINO™ team at Intel® to increase the model performance using OpenVINO™ toolkit and made significant savings on cloud costs.
Background
The global artificial intelligence in the sports market was valued at $1.4 billion in 2020 and is projected to reach $19.2 billion by 2030, growing at a CAGR of 30.3% from 2021 to 2030. Artificial intelligence in sports is emerging all over the industry, covering training and performance analysis, maintaining player health, and fitness, scouting and recruitment, and broadcasting and advertising.
Video data is being collected and analyzed to help athletes perform better. For example, by tracking players’ movements on the field, coaches can see where they need to adjust to improve their overall performance that in turn impacts the team.
Computer Vision based video analysis for coaches and athletes is one of the most common applications in sports for automated athletes’ movement analysis. It involves using machine learning software and pattern recognition to automatically extract data from video footage of athletes in action. This data can then be used by sports teams, coaches, and athletes to quantify, assess, and improve player and team performance.
Computer vision models that track a player’s movements can identify potential risk factors for injuries. For example, if an athlete consistently lands awkwardly during a run, this could be a sign that they are at risk of an ankle injury. By identifying signs of risk factors early, coaches and athletes can take steps to prevent injuries.
Intel® 3D Athlete Tracking (Intel® 3DAT) technology uses computer vision-based models to extract and analyze athlete movements. The key points are used to calculate biometrics relevant to athletes such as velocity, acceleration, and posture. The 3DAT SDK is a SaaS platform that enables its partners to leverage the powerful 2D/3D and biometrics capabilities of 3DAT AI in building consumer-facing products. They can seamlessly integrate, deploy, and scale their application to meet their consumer needs.
The Challenge
Bringing personalized coaching to athletes at an affordable cost.
Sports coaches assist athletes in developing to their full potential. They are responsible for training athletes in a sport by analyzing their performances, instructing in relevant skills, and providing encouragement. Coaches teach amateur or professional athletes the skills they need to succeed at their sport. Athletes depend on their coaches for advice and guidance on each step of the way to do well.
Sports injuries are common, and the athletes returning to play after injury are especially at high risk. As per a few studies, the rate of reinjury can be reduced using a coach-controlled rehabilitation program.
Most amateur athletes, especially in developing countries, cannot access good coaches.
The Solution
Intel-3DAT has developed an AI-based fully automated platform to provide an equal opportunity to all athletes so that athletes can perform to the best of their abilities. This platform supports the requirements of the athletes, scouts, and coaches.
Training
Primarily, we are leveraging AWS EC2 DL1 instances to train 2D and 3D models for 3DAT pipelines. We are consistently seeing cost savings compared to existing GPU-based instances across model types, enabling us to achieve much better Time-to-Market for existing models or training much larger and more complex models.
Amazon DL1 instances with Gaudi accelerators offer the best price-performance savings compared to other GPU offerings in the market. The models were trained using the PyTorch framework. We have trained the following models for the 2D and 3D pipelines.
- Person + Object Detector
- 2D Pose Estimation
- 3D Pose Estimation
Inference and Optimization
Intel® OpenVINO™ is an inference solution that optimizes and accelerates the computation of AI workloads on Intel® hardware. The trained 2D and 3D PyTorch models were converted to ONNX (Open Neural Network Exchange) model representation format and then further optimized to the OpenVINO format or Intermediate Representation (IR) of OpenVINO™ using the model optimizer tool from OpenVINO™ toolkit.
The FP32-optimized IR models outperformed using OpenVINO™ runtime in terms of throughput compared to other Deep Learning framework runtimes on the same Intel® hardware.
As a next step, the FP32 IR model was further optimized and converted to lower 8-bit precision with post-training quantization using the default quantization algorithm from the Post Training Optimization Tool (POT) from OpenVINO™ toolkit. This inherently leads to a jump in the model’s processing speed and throughput. The user also gets a higher FPS when dealing with video streams with a negligible loss in accuracy.
The INT8 IR models performed extremely well for inference on Intel® CPU (Central Processing Unit) 3rd Generation Intel Xeon.
Integration with 3DAT
This final optimized solution with OpenVINO™ runtime using the INT8 IR model was experimented with various OpenVINO™ runtime configurations to produce the best combination for fast inferencing.
This solution was then integrated with 3DAT’s 2D and 3D pipelines by modifying the inference definitions to use OpenVINO™ APIs for inferencing rather than the existing PyTorch APIs. The pipelines were experimented with different batch sizes to produce the best combination using the new OpenVINO™ solution. The OpenVINO™ runtime not only offered faster inferencing but was also able to support advanced requirements from 3DAT like the support for dynamic input shapes on the Xeon CPUs.
The integration was smooth, and the new pipelines were tested on 3DAT’s pre-production environment, and the results were quite amazing.
This transition helped 3DAT with great cost savings on cloud inferencing, therefore making OpenVINO™ inferencing the most viable solution for 3DAT products.
“The OpenVINO team has been incredible to work with as we grow the Intel 3DAT business. Their solutions have allowed us to reduce the cost of running our models in the cloud while maintaining performance and providing a better product to our customers.”
Testimonial by Jonathan Lee, Senior Director of Sports Performance Technology, Intel
Result
Overall, a better price performance on Intel® Xeon Scalable processors (code-named Ice Lake) than other popular GPU vendors.
Conclusion
This work has enabled coaches to analyze athletes’ 3D and 2D pose estimation over time to measure biomechanics metrics, such as velocity, and monitor the athletes’ performance using quantitative and qualitative methods.
This blog demonstrated how OpenVINO™ was leveraged to provide the same functionality with faster inferencing on Intel® CPU’s leading to great cost savings on the cloud and making the product more cost-viable to continue serving many athletes, coaches, and helping to land more potential customers to the product.
Notices and Disclaimer:
All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps.
1. Results based on 3DAT’s internal testing data Performance varies by use, configuration, and other factors. Learn more at https://intel.com/PerformanceIndex
Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.
No product or component can be absolutely secure.
Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.
Intel does not control or audit third-party data. You should review this content, consult other sources, and confirm whether referenced data are accurate.
2022 Copyright © Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Distribution of OpenVINO™ toolkit, Core, Movidius, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
* Other names and brands may be claimed as the property of others.
