Tencent ML Team Trains ImageNet In Record Four Minutes
Since 2010, the annual ImageNet Large-Scale Visual Recognition Challenge has been the most widely recognized benchmark for testing image recognition algorithms. Tencent Machine Learning picks up the challenge with its new paper Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes.
Due to continuous efforts from researchers around the world, ImageNet’s Top-5 error rate dropped from around 28 percent in 2010 to 15.4 percent in 2012 (AlexNet), and finally to approximately 3 percent in 2017 — eclipsing the human rate of about 5 percent.
Through this iterative process, two neural network models were developed which proved very effective for image recognition tasks: AlexNet and ResNet-50. Although the trained model can usually achieve high accuracy, the training process is very time-consuming. For example, it takes 14 days to train ResNet-50 on ImageNet using an NVIDIA M40 GPU, a process that would take decades using only a single-core CPU with a serial program. Accordingly, how to train AlexNet and ResNet-50 on ImageNet in a shorter period of time has become a focal point for researchers.
The team from Tencent Machine Learning (腾讯机智, Jizhi) trained ResNet-50 in 6.6 minutes and AlexNet in just 4 minutes on the ImageNet dataset. The previous record time for training ResNet-50 was 15 minutes (by Preferred Network’s Chainer team); while training AlexNet had been achieved in 11 minutes by a team of UC Berkeley scholars.
The Tencent Jizhi team says improving training speed on ImageNet is just a small part of their work in AI development, and plan to leverage their accelerated training capability in other AI businesses and services, such as game AI.
On July 31st, Tencent’s AI Go program “Jueyi” (绝艺, FineArt) swept its opponent “Golaxy”(星阵) 6–0 to win the World AI Go Competition, which featured 11 AI Go teams from China, Japan, South Korea, Belgium, and the US.
The Tencent Jizhi Machine Learning Platform was established by the Tencent Engineering Group’s Architecture Platform Department and Operations Management Team, and collaborates closely with Professor Xiaowen Chu’s team from the Computer Science Department at Hong Kong Baptist University.
The paper related to this study is now on Arxiv.
Author: Mos Zhang | Editor: Michael Sarazen
Follow us on Twitter @Synced_Global for more AI updates!
Subscribe to Synced Global AI Weekly to get insightful tech news, reviews and analysis! Click here !
