Facebook Open-Sources PySlowFast Codebase for Video Understanding

Synced
Synced
Jan 10 · 3 min read

Facebook AI Research (FAIR) has been contributing heavily to video understanding research in recent years. At October’s ICCV 2019 the team unveiled a Python-based codebase, PySlowFast. FAIR as now open-sourced PySlowFast, along with a pretrained model library and a pledge to continue adding cutting-edge resources to the project.

The name “PySlowFast” derives from a novel duality — the model has both a slow pathway that operates at a low frame rate to capture spatial semantics, and a lightweight, fast pathway that operates at a high frame rate, captures motion at fine temporal resolution, and can learn useful temporal information for video recognition.

The introduction of PySlowFast addresses a couple of needs for ML researchers. First, there was no concise, efficient and easy-to-modify video understanding codebase in the open source community. Second, recreating today’s state-of-the-art deep learning models can be a headache, as such models often require tens of GFlops, several days of training, and repeated experimental tuning to make every detail correct. This can be prohibitively time- and resource-consuming for many researchers.

PySlowFast will enable researchers to easily reproduce video classification and action detection algorithms, whether they are basic or cutting-edge. FAIR has also open-sourced a number of pretrained models to save researchers the trouble of repeatedly training sessions.

PySlowFast performance on video classification database Kinetics 400

PySlowFast also includes a dedicated interface to support multi-modal video understanding, video self-supervised learning and other tasks through simple editing. FAIR says PySlowFast will be actively updated with cutting-edge algorithms in real time to ensure it remains a current and reliable benchmark in the field of video understanding.

After completing the installation, users can test performance on different video databases by downloading the pretrained model and corresponding configuration file provided by MODEL_ZOO and running the following code:

python tools/run_net.py \
--cfg configs/Kinetics/C2D_8x8_R50.yaml \
DATA.PATH_TO_DATA_DIR path_to_your_dataset \
NUM_GPUS 2 \

The PySlowFast Codebase is on GitHub. The related papers are on arXiv: SlowFast Networks for Video Recognition and Non-local Neural Networks.


Author: Reina Qi Wan | Editor: Michael Sarazen


We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.


Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!


2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.

Synced

Written by

Synced

AI Technology & Industry Review — syncedreview.com | Newsletter: goo.gl/Q4cP3B | Become Synced Insight Partner: goo.gl/ucXZDw | Twitter: @Synced_Global

SyncedReview

We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade