Transformers are RNNs | Fast Autoregressive Transformers with Linear Attention

Data2Vec | A General Framework for Self-supervised Learning in Speech, Vision, and Language

ViT-Lite | Escaping the Big Data Paradigm with Compact Transformers

StyleGAN | A Style-Based Generator Architecture for Generative Adversarial Networks

Problem. SyleGAN is about understanding (and controlling) the image synthesis process in the generator of convolutional GANs. More specifically, this paper aims to improve the…

Sparrow | Improving alignment of dialogue agents via targeted human judgments

EfficientNet | Rethinking Model Scaling for Convolutional Neural Networks

AlphaTensor | Discovering fast matrix multiplication algorithms with reinforcement learning

GMA | Learning to Estimate Hidden Motions with Global Motion Aggregation

RAFT | Recurrent All-Pairs Field Transform

Princeton is awarded the ECCV Best Paper Award 2020. Zachary Teed and Jia Deng developed a novel architecture that efficiently estimates a dense optical flow field, beating current state-of-the-art across multiple benchmarks.

Scaling Vision Transformers

Scale drives transformer’s performance. In this paper, Google Brain team explores the scaling properties of…