Papers with Code 2021 : A Year in Review

Papers with Code indexes various machine learning artifacts — papers, code, results — to facilitate discovery and comparison. Using this data we can get a sense of what the ML community found useful and interesting this year. Below we summarize the top trending papers, libraries and datasets for 2021 on Papers with Code.

Top Trending Papers of 2021

ADOP was the most talked about ML arXiv paper on social media.

We looked at several metrics, including Papers with Code page views, GitHub stars and social reactions. We found social media reactions produced the most interesting and diverse list of papers and topics, so we share this list below.

ADOP was the most talked about paper on social media — a strong reaction that hit the zeitgeist with VR and the metaverse in Q4. More generally, the list highlights broad progress across machine learning — from new self-supervised learning approaches, to new multi-modal architectures, new approaches to save memory when training large models, and more.

1. ADOP: Approximate Differentiable One-Pixel Point Rendering — Rückert et al — https://paperswithcode.com/paper/adop-approximate-differentiable-one-pixel
2. The Bayesian Learning Rule —Khan et al https://paperswithcode.com/paper/the-bayesian-learning-rule
3. Program Synthesis with Large Language Models — Austin et al https://paperswithcode.com/paper/program-synthesis-with-large-language-models
4. Masked Autoencoders Are Scalable Vision Learners — He et al https://paperswithcode.com/paper/masked-autoencoders-are-scalable-vision
5. 8-bit Optimizers via Block-wise Quantization — Dettmers et al https://paperswithcode.com/paper/8-bit-optimizers-via-block-wise-quantization
6. Revisiting ResNets: Improved Training and Scaling Strategies — Bello et al https://paperswithcode.com/paper/revisiting-resnets-improved-training-and
7. Image Super-Resolution via Iterative Refinement — Saharia et al https://paperswithcode.com/paper/image-super-resolution-via-iterative
8. Perceiver IO: A General Architecture for Structured Inputs & Outputs — Jaegle et al https://paperswithcode.com/paper/perceiver-io-a-general-architecture-for
9. Do Vision Transformers See Like Convolutional Neural Networks? — Raghu et al https://paperswithcode.com/paper/do-vision-transformers-see-like-convolutional
10. Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions — Niepert et al https://paperswithcode.com/paper/implicit-mle-backpropagating-through-discrete

Did you know? you can sort papers on Papers with Code by social reactions.

Top Trending Libraries of 2021

PyTorch Image Models was the most viewed library on Papers with Code for 2021.

We looked at outbound link traffic: i.e. what were the most visited libraries from Papers with Code?

This year TIMM (PyTorch Image Models) was the top trending library on Papers with Code. This reflects the continued growth of the library, and its number of implementations, and the increased usage of the the library as a framework to use and extend the latest vision architectures.

1. PyTorch Image Models — Ross Wightman — https://github.com/rwightman/pytorch-image-models

2. Transformers — Hugging Face — https://github.com/huggingface/transformers

3. PyTorch-GAN — Erik Linder-Norén — https://github.com/eriklindernoren/PyTorch-GAN

4. MMDetection — OpenMMLab — https://github.com/open-mmlab/mmdetection

5. Darknet — AlexeyAB — https://github.com/AlexeyAB/darknet

6. Vision Transformer PyTorch — lucidrains — https://github.com/lucidrains/vit-pytorch

7. InsightFace — DeepInsight — https://github.com/deepinsight/insightface

8. Detectron2 — Meta AI — https://github.com/facebookresearch/detectron2

9. PaddleOCR — PaddlePaddle — https://github.com/PaddlePaddle/PaddleOCR

10. FairSeq — Meta AI — https://github.com/pytorch/fairseq

Top Trending Datasets of 2021

MATH Dataset was the most viewed new dataset on Papers with Code for 2021.

One of our flagship feature launches this year was a dataset index, which allows us to track dataset usage and evolution in the community. We looked at new datasets with the most views in 2021 on Papers with Code.

MATH was the most viewed new dataset on Papers with Code. This reflects a growth in research looking at Transformers for mathematical problem solving, and a gap in performance between Transformers on these kinds of tasks compared to other tasks. MATH is our benchmark to watch for 2022.

1. MATH — Hendrycks et al https://paperswithcode.com/dataset/math
2. UAV-Human — Li et al https://paperswithcode.com/dataset/uav-human
3. UPFD (User Preference-aware Fake News Detection) — Dou et al https://paperswithcode.com/dataset/upfd
4. OGB-LSC (OGB Large-Scale Challenge) — Hu et al https://paperswithcode.com/dataset/ogb-lsc
5. CodeXGLUE —Lu et al https://paperswithcode.com/dataset/codexglue
6. AGORA — Patel et al https://paperswithcode.com/dataset/agora
7. BEIR (Benchmarking IR) — Thakur et al https://paperswithcode.com/dataset/beir
8. WikiGraphs — Wang et al https://paperswithcode.com/dataset/wikigraphs
9. Few-NERD — Ding et al https://paperswithcode.com/dataset/few-nerd
10. PASS (Pictures without humAns for Self-Supervision) —Asano et al https://paperswithcode.com/dataset/pass

Congratulations to all the authors and developers for making an impact in the ML community this year!

Thanks also to all our users who’ve used and trusted us in 2021! We will be building on the improvements we made to the platform this year with some exciting new features and tools in 2022. To stay connected with our work, and get updates on ML progress, you can subscribe to our newsletter.

We hope you have a happy new year and a healthy & productive 2022!

The Papers with Code team