Iclr 2021

Small synthetic data set can be enough for training a model. — The topic of data-efficient learning an important topic in Data Science and is an active area of research. Training large models on big data could take a lot of time and resources, so the question is can we replace a large data set with a smaller one, that will nevertheless…

Learned optimizers are algorithms that can be trained to solve optimization problems. Although learned optimizers can outperform baseline optimizers in restricted settings, the ML research community understands remarkably little about their inner workings or why they work as well as they do. …

Transformer-like neural network architectures have played a game-changing role in natural language processing (NLP), and are now increasingly being applied in the field of Computer Vision (CV) and related research. …

One of the most popular approaches to unsupervised learning of complicated distributions, Variational Autoencoders (VAEs) consist of an encoder and a decoder built on top of standard function approximators (neural networks). …

A new research paper, An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale, has the machine learning community both excited and curious. With Transformer architectures now being extended to the computer vision (CV) field, the paper suggests the direct application of Transformers to image recognition can outperform…