#3: RLiable to Save RL Research, NeurIPS is Here, Offline RL Gains Steam, Check out COMARL

RLiable to Save RL Research

Deep RL is notorious for the variability in training, and it turns out many benchmarks have long neglected that. In their recent paper “Deep RL at the Edge of the Statistical Precipice”, Rishabh Agarwal and researchers from Google Brain and MILA discuss the subjectivity that goes into RL comparisons when this variability is not reported. To remediate that, their paper, an Outstanding Paper awardee at NeurIPS 2021, proposes more descriptive statistics for better evaluation of RL algorithms. Also to this end, the researchers introduce RLiable, an open-source Python library.

The distribution of median human normalized scores on the Atari 100k benchmark, which contains 26 games, for five recently published algorithms, along with the reported results in the corresponding papers. Most overestimate the success of the algorithms. Source: Google AI Blog.

NeurIPS is On and RL is Making a Splash

NeurIPS is progressing at full speed and there is a lot going on when it comes to RL. Here is what not to miss:

Outstanding Paper Awardees:

• Deep Reinforcement Learning at the Edge of the Statistical Precipice
• On the Expressivity of Markov Reward

Oral Sessions:

• An Exponential Lower Bound for Linearly-Realizable MDPs with Constant Suboptimality Gap
• The best of both worlds: stochastic and adversarial episodic MDPs with unknown transition
• Efficient First-Order Contextual Bandits: Prediction, Allocation, and Triangular Discrimination
• Bellman-consistent Pessimism for Offline Reinforcement Learning
• Interesting Object, Curious Agent: Learning Task-Agnostic Exploration
• Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification
• Sequential Causal Imitation Learning with Unobserved Confounders

Workshops [registration required]:

• Deep Reinforcement Learning Workshop
• 2nd Offline Reinforcement Learning Workshop
• Political Economy of Reinforcement Learning (PERLS) Workshop
• Ecological Theory of Reinforcement Learning

Offline RL Gains Steam

RL Datasets by Google. Source: Google AI Blog

Offline RL has a lot of promise as it learns directly from datasets. One challenge, though, is to collect, record, annotate and share the data that can be then easily used in an offline RL setting. Google Brain takes on this challenge with their new Reinforcement Learning Datasets (RLDS) ecosystem, which offers a lot of convenience in every step of the data pipeline. New to offline RL? Get started here.

COMARL: Challenges and Opportunities for Multi-Agent Reinforcement Learning

If you are into multi-agent RL, you will find this seminar series highly insightful. COMARL features talks about multi-agent RL from top researchers.

Source: Learning to Communicate with Deep Multi-Agent Reinforcement Learning GitHub Page

And More…

That is not all — there have been other great blog posts and papers that came up recently. Check them out.

Blogs:

• Permutation-Invariant Neural Networks for Reinforcement Learning
• DeepMind RL method promises better co-op between AI and humans
• Hugging Face’s Snowball Fight

Papers:

• A Survey of Generalisation in Deep Reinforcement Learning
• Physics-informed Dyna-style model-based deep reinforcement learning for dynamic control
• Adversarial Reinforcement Learning for Procedural Content Generation
• MoleGuLAR: Molecule Generation Using Reinforcement Learning with Alternating Rewards

RL Opportunities in Academia

Photo by Vadim Sherbakov on Unsplash

• Vector Institute has many affiliated faculty working on RL and hiring graduate students for the upcoming academic year. Check out the work of Amir-massoud Farahmand, Angela Schoellig, Animesh Garg, Daniel M. Roy, Florian Shkurti, Jeff Clune, Joseph J. Williams, Pascal Poupart, Scott Sanner, Sheila McIlraith.
• The Cognitive Robot Autonomy and Learning (CoRAL) Lab is looking for PhD applicants for Fall 2022. Applications are through Purdue’s CS admissions portal.