#2: Teaching Robots Hand Manipulation, Chip Floorplanning Explained, RL Frustrates Humans in Teamplay, Break into RL
Teaching Robots Dexterous Hand Manipulation
Dexterous in-hand manipulation is one of the grand challenges in robotics with many real-world applications. You may remember OpenAI’s earlier success with solving Rubik’s cube with a robot hand. A similarly impressive work came out of MIT, by Tao Chen and colleagues, in which the researchers train a model-free RL agent that can reorient over 2000 types of objects with strong zero-shot generalization. The work involved a teacher-student architecture to distill the policies so as to use only sensory information. The authors performed the experiments in simulation, but they provide evidence for real-world applicability. Check out the exciting demonstrations on YouTube.
Chip Floorplanning with Deep Reinforcement Learning — Explained
A recent video by the TensorFlow group explains how Google achieved a breakthrough in computer chip design using deep RL. Chip floorplanning is considered one of the toughest design problems and normally requires human experts in the loop on top of many special optimization algorithms. DRL’s success in beating the benchmarks was earlier reported in a blog post and was recently published in Nature. The TensorFlow video gives interesting insights into how the researchers used supervised learning for the value network for better reward estimation and employed action masking to facilitate training.
RL Agents Frustrate Humans in Teamplay
It turns out that teaming up with RL agents can be frustrating, especially when they are not budged by the in-game tips you give to them. A TechTalks post explains a research paper that came out of MIT and the U.S. Air Force AI Accelerator that investigates human-AI collaboration via RL and Hanabi. The work discussed the need for alternative objectives, as opposed to the ones focused on winning, we might want to train RL agents and AI models with if we want to enjoy them being around us.
Resources to Break into Reinforcement Learning
If you have been wondering how you can get started with or get more involved in RL, here are some recently compiled lists of RL resources for you.
- A Neptune blog lists RL tutorials, courses, and projects.
- A Coder One post features active and upcoming RL competitions.
- An AIM article compiles the most important developments in RL for 2021.
Recent Papers & Talks
There is a lot going on in RL research. Here is some recent work that you might find interesting.
- There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning
- Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning
- Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention
- Model-Free Risk-Sensitive Reinforcement Learning
- Procedural Generalization by Planning with Self-Supervised World Models
- PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems
- Goal-directed graph construction using reinforcement learning
- Instance-based defense against adversarial attacks in Deep Reinforcement Learning
RL Opportunities in Academia
If you are interested in RL positions in academia, here are a couple:
- Dr. Sarath Chandar of Polytechnique Montréal has multiple master’s and Ph.D. positions for Fall 2022.
- Dr. Sharan Vaswani of Simon Fraser University is looking for master's and Ph.D. students for Fall 2021.
If you are interested in featuring your RL positions in academia, reach out to us.
We hope you have enjoyed this issue of the Reinforcement Learning Newsletter. If you have, consider subscribing, following us on Twitter, and sharing it with your network. If you are interested in contributing stories, reach out to us at editor@rlagent.pub.
