#6: RL Goes Nuclear, RL Assists Anesthesiologists, MuZero Powers YouTube, Outracing Champion Gran Turismo Drivers
Deep RL Model Controls Nuclear Fusion Plasma
As yet another breakthrough in RL; DeepMind announces that they have developed a deep RL model to control the nuclear fusion plasma in a tokamak in partnership with the Swiss Plasma Center of EPFL.
Fusion power has major advantages over traditional nuclear power based on fission, such as reduced radioactivity, minimal nuclear waste, and increased safety. Yet, the technology has not been practical due to multiple challenges, one of which is the instability in the control of the process that produces Sun-like heat. The success of DeepMind’s RL model, therefore, made a big splash across the media, academia, and industry. The study is published in Nature and covered in many outlets:
- CNBC: DeepMind scientists say they trained an A.I. to control a nuclear fusion reactor
- Wired: DeepMind Has Trained an AI to Control Nuclear Fusion
- PHYS ORG: EPFL and DeepMind use AI to control plasmas for nuclear fusion
- MIT Technology Review: DeepMind’s AI can control superheated plasma inside a fusion reactor
RL Model Controls Doses of Propofol, Assisting Anesthesiologists
Researchers from MIT and Massachusets General Hospital published a study in which they use deep RL to control propofol dosing during general anesthesia. The model is based on an actor-critic architecture and it significantly outperforms a PID controller baseline. Healthcare is expected to benefit from RL research in a significant way, and this news is a step in that direction. You can read more about the story on MIT News.
MuZero Powers YouTube to Optimize Video Compression
The MuZero model, a descendant of the famous AlphaGo that beat the world Go champion, is used to optimize the compression of YouTube videos, DeepMind announces. As YouTube accounts for a major portion of the world’s internet traffic (35% in mobile), the 6.28% reduction in compressed video size MuZero has achieved is pretty significant. The details of the work are available in an ArXiv preprint.
Outracing Champion Gran Turismo Drivers
Sony A.I. announces a deep RL model that beats world champions in the famous PlayStation game Gran Turismo, which is known for its accurate representation of race car dynamics. The work continues the series of RL outperforming humans in competitive e-sports. The study is published in Nature and covered by NPR.
Microsoft Research Summit Videos on YouTube
If you missed the Microsoft Research Summit late last year, you can catch up on YouTube. There are excellent discussions and presentations on RL by world-renowned experts:
- Opening Remarks by Katja Hofmann
- Keynote: Key research challenges for real-world reinforcement learning by John Langford
- Panel: The future of reinforcement learning
- Dead-end Discovery: How offline reinforcement learning could assist healthcare decision-makers
- Panel: Generalization in reinforcement learning
- Research talk: Safe reinforcement learning using advantage-based intervention
Recent publications on Reinforcement Learning
- TransDreamer: Reinforcement Learning with Transformer World Models
- Don’t Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
- Real-time model calibration with deep reinforcement learning
- It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generation
- VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learning
- Learning to Control Partially Observed Systems with Finite Memory
- Selective Credit Assignment
- Improving Sample Efficiency of Value Based Models Using Attention and Vision Transformers
- Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error
- The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning
- Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization
- BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
- Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control
- A Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalization
- A Survey of Explainable Reinforcement Learning
- Open-Ended Reinforcement Learning with Neural Reward Functions
- Domain Adaptive Fake News Detection via Reinforcement Learning
- Safe Reinforcement Learning by Imagining the Near Future
- Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand
- Reward is not enough: can we liberate AI from the reinforcement learning paradigm?
- The Challenges of Exploration for Offline Reinforcement Learning
- Learning to Guide and to Be Guided in the Architect-Builder Problem
- SAUTE RL: Almost Surely Safe Reinforcement Learning Using State Augmentation
Academic Positions
- BIOLab PhD Position 1/4 Sample efficient reinforcement learning in neuroscience
- Postdoc in reinforcement learning at the Chalmers University of Technology, Sweden
- Full-time post-doc position (2 years) in machine learning in Sweden, The School of Science and Technology at Örebro University
- Two Full-time PhD Candidates in RL / Cognitive Science / HCI, University of Jyväskylä, Finland
If you have found this newsletter useful, consider subscribing on Medium and LinkedIn, following us on Twitter, and sharing it with your network. If you are interested in contributing stories or have academic positions to feature, reach out to us at editor@rlagent.pub.
