Member-only story

RL meets hyperbolic geometry

Hyperbolic Deep Reinforcement Learning

Many RL problems have hierarchical tree-like nature. Hyperbolic geometry offers a powerful prior for such problems.

Michael Bronstein
TDS Archive
Published in
17 min readApr 30, 2023

--

Many problems in Reinforcement Learning manifest a hierarchical tree-like nature. Hyperbolic spaces, which can be conceptualised as continuous analogies of trees, are thus suitable candidates to parameterise the agent’s deep model. In this post, we overview the basics of hyperbolic geometry, show empirically that it provides a good inductive bias for many RL problems, and describe a practical regularisation procedure allowing to resolve numerical instabilities in end-to-end optimisation with hyperbolic latent spaces. Our approach shows a near-universal performance improvement across a broad range of common benchmarks both with on-policy and off-policy RL algorithms.

Stable Diffusion prompted with “Hyperbolic Atari Breakout game, icon design, flat design, vector art” (courtesy of David Ha)

This post was co-authored with Edoardo Cetin, Ben Chamberlain, and Jonathan Hunt and is based on the paper E. Cetin et al., Hyperbolic deep reinforcement learning (2023) ICLR. For more details, find us at ICLR 2023!

Basics of Reinforcement Learning

RL problems can be described as a Markov Decision Process (MDP), where the agent observes some state sS from…

--

--

TDS Archive
TDS Archive

Published in TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

Michael Bronstein
Michael Bronstein

Written by Michael Bronstein

DeepMind Professor of AI @Oxford. Serial startupper. ML for graphs, biochemistry, drug design, and animal communication.

Responses (4)