Member-only story
RL meets hyperbolic geometry
Hyperbolic Deep Reinforcement Learning
Many RL problems have hierarchical tree-like nature. Hyperbolic geometry offers a powerful prior for such problems.
Many problems in Reinforcement Learning manifest a hierarchical tree-like nature. Hyperbolic spaces, which can be conceptualised as continuous analogies of trees, are thus suitable candidates to parameterise the agent’s deep model. In this post, we overview the basics of hyperbolic geometry, show empirically that it provides a good inductive bias for many RL problems, and describe a practical regularisation procedure allowing to resolve numerical instabilities in end-to-end optimisation with hyperbolic latent spaces. Our approach shows a near-universal performance improvement across a broad range of common benchmarks both with on-policy and off-policy RL algorithms.
This post was co-authored with Edoardo Cetin, Ben Chamberlain, and Jonathan Hunt and is based on the paper E. Cetin et al., Hyperbolic deep reinforcement learning (2023) ICLR. For more details, find us at ICLR 2023!
Basics of Reinforcement Learning
RL problems can be described as a Markov Decision Process (MDP), where the agent observes some state s∈S from…