Three generals are voting on whether to attack or retreat from their siege of a castle. One of the generals is corrupt and two of them are not. What happens when the corrupted general sends different answers to the other two generals? A Byzantine fault is “a condition of…

What motivates cooperative inverse reinforcement learning? What can we gain from recontextualizing our safety efforts from the CIRL point of view? What possible role can pre-AGI systems play in amplifying normative processes? Cooperative Inverse Reinforcement Learning with Dylan Hadfield-Menell is the eighth podcast in the AI Alignment Podcast series…

Humanity is at a turning point. For the first time in history, we have the technology to completely obliterate ourselves. But we’ve also created boundless possibilities for all life that could enable just about any brilliant future we can imagine. …

By Kirsten Gronlund In both 2016 and 2017, genome editing made it into the annual Worldwide Threat Assessment of the US Intelligence Community. One of biotechnology’s most promising modern developments, it had now been deemed a danger to US national security — and then, after two years, it was dropped…

How can humanity survive the next century of climate change, a growing population, and emerging technological threats? Where do we stand now, and what steps can we take to cooperate and address our greatest existential risks? In this special podcast episode, Ariel speaks with Martin Rees about his new…

By Tucker Davey At the core of AI safety, lies the value alignment problem: how can we teach artificial intelligence systems to act in accordance with human goals and values? Many researchers interact with AI systems to teach them human values, using techniques like inverse reinforcement learning (IRL). In theory…

How are we to make progress on AI alignment given moral uncertainty? What are the ideal ways of resolving conflicting value systems and views of morality among persons? How ought we to go about AI alignment given that we are unsure about our normative and metaethical theories? …

By Jolene Creighton Our AI systems work remarkably well in closed worlds. That’s because these environments contain a set number of variables, making the worlds perfectly known and perfectly predictable. In these micro environments, machines only encounter objects that are familiar to them. As a result, they always know how…

By Ariel Conn The following article was originally posted on Metro. Killer robots. It’s a phrase that’s both terrifying, but also one that most people think of as still in the realm of science fiction. …