One of the biggest challenges facing AI researchers today is an absence of explainability from their models. Neural networks can crunch data and churn out remarkable predictions, but humans are unable to understand the thought process that led to those conclusions.
Explainability will become increasingly important as AI is deployed into more consumer facing products (as will the need to define what we hope to learn from explainable models). And there are a number of techniques being pursued to gain visibility into deep learning models (you can read about the Deep Explanations approach here, and the Model Induction approach here.)
But today I want to shed some light on Bonsai’s approach to building intelligent systems, and how that approach results in more explainable AI models.
Machine Teaching
In my deep dives of both the Deep Explanations and the Model Induction approach, I touched on the importance of subject matter expertise and machine intelligence.
We know machine learning is very flexible, very powerful, and can learn dynamic representations.
We also know subject matter expertise enables us to give ontology to machines, and tell machines about which subjects matter to a specific problem we’re trying to solve.
Integrating both together results in a new technique — Machine Teaching.
Machine Teaching is used to build a conceptual hierarchy. It’s a fuzzy hierarchy; it’s not like old expert systems with very strict sets of rules. But it enables you to teach specific expertise that matters, and build that expertise into a conceptual model.
Let’s go back to the zebra example from the Model Induction experiment. Using Machine Teaching, you can teach the system that a zebra has four legs, and a head, and stripes, and a tail. (You don’t need to tell the machine what those things are; machine learning techniques and data and simulations enable the machine to learn what a head is). Those ideas are built into a conceptual hierarchy to drive predictions from models that can be explained.
Machine Teaching is particularly important when talking about deep reinforcement learning systems and not just traditional classification systems.
Recomposability and Behaviors
In a classic reinforcement learning system, you have an actor taking an action in an environment. The actor gets some assessment of reward for its behavior, and that reward is used to drive the learning loop.
It’s important to note that, unlike circumstances with quite readily composable features (ie: a leg, a head, and a tail of a zebra image), RL systems must predict actions to be taken. And everything is fighting over the exact same action prediction.
For example, in the game Lunar Lander, you can teach the system lots of concepts in order to win the game, but each concept is still ultimately trying to fire the thrusters. They’re all thinking the same thing (ie: Concept 1 thinks they should fire one way, Concept 2 thinks they should fire another way.) The ability to understand how to deal with those conflicting ideas becomes very important for RL systems.
One approach to this problem, which informs a lot of the work we do at Bonsai, is the subsumption architecture put forth by Rodney Brooks when he was building RL models for robotics in the mid 80’s.
In this approach, the concepts you teach the machine are not features of the environment, but behaviors for the agent to take in an environment. You can decompose behaviors into sub-behaviors that correspond to a hierarchy of available actions to take.
For example, if you have sensory information coming in, you could have behaviors for ‘explore world’, ‘wander around’, and ‘avoid objects’. If you’re activating the ‘avoid objects’ behavior, you’re inherently also activating the other two behaviors, but it forms a hierarchy of needing to do certain things in order to do other things.
This approach helps us derive a way to decompose and recompose when we talk about behaviors.
Recomposability in the Bonsai Platform: Concept Graphs and Inkling
In our platform, we have a new programming language called Inkling that lets you build a concept graph. You literally codify concept hierarchies.
In our system, these concepts can be features or actions. But as you build them, you’re giving each concept its own training regime.
To understand concept graphs, think about teaching a 5 year old to play baseball. You, as the teacher, already know the skills that the 5 year old will have to learn in order to understand the game: catching, hitting, running bases.
You would never teach a 5 year old to master the skill of hitting a ball by taking them outside and throwing fastballs. But that’s exactly what we do with modern machine learning systems; we take the absolute end result and throw lots and lots of examples at them.
Instead, we start with a wiffle ball on a tee. Then the tee pops up the ball. Then underhand, etc. It’s totally fake to start, but it’s fake in a way that’s designed to be instructive. And it genuinely works.
This increases rate of acquisition of skills dramatically; around 2 orders of magnitude faster. You can see what I mean in the Lunar Lander example below:
The goal of the game is to get the spacecraft to land on the pad. We used Bonsai’s Platform to teach the game 3 concepts to win the game : stability, going to certain points in space, and landing at a graceful rate of descent
To be clear, you don’t have to teach these concepts. You can use powerful reinforcement learning algorithms and it will learn how to solve this problem. But just using RL takes around 2,000 episodes. Using concepts, it beats the games in 50 episodes. It’s not just a little better; it’s a lot better.
And on top of performance increases, now it’s explainable.
To get explainability, it needs to learn how to select between behaviors that we want system to learn, and synthesize novel behaviors that are informed by those behaviors in pursuit of the objective. Let’s see it in action:
In this first video, the craft is learning. In the upper left, you can see the three different concepts it’s learning between and it’s reason for using any one concept at a given time. In this training example, ‘going to a certain point’ is dominating because that the concept it’s currently learning.
In this video, the system is already trained. It’s trained first to go to a specific point at the top, then a specific point at the bottom, then land gracefully.
It learns all behaviors but then the systems learns how to select between them at any given time. And we can offer visual explanations for those selections.
When you integrate these together and the system is fully trained, you get perfectly reasonable, human-like behavior. And because you can explain it, and you’ve introduced concepts, you get generalizability in that system.
The system has not just learned how to keep the craft stable, go to a point, and land. It means that if I tweak the environment in ways that it’s never seen before — if I add wind, for example, it will continue to expect human-like behavior and solve the problem.
A normal neural network would no longer work with added wind. But since the system learned these generalized concepts, and it understands the subsumption hierarchy that getting to a point and stability takes precedence over landing, it will still exhibit intelligent behavior.
Concepts — both feature or behavioral — give you benefits around training times and offer explainability in a way that lets you add increasingly complex capabilities and verify behaviors by testing and setting up new environments that might not have been seen in training.
Bringing It Together
We’ve now explored three approaches to building more explainable AI systems:
- Deep Explanations
- Model Induction
- Machine Teaching & Recomposability
Taken together, we can start to build explainability and debug systems to build more sophisticated models.
Bonsai is building a general purpose deep reinforcement learning platform for control and optimization problems including robotics, HVAC, and supply chain. To learn more about our work, visit bons.ai. You can also view my Explainable AI talk in full at the link below:
