How we taught an AI to play Breakout

We all have some unique expertise that we’ve learned over years or even lifetimes. To create truly intelligent systems, we need a way for every developer to easily teach their own individual expertise and ideas to machines. At Bonsai, we’ve done this by taking a fundamentally different approach to programming AI — express the solution to a problem in terms of how to teach it, as opposed to how to calculate it.

Teach computers the same way we teach people

Just think about how we teach kids math. We don’t shove them in a room with a calc textbook and ask them to learn it themselves. Instead, we teach them the concept of numbers, and the concept of numbers having value, and the concept of being able to add together those numbers for a greater value and so forth.

We’re applying this logic to programming AI. Our new programming language, Inkling, enables a teaching-oriented approach by providing a set of foundational primitives that can be used to represent AI without specifying how the AI is created. By abstracting away the underlying algorithms and techniques, you can focus on creative ways to describe your problems and leave the heavy lifting to us.

We will learn more about Bonsai’s platform through the example of teaching an AI agent to play Atari’s Breakout.

Bonsai’s Breakout

Teaching an AI to play Breakout

Mental Model for Breakout

First, we create a mental model (a high level description of the problem) for Breakout with schemas to represent the input and output and a single concept, get_high_score.

The schema keyword declares a set of named types that can be used throughout the system. In addition to common basic data types, native data types exist for working with common media formats (e.g. images, audio recordings). The metadata from native data types is used to guide and improve learning.

The schemas are represented in Inkling as follows:

schema GameState
(
    Luminance(84, 84) pixels  # board state 
)
schema PlayerMove
(
    String{"left", "no move", "right"} move
)

In this example, the Luminance data type declares that the input is a grayscale image of the gameboard where the values are floats between 0 (black) and 1 (white) and PlayerMove moving left, do nothing or right.

Concepts
A concept is something that can be learned. Once learned, it can provide intelligent output.

get_high_score

concept get_high_score : (PlayerMove)
    is classifier
    follows input(GameState)
    feeds output

This first base-level concept is get_high_score, which takes a screenshot of the game as input and and outputs a PlayerMove that feeds the output. When paired with a curriculum (which we’ll dive into in our next post), this simple mental model takes the image of the board and makes moves that maximize the score.

Using multiple concepts

While a Mental Model consisting solely of get_high_score would be sufficient to train an AI to play Breakout, adding extra concepts decreases the training time. For other simulations, as we’ll discuss in future blog posts, having more concepts makes the AI agent give smarter, more accurate predictions.

Multiple concepts mental model

Our new mental model has multiple concepts that can receive input data from other concepts or send output data to other concepts or a final result.

Let’s go over these in more detail:

concept ball_location : (Int32 x, Int32 y)
    is estimator
    follows input(GameState)

ball_location takes a grayscale image of the game board as input and estimates the location of the ball.

concept keep_paddle_under_ball : (PlayerMove)
    is classifier
    follows ball_location, input(GameState)

keep_paddle_under_ball is a classifier that outputs a player move (left, no move, right) based on the ball_location and a grayscale image of the game board. This concept encapsulates the idea that the paddle should follow the ball location using the output of the previous network and the state of the game.

concept get_high_score : (PlayerMove)
    is classifier
    follows keep_paddle_under_ball, input(GameState)
    feeds output

Our augmented get_high_score concept takes input from the keep_paddle_under_ball concept and an image of the game board and outputs the player move.

After defining a mental model for Breakout, we use a curriculum to teach these concepts to an AI agent. The curriculum is the second of the foundational primitives that enable Bonsai to create a teaching-oriented approach to creating AI.

We will talk about curriculums, lessons, simulations, algorithm selection, training, deployment and more in the coming posts. Stay tuned!

Find the part two here

Sign up here learn more: http://bons.ai/