Bonsai’s BRAIN

This is the third in our series of posts of how to train an AI agent to play Breakout using Bonsai’s platform. You can read our earlier posts here and here.

In the last couple of blog posts, we went over schemas and concepts that together define the mental model. We also defined curriculums with attached lessons to teach the mental model to the learning agent.

Once the mental model and curricula have been coded in Inkling, the code can be sent to the BRAIN server for training. Bonsai’s Basic Recurrent Artificial Intelligence Network Server, or BRAIN Server for short, is a platform-as-a-service for creating AI agents. The BRAIN Server contains implementations of many learning algorithms, has logic for picking learning algorithms and guiding training, and is built with an infrastructure that supports streaming data efficiently through the system. In short, BRAIN Server computes a learned solution from a problem described in Inkling.

To start, let's pick up where we left off, and imagine a file,breakout.ink that includes the mental model and curricula needed to play breakout.

Let’s go over it in a series of steps:

Create a new BRAIN:

We create a new BRAIN either by using Bonsai’s CLI tools or through an AWS-like console.

Once the brain is created, we load the inkling file, breakout.ink, containing the mental model and attached curriculums and lessons.

$ bonsai brain load mybreakout breakout.ink
  loaded.

Algorithm Selection

The first step the BRAIN Server takes is to pick an appropriate learning algorithm to train the Mental Model. Choosing the right algorithm is a critical step in training any AI model; it takes data scientists years of expertise and study to learn to do this effectively. The BRAIN Server has knowledge of many of the available learning algorithms and has a set of heuristics for picking an appropriate algorithm as well as an initial configuration from which to train.

For example, if the BRAIN Server were to pick Deep Q-Learning for training a Mental Model, it would also need to pick an appropriate topology, hyper-parameters, and initial weight values. A benefit of having the heuristics available to be used programmatically is that the BRAIN server is not limited to a single choice; it can select any number of possible algorithms, topologies, etc., train all of them in parallel, and pick the best result. The only limit is the amount of computing resources the user wants to throw at the problem.

Next, we connect our breakout simulator to the BRAIN server using the Bonsai Python SDK.

from math import sqrt

from bonsai import Simulator
from bonsai.inkling_types import Luminance
import pygame

from breakout_game import BreakoutGame

class BreakoutSimulator(Simulator, BreakoutGame):

    def score(self, move):
        return self._score

    def ball_location_distance(self, x, y):
        """ Gets the distance between the predicted x and y coordinates and the actual coordinates.
        """
        return sqrt((self._ball_left - x)**2 +
                    (self._ball_top - y)**2)
  
    def next_data(self, move):
        # Step 1: Perform the action.
        self._perform_action(move)

        # Step 2: Update the game
        self._update()

        # Step 3: Draw the game.
        self._draw()

        # Step 4: return the latest game state to the AI agent
        return {"pixels": pygame.display.get_surface() }


if __name__ == "__main__":
    bonsai.run_for_training_or_prediction(
        "breakout_simulator", BreakoutSimulator())

Here we hook the simulator to the BRAIN:

$ python breakout.py — brainnpc= <http link to BRAIN server>
$ bonsai sims list
  breakout  1  instance  ready

Finally, we give the command to start training:

$ bonsai brain train mybreakout
training started.

Training is initiated with an explicit command from the user. In order for training to proceed, the user needs to have already submitted compiled Inkling code and registered all simulators with the BRAIN Server as we did above.

Once an algorithm is chosen, the BRAIN Server will proceed with training the BRAIN’s Mental Model via the Curricula. The BRAIN Server manages all of the data streaming, data storage, efficient allocation of hardware resources, choosing when to train each Concept, how much (or little) to train a Concept given its relevance within the Mental Model (i.e. dealing with the common problems of overfitting and underfitting), and generally is responsible for producing a trained BRAIN based on the given Mental Model and Curricula.

As is the case with picking an appropriate learning algorithm, guiding training — notably avoiding overfitting and underfitting — to produce an accurate AI solution is a task that requires knowledge and experience in training AIs, and the BRAIN server has an encoded set of heuristics to manage this without user involvement.

The process of picking the right algorithm and guiding training is itself a BRAIN that Bonsai has trained (and will continue to train), meaning Bonsai BRAIN will get better at building BRAINs each time a new one is built.

Deploy

$ bonsai brain deploy mybreakout
deploy success.
Connect to https://brains.bons.ai/<link to trained BRAIN>

Once a BRAIN has been sufficiently trained, it can be deployed such that it can be used by the user’s production application. The interface for using a deployed BRAIN is simple: the user submits data (of the same type as the BRAIN was trained with) to a BRAIN Server API and receives the BRAIN’s evaluation of that data.

Bonsai’s platform consists of three pillars — a special purpose programming language called Inkling, a BRAIN server backend that abstracts the implementation details for programming AI, and a Mastermind IDE that allows a visual studio-like experience for authoring and debugging. We will talk about the Mastermind experience in a later blogpost.

Our mission is to enable any software developer to build AI solutions regardless of their AI expertise. This is the promise of Bonsai, handing the keys of the kingdom to anyone who needs and wants to use it.