A new way to blend existing models or classical controllers with state-of-the-art reinforcement learning.
Today we announced a new feature, Gears, to help programmers leverage their existing AI models and code to solve reinforcement learning problems with the Bonsai Platform.
Using Gears, a Bonsai Platform programmer can replace any “concept” in an Inkling program with existing Python code. In effect this means you can transform data flowing into our Bonsai Platform generated reinforcement learning algorithms. These transforms can include other machine learning models built with frameworks like Google’s TensorFlow and even “skills” that incorporate classical control operations into a more complex AI model. At Bonsai, we’re building Gears to:
- Put previously developed machine vision models in the concept network as a perception step prior to the agent action decision step
- Automatically blend deterministic behavior defined by inverse kinematics software like MoveIt! with behavior learned by algorithms like TRPO for robotic control
To be clear there are still many restrictions on this feature. As such, Gears are currently only available to those that are part of the Bonsai Early Access Program. We will, of course, relax these restrictions over time.
Let’s explore how one could use a Gear in a Robotics application. For this application, we will teach a robot arm to pick and place blocks.
Our first step is to break this problem down into four discrete skills which we can define in Inkling: reach, grasp, move, and stack. In this particular example we’ve decided to code the more dextrous manipulations, grasp and stack, using concepts (and accompanying curriculums, not shown in the code sample below) so the AI Engine will generate and train a model to learn those skills. Move and reach are simpler skills and can be coded using a classical controller, so we define these as Gears. Lastly, these four concepts are blended together using the AI Engine’s underlying selection and synthesis mechanisms.
In inkling we promise a Gear for reaching and moving the arm. We resolve that with some Python code using MoveIt!:
Additional capabilities we are working on enabling within gears include:
- Training models defined as a Gear using the Bonsai Platform
- Using the Bonsai Platform purely as a way of hosting existing models
- Collaborating between Inkling script authors and data science experts who are familiar with learning algorithms
To learn more about Gears and the Bonsai Platform, sign up for an in-depth webinar and demo on July 13th with Bonsai co-founder and Head of Product, Keen Browne: https://bons.ai/gears-webinar-signup. For more information on how to join Bonsai’s Early Access Program, visit: https://bons.ai/getting-started.
The Gears feature is available today for partners in the Early Access Program. If you have a control or optimization use case within your organization, learn more about the Early Access Program at https://bons.ai/getting-started.