Blackboard Pattern

Why use the blackboard pattern

So I realize its been over a week since my last article — I haven’t started to run out of ideas, I just took a week off from robotics to catch up with other things.

Today’s subject will look at the Blackboard pattern and how it can be used in Robotics. Roughly speaking, the Blackboard pattern is a software design pattern with three components:

1. A shared space for storing information about the world (The Blackboard).
2. A set of algorithms (machine learning or otherwise) that read in information from the world and post their results to the Blackboard.
3. A set of modules that read the results from the Blackboard and update the robot accordingly.

For example, a robot could have two cameras, one pointing at a table, the other pointing at some target area (maybe a basket). The vision algorithms for both cameras can run independently, each posting their results to the Blackboard and allow a central module to simply query the current state whenever it needs to make decisions without having to know any of the details regarding how the camera’s extract their features (or that there are in fact cameras for that matter).

Adding Structure to the blackboard

For my own robot, I’m using my own spin on the Blackboard pattern. Rather than having a simple “shared space” for storing the results of various machine learning algorithms I’ve imposed a structure on the how the data arranged. This structure is itself based on the results of the mapping algorithm. Essentially in my case the Blackboard is an undirected graph, with each vertex representing a physical position in the robots world.

When updating the Blackboard, the results of any algorithms that are currently running are merged with the data belonging to the vertex on the Blackboard that represents the robots current location. This gives a few advantages:

1. It provides the robot with a “BUS MAP” view of the environment — so when it wants to reason about how to get from place to place, or where different objects are it doesn’t need to work directly with Occupancy Maps.
2. Since it is intended to be a service robot, most of its attention will be focused on the area directly around it (where’s my human? Do I see a couch? etc?), by keeping a reference to the current vertex the robot can quickly extract an up to date model of its current location, including data extracted the last time it was in the area.

In addition, the other previously mentioned advantages of using a Blackboard are kept. The advantages are not free however, since all of the robots information about the world are tied to different locations in the world, when the robot needs data that isn’t in it immediate area there is a requirement to crawl through the graph to query each node for the data it needs. If the robot is in a large environment with a lot of nodes this could be inefficient.

Implementation

So how do we build a Blackboard in ROS? The most versatile way I’ve been able to come up with is to use a Service. A ROS Service allows a node to submit a request and get a response to that request, contrasting the usual behavior where a node will broadcast to a topic with no guarantee anything will ever respond to it.

A downside to this is that a Service appears to require a preset message type, which means you’d need to know the format of all data needed for your Blackboard to make use of this pattern. There is a way around this however — we can simply communicate with the Blackboard using json. Since json can be parsed into and out of a string, and can be used to represent nearly any type of data we might care to use this makes for a versatile data structure to represent the messages with our Service.

So what are the basic steps for creating such a thing? Well first we need create a service definition in ROS. Our service will take in string type data as input and spit out string like data as output (which will then be parsed back into json by the client). We represent this in a .srv file as follows:

string query
---
string response

Next you’ll need to add the .srv file you create above to your CMakeLists.txt file in your project. This will get added in the “add_service_files” section similar to the example below:

add_service_files(   
  FILES    
  blackboardQuery.srv
)

In addition to the above, we’ll also need to let catkin_make know that we wish to generate the header files we’ll need for using this service when we build the application. To do this, simply ensure that a line similar to the following is in your CMakeLists.txt file:

generate_messages(    
  DEPENDENCIES    
  std_msgs
)

Now we can use catkin_make to generate the messages and the dependencies we will need for the next step. Once this has been done, we can build the basic ROS Service. Assuming you have named the service “blackboardQuery.srv”, the following code should form a good template for initializing the service in C++.

#include <ros/ros.h>
#include <string>
#include <iostream>
#include "rover_platform/blackboardQuery.h"
#include "Blackboard.cpp"

using namespace std;
using json = nlohmann::json;

Blackboard g_blackboard;

bool blackboard(rover_platform::blackboardQuery::Request &req, rover_platform::blackboardQuery::Response &res) {
  cout << "BLACKBOARD: " << req.query << endl;
  res.response = g_blackboard.performQuery(req.query);
  return true;
}

int main(int argc, char* argv[]) {

  ros::init(argc, argv, "blackboard_server");
  ros::NodeHandle n;

  ros::ServiceServer service = n.advertiseService("blackboard", blackboard);
  ROS_INFO("BLACKBOARD READY");
  ros::spin();

  return 0;
}

To facilitate parsing json strings, we can use helpful library provided by nlohmann, this libary provides an elegant interface for creating and parsing json objects. Furthermore it allows us to easily convert the json objects we create to and from string format. I highly encourage you check out the documentation on this library before reading any further.

Ok, I shall assume now you have at least lovingly glanced at documentation for the json library we are about to employ. Our next step is to decide on a communications format. For my Robot, I decided to use a format like the following:

{
    "Command": "setCurrentLabel",
    "data": {
        ...
    }
};

This allows me to route each service call to an appropriate sub routine on in the Blackboard and handle the provided data accordingly. (In some cases there is no data to handle, but instead a request to return data). Below is some sample code that shows how to extract the command from the provided json and call the appropriate function:

string Blackboard::performQuery(string query) {
  auto queryData = json::parse(query);

  string command = queryData["Command"];

  cout << "Command: " << command << endl;

  if (command == "addVertex") return this->addVertex(query);
  ... Other Commands ...

  return "OK";

}

Now if we want to send data back, we need to create our own json object using the module we discussed earlier. Thankfully this is fairly intuitive, as the code snippet provided for creating a Vertex’s json provided below should demonstrate:

string Vertex::getJson() {
  json response = {
    {"Vertex", {
      {"x", this->x},
      {"y", this->y},
      {"index", this->index},
      {"parent", this->parent},
      {"label", this->label}
    }}
  };

  return response.dump();
}

As you can see, the notation for creating a json object using the provided plugin is fairly intuitive — nearly a mirror of the actual json we would want to create.

This should provide you with a good starting place to thinking about adding a Blackboard Service of your own for your own robot. If you are interested in my precise implementation it can be found on my github page here.

Next article will cover the physical construction of the Robot itself (the final part is arriving soon!). Until then

Share and Enjoy!

Note: Some may notice its been a bit of a delay since my last article — I took a week off from “Roboting” as it were to give my head a bit of a rest. Furthermore, in order to keep this blog manageable, I am going to limit it to one article per week. After all, I’m an amateur roboticist, not a writer.