Aye Aye, Captain

A few weeks ago, we followed the 2-day AI Captain Crash Course in Machine Learning and learned how to program a boat to steer itself. Here’s a short recap.

What’s AI Captain?

The AI Captain Crash Course in Machine Learning is a new course aiming to teach anyone interested in AI how to apply machine and deep learning to program a self-steering sailing boat in two days.

The course revolves around AI Captain — a mechanical box that you can connect to the boat’s rudder, helping it maintain the optimal route based on an AI model and input parameters — and a Python-based boat simulator, programmed by organisers and hosts Daniel Versteegh and Wouter de Winter.

Day 1: theory, data & creating models

On September 20th and 21st, we attended the first edition at the Regatta Center Muiderzand in Almere: an informal spot, close to the marina and home to several sailing groups. In other words: perfect for the purpose.

First: what’s the history behind AI Captain, and how did the idea for creating the course come to be? The highlights:

  • Daniel and Wouter wanted to create a course to get to know AI in the most fun way they could imagine
  • They own a boat, so merging AI with sailing made sense (and it’s fun!)
  • It’s relatively easy to apply and test this tech in the water: there are no regulation rules, more freedom of movement, less risk and automatic navigators available. Also, the concept of a self-sailing boat is still unique.

Next: the theory on machine learning. What is machine learning, what are the most popular algorithms to create a model, and which applies to which situation? We discussed two types:

  • Classification: the outcome is the probability of the result fitting predefined classes
  • Regression: the outcome is numeric

Classification examples:

  • A decision tree that can decide if someone is physically fit, based on two simple parameters (e.g. age and amount of exercise)
  • A house price range based on its location, number of rooms, condition and other parameters

As for AI Captain, our goal was to calculate the rudder angle (i.e. a number) based on various input parameters — which is a regression problem. We also discussed pandas in Python, and how we could potentially use them to analyse the data we’d be using for our AI model.

A few words on AI and data:

  1. Every AI model is driven by data. Partially to train the model based on the algorithm that best fits the problem, and partially to test how accurate the model actually is.
  2. The marktplaats rule: if the outcome is too good to be true, it’s probably false. If a models accuracy is too high, it’s likely incorrect or overfitting — trained in a way that overfits the given data. The model will still produce the right results for the given data, but if the data changes results will be incorrect, as it didn’t actually ‘learn’ anything. Sometimes your model will do better if you train it with less data.
  3. It’s not possible for an AI model to be 100% accurate. If it is, this again means it’s probably too good to be true.


After fuelling up, it was time to start doing cool stuff.

To compile our model, we collected data of environment parameters and boat movements. As mentioned, AI Captain provides a Python-based boat simulator to mimic position and movement based on parameters such as boat angle, heel and speed, wind direction and speed, and angle of attack.

In the process of simulation, these parameters would change constantly. The simulator showed a pointer for the boat to point at in order to stay on track. Our challenge was to make the boat aim correctly by changing the rudder angle with any of these options:

  • Apply one of the ready-to-use strategies
  • Use the manual mode (move the boat left and right with the arrow keys)
  • Create your own strategy

Some of the participants had real-life boating experience, so they chose to create their own strategy (extending the existing Python code) and find the best way to make the boat decide on the right rudder angle. Others — like us — just went with one of the existing strategies, and adjusted them if needed.

We collected all data to make the simulator run (parameters, date times, rudder angle) in a CSV log file. After letting the simulator run for about 5 minutes, we gathered enough data to start training our AI models.

Creating the machine learning model

Time to create!

First, we imported our CSV data file in a Python data field, which helped us understand how the data is interrelated. We then decided which parameters would be useful to generate the correct rudder angle.

It’s not necessary to use all parameters — and their relevance strongly depends on the strategy you choose when collecting the data. In fact, adding redundant parameters can generate a less accurate result. Within our group, the choices varied from five to three to just two, based on the raw data.

Next, we split the data into 2 parts: one for training part, the other for testing. We then used an algorithm to create our models and test them in the simulator. The key was to match the boat’s direction as closely as possible with the pointer’s position, which determined how good (or bad) the model was. We could play around with the settings, generate new models and try again as often as needed.

Note: many things can influence the accuracy of an AI model — from the input parameters to the algorithms, and from the algorithm settings to the number of training steps. This makes it difficult to know the exact best approach and settings beforehand. The best way to learn is through trial and error— but even then, you’ll need to keep experimenting for better results. In our case, calculating the mean absolute error would help measure the average loss of the model and determine if you’re working towards the right solution.

Day 2: deep learning & testing, testing

Day 2 was all about deep learning — for most of us the most interesting part of the course. We got to talk about more complicated AI systems such as image recognition systems, and how they work from input (image) to final decision (e.g. if the photo shows a dog or a cat). Systems like these have a lot of real-life potential, especially when paired up with a mobile app.

To create a network, many layers of neurons work together. In case of classification, the output of the network is the probability of the input matching each of the predefined classes (e.g. the probability of the image showing a dog or a cat). The system defines this depending on which neurons are triggered.

TensorFlow is a nice tool to make this process more tangible, allowing you to play around with different inputs, algorithms and network settings to create a deep learning system that learns to decide if the color of a given dot is blue or orange based on its location. TensorFlow immediately visualises your choices and makes it easier to grasp how deep learning works.

Ready, set, action

With the data collected on day 1, we could now use different algorithms and create a deep learning model for the boat with convolutional or dense layers. The problem we were solving for AI Captain wasn’t too complex, so dense layers could do the work — however, for image recognition purposes convolutional layers are the way to go.

We played around with various settings — input parameters, number of dense layers, number of neurons per layer and so on. After training the model with a subset of the data and testing it against the remaining data, we were able to determine the loss of the model and decide which to toss and which to keep. Just as the day before, we got to test the model in the boat simulator.

And now: time for action! Unfortunately, the weather on day 2 was pretty bad, with lots of wind and rain. The group decided last minute that it was safe enough to use the actual boat. We opted out as we don’t have any sailing experience, but we did get to chat afterwards with the people who joined and tested their model at work. Their models worked pretty well — the deep learning models better than the machine learning ones — and most importantly, they lived to tell the tale.

The AI models in action:

And that’s a wrap

The AI Captain crash course was a good experience and introduction to the exciting world of AI. We love being challenged to learn new things, and the course sparked interesting ideas of how to implement AI in our work and the new doors opening up because of this technology becoming more widespread and applicable to real-life scenarios.

If you’d like to learn more about programming a self-steering boat, check out the course on AI-Captain.com.

Maria Tzaneti
Greece-born Backend Developer at The Mobile Company. Loves solving problems and fixing bugs — the digital ones — and is secretly super fluent in Dutch.

Maiko Hermans
Mobile Developer at The Mobile Company. Used to dream about training sled dogs as a career. Now trains his pawesome Belgian Malinois on the weekends.

The Mobile Company

We’re an award-winning Amsterdam-based mobile agency, working with leading companies and well-known brands to create apps for millions of users. We post about cool tech, awesome projects and other fun stuff. Want to chat? Get in touch. Learn more at www.themobilecompany.com.

If you’ve enjoyed this article, hit 👏 and help others find and read it!

Exclusive rights to selected visual footage belong to AI Captain.