In Melbourne’s COVID-19 lockdown, I’ve wheelied over 17km. Not all at once, though.
Over three months, I’ve spent 90 minutes with my front wheel raised. I’d like to keep it up, but as lockdown has gradually relaxed, and routines have changed, so have I landed the wheelie project, for now.
I’m pretty happy with my best 7.5s effort against my initial 8s target, and as such, the focus of this instalment is on quality over quantity.
In this post, we’ll extend the capabilities of the AI coach from Part 2. We’ll explore ML models to gain deeper insight into what makes a wheelie, getting the right balance, and how they come to an end.
In Part 2, there was some controversy about my foot touching down before the front wheel was grounded. I’ve resolved that by adding an extra “sensor” to my rig, so I can manually record my foot touching down…
all_foot_down_events = [(1599474599.923785, 1599474601.228138)]
… and adjusting wheelie state calculation accordingly …
ts_df['wheelie_state'] = ts_df['wheel_up'] - ts_df['foot_down']
… so that everything else just works (extracting events, Part 1).
Now my legitimate record is a clean 7.5s wheelie from 13 September.
I look forward to bettering this one day!
What’s in a Wheelie?
When I’ve reviewed individual wheelies so far, I’ve presented a trace of the whole wheelie — typically 100 data points, representing 10 seconds of bike pitch — plotted over time or distance. To get some simple coaching insight, I’d like to work with far fewer than 100 numbers. How about just three?
If I had to pick three numbers to describe a wheelie, I’d probably choose duration, amplitude (avg or max pitch), and style (to be defined!) But this is a problem machine learning can help with too. A neural network architecture called an autoencoder can learn a simple representation for a wheelie.
Here’s how it works: an input wheelie trace is mapped (encoded) to “latent” representation from which the trace is reconstructed (decoded), and compared to the original input.
The neural network learns to minimise the difference, or “loss”, between the input and reconstructed traces. If we plot latent representations of all wheelies in 3D, we get a point cloud with an interesting shape.
So what do those three numbers — the “latent dimensions” — actually mean? Do they even have a meaning for people? Let’s take a wheelie instance and see what happens when we vary each of the latent dimensions by up to +/- 3 standard deviations (of their distribution over the training set).
The meaning of the latent dimensions is not as obvious as duration, amplitude and style. What I see is variation around the instance in terms of:
- L0 = Long duration & wiggly / short duration & smooth,
- L1 = Sharp landing / gradual landing
- L2 = Long duration & high amplitude / short duration & low amplitude
And this is great, because it gives me additional insight into what’s in a wheelie! As a result of this exploratory analysis, I am going to look further into the “wiggliness” of my wheelies, and the landing phase of wheelies.
However, duration itself remains really interesting. The point cloud above has a distinct “spine”, along which duration varies smoothly. To define a wheelie by duration, we can fit a 3D curve to this spine. Duration is short at one end of the curve and long at the other.
One of the great properties of autoencoders is that — once trained — they can be used to generate complex outputs from simple inputs. Now that I can plot a point in latent space with duration alone, I can dial up new, convincing looking wheelies of any desired duration! (within the limits of my data)
So after all that, here are a bunch of wheelies that do not exist, except in the imagination of a decoder neural network!
This could potentially be used by an AI coach to show how a wheelie might look different with minor changes in key features, or what would be achievable with improved technique.
Getting the Right Balance
Wiggles in the balance phase are interesting — you might have noticed the autoencoder even adds them to the synthesised wheelies of longer duration!
I think wiggles indicate two things. First, that I’m moving backwards and forwards across the balance point, so the more wiggles observed, the better my ability to regain balance. Second, from a style perspective, it looks like I’m constantly about to lose my balance (!), so the fewer wiggles observed would point to higher quality wheelies. Some tension here…
For now, I just want to capture wiggles in my training diary. We can detect wiggles with some simple filters. The first stage is to split wheelie traces up into phases: pre, raise, balance, land, post.
Then apply the following filters (not perfect, but a start) to the traces from the balance phase alone:
- Change from trending up to trending down? (sign of first difference)
- Change direction sharply? (magnitude of second difference)
This gives a measure of wiggliness per wheelie. I feel like I had more wiggles in my last few sessions, especially after I literally took things up a gear (raising in 4th, 28:28 instead of 3rd, 28:32). This chart added to my training diary shows the rolling mean of wiggle frequency, and the point where I changed gear — you can see the change in the data too.
Coming to an End
Some wheelies end gradually, and some sharply. The gradual ending is typically due to “underbalance”, where I can no longer accelerate fast enough (by pedalling furiously) to keep the wheel up. The sharp ending is typically due to rapidly overcorrecting “overbalance” by applying the rear brake.
I wonder if I can predict from a segment of the balance phase whether I’ll continue balancing, or whether the wheelie will come to an end. We could tackle this in a similar way to the raise-duration prediction from Part 2, but given the variable length of the balance phase, I’ll use a multi-step time series forecasting approach instead. For any 1s window of the balance phase (10 samples at 10Hz), I’ll predict the next 1s of pitch values. For this I’ll use a neural network called an LSTM.
This requires chopping the training data up into a lot of windowed pairs — for a point in time, 1s prior “input” and 1s following “labels” — looking something like this:
The inputs can be univariate — just pitch — or multivariate — including other sensor values as inputs too. After training, the model makes predictions like this:
To understand how well it predicts the land phase, vs continue balance, I classify each labelled “true” trace and each prediction as representing continued balance (wheel still up and the end), or a landing event (wheel down at the end). Then we can compare how the predictions compare to true outcomes using a Sankey diagram, which shows most predictions agree with the true result, but that landing is harder to predict correctly (we include remaining grounded “ground” predictions and “raise” predictions for completeness).
We can show this more precisely with a confusion matrix and class f-scores.
It’s a reasonably hard challenge to pick between the end of the wheelie or continued balance, and my experience gels with this, especially as one gets better at recovering balance. Generally, improvements in “land” predictions in the above come at the expense of “balance” predictions.
But overall, those predictions are far better than a baseline of assuming one case or the other, or guessing at random, so this additional mini AI could have coaching value too.
The balance and landing analysis has been informative and fun, and could be incorporated into a coaching app, given the end-to-end process already demonstrated in Part 2, but that will come in time. As per previous posts, there are still many directions to take this project, and it’s been a great distraction during COVID-19 lockdown.
For now, I can predict with nearly 100% certainty that this is the last post for a little while. I hope it’s been entertaining and educational. Thanks for reading!