I now have an AI coach for my wheelie project. Coach has seen over 1,500 of my wheelies, and reckons they can tell pretty quickly whether my effort will be wheelie good or bad. Coach also fits on my phone, so they come on rides when I want real-time advice.
I have also recorded a wheelie at 7.9 seconds! (against target 8s). But wait — controversy! On reviewing the wheelie trace I realised that this was an effort where I had put my foot down before the front wheel “landed”. I estimate my foot went down at 6.6s — surely this marks the end of the wheelie? What will the UCI Technical Subcommittee for Wheelies decide?
The Lockdown Wheelie Project continues in Part 2, “2 Lock 2 Down”. More on both developments later, after some Q&A from the first installment.
Question and Answer from Part 1
But what about X? Asked a bunch of people in response to my first post…
Is this really the best lockdown-compliant exercise program ever developed?
Asked no-one. But I realised this was a key tenet of the program and I should have shared that story with data. Hence, the Victorian Stage 4 Restrictions Compliance Report 😂😭.
What is the right amount of effort to put in to wheelie sessions?
Until I was asked, I assumed I got my best results earlier in each session, but the data told another story: both the longest individual wheelies and especially the rolling median more frequently occurred towards the end of the session. Since my first review, this has become even more pronounced — I think I like to finish on a high!
Could you detect accidents and other events too?
Yes, by labelling when events occur, and then training a Machine Learning (ML) model to learn the connection between sensor data at the time and the event. Then the model can make predictions on new sensor data about whether the event has occurred; be it an accident, or — much more fun to label — a wheelie.
Aside: I talked about an “AI” coach above, is that different to “ML”? To conform with general usage, I typically use “AI” to describe human interactions, while “ML” describes the technical implementation of solutions.
Building a Machine Learning Coach for Wheelies
Almost all of the analysis in Part 1 was based only on the pitch angle data from my phone, and even then, only whether or not it exceeded the wheelie threshold angle. I otherwise ignored more than 40 sensor values (including accelerometers, magnetometers, gyroscopes, and various sensor fusions) that I could use to gain insight. However, all of these sensor readings are now labelled with whether or not I was doing a wheelie at the time. I can use this data to build a coach with ML that I can take with me on my rides.
ML can be complex, so let’s start simple, by detecting wheelies with the accelerometers. This should work, because pitch is calculated using accelerometers. A quick look at the data confirms wheelies (1) can be to some extent distinguished from non-wheelies (0).
I trained a simple model to reach 98% accuracy at predicting wheelie state from accelerometers, and visualised the correct and incorrect predictions as a confusion matrix. See this notebook and data for details.
Great, I’ve trained a neural network to do trigonometry! Now, how do I turn this into a product that I, or anyone else, might want to use?
Build a Product
Building the whole product as soon as possible with a simple model helped uncover and resolve issues in the ML pipeline — I had to resample sensors, convert units, compensate for Android quirks and figure out Keras to TFJS model conversion. It also confirmed that real-time feedback was useful when out riding (though the duct tape does obscure low battery warnings :).
ML in this case doesn’t give a better experience than detecting wheelies with the pitch value, so I won’t take this accelerometer version any further just now. However, it has given me confidence that a coach app could be useful too, and successfully deployed, before I invest too much time in designing and training the coach ML model.
Design the ML Coach
A truly great wheelie coach could help me with all aspects of my wheelies. But again I’ll keep it simple, and break down the problem of doing a good wheelie into:
- Getting the wheel up
- Keeping it up
Sometimes I raise the wheel cleanly, and sometimes I lean or veer to one side or the other, which makes it much harder to balance and keep the wheel up. So let’s give Coach a simple assignment first up: predict how long I’ll be able to balance a wheelie, based on how well I raise the front wheel.
This first iteration won’t be a truly great coach, but at least this coach can travel in and out of my 5km bubble and join me on every ride!
Train the Coach Model
For making predictions, time series data is kind of magical: for all the data collected in the past, we can know the future of each point, and learn how to predict it. The duration to predict comes from my training diary data (described in Part 1). To capture the nature of the raise, I use as inputs the phone’s pitch, roll (for lean) and yaw (for veer) angles for the 2s prior to a wheelie, and the first 0.5s of the wheelie. This is 25 timesteps at 10Hz.
Here is the cleaned input data, aligned to the same start time, for 100 timestep windows (which covers the whole duration of all wheelies) and 25 timestep windows (just the raise).
After a few passes, the trained coach model can predict the wheelie duration to within 1 second nearly three quarters of the time (as shown below). This could be improved further, but it doesn’t seem ineffective, so I’ll use it for my first release of the coach.
Indeed, this model is significantly better than just guessing according to the distribution of wheelie durations, which gets just over half within 1 second (as shown below). This is the definition of ineffective and the baseline that any coach must exceed.
But it’s not as good as observing the wheelie for longer (although this goes beyond just the raise)… the longer you observe, the easier it is to predict the duration! :)
Coach Goes Live!
Now it’s time to build the coach into the app. To turn the prediction into some form of coaching, I come up with some pithy encouraging words based on the combination of predicted (at 0.5s) and actual duration (once the wheelie is finished). The app continuously maintains the most recent 25 time steps for pitch, roll and yaw deltas, and makes a single prediction 0.5s after a wheelie starts.
I take coach for a ride, and I’m actually pretty happy with the results. I haven’t yet observed a huge performance benefit to myself, and any I do may just be the Hawthorne effect. I also have a lot of ideas about how to improve Coach — who still has some rough edges — especially because I still have more data to exploit, and different framings to explore. I also expect the raise to become less predictive of duration as my balance improves. However, I’m quite pleased with this first release, and having an AI wheelie coach!
I’ve logged 770 additional training wheelies since Part 1.
Until my disputed 7.9s wheelie, I had been reasonably pleased with a max 6.7s achieved on a couple of occasions. The wheelie in question occurred on the last session of this round, just before publishing.
Most wheelies end with me “landing” the wheel cleanly — a controlled response to under-balance or over-balance — but some end with the bike “looping out” and me putting a foot down. I haven’t done the full analysis (it’s coming) but I have assumed this is a minority of shorter wheelies and I usually land the front wheel quickly after putting a foot down. I hadn’t considered that time delay as material in my analysis, until this one fateful wheelie! So I’ll need to update the wheelie definition to accommodate these loop outs better. I’ll report how that went next time :)
For now, I’ll leave out that last session in the training diary recap.
It’s good to see continued improvement too, though sometimes two pedals forward and one back. I’m planning to experiment more in the next round with some changes to training inputs that were otherwise held constant throughout these first two rounds — such as gear ratio — and expect this will have a mixed impact on the view below.
And, finally, my gallery of top wheelies is looking neat, with all of the top 24 over 5s now.
There are still many directions to go from here, but three most likely for the next round are:
- Update the definition of a wheelie, as above
- More study of the wheelie profiles, extracting more salient features of a wheelie, including how well balance is achieved, for the next iteration of Coach
- Rider pose analysis from video (of which I have more now) and combining this with sensor data
And, of course, closing that last 1.3s gap with my feet up!