One year of Byrd: it’s been harder than expected
As part of owning my mistakes of 2020 I vowed to write more about what we’re doing at Byrd. We just celebrated our first birthday and it seems like a pretty good time to explain why we’ve still not yet released our product. It’s the question we’re asked most often given how hard it is to survive as a start-up whilst being pre-revenue.
tl;dr we knew it’d be hard, but it’s been even harder than we expected.
Going into the depths of why narrow AI and human-centred design is hard is out of scope for this article. What we’re trying to do with Byrd is what Horst Rittel defined as a wicked problem. We’ve no definitive formula and no binary solution, we’ll be judged subjectively on whether what we’ve created is good or bad and we have to own whatever we propose. It’s the reason we’re taking an iterative — design-thinking — approach to creating Byrd. It allows us to test, prototype and validate our solutions as we learn new things.
We’re doing this because our vision is to build a business that enhances the health and wellbeing of runners. Just as with running there’s no shortcuts to get to our destination more quickly. Often in AI development a human-centered approach is side-stepped and products are shaped around the way the computer processes data. This is a mistake. Computers are entirely procedural, they do exactly what they’re told, even when that’s incredibly stupid as judged by a human. We humans aren’t procedural, we’re instinctive, emotional and gifted with high-levels of reasoning.
At Byrd we’ve put ourselves between these two domains. On one side pattern-matching algorithms give semantic meaning to running data, probabilistic models produce confidence scores for future behaviour and adversarial algorithms ensure balance. On the other side, there are humans who find all that pretty boring. As we’ve discovered in testing no-one cares about the AI, but they care deeply about being able to find joy in their running. If the deluge of data that running produces can be communicated in a way that’s personal and relevant to an individual’s life our belief is that it will lead to better running experiences.
This is a brief exploration of some of the things we’ve been spending time on, why we think they’re important and why we’re happy to put in the time to make sure they’re correct.
We had to talk to more runners
We want to make running a joyful experience for as many people as possible. To do that we needed to talk to runners. Many, many runners. Because otherwise we’d just be building a business for people who approached running in the way we did. We’ve spent many hours through 2020 making sure we have deep conversations with how people approach their running, goals and the universe. At current count we’ve clocked up at least 200 hours talking to runners about running and how Byrd can improve that experience.
Talking to runners confirmed that everyone’s an outlier. We had initially started with quantitative survey data where everyone was neatly plotted against the amount they ran or their years running or their motivation for running. Talking to people made it clear that trying to think of that experience on bell curves, whilst convenient, would be impossible.
There’s a diversity around why people go running. Some have incredibly high-specificity goals — a certain time, on a certain date at a certain event — others have much looser goals around finding headspace or feeling better. Digging deeper within those motivations there was a holistic approach that runners were taking to consider their running. People weren’t trying to smash a personal best (PB) simply for the sake of a time. The PB was an excuse to get out running because it made them feel good. Conversely, those runners that, on the surface, simply looked like they were trying to maintain some time for themselves still wanted to progress. They were just as likely to be looking to have a sense of flow within their running as more performance driven users.
Digesting that diversity has been a very deep design challenge through 2020 for us on Byrd.
Byrd can only track a user with their running data. This makes holistic progress very difficult to track directly. A lot of the heavy lifting we do with pattern matching algorithms is to try and detect this progress. We augment that data, and resolve fuzzy data that’s difficult to match, by talking directly to the user with dialogue flows.
It would be easier if we had just used the Training Stress Balance model or the Bannister model to use aerobic stress as a proxy for progress. But talking with people it’s clear that running and human behaviour has more complexity than an aerobic fitness score.
Ensuring autonomy
A few days after we launched one of the beta versions I had a great video call with one of the people that was running with Byrd. They were loving the experience of it and the fact that every run felt like it had a purpose. It was really exciting to hear! They mentioned they’d felt a little stretched by Byrd but were enjoying the push. After the call they messaged with what Byrd had proposed for the weekend. They asked me whether it was a little bit too intense, I responded immediately with “Do. Not. Do. That. Run.” It was a heart sinking moment. This was a relatively knowledgeable runner but they had deferred almost entirely to a machine. In this specific scenario the machine was wrong because of a bug — a primitive Monte Carlo tree search algorithm had gone rogue. But it was a wake-up that any AI system needs to be built in such a way that users can navigate around erroneous data, even if they never have to.
To be human-centred we need to ensure that anyone using our technology is autonomous and independent of that technology. That is, it should be a user-defined outcome facilitated by a machine rather than the machine as driver. I like David Epstein’s way of framing it as the human being as the strategist and the computer as the tactician in any relationship.
Our challenge was to reinforce that an adaptive training plan means the runner can choose their own adventure. It’s for the user to own their goals and how they want to get there. Byrd will plan the best way to get there but the user should always be able to change their mind about exactly what to do. It’s a little like someone using Google Maps and trying to get to their destination. If they chose to take a right when it was a left suggested they’ll be rerouted taking in to account their deviation. Fundamentally if someone makes it out of the door and goes for a run, and they keep doing that, they’re making progress.
Committed to data stories not dashboards
The standard way to present data is in some sort of tabulated form. More often than not, in a consumer product, that table is represented as a dashboard. The problem with dashboards is that, as designers, we’ve only done part of the work. It’s being left to the user to infer the quality and meaning of that data. And that design gap assumes that the user has the experience, knowledge and tools to translate the data into something useful.
In the context of running, at a top level, it means someone is forced to parse their pace, distance, elevation and various other metrics to then decide whether that run was ‘good’ or ‘bad’. Don Norman, in The Design of Everyday Things, talks about the gulf between execution and evaluation, and being presented with a dashboard makes that gulf much more difficult to hurdle.
At Byrd we think a better experience can be designed for runners. Through Byrd we use narrative extensively and use Natural Language Generation to create stories that are contextually relevant for the user and their running. Rather than a deluge of data hitting you in the face immediately after the run you get a more human response of having the run reflected back using words in addition to numbers. It facilitates better remembering and greater confidence in their running. It means you don’t need a maths degree to understand whether you’re making progress or not.
But natural language and data stories are difficult tasks. There’s a fine balance between phrases that we generate declaratively (e.g. the run was at a certain time of day so pick one of these phrases) or that need to be created imperatively (e.g. the run is at a certain cycle within training and it needs to be clearly communicated to the user why they’ve got this specific task to complete).
In early testing we generated all of the meaning declaratively. But the feedback, and our own experience using the product, was that it led to it feeling repetitive and wooly. It was a neat trick the first few times it was used but after more regular interaction people started losing confidence in the runs they were being set because they could see the repetition. As Jakob Nielsen might say, the stories weren’t doing a good enough job of giving visibility to the system status for our users. But the way the stories resonated did allow us to take confidence in them as a central design pattern for the product.
Avoiding adding to digital noise
For pragmatic reasons around interoperability — to get something released — the alpha version of Byrd and the two beta versions all had a refresh button in the top-right corner of the homepage. It allowed the user to sync their runs with Byrd, from other platforms, and saved us some pain around asynchronous processing.
It seems like a small detail, but that refresh button fundamentally changed how we had hoped people would experience Byrd. In hindsight it’s obvious that the user’s relationship with the product would be distorted if they had to manually open the app, tap a button, wait and then get a result. But it was fascinating just how distorted it became.
Golden Krishna’s The Best Interface is no interface came out almost five years ago now, but it still stands up as an interaction design masterpiece advocating for fewer, more rewarding, digital experiences. Our ambition with Byrd is that it sits in the background. To borrow words from Bill Buxton we want to build the right tool, that’s useful for a runner at the right time. We’re aiming for ubiety not ubiquity. That means it’s not a product demanding attention or becoming one of Jake Knapp’s infinity pools.The refresh button took it very close to the world of infinity pools with a number of users getting fixated on the refresh button and the post-run analysis it brought.
Measure twice, cut once
Rewind a year. It’s drizzling outside and I’m sitting in a café with a good friend. I regale them with the fact that we’ve done most of the hard stuff, we’ve worked out the maths — or at least much of the maths — and now it’s “just” the design work. Byrd would quickly be post-revenue and flying high. Today, I’m jealous of that overconfidence but I’m excited about everything we’ve learnt.
We do weekly demos to show our work and get feedback from the wider team and external advisors. It’s for another post, but as a team we believe strongly that it’s through authentic feedback that Byrd can become as good as we want it to be. Regardless, at one of the demos I said I felt we were at the final checkpoint of our first ultra. I can’t remember who, but someone on the call said I was wrong. In their opinion we’ve not started running yet. We’re just before the event, at the awkward meet-up point in the middle of the night where everyone’s half-asleep waiting for the bus to take them to the start line. Through 2020 we’ve been doing the training, learning unknown unknowns and reducing our blindspots. 2021 is when we finally get to push forwards across the start line.
