Athletes, Prototypes, and Testing

What happens when you get your prototype in the hands of the top athletes at the Nike Football Combine?

That was the question on my mind and the minds of our team at Hudl.

We’re in the early prototyping stages of a new app and we have a lot of questions that we need answered.

So I packed up our prototype and took it to the Nike Combine at AT&T Stadium in Dallas, an event where thousands of high school football athletes come every year to get tested in 4 athletic events(40 yard dash, shuttle, poweball toss, and vertical).

After nearly 40 back-to-back user tests+interviews, I could practically predict how the next test would go.

There were 3 lessons I took away from this experience:

1. Fail Early

Failing is learning, and if you’re not learning, you’re up a creek.

Watching athlete after athlete struggle through the same parts of the prototype was painful and eye-opening. The motto for our squad(what we call our small cross-functional teams at Hudl) is “The Truth Hurts”, and this is very apparent when you test a prototype.

Thankfully, we were able to notice these parts of the user experience before any code was actually written. It would have been a shame to commit dozens of design/development hours to this part of our product only to have to rip it all out later.

If you get to the end of a project and you don’t have a wastebasket full of failed versions, you likely did something wrong, you just haven’t noticed it yet.

2. Observe everything that’s not “on-screen”

One of our designers Craig Zheng, also calls this “Thinking beyond the UI”

There’s always a place for quantitative research(A/B testing, analytics, etc) but being in the environment with a user will show you things you never would have seen otherwise.

Like an athlete putting down his cleats and gear because something in the app really got his attention.

Or his reaction when he sees his score(which was faked) was lower than what he thought.

Embedding with users

See, these things are cues into the emotional journey of a user going through your product. They’ll never show up in analytics, but they’re hard to miss when you actually sit down with your user.

3. Design enough to learn from, but not so much that it becomes indispensable.

In the early stages of a product, it’s crucial to design only enough to learn from. We’re not creating Dribbble shots at this point, we’re trying to find answers to questions.

Is this actually solving the users problem?

Do they even know what this feature does?

If you pour too much of yourself into the pixels and border-radius of your designs in the early stages, you’re a lot less likely to scrap it if it fails in user testing(or you’ll be less likely to test at all). Until you have feedback that your design is actually solving a problem, think of your designs as ephemeral.

After several tests I learned that our scrubbing interface didn’t make sense to anybody(I was probably trying to be too clever). Right there on the sidelines I quickly mocked up a replacement in Sketch, made some edits to the Form prototype, and tested it on the next batch of athletes. Problem solved and solution validated.

Old(confusing) scrubbing interface. Just enough design to learn from.
A very quick iteration. Mocked up, tested, and validated on the sidelines.

So to wrap it up…

Getting in front of your users can be scary, it can be vulnerable, it can be validating, but just remember, that’s what it is supposed to be.

We do it so we can fail.

We do it so that we can learn.

We do it so that we can make a better product.

What have you learned from getting out in front of your users?