Adventures in Centralized Prototyping
„Centralized prototyping?”, I hear you ask, and I see your raised eyebrow. Yes, I agree, every team should build their own prototypes and collect regular user feedback. At XING, however, we were preparing a broader, more holistic change and needed to test a new product architecture that would affect user journeys across a whole variety of our products. A centralized approach was in order.
XING being a social network where every user’s experience highly depends on their individual data (such as their contacts, messages and activities), clickdummies alone just wouldn’t do the trick. We needed live-data prototypes to get the necessary answers to proceed, so we fully committed to user-centric development and put an engineering team on the topic.
I was lucky enough to spend the last 9 months in the very special role of product owner to our great Prototyping Team. Here I want to share some of the main lessons we learned along the journey (some of which the hard way)…
#1 Be Pragmatic
Speed is your biggest asset. Prove your value by optimizing for fast turnaround times.
On paper, we all agree that prototyping is important. When it comes to dedicating an entire engineering team to the topic full-time, however, this belief is put to a real test across the organization. It’s a big investment, so you’ll want to deliver actionable results as quickly as possible and prove your team’s worth. If you’re lucky enough to collaborate with user research colleagues who conduct the tests and analyze the results, they are your main partners in crime when it comes to optimizing and speeding up the overall process.
This speed goes hand in hand with a certain pragmatism: In prototyping for qualitative user testing, you have to rely on smoke and mirrors, cut corners where needed and boldly ignore edge cases. And from time to time, you just need to cut through the requirements noise and show a little bit of a “screw it, we’re testing it like this” attitude. Speaking of which…
#2 Preach the Mindset
People need to understand prototyping — and you should be the one to tirelessly champion the topic.
Everyone involved — from the engineers and the teams that you test with to the management — needs to know what prototype testing can and cannot do. And even if everybody technically knows, this info gets lost in the promises of a shiny prototype and quick and early user feedback all too easily. Teach people. Evangelize. Repeat yourself, often.
Work together with your research colleagues to advocate for a “quick and dirty” prototyping mindset around throwaway code. Clearly point out the boundaries of this type of testing and manage expectations as to which questions prototype testing can help with.
#3 Commit, Don’t Stray
Avoid spreading yourselves thin to try and please everyone.
Qualitative user testing, like any methodology, has its limitations. There will always be people who will doubt the insights you’re generating (“But it’s just 8 users! The interview situation is so artificial!”, etc.). In an effort to make our results more reliable and convincing, some bigger, longer studies were planned. Instead of conducting interviews, we would ask a couple of hundred users to integrate our prototype into their routines, using it on their own phones for several weeks. This was accompanied by regular online surveys and behavioral data.
Because there was no moderator to navigate and explain away bugs and missing features, the prototype had to become more real and stable than the smoke-and-mirrors one we’d used before. We put a lot of work into it and ended up maintaining two different prototypes for months, having to adapt both when there was a design or copy change.
While in theory a good idea (the more insights the better!), the cost/benefit ratio was off: The tests with the new prototype verified what we had heard with a larger sample, but didn’t warrant the work we put in. They were a huge amount of effort for everyone involved — and unfortunately didn’t go a long way in convincing the skeptics. It turns out: The people who don’t believe in the insights generated from 8 user interviews likely won’t be convinced until there are full-blown A/B tests with large sample sizes.
I believe that, instead of trying too hard to get absolutely everyone on board, it’s better to double down, concentrate on your “core business” and stick to the methods and tools you chose to focus on.
#4 Stay on Strategy Level
Focus on the big picture and don’t get lost in the details.
This also falls in the pragmatism category, but it’s important enough that it bears repeating: You cannot test everything.
Round up the main stakeholders to agree on a clear prioritization of the main hypotheses and research questions that need testing. Remind everyone involved that you’re trying to figure out the big stuff — now is not the time for the nitpicky details. Can it be tested in a clickdummy? Then it should not take up your engineering team’s time. Is it a “which design works best” question you cannot answer until it’s part of an A/B test? Don’t bother implementing it, the results won’t help you.
#5 Make Sure You Have Autonomy
Get all the right people on the team to be able to act independently.
This point is important in any team, but all the more so in prototyping. Your team should ideally have the autonomy and mandate to:
- prioritize the broad research topics,
- conceptualize the features,
- design the UI,
- and implement the prototype.
With any one bullet point you take out of this equation, you increase dependencies, create bottlenecks and decrease speed (Remember? Your biggest asset?).
I’m a product manager and still a researcher at heart and I will always, always believe in user testing and prototyping. I do believe every team should be able to prototype and quickly test their own product, but I also see a lot of value in the centralized approach: zooming out, looking at the bigger picture and connecting the dots.
Raised eyebrow or not: A central Prototyping Team can add a lot of value to your organization if you’re willing and able to lay the groundwork: set your team up to be autonomous, practice pragmatism, don’t spread yourselves too thin, stay on a strategic level — and keep preaching the prototyping gospel…
