Frameworks for Rapid Piloting
After a decade of designing, testing and scaling new products, I’ve learned what it takes to deliver a successful pilot (also sometimes called a “test” or “experiment”)— one that resonates with customers enough to build an organisation’s confidence to deliver new and novel concepts to market.
A well-executed pilot requires a well-calibrated team that understands the subject-matter, market, their customers, and their executive stakeholders.
A successful pilot requires a team to strike a delicate balance, to nimbly and cost-effectively deliver a “believable” experience to customers, and quickly deliver “believable” evidence to executives. Off-kilter pilots often yield data that is too poor quality to make effective design and strategy decisions, or end up fatiguing the organisation by sucking up resources due to over-engineering. Pilots that drag on simply lose executive interest and miss the market opportunity.
After a fair bit of trial and error, I’ve found the following two frameworks useful to plan the pilot and calibrate the team.
Framework 1: The Customer Line of Believability
This requires working with a team, internal stakeholders and external partners to deliver to the threshold of “believability” to conduct an effective market pilot. Discerning what can be done with “smoke and mirrors” manually versus what must look and feel like the “real thing” depends entirely on what is being tested, timelines, resources and other factors.
Framework 2: The Executive Evidence-to-Intuition Ratio
Launching a suboptimal product can cost an organisation millions of dollars , hurt the brand and kill careers — there’s a lot at stake. Most executives rely on a blend of intuition (based on wisdom built up over years) and hard evidence when it comes time to make decisions— even in organisations that are reputed to be data-driven.
A successful pilot recognises the balance between evidence and intuition, and requires the team have a keen ability to adjust on-the-fly based on what the pilot reveals day to day. Exactly how much data and evidence is required to make the leadership comfortable with a decision to go/no-go dictates how long the pilot should run, how to report, which measurements must be taken, and who must be involved. When the ratio is too high in either direction, it’s a signal that piloting is likely not the right approach.
If you’re interested in running better pilots, and want to meet folks from the the community facing similar challenges to you, I’ll be in San Francisco on May 15th to run a Rapid Piloting Masterclass. It draws on ten years of working with fast-paced clients from my time at frog, Studio D and most recently as the Head of Service Design at Yoma Bank. Look forward to seeing you there.
Finally, if you’re interested in how other people run pilots, and are willing to share some of your own experience, please take the Rapid Piloting Survey.
