Iterating towards a fruitless goal
I recently attended a Product & UX workshop where the focus was building experiences your customers will love. One key talking point was the concept of performing multiple, small discovery projects every week.
Think 10+ a week for small teams.
For me, this was a compelling, if not slightly idealistic way of working.
In principle it’s clear how that principle can help suss out the wheat from the chaff; to have a much better, maybe even proven understanding of a change before you spend weeks or months building a production-ready solution.
However, even discovery tests can take time to build and to see conclusive results. Running 10+ discovery tests a week felt almost all consuming in its need for attention – constantly creating tests, qualifying results and making decisions on what to progress. whilst drawing up new tests to run. And in a startup that potentially sees few visitors to their site (and fewer converters), how do you quickly gather valuable data?
But the wider risk is that you’re so engrossed in small discovery projects that you miss the bigger picture.
What if you’re iterating a feature that’s not desirable?
What if you’re optimising towards a product that doesn’t fully take advantage of your target market?
Missing the need to pivot
One of the most successful examples of a pivot is Instagram. Starting out as Burbn, their app focused on check-ins, with photos only a small part of their product. Noticing the app had too much clutter and too many options, they pivoted their core model and built a version of the app that focused solely on photography – a decision that eventually led to their $1bn acquisition by Facebook.
Had they focused purely on optimising their check-in app, they’d have iterated towards a goal that was ultimately likely to be far less fruitful.
By understanding their core direction was wrong and pivoting towards a more valuable goal they found huge success; success that a/b testing in isolation would unlikely have discovered.
Questioning the bigger picture
I’m certainly not arguing that testing isn’t valuable – I’ve seen many examples where a small test has seen a much larger impact than anyone expected. However I do argue that narrow tests aimed at optimising existing journeys must be paired with far broader tests that continually question and validate the core direction of a business.
Delivering optimal value for the customer is incredibly difficult, and whilst going completely outside of the box you operate in can be scary, it can often lead to the biggest step changes in the success of businesses.