The Novelty Effect: An Important Factor to Be Aware of When Running A/B Tests
Originally published on www.Ben-Staples.com
Last week, I wrote an article running through a framework product managers can use to gain more confidence in the success of their feature without building the actual functionality behind it. This is an incredible game-changing concept for any Product Manager looking to move faster and get learnings more effectively.
To learn more about this way of thinking more broadly, a few people have recommended The Lean Startup as a great book that walks you through the concept of experimentation and ways you can apply it to building anything.
In discussing last week’s article, Stephen Delaney brought up a great point; while this approach relies on light weight experimentation to reduce cost and increase learnings, there are distinct risk factors with this approach, especially around the Novelty Effect.
The Novelty Effect
The novelty effect is the idea that the people who generally interact with your website or service may notice what you’re testing, and specifically interact with the feature just…
