While the topic of qualitative user research and the necessity of talking to your customers is tackled a lot lately, using quantitative data and experimentation often falls a bit short. We want to change this, and with this collection, advocate how metrics can help you make informed decisions, and increase understanding of your product’s performance and customer behaviour. Running the right experiments can be crucial to the success of your product, save time and arguments, help you to make data-driven decisions, and most importantly make you learn constantly.
We’ve collected 9 great books that will help you increase your knowledge in working with data and show you techniques for experimenting.
Trustworthy Online Controlled Experiments
A Practical Guide to A/B Testing
by Ron Kohavi, Diane Tang and Ya Xu
Getting numbers is easy; getting numbers you can trust is hard. This practical guide by experimentation leaders at Google, LinkedIn, and Microsoft will teach you how to accelerate innovation using trustworthy online controlled experiments, or A/B tests.
Based on practical experiences at companies that each run more than 20,000 controlled experiments a year, the authors share examples, pitfalls, and advice for students and industry professionals getting started with experiments, plus deeper dives into advanced topics for practitioners who want to improve the way they make data-driven decisions.
289 pages, Cambridge University Press 2020
The Surprising Power of Business Experiments
by Stefan H. Thomke
When it comes to improving customer experiences, trying out new business models, or developing new products, even the most experienced managers often get it wrong. They discover that intuition, experience, and big data alone don’t work. What does? Running disciplined business experiments. And what if companies roll out new products or introduce new customer experiences without running these experiments? They fly blind.
“Experimentation Works is required reading for all leaders who want to know how companies like Amazon, Google, and Netflix win through fast-cycle scientific experiments―and how it can be done in their organizations. Making business decisions without experimentation is like bungee jumping without testing the cord. Yet far too many organizations still do this. With Thomke’s clear explanations and examples, yours won’t be one of them.” — Scott Cook, cofounder and Chairman of the Executive Committee, Intuit
288 pages, Harvard Business Review Press 2020
The Power of Experiments
Decision Making in a Data-Driven World
by Michael Luca and Max H. Bazerman
Have you logged into Facebook recently? Searched for something on Google? Chosen a movie on Netflix? If so, you’ve probably been an unwitting participant in a variety of experiments―also known as randomized controlled trials―designed to test the impact of different online experiences. Once an esoteric tool for academic research, the randomized controlled trial has gone mainstream. No tech company worth its salt (or its share price) would dare make major changes to its platform without first running experiments to understand how they would influence user behavior. In this book, Michael Luca and Max Bazerman explain the importance of experiments for decision making in a data-driven world.
232 pages, The MIT Press 2020
The Lean Startup
How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses
by Eric Ries
Most startups fail. But many of those failures are preventable. The Lean Startup turned into a classic and must-read book when it comes to changing the way companies are built and new products are launched.
The Lean Startup approach fosters companies that are both more capital efficient and that leverage human creativity more effectively. Inspired by lessons from lean manufacturing, it relies on “validated learning,” rapid scientific experimentation, as well as a number of counter-intuitive practices that shorten product development cycles, measure actual progress without resorting to vanity metrics, and learn what customers really want. It enables a company to shift directions with agility, altering plans inch by inch, minute by minute.
Rather than wasting time creating elaborate business plans, The Lean Startup offers entrepreneurs — in companies of all sizes — a way to test their vision continuously, to adapt and adjust before it’s too late. Ries provides a scientific approach to creating and managing successful startups in an age when companies need to innovate more than ever.
338 pages, Currency 2011
Use Data to Build a Better Startup Faster
by Benjamin Yoskovitz and Alistair Croll
Marc Andreesen once said that “markets that don’t exist don’t care how smart you are.” Whether you’re a startup founder trying to disrupt an industry, or an intrapreneur trying to provoke change from within, your biggest risk is building something nobody wants.
Lean Analytics can help. By measuring and analyzing as you grow, you can validate whether a problem is real, find the right customers, and decide what to build, how to monetize it, and how to spread the word. Focusing on the One Metric That Matters to your business right now gives you the focus you need to move ahead — and the discipline to know when to change course.
440 pages, O’Reilly Media 2013
Designing with Data
Improving the User Experience with A/B Testing
by Rochelle King, Elizabeth F Churchill and Caitlin Tan
On the surface, design practices and data science may not seem like obvious partners. But these disciplines actually work toward the same goal, helping designers and product managers understand users so they can craft elegant digital experiences. While data can enhance design, design can bring deeper meaning to data.
This practical guide shows you how to conduct data-driven A/B testing for making design decisions on everything from small tweaks to large-scale UX concepts. Complete with real-world examples, this book shows you how to make data-driven design part of your product design workflow.
369 pages, O’Reilly Media 2017
Data Science for Business
What You Need to Know about Data Mining and Data-Analytic Thinking
by Foster Provost & Tom Fawcett
Written by renowned data science experts Foster Provost and Tom Fawcett, Data Science for Business introduces the fundamental principles of data science and walks you through the “data-analytic thinking” necessary for extracting useful knowledge and business value from the data you collect. This guide also helps you understand the many data-mining techniques in use today.
“A must-read resource for anyone who is serious about embracing the opportunity of big data.”
— Craig Vaughan, Global Vice President at SAP
414 pages, O’Reilly Media 2013
Harnessing Data and AI to Reinvent Customer Engagement
by Tom Chavez, Chris O’Hara and Vivek Vaidya
A tectonic shift in the practice of marketing is underway. Digital technology, social media, and e-commerce have radically changed the way consumers access information, order products, and shop for services. Using the latest technologies―cloud, mobile, social, internet of things (IoT), and artificial intelligence (AI)―we have more data about consumers and their needs, wants, and affinities than ever before.
You’ll discover the three principles for building a successful data strategy and the five sources of data-driven power. You’ll see how top companies put these data-driven strategies into action: how Pandora used second- and third-hand data to learn more about its listeners; how Georgia-Pacific moved from scarcity to abundance in the data sphere; and how Dunkin’ Brands leveraged CRM data as a force multiplier for customer engagement. And if you’re wondering what the future holds, you’ll receive seven forecasts to better prepare you for what may come next.
256 pages, McGraw-Hill Education 2018
The Power to Predict Who Will Click, Buy, Lie, or Die
by Eric Siegel
An introduction for everyone. In this rich, fascinating — surprisingly accessible — introduction, leading expert Eric Siegel reveals how predictive analytics (aka machine learning) works, and how it affects everyone every day. Rather than a “how to” for hands-on techies, the book serves lay readers and experts alike by covering new case studies and the latest state-of-the-art techniques.
Prediction is booming. It reinvents industries and runs the world. Companies, governments, law enforcement, hospitals, and universities are seizing upon the power. These institutions predict whether you’re going to click, buy, lie, or die.
356 pages, Wiley 2016