Applying the Scientific Method in Product

How experimentation and data-backed decisions can help our product deliver value.

Mike Jimenez
Mar 3 · 7 min read
Photo by National Cancer Institute on Unsplash

Our good friend John Doe has been continuously improving his product, processes, and work relationships with each iteration at ProductNation. The whole team has gained significant knowledge and skills that help deliver value, boosting morale to its highest level since John joined the company! Stakeholders and management both agree that the team has matured and are comfortable with its current performance.

As a result, the Scrum Team is more engaged with their product. Everybody seeks new opportunities to collaborate and participate in ceremonies, with John taking the lead as new ideas are discussed and presented to Stakeholders.

Developers feel that the time is right to implement the newest technology trends to keep the product competitive and their skills up to date. John spends more time talking with customers, asking them about their experience and suggestions for improvement. Stakeholders are also very active, investigating competitors and reading industry forecasts.

As with any new endeavor, there is a learning curve. Stakeholders hold back on ideas they find risky while developers take a biased stance about what would deliver more value. As Product Owner, John intervenes by asking probing questions that will allow him to craft a Sprint Goal that maximizes value:

Surprisingly, nobody knows! The team agrees to investigate and reconvene to make an informed decision, which leaves the next Sprint shorthanded with spikes and maintenance tasks. It’s time to go to the drawing board!

Science

Based on his recent experience engaging with customers, John has developed a theory. Customers usually don’t know what they want until they receive something functional to try out. With this in mind, it’s up to the team to develop a value proposition and ask customers for feedback.

John is ready to take new challenges as a Product Owner. This time he’ll need to discover what to build! He is aware that Product Management is not just about launching great products. It’s about delivering a great experience that addresses a particular need in a feasible and viable way. He will have to perform some tests to determine if the proposed idea does deliver the expected value.

Gilligan, V., Johnson, M., Moore, K., Toll, J., Villalobos, R., Porter, D., Cranston, B., … Sony Pictures Home Entertainment (Firm),. (2009). Breaking bad: The complete first season.

Merrian-Webster defines the scientific method as:

Principles and procedures for the systematic pursuit of knowledge involving the recognition and formulation of a problem, the collection of data through observation and experiment, and the formulation and testing of hypotheses

To begin experimenting, the team will have to make an assumption and elaborate a hypothesis that will then be proven or disproven via testing. The results can elaborate the idea via another hypothesis or change direction towards a different one.

Hypothesis

Merrian-Webster defines a hypothesis as:

An assumption or concession made for the sake of argument.
An interpretation of a practical situation or condition taken as the ground for action.
A tentative assumption made in order to draw out and test its logical or empirical consequences.
The antecedent clause of a conditional statement.

Jeff Gothelf and Josh Seiden elaborated a framework in Lean UX: Applying Lean Principles to Improve User Experience:

1. Declare assumptions

Start by declaring assumptions. This process will help to identify risks and prioritize them.

2. Create MVP

Create a Minimum Viable Product that addresses a customer need. This process converts the initial hypothesis into a functional product.

3. Run an experiment

Test the validity of the initial assumption via the MVP. Release it to customers and ask for feedback. Run in a controlled environment.

4. Feedback and research

Gather feedback, confirm or deny assumption, inspect and adapt, reiterate.

John champions the idea of a safe environment where the team can fail fast to gain the necessary experience to iterate towards success. In this scenario, concepts like MVPs, Prototypes, A/B tests, and Mockups are helpful to confirm or disregard hypotheses with no customer impact.

John realizes that with every prototype, mockup, or test, his theory continues to prove itself. Customers usually don’t know what they want until they receive something functional to try out; At this point is when the magic happens. Behaviors are studied, use cases arise, and feedback begins to flow.

Way to go Mr. Doe! Keep on eliciting, iterating, and delighting customers!

Our other good friend Mary Jane is also excelling at WeAreProducts. After the engineering team’s Definition of Ready update and continuous improvement policy, she started reviewing all her processes, looking for optimization opportunities.

As Product Manager, she continuously speaks with internal departments, customers, and stakeholders, which sometimes becomes overwhelming. All the information, requests, and tasks she receives aim to influence her to make product decisions.

Mary knows that, even though there might be strong opinions or “gut feelings” towards a decision, she must always keep an objective mindset that puts customers first. If customer expectations are met, by consequence, all other OKRs will do as well.

It’s time to put on your lab coat. We’re going to science the hell out of data!

Data oriented decisions

WeAreProducts requires all documentation to be available to team members and stakeholders. Mary has access to several documents and databases related to her products and services. Business-critical and customer-sensitive data is safeguarded by encryption and controlled access. Authorized personnel has limited access to certain sections only.

The engineering team proposed to store information in a data lake so that business intelligence could access and query it on demand. Mary relies on them and other departments when she needs data analysis. The majority of her requests are statistical reports about customer usage and infrastructure health.

This time Mary wants to dig a little deeper. She knows that proper analytical tools can deliver valuable insights. Customer behavior models, personalized marketing, and intelligent suggestions are defined using data. All of these can help her make appropriate product decisions.

Data science

Photo by Franki Chamaki on Unsplash

Wikipedia defines data science as:

An inter-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from many structural and unstructured data.

The OSEMN framework

Dr. Cher Han Lau describes the OSEMN framework in his post 5 Steps of a Data Science Project Lifecycle

1. Obtain Data

Gather data from relevant data sources. Mary has a data lake ready to query!

2. Scrub Data

Pre-process data to a clean format that machines understand. Data lakes are repositories for storing data in its natural “raw” form. A pattern or data format will facilitate the easy processing of data.

3. Explore Data

Find significant patterns and trends using statistical methods. Roll up those lab coat arms; it’s hypothesis formulation time! Mary can ask a scientific question starting with an assumption. Do customers with canceled subscriptions renew more when offered a promotion than when new features are launched?

4. Model Data

Construct and train models to predict and forecast. Analyze how you answered your exploratory questions. If that particular context is interesting enough to explore further, then optimize your data to generate insights.

5. Interpret Data

Put the results into good use. Actionable insights are key outcomes in prescriptive and predictive analytics. With them, Mary can now more confidently attempt to repeat good outcomes and prevent negative ones.

Justified decisions

Photo by LinkedIn Sales Navigator on Unsplash

Mary is a great Product Manager. She can speak to customers, business stakeholders, and engineers with the same ease; This allows her to navigate through the three dimensions of product management, reach agreements and maximize value delivery. With data-backed decisions and constant customer feedback, she has positioned WeAreProducts in a privileged spot. We’re proud to have you onboard Madam!

Final thoughts

John and Mary are growing with their products! They apply the same empirical practices used in development to their careers, inspecting and adapting through every situation.

John is constantly learning and evolving. In recent Sprint Reviews, stakeholders openly share their ideas and ask for opinions before making a request. Requirement elicitation and prioritization keep John busy as he fosters everybody’s collective knowledge. With all of this input, he creates Sprint Goals that help the team focus on value delivery.

Mary takes new challenges and promotes a healthy environment for the team to thrive. Her customers receive value regularly, the business has steadily grown for the past year, and engineering is satisfied with its product’s maintainability, robustness, and performance.

Don’t stop now my friends, as there will always be more to achieve. I am confident new challenges and opportunities will come shortly. Keep it up!

The Startup

Get smarter at building your thing. Join The Startup’s +786K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Mike Jimenez

Written by

Product enthusiast, agilist, continuous learner - Technology development & innovation

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +786K followers.

Mike Jimenez

Written by

Product enthusiast, agilist, continuous learner - Technology development & innovation

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +786K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store