Hypothesis Testing: Save Time and Money by Test Driving Your Ideas
Have you ever encountered someone in your organization who always seems to have an unwavering opinion on how things should be done? They confidently rely on their past experiences, citing “that very specific situation in a company that doesn’t resemble yours at all.” While these anecdotes might sound convincing, they can actually be detrimental to your business.
Now, don’t get me wrong, experience is undoubtedly valuable when making important decisions. The challenge arises when we assume that expertise in one area automatically translates to making sound choices in another. This is where hypothesis testing comes into the picture.
Intuition and anecdotes can serve as the foundation of a hypothesis, but without data to substantiate them, they remain merely ideas. Hypothesis testing provides the evidence needed to support or reject an idea based on data from a sample of a population (1). The value of hypothesis testing is that it allows a business to see what will happen on a small scale before it does happen at a, potentially costly, large scale. Think of it as a test drive of that idea.
Imagine you visit a car dealership with a specific car in mind and the hypothesis that you will like it and want to purchase it. You have the opportunity to get behind the wheel and take the car for a spin around town. As you drive, you pay attention to key aspects that matter to you: How does the car handle? Is the seat comfortable? Does the car have assisted driving features to make that long drive with the kids easier? During this test drive, you’re essentially collecting data points based on your first-hand experience to determine if the car is ultimately something you want to buy.
It’s important to form a hypothesis so that it is easily falsifiable. In the car situation, the hypothesis is “I am going to want to buy this car.” Then, the data collected from the test drive allows you to accept or reject that hypothesis. One important consideration, however, is to make sure to not just finish the test with a pass/fail mentality (2). Maybe you decided the car wasn’t for you. Why did you decide that? What about the car didn’t you like?
Sometimes a failed hypothesis test is even more helpful than a successful test when you ask why. Maybe you didn’t like that the car couldn’t get to speed quickly when you got on the freeway and this was something that you never thought of. You can now use this information to research more and form a new hypothesis to test.
Something to remember in all of this is that the accuracy of the results of your test depends a lot on the design of the test and the quality of the data that you collect (3). Imagine that you had never tried driving on the freeway. Maybe you decided to get the car and then regretted the decision as soon as you drove to work. It’s important to make sure that the test you create will get valuable data to make a good decision.
Hypothesis testing is a powerful tool for drawing meaningful conclusions and making informed decisions based on data analysis. Tests need to be thoughtful in what data they collect and always ask “why” results were found. A smart approach to hypothesis testing allows businesses to see the results of a decision before implementing it at a large scale, saving time and money in the process.
Sources:
- “Hypothesis Testing: Putting Assumptions to the Test.” Harvard Business School. Available at: https://online.hbs.edu/blog/post/hypothesis-testing
- “Hypothesis Testing for Entrepreneurs.” The Startup Advantage, Oregon State University Blog. Available at: https://blogs.oregonstate.edu/thestartupadvantage/2014/10/14/hypothesis-testing-entrepreneurs/
- “Hypothesis Testing (Significance Testing).” Investopedia. Available at: https://www.investopedia.com/terms/h/hypothesistesting.asp
