How to Science 2: Making Hypotheses and Testing Them

How we know what we know about the natural world

In our last How to Science post, we talked about forming good research questions and applying that to Experiment proposals. In this post, we’ll zoom out of the scientific process and enter the realm of hypotheses, statistics, reasoning and assumptions.

Whether you strive to become a scientist or not, it’s important to know how scientists come to the conclusions they do, so you can understand the effects of their products. How does a scientist decide what information is true?

To answer this question, we have to understand hypotheses.

Hypotheses

Once you’ve solidified your research question, you should try to develop hypotheses. Hypotheses are potential answers to your research question, or parts of your research question that are supported by prior knowledge. This is why a lot of people refer to hypotheses as ‘educated guesses’, meaning you use information to predict what your study might find.

In science, it’s important to not swap this term with the term ‘theory.’ While hypotheses are based on prior knowledge — ideas that have already been supported with evidence — we don’t know whether they’re true yet. Meaning, we don’t know whether our idea will match what happens in experiments, or in nature.

Theories, on the other hand, are substantiated with evidence, reproducible, fact based, and repeatedly offer reliable explanations of the natural world. Even if you find a lot of evidence in support of some hypothesis, it still doesn’t make it a theory. It’s a very hard club to get into!

Hypotheses are formed before you start gathering data, but they should be supported with data from previous studies or observations. Science is a living thing that builds upon itself, revises, and renews constantly. If your research question is a person, existing data make up it’s family tree. And a family history is crucial to making the best predictions and designing the best experiments!

Perhaps most importantly, hypotheses need to be testable, and so, they can’t be questions of what’s moral or ethical. To form one you need to consider how you’ll approach the research question, and what factors (called variables) you are comparing or monitoring.

Post-testing, it doesn’t matter whether your hypothesis is supported by the evidence you gather or not. Even a refuted hypothesis can provide valuable information about a topic and define a reproducible experiment for other scientists to work from.

Types of Hypotheses

Null hypotheses assume that two variables are not related in any significant way.

Ex. There is no difference in the speed of plant growth when plants are grown with or without fertilizer.

It’s customary to make this hypothesis before the alternative hypothesis, because it allows for a lot less ambiguity in the researchers expected results. The researcher seeks to nullify, provide evidence against, or disprove the null before moving forward.

Alternative hypotheses assume that two variables are related in a significant way. They’re setup to establish a conclusion through testing.

Ex. Plants grow more quickly when fertilizer is used than when it’s not used.

These hypotheses assume, after the null hypothesis is rejected, that there is some significant relationship between two variables.

xkcd

How to test hypotheses

The point of testing is to gather evidence that can help you decide if your hypothesis is true. And we test through experiments!

This step can get a bit tricky, so let’s first look at an example and then come up with some rules for good experiment making. Here’s our sample hypothesis:

Plants grow more quickly when fertilizer is used than when it’s not used.

You can start to test your hypothesis by getting two, near identical plants and growing one with fertilizer and one without fertilizer, but you’ll also have to make sure the test is reproducible and can provide the necessary data.

The first thing to consider is that the conditions the plants are growing in are identical (or at least as close as you can get), and that the plants themselves are of equal health. If one plant has more sun and fertilizer than the other, you won’t know what caused your results. If one plant has a leaf eating parasite, then it might’ve been doomed regardless of fertilizer levels.

Secondly, if you use the same species of plant then your results will be limited to that species. To make the claim that the above hypothesis is correct (to apply your data to all plants), you’d have to show that the fertilizer influences a trait that is ubiquitous in all plants, like the Calvin Cycle.

Thirdly, since not all fertilizers contain the same ingredients in the same ratios, it’ll be best to pinpoint an active ingredient in all plant fertilizers that you can suggest influences growth (as opposed to all the other ingredients). Testing multiple fertilizers would yield an even more significant result!

Then, of course, you’re going to need some way to keep track of changes to your variables. There are two kinds of variables in most experiments, independent and dependent.

Independent variables are factors a scientist modifies during an experiment (fertilizer, in this case), and dependent variables change because you’ve altered the independent variable (plant growth). They literally ‘depend’ on their independent half in order to change.

Decide on the unit you’ll be using to track changes in your variables, and then plot the relationship between them. If your data doesn’t confirm your hypothesis or you make a mistake in testing, you must record it! Research isn’t about getting everything right, it’s about exploring. Only good things can come from a refuted hypothesis.

Based on this example, we can come up with some general features that are crucial to a good experiment.

1. You need something to compare your tests to. In science, this is called a control.
2. If there are multiple variables involved, and you’re only comparing two of them, keep the others the same.
3. You need to be honest about the limitations of your experiment and your data
4. You need to keep track of your data and any errors in experimentation.

Most examples you see of experimentation and hypotheses tend to be very centered in life sciences, but physical scientists (like geologists, and physicists) use this method, too!

Physics asks especially big questions about the entire natural world. While biologists might look at a tree and chemical reactions, microbial activity, transpiration, and a species names — things that distinguish the tree from its surroundings, a physicist might notice the things that apply to all matter in the universe — like forces.

Since most things in physics can be described mathematically, the subject relies on a lot of deduction (we’ll learn what that is further down) and can come to conclusions that are logical necessities. Procedurally, the process of testing a physics hypothesis can be complex and often don’t make the best examples, but physicists still rely on making predictions, seeing what those predictions would lead to, and then comparing that result with nature and experiments.

Evaluating your results

The superstar term of evaluating results is statistical significance, which uses mathematics to assess how valid the results of an experiment are. If you perform an experiment multiple times and the data you collect shows roughly the same pattern, your results are more statistically significant than if multiple experiments have completely different patterns.

In addition to all this formal stuff, we can take a few steps back and answer the simple question: does it work? Do your results match what’s happening in the natural world? Is it effective in application? While you can’t coast on this alone, good science should withstand nature’s scrutiny.

As any scientist will eagerly tell you, science relies on evidence (which we’ve collected in the testing phase) and reasoning (how we interpret the results of our test). But, what is reasoning? Loosely, it’s thinking about things logically and rationally. But, those definitions only lead to more questions about what logic and rationality is, and who decides what it is. A tempting answer to these questions is: ‘you’ll know it when you see it’, but a more effective way to explain reasoning is by exploring it’s many forms.

Reasoning method #1: Inductive

A lot of scientific research is based on this method! It’s all about gathering evidence to support a likely conclusion. However, the conclusions a lot of research relies on aren’t logical certainties, just extreme likelihoods. This method is based on overwhelming evidence. Present what you’ve found in your study, and then tell us what it could mean.

This doesn’t mean that it’s impossible for science to come to conclusions about anything (you can do this mathematically, or by gathering observable things in nature), but most research relies on this method in the initial stages. That’s not necessarily a liability, however, as evidence is crucial to arriving at honest answers. The most important thing about this method is that it’s dynamic — constantly changing. New or contradictory evidence always has the potential to come along, and may cause your conclusions to change.

Reasoning method #2: Deductive

Mathematicians rely heavily on this method to find out what is true in their fields, and scientists use it, too! This method of reasoning starts with assertions — statements that say something is true. Then, from those assertions, you can come to logical conclusions that are guaranteed true.

For example, if you assert that x = 10 and y = 20, then x + y = 30. Any other answer is illogical.

Similarly, if you assert that all people named John wear blue pants, and you meet John Legend at one of his concerts, he has to be wearing blue pants. This example shows why it can be hard for some natural sciences to use this method. In nature, John Legend might not want to wear blue pants, so it’s hard to come up with assertions that would make deduction possible.

Reasoning method #3: Abductive

This is a kind of reasoning science doesn’t rely on too heavily. Instead, it’s the process most of us use to come to conclusions in our everyday lives. The problem, and the reason I’m including this method, is that the observations used in abductive reasoning are incomplete, while the conclusions are typically stated definitively. But, this method isn’t all bad!

Medical diagnoses and judicial cases rely on this kind of reasoning, and thinking this way can throw imagination into the picture, which can be useful for scientists, too.

Coming to conclusions

Conclusions usually begin with an equally pretentious and informative introduction, “The data suggest…”

This is because your conclusions need to be based on the data you collect! What relationships did you find within variables? What can then be said about your test subjects?

It’s important to not make conclusions that are so broad that they’re no longer supported by your evidence. For example, if you’ve found that your plants grow better with fertilizer, it might be best to hold off on saying that fertilizer could help all living things grow. Keep your statements honest. There’s no pressure to cure diseases or solve the world’s issues — every little bit of reproducible research helps!

There we have it! A summary of how scientists find truth in the world. There are certainly many ways people decide what to accept as true, and they’re usually based on some information that person has stored in their brain or some method that they’ve internalized. Although I am slightly biased, I like to think science is special because of its vigor. Regardless of the results, science strives to provide as complete information as possible, report data honestly, verify their results with nature, all the while remaining flexible enough to adapt to new information. And I think that idea can be far more valuable than memorizing a list of science’s products.