Lesson #2 | The Citizen’s Guide to Research

Credibility, shmedibility

When I was teaching English 101, my students had to write a research paper on the topic of their choosing. As part of this assignment, they had to have a minimum of ten “reliable and credible sources.” There was immediate panic. Does the local newspaper count as a “credible” source? Does an article in a medical journal? Does a blog? What’s the difference, and how will I know if a source fits the criteria?

But after the research project was complete, one of my students came up to me and said: “I feel like I’ll never be able to read anything ever again without questioning where the information came from.”

Success! Welcome to the wonderful, frustrating, but ultimately satisfying, world of research.

Last week, we talked about looking for the source in an article that claims science “proved” something. But what happens next once you’ve found a source? How can you tell if the research has been credibly produced?

We’ll discuss credibility more over the coming weeks. Today, let’s talk specifically about published research articles found in research journals. This refers to the actual source of the data, where the researcher(s) share:

  • An overview of their study, called the abstract;
  • The purpose of the study and their research questions;
  • A review of existing research on the topic, called the literature review;
  • The steps and process of the study, called the methods;
  • The findings of the study, called the results;
  • A discussion about the findings and further implications of research;
  • and the references cited throughout.

If you remember last week, we found a research article referenced in a Mic.com article about the potential weight-loss capabilities of pasta. (Here’s the article. You’ll have to read it to find out if that claim is accurate. ;) )

Here are three basic questions you can ask yourself when you find a research article to help evaluate its credibility:

Question #1: Where is the research published?

The first step is to see where the research article was published. Not all journals have the word “journal,” in the title, but many do. See what you can find from the journal’s main website. Is it connected to a university or research institute? Is it part of an industry? Is it an open access journal? (Hint: if you don’t know what that means, you’ll find out next week!)

Question #2: Is the journal “peer-reviewed” or “refereed”?

Peer-review means that other researchers in that field review a research article before it is published in the journal. This ensures that the process and findings make sense, and the researchers followed the scientific method and proper research ethics. By refereeing research like this, it helps to hold researchers accountable for their results and their methodology.

If you’re a student or community member with access to a library, an easy way to see if a journal or publication is peer-reviewed is to use this website:http://ulrichsweb.serialssolutions.com/login. You can search for a journal title, and it will tell you if that journal is refereed. If you don’t have access to that website, you can look in the journal itself, or on the journal’s website. (Or, ask a librarian! A good tip for most things in life.)

Keep in mind that not every single article in a peer-reviewed journal may have actually been peer-reviewed; sometimes journals publish book reviews or opinions that don’t require a committee review process.

Question #3: Who conducted the research?

The names of the authors should be easily accessible, even if the article itself isn’t. Can you find out where the authors are affiliated (for instance, their university of employment)? If there isn’t any author data, what else can you find out about the people behind the study?

Optional question: Is the raw data available?

This isn’t a deal-breaker, but for many researchers, being able to see the original data (such as the notes from a group study, spreadsheets with numbers, etc.) helps when determining accuracy and replicability in the findings. Replicability means that other researchers in the field could follow the same research process as outlined in the article, and hopefully find similar results to the original study.

*I want to clarify one thing: if you find articles that aren’t in peer-reviewed, academic journals, it’s OK! It doesn’t mean that information doesn’t matter or that it isn’t good research. However, you will want to take into account potential biases or problems with the research methodology that haven’t been addressed by the greater research community.*

Step 1:
We’re going to repeat last week’s activity. Find an article in a magazine or newspaper that references a “study,” and see if the article links to the actual study.

Step 2:
Using the three questions above, see what you can find out to determine if the study is “credible.”

Bonus activity:
I’m willing to bet you’ve heard a lot about “fake news” lately. It may have even been one of your reasons to subscribe to this newsletter! If you’re concerned about this trend, I highly recommend reading these two articles:

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.