Evaluating information sources
In this post, you will explore different types of information sources, and consider how to evaluate them so that you are selecting the most reliable, objective and relevant sources for use in your academic work.
Contents
Introduction
In this post, you will consider different types of information sources. We will introduce you to a questioning approach to inform your judgement on the reliability, relevance and objectivity of the source. You will also find out how to document your evaluations in an ongoing way using an annotated bibliography.
Understanding your purpose
The information landscape is vast and made up of a wide range of information types and sources. As part of your searching process, it can help to recognise what types of information are available.
Before you begin to search for information sources to use in your academic work, consider exactly what type of source you are looking for and how you intend to use the information you find. In other words, identify a clear purpose for your ‘information need’. This will help you to evaluate the effectiveness of a source, in relation to your purpose or the assignment you are working on.
Your ‘information need’ may be to gain a better understanding of a topic, to find specific data or a methodology that fits your purpose or it may be to identify the debate around a particular topic.
Consider the following questions:
- What type of information am I looking for?
- How will I use the information after I have made notes?
- What do I need to take notes on? (E.g. do I need to note down bibliographic information in order to reference the source later?)
Before you continue, it may be useful for you to bring to mind an upcoming academic task or assignment. Bear this task in mind as you complete each activity, considering how the aims of this assignment might affect how you evaluate the effectiveness of different sources.
What types of Information are you looking for?
Identifying the type of information that you are looking for will help you search more efficiently. There are many different types of sources that you can use in your studies, so pinpointing exactly what types of sources will help develop your understanding and argument is important.
Consider what information types you are searching for beyond just books, such as:
- Journal articles
- Websites
- Reports
- Conference proceedings
- Encyclopaedias
- Audio-visual materials
- Blogs
- Archives
- Statistics
- Patents
For further types and explanations of each, take a look at this glossary of information types.
Popular vs scholarly sources of information
Identifying whether a source is popular or scholarly can give you an indication of how authoritative it is. Scholarly literature is usually written by researchers who are experts in the subject area. In order to get their work published, there are standards researchers have to adhere to and processes designed to check the quality of a piece of work before publication.
For instance, to get an academic article published, the author must first submit a draft to an academic journal, where it will be reviewed by one or more people with similar experience in the field. This process is called peer review and it helps ensure the quality of academic sources.
If a source has been peer-reviewed, you can be reasonably confident that it is of a good standard. However, this does mean you should take everything it says at face value! Equally, if a source is ‘popular’, it doesn’t mean you can’t use it in your work — you might just want to think carefully about the purpose you are using it for.
A blog post written by a newspaper columnist, for instance, might provide useful evidence for one side of the debate around a particular topic or event. If you are aware of specific limitations in the sources you are using, then make that explicit in your analysis so the marker knows that you recognise these limitations.
For example, when including the blog written by a newspaper columnist, you could use the source to explore bias and unpack how the author came to hold that point of view.
The table below highlights some further characteristics of popular and scholarly sources:
The distinction between popular and scholarly sources of information is not absolute — some sources are more popular, some are more scholarly and some sources contain characteristics from both. The diagram below plots where some common types of source sit on the scale:
Primary, secondary and tertiary sources
The difference between primary, secondary and tertiary sources is determined by how far removed the information provided is from the subject of study.
- Information is considered a primary source when it is collected first-hand, or provides someone’s first-hand account of the subject matter. Primary sources are original and have not been previously filtered, interpreted or evaluated by other people. Archives, such as those held at the John Rylands Library, often include primary information sources like letters written by historical figures or old photographs. Other examples of primary sources are raw data, interviews, statistics, survey results, patents, financial records, and works of art.
- Secondary sources of information provide analysis, interpretation or commentary on existing, primary information or data. Examples of a secondary source would be newspaper, magazine or journal articles (those that review content), reproductions of art work, textbooks, biographies and histories.
- Tertiary sources are summaries or collections of primary and/or secondary sources. Examples of tertiary sources include dictionaries, encyclopaedias, Wikipedia, bibliographies and directories.
A lot of the sources you will come across during the course of your studies will include several of the information types we have outlined. For example, a company report might include primary data, such as survey responses and statistics, and secondary analysis of that data.
Whether a particular source is considered primary, secondary or tertiary may also change depending on what is being studied. For example, if the subject you are studying is the works of the Roman historian Tacitus, then his book about the history of the Roman Empire “The Annals” should be considered a primary source.
The Annals is a direct example of the subject of study: Tacitus’ literary works. However, if your subject of study is the lives of the Roman emperors, then Tacitus’ book “The Annals” would be considered a secondary source. In this case, Tacitus is providing a second-hand commentary on evidence he collected in the course of his research. The diagram below illustrates the idea of the proximity to the subject of study:
Questioning information sources
If you want to be able to put together strong, critical, academic work and achieve higher marks, it is important to make a habit of asking questions of your source material, rather than taking things at face value.
The following six questions are a great starting point:
- What?
- Why?
- When?
- How?
- Where?
- Who?
You can often find the answer to some of these questions straight away (in an article abstract, for example), which can help you decide whether the source in question is relevant to your needs.
Once you know whether the source in question is going to be relevant, you can start to devise further questions which will help you evaluate it.
We will now look at each of these questions in turn.
What… type of source is it?
Identifying what kind of source you have can help you in indicating the source’s quality. This is important if you are using the source to establish an argument. A high quality information source can enhance your argument or research.
Here are some questions that you could ask:
- What are the main arguments that are contained in the source?
- What is the content of the source that is relevant to your research?
- Has the information been verified in any way?
Why… was the source produced?
Just as you should pay close attention to your purpose when searching for information, you should also consider the purpose of each source you come across.
- Why was the source written?
- What was the author hoping to achieve?
Different types of source are often written with a different purpose in mind. Let’s return to some of the source types mentioned earlier in this resource:
- Conference proceedings are written to inform, giving those who couldn’t attend the conference a basic outline of what was discussed.
- Journal articles are written to inform others of new research in a certain field but they are also designed to persuade the reader of the value of the author’s research and validity of their arguments.
- Blogs are usually written to entertain readers and often feature the author’s subjective opinion on a certain subject, without necessarily including evidence for those opinions.
- Newspapers are written to inform readers about the news and current events, however, particular newspapers tend to have a political leaning which influences the way they present information. Some information within them is also to entertain.
When… was it created?
In order to decide whether a source is relevant to your purpose, you will need to gather information about when it was created. Different types of information sources are created in different ways, so the exact information you gather may differ depending on what you are looking at. Here’s what you should consider in this regard:
- When the source was published?
- When was it last updated?
- How often is it updated? (especially important for websites)
- What is the time between any events discussed and the source’s publication?
Often, your academic work will require you to demonstrate your understanding of the current debate surrounding a particular topic, so you will want to make sure that the information you use is up-to-date.
Pro tip: Academic journal articles are often a good place to look for up-to-date research on a particular topic as they take less time to publish than an academic book. Academics will usually use a journal article as a way of sharing their latest research and ideas to see what the reaction is, then they may build on these ideas in a book later on.
Who… wrote or produced the information source?
Finding out more about who wrote a particular source can help you make an assessment of how reliable it is. Consider what the author’s background is — are they well-qualified to talk about the subject? Have they written anything else? Some sources, such as academic journal articles, are more likely to make information about the author(s) readily available (often, this can be found in the article abstract). Other sources, like websites, may require you to investigate further in order to find this information.
Learning about the author of a source can also give you an indication of how objective it is.
- Does the author have a bias?
- Has the author been funded by or affiliated to an organisation?
- Who is the intended audience and does this affect its relevance?
When searching for sources to use in your academic work, you will usually want to find sources which have been produced by experts in the subject area. However, in some cases, your assignment brief might specifically ask you to comment on public opinion during a particular time period, in which case, newspapers and other primary sources might be helpful.
Where… was the source produced?
You can look at this from different perspectives. You might want to look at the geographical location of where the information source was made available. This might be relevant to your research area. You may also want to look at where the source was published and the process that the information went through to be made available.
The second perspective is to look towards where the source was published; this can support you in defining the objectivity and reliability of the source. If you are looking at a research journal article, then you might want to look if the journal is peer-reviewed. Peer review is a publishing process where, before research is published, the article will be looked at by experts in the discipline. This process is considered by some to enhance the quality of the research.
Questions you could ask are:
- What is the purpose of the source and does the purpose influence the objectivity?
- Where is the source published?
- Was the source peer-reviewed?
How… was the information produced?
When you examine how the information was produced, you are looking in more detail at the source. You want to know how the author got to their main idea that they are sharing. So you might be asking questions to establish the reliability of the source by looking towards any references that they might have provided to substantiate their ideas.
If the source is a piece of research, then look at how sound their methodology is to establish reliability as well as relevance. If the source is not a piece of research, then has it been informed by research? Can you identify that research and examine the credibility of the original work and then analyse the purpose of the secondary source?
So you may be asking questions like:
- Does the source provide references that have informed its current form?
- What methodology was used?
Reliability, objectivity, relevance
While thinking like a detective is a good way to approach evaluating the sources you read, consider how you will have those sources earn their place in informing your opinion. Studying at university, you will be expected to make judgments on the information that you give credence to, that you use to inform your thinking and that you go on to cite in your work.
You can use the previously outlined questioning approach to make an evaluation that will lead you to think about the information source from the three following perspectives. Thinking about information sources and their reliability, objectivity and relevance can help you to elaborate on the evaluation that you have made and support you in writing and talking about the information that you have identified.
Reliability: Is the evidence trustworthy? There needs to be a good reason to believe that the information presented is accurate and complete in order for a source to be considered reliable.
Objectivity: Is the evidence objective/neutral? Note that a source doesn’t need to be objective for you to use it in your work. In some cases, you may be seeking sources from a particular perspective to illustrate a point, or provide a counter-argument. However, it is important that you are aware of any bias when using a source.
Relevance: Is the evidence useful/relevant? A source can be reliable, objective and of a generally high quality, but if it’s not relevant to your work, then there’s no point in using it.
This questioning strategy handout cross-references the questioning approach with reliability, relevance and objectivity.
Annotated bibliographies
One way to capture your thoughts and words that evaluate a source effectively is by creating an annotated bibliography of everything you have read.
An annotated bibliography keeps all of the important reference data that you need and also your short summary evaluation of each information source.
The advantages to doing this are:
- You keep a record of the material that you read across the depth and breadth of your course.
- It is easy for you to locate items that you have read and that have informed your opinions and ideas.
- It will help to inform potential future research or project areas of interest.
Below are two examples of annotations that capture the three perspectives we have outlined:
Annotation example 1:
“Provides a discipline specific view of students transitioning to a nursing degree and the feelings that students have experienced. The literature review is thorough. The authors used a thematic analysis qualitative data to build a clear view of the student’s experiences. This is relevant to use with students as it shares the experiences of students at Manchester but this could limit its reliability in terms of generalising and it was funded by a grant from the University of Manchester so may not be fully objective.”
Reference: Pryjmachuk, S., McWilliams, C., Hannity, B., Ellis, J., & Griffiths, J. (2019). Transitioning to university as a nursing student: Thematic analysis of written reflections. Nurse Education Today, 74, 54–60.
Annotation example 2:
“The paper is 12 years old now and while this could compromise its reliability it forms part of a body of work that captures and describes the identities of ‘professional’ staff and not academics that work in higher education and span a range of different professions that work across boundaries. There is a sense that there is very little bias in the research and the body of work. Relevant in terms of the different areas that HE staff circulate the study is explicit that it omits information professionals and library staff.”
Reference: Whitchurch, C. (2008). Shifting Identities and Blurring Boundaries: the Emergence of Third Space Professionals in UK Higher Education. Higher Education Quarterly, 62(4), 377–396
If you use reference management software such as EndNote or Mendeley, you can easily keep track of your reading and annotations in the notes feature, which keeps everything together in your reference library.
Summary
Thinking about information in terms of reliability, objectivity and relevance can help you in writing and talking about information sources and progress your evaluation of a source from the questioning approach to making informed judgements. Recording your evaluation of information sources in an ongoing way using an annotated bibliography will support you to see what has informed your thinking over the duration of your course and serve as a reference point to revisit when writing and conducting research.
