I Want To Believe
Confirmation Bias, Predatory Journals and Open Knowledge
Open knowledge encompasses free journals, video courses and other mediums in the effort to spread knowledge for free. The University of Manchester uses sites called Coursera and Futurelearn to put it’s MOOC (Massive Online Open Content) courses on. Anyone, anywhere can supposedly access these courses and hopefully gain a qualification or extend their knowledge in their chosen topic.
I want to look at the challenges presented by the Open Knowledge format for content creators in Higher Education. Specifically I’ll be looking at the pitfalls of written content in Open Knowledge and how easily those problems could be transferred over to MOOCs and other video content Universities produce for the Open sector.
There are hundreds of bad article examples on Bad Science. These examples are excellent in that they can show you how data can be misinterpreted by the media and misreported. The writer Ben Goldacre is a GP he does a TEDtalk here which is very funny. He talks about the amount of ‘amazing new cancer drugs’ that get reported because of a basic misunderstanding of how science papers should be read. How does this relate to Open Knowledge I hear you ask… Well that’s where it gets complicated.
I work on how the data is presented in videos in much the same way that Ben has to deal with ‘science communicators’ in that we understand what is watchable and how to create relatable media but we don’t specifically understand the science. When I make videos I never alter what the Academic has sent us but other creators have no such scruples. This is where there becomes a problem with Open Knowledge and it’s in how it is consumed and read. Ben Goldacre gives his examples and even includes an example of him prescribing medication where the research had been skewed. Essentially negative reviews of the drugs are removed causing a bias towards the positive reviews. To counter this Ben plots the data on a graph which shows the missing data, this is not something everyone has the skills to do. Does open knowledge teach you how to read, analyse and interpret data or does it just give you the data? Sometimes this will mean you interpret something to suit your own agenda and that’s when open knowledge becomes a problem.
Confirmation Bias
With the proliferation of ‘fake news’ in the media more and more people are succumbing to something known as confirmation bias and I would argue that Open Knowledge is equally as susceptible. Confirmation bias basically means that you read and interpret information in a way that supports your current beliefs. You’re like Mulder in the X-files, you want to believe and that is an incredibly dangerous standpoint to take when you read or watch content. An argument should convince you with facts rather than you already assuming the facts are correct.
As a member of The University of Manchester media production team I regularly create videos where I have little to no understanding of the content. To me it would make sense that these were peer reviewed like a Journal so they are ready for release. I’m sure all the MOOCs I have made have been accurate but with no filter to check it’s difficult as a production team to know about the content. When videos are Open Access and anyone can watch them even accidental misinformation could be problematic. MOOCs often have an obvious source though as they are hosted or branded by University’s.
The problem that then comes with Open Access is that there are a limited number of Journals and resources that are open to everyone. Not everything ever written on a topic is freely available for everyone all the time, there are serious limits on the amount of Open Knowledge Journals and articles that are available. The toll access model (where you pay to view content) is still much bigger than Open Access as well though there are no clear figures on how much information is behind paywalls. At the moment the DOAJ have 9,661 Journals on record but there isn’t a definitive total that can be made a percentage of. Essentially if you’re researching something and you aren’t paying for Journals you will only be able to access a small sliver of information to guide your thinking. There is no count of how many MOOCs exist so that’s also impossible to track.
As I said in my previous post a little knowledge is a dangerous thing especially when it is being interpreted by someone who isn’t trained in assessing the sources and content. This applies to all Higher Education fields not just science, though science is the easiest for me to give you examples on. What is being written often depends on the writers agenda. Has research been skewed to sell more drugs? Has bad research been hidden or glossed over? In a peer reviewed Journal this kind of thing shouldn’t happen. For example if a journalist reads a story they don’t fully understand then their misinterpretation can then spread through the internet. Confirmation bias also means you don’t necessarily believe the facts or may dismiss something because you disagree with it. A lot of Open Access articles and Journals can be found here at the DOAJ where they have been peer reviewed but you can equally find some things through google scholar and other sites. This is where it gets difficult to tell if they are reviewed or not. So there is lot of information with some filtering systems but it gets very difficult to tell a peer-reviewed articles from biased articles.
I think this is a dangerous combination of information and potentially a lot of ways to misinterpret it. I’m talking more about public access over access for people within the Higher Education community but these problems could apply to both. I’ve read quite a few articles about the psychology of analysis and apparently we don’t like having our minds changed, even by facts, which is slightly worrying when many people only read things to confirm a theory they already have. Essentially it means that people cherry pick the articles that agree with them rather than seeing if anything disagrees with them and has better evidence. I would say as well that this bears a passing resemblance to the phenomena of ‘fake news’. It’s searched for on the internet and rarely fact checked. I’m even starting to question if I’m getting my own research right because of the sheer amount of research and the lack of filters surrounding it.
Confirmation bias is described in the article above as being a an evolutionary problem that technology has surpassed. Essentially our technology is evolving faster than us and our way of believing things is still based on working in a group mentality not assessing things individually to prove them right or wrong. In the experiments described in the article above people rated themselves to be more correct even though they had been told that the information they had be given was fictitious. Which is disturbing to say the least. People also tend to interpret inconclusive or ambiguous data bias meaning they will bend it to what they think it should probably mean which is spectacularly unscientific.
So how can Open Access help to prevent confirmation bias? Well, first of all it allows anyone to check the sources within articles. Which is good, if everyone can have access to a source it means they can check the legitimacy of articles. If a Journal is behind a paywall then it’s likely to put off a standard reader from checking the source. Which would mean that you can reference something a limited number of people are able to check.
This line of thinking relies on people actually fact checking a lot of what they read which according to this great website Hoaxy just doesn’t really happen before the article is shared and spread around the web. These stories are more journalism but the principles are the same. This becomes a problem when legitimate or trusted sites share fake stories by accident. Snopes recommends checking the source and they are a fact checking website so that’s a good start but it’s even more of a problem if people only read the headline and don’t even view the content. Like Sean Spicer the former White House Press Secretary. If people can’t be bothered to check the full story will they ever check the source? Which brings me to my next point…
We don’t Read the Full Thing
’59 percent of links shared on social media have never actually been clicked: In other words, most people appear to retweet news without ever reading it.’
https://hal.inria.fr/hal-01281190
That is a quote from a study by Columbia University and the French National Institute. Ironically the full Journal article is behind a paywall. So I can’t properly check that my research is fully correct. This is the problem when some articles are paywall and others are free, I’ve read an article that says something I believe but the actual article is unreadable unless I pay (I won’t) so I can’t check the maths or the sources so is it untrue? I don’t know. The article I was originally reading was The Washington Post. Who are a reasonably reliable source(I hope). You have to think though if 59% of people share without even checking the article then what’s the point of open access for the 41% left who will read the article, will they check the facts? The answer is probably not. Some people will though and that has to be worth something.
When the production team create videos we know that they are a passive medium. Often a viewer will only truly watch and take in the minute of a video before they are distracted. Depending on a how a video is structured could potentially only show one side of a debate and give a viewer an unbalanced view on the topic. Video viewers also have a similar quirk which mean they watch a video and think they already know what’s being said it’s similar to confirmation bias but just a little bit weirder. Essentially people don’t watch the video properly because they think they already know the answer leading to them not actually learning anything. We try to avoid this but often MOOCs are long and unbroken like essays which could lead to viewers not watching them thoroughly and getting the wrong impression.
Confirmation Bias then seems to override the urge to fact check or in a lot of cases even read past the headline and if it’s a video they often won’t go past the first thirsty seconds.
Open access can fight this by providing a reliable source to trust. I would say though that there is a problem with actually identifying what is a peer reviewed journal, MOOC, not a blog or other source that is Open Access.
Predatory Journals
The practice of creating false Journal articles is not unheard of. In an experiment researchers submitted a scientific paper with ‘fatal flaws in the research’ and fake sources to open access journals which were then not checked and released by them. The experiment was by a researcher at Harvard called John Bohannon.
‘The “wonder drug paper” as he calls it, was accepted by 157 of the journals and rejected by 98. Of the 255 versions that went through the entire editing process to either acceptance or rejection, 60% did not undergo peer review. Of the 106 journals that did conduct peer review, 70% accepted the paper.’
One Journal did call out the flaws but that’s one in two-hundred and fifty-five. If you were reading the paper in a Journal and it confirmed something you were already thinking would you believe it or would you question the research? This practice can also be done maliciously by researchers trying to pass off small studies as conclusive and for a fee they can pad out their CV’s. It can also be done to sell more of your product by sponsored studies and to me this seriously brings into question the legitimacy of Open Access Journals especially for public consumption.
The Journals that publish bad science and aren’t peer reviewed are known as predatory Journals, at the last count at the end of 2016 there were nearly a thousand. Essentially they are profiteering because they ask the scientists for money to publish their research. They look respectable and seem legitimate but anyone with any kind of scientific acumen should be able to tell they are false. There’s a website with a list of predatory Journals here.
However they do add a problem into the Open Access world because they are usually free to view and if you’re like me and you’re looking for evidence and you’re not a scientist it’s easy to be grossly misled. Again this is happening within all disciplines but the research exists for science Journals because they are monitored more. There’s a list on the website above of predatory Journals that claim to be on the DOAJ but aren’t. As Keith says here who’s guarding the guards?
There doesn’t seem to have been any instances of fake MOOCs so far but there is a lot of unsubstantiated education videos on open video sources like Youtube. These are often well made but have poor sources or only offer one side to an argument which will be extremely biased. These sources are often taken at their word without proper research which echoes what is happening with Open Access Journals. I think Universities need to increase their web presence on more popular sites and prove that well rounded content is available for viewers to learn from.
Summing Up
To sum up: people don’t change their minds when you present them with facts, they don’t read whole articles and potentially the articles you’re reading are pseudo science published by predatory Journals. Great! You may be finding all this disheartening, I certainly am.
Something as useful and wide reaching as Open Knowledge should be celebrated but to assume it is flawless would be wrong, very unscientific in-fact.
The Higher Education needs to be mindful to take into account the current flaws built into the Open Knowledge system as it stands. While there are regulatory bodies for Journal content such as the DOAJ there is no such regulator for video content or MOOC courses making it difficult for potential learners to differentiate between real and fake content. I would suggest that The Higher Education sector endorses certain sites with links on from their own sites to make real content clearer. Coursera and Futurelearn are already part of this though again I think a specific branding system or hub site of official MOOC content like the DOAJ would help to put off would be fake content.
How the information we produce is consumed is nearly as important as the information itself. If Universities are producing content I think it would be good to include how information can be read and interpreted as well as the information itself. The way Open Access materials are consumed is similar for videos and written articles. The way the content is formed plays a large part in how content is consumed. I think that more vigilance on the part of the Higher Education publishers is needed but also anyone reading Open Access or indeed any kind of content needs to be more discerning. Assuming something is true is dangerous and can cause more problems later on. Education on sources and research could also combat ‘alternative facts’ and ‘fake news’ though that’s an article for another day. Often videos are supported with tests to make sure the video has been understood and I think this works really well.
I don’t think MOOCs have reached the levels of fakery that Journals have but it’s something to be wary of. There’s definitely a lot of pseudoscience in the public spectrum but little of it is pretending to be from a Higher Education source like journals are. I think more vigilance and cohesion in how Universities produce content for MOOCs could avoid some of the problems being experienced by the Academic Journals in Open Knowledge. MOOCs could potentially reach new people but if new readers can’t analyse new information in an academic way it could become counter productive.
P.S Did you check all my references?
