Factitious News Game Helps Students Spot Pandemic Misinformation
Students have played the popular news game nearly one million times in classrooms around the country
The Factitious news game has challenged hundreds of thousands of students to correctly identify real and fake online articles since 2017. And as of 4/04/20, nearly one million games have been played in classrooms around the country (975,454 and rising, see graph down below).
During February of this year, as the early danger signs of the pandemic erupted in Italy, Factitious Co-designer Maggie Farley and I researched and fact-checked 9 new articles related to Covid-19 pandemic (3 real, 6 fake) and “inserted” them into the game on 3/11/20. Over the past three weeks, the collection of new articles has been played nearly 14K times (an average of 1,663 time per article).
Students who viewed a hint that shows the source of the article (the site’s url) correctly identified fake articles 80.8% of the time. Students who viewed only the article performed somewhat better than chance (57.8% vs. 50%).
We’ll keep collecting data on these articles through the rest of the semester (teachers are still running the game in their virtual class lessons). Our data analysis will also explore several new questions that have emerged.
Background
This is the fifth article in the continuing story of the Factitious news game. The others can be found here, here, here, and here (or at the end of the article).
The Factitious game, originally designed by Bob Hone and Maggie Farley[i] and updated with the expertise of Chas Brown and Joyce Rice in 2018, was released by the American University Game Lab on July 3rd, 2017. The game took off like a rocket with more than 100K games played in the first 36 hours. After this initial surge subsided, a second phase emerged as thousands of teachers in classrooms around the United States adopted it for their media literacy lessons.
Benefits of Investigating the Source
One of Maggie’s and my educational goals for the game was to encourage players to examine the source of online articles as a measure of the story’s veracity. Each article includes an optional “show source” button that provides this clue.
We have analyzed the benefit of this hint by comparing the success rates of players who view the hint with those who don’t. These success rates are shown in the table below (along with the article headlines and sources). The data was collected from games played during school hours, Monday-Friday, from 3/11/20 to 4/3/20[ii]. These articles are still in the active game, so I guess a spoiler alert is warranted (here’s the link to the game).
Somewhat Nerdy Stat Stuff
The Factitious game is powered by the Factitious Game Engine, which presents a collection of articles and then anonymously collects data on the player’s performance on each article. Two important pieces of binary data — right/wrong and hint/no hint — produce a useful table for analyzing situations like these:
Focusing on just those students who view a hint, we can see their success is nearly 80%; whereas the success for students who don’t see the hint is only somewhat better than chance. Because of the binary real/fake decision point in the game, we were able to use a simple N-1 Chi Square test to determine statistical significance with the p-value shown.
The table shows the aggregate of student responses to all six fake articles in the game from 3/11/20 to 4/4/20 (M-F, 8am-4pm, 3 US time zones). We conducted this analysis on each article separately to create the value shown in the earlier table.
Part of strong correlation between viewing the hint and success could literally be a selection bias (kids who click hint buttons do better) and another part could be a learned behavior of seeking more information (which achieves our educational goal for the game). We’re continuing to examine the data to gain some insight on these statistical contributions.
Next Steps
We’ve just released the Factitious: Pandemic Edition — a special version of the game devoted to surprisingly real and deceptively fake/false articles about the COVID-19 pandemic. We created this to help teachers who are looking for engaging and educational virtual classroom activities that can help stem the rising tide of misinformation about the pandemic. We will be posting a suggested lesson plan for a team-based activity in the coming weeks.
We’re also fleshing out the design for a possible live, teacher-facilitated, virtual team version that’s based on a classroom team activity described in a previous Medium article: The Hidden Audience of the Factitious News Game. We hope to have an early and rough prototype of this version ready by late May.
Special Thanks
The current members of the “Factitious team,” who have been sustaining the game since 2017, have enjoyed a supportive collaboration with the American University Game Lab, which launched the original game back in 2017 and is still supporting the game at factitious.augamestudios.com. Game Lab Director Andy Phelps and Associate Dean Kristi Plahn-Gjervold have been tremendously supportive of our ongoing efforts on the surprisingly popular and informative Factitious game.
Previous Articles on Factitious
I’ve been reporting on the Factitious news game since the fall of 2018 as it became clear that teachers had incorporated our game into their lesson plans (fall traffic in 2018 was 20% higher than 2017–with no promotion). Here are the headlines and links of the previous articles.
How the Factitious News Game is Revealing the Ways People Read Online News (Sept 2018)
The Hidden Audience of the Factitious News Game (Nov 2018)
Factitious News Game Tops 1 Million Plays! (July 2019)
Factitious News Game Helps Students Spot Fake News Better (Jan 2020)
[i] The full Factitious team includes: Bob Hone (producer/co-designer), Maggie Farley (co-designer/editorial lead), Chas Brown (developer/designer), Joyce Rice (artist/game designer), Kelli Dunlap (game designer), Cherisse Datu (designer/producer), Amy Eisman (editorial consultant), and Lindsay Grace (game design guru).
[ii] The “virus” articles were distributed to four of the game’s six game levels. Teachers select which level is appropriate for their students: middle-school, high-school, or college. The varying totals indicate their choices.
