Why We Don’t Have Amazing Games in Schools and Universities

There have been people working on learning games for decades now and, while there is growing use in schools, they aren’t embedded into the learning activity of the majority of classrooms. This isn’t a distribution problem. This isn’t an “anti-games” sentiment problem. This is a problem with integration at scale.

If anyone has evidence that a majority of teachers are using well-designed games integrated into instruction, I am happy to consider that. I know in a recent survey 55% of the 519 K-8 teachers surveyed said they used games at least once per week. People in game development certainly see teachers who are excited to use games in classrooms. However, what I see anecdotally is that many teachers, who surely would have said yes in that survey, are sending individual kids to gaming sites that are filled with drill-and-practice games as a filler when other students are doing something else. I also see a lot of game developers of good games at conferences bemoaning the difficulties getting into schools.

For many small game makers, distribution is a key challenge. Our education system in the U.S. is so de-centralized that it is very difficult to actually address the tens of thousands of decision makers. However, I work for Pearson. We have a huge and amazing sales force ready to sell. Getting in front of buyers is not our problem. Scaling game development with curriculum integration is our problem. And I think it is a problem for educational games generally.

This post is inspired by two things. First, was a Tweeststorm by Lars Doucet (@larsiusprime) that focused on game developers working with researchers. Second was this presentation by Erin Hoffman (@gryphoness) They each come from a game developers’ perspective. Here is my perspective as someone who works in research but at a big education company.

Different Kinds of Games

There are many ways to classify learning games. One is drill-and-practice games versus conceptual learning games. Drill-and-practice games are designed to build what educators call “fluency” with things like math facts and spelling through repetition. Research shows us this fluency is necessary to make basic things automatic and free up thinking space for more complex tasks. If games can make it so the practice necessary for fluency is a little less boring and get students to spend more time on it, that is likely a good thing. There are a lot of drill-and-practice games available and they can be made reasonably inexpensively.

However, most game folks who do research on games and present/ attend the major learning games conferences are interested in games that introduce and teach concepts. They are interested in making games that build on students’ curiosity and need to discover to teach new things.

The Goal

So, what would I love to see? A game that game designers would recognize as having solid game play, integrated into a curriculum that is aligned to a research-based learning progression, in which the actions of the players also serve to tell us about what they know and can do (assessment baked in). I’ll dive a little more into two of those elements: integration and game-based assessment. I’ll save the discussion of good game design for the game designers.

Integration
There are amazing teachers out there. They scour the ecosystem for individual programs, including games, to bring into their classrooms. Some become experts at using particular games in their classrooms. However, there are 3.6 million primary and secondary teachers and 1.3 million post-secondary teachers in the U.S. Asking them all to go out and individually source games for their classrooms and then do all the work integrating them into their curricula is not sustainable. This is why I have long believed that to get games into classrooms at scale, they need to be integrated into a curriculum.

This comes in part from an experience I had with the Cisco Networking Academy. We had an amazing stand-alone computer network simulation tool called Packet Tracer. You could simulate all kinds of networks and create all kinds of network designs and troubleshooting tasks. Teachers could create their own activities. And we got amazing feedback. Our power user instructors loved it and evangelized it at conferences and to other instructors. However, our data showed that only about 25% of our 10,000 instructors actually used it in classes. In our next revision of the curriculum, we integrated the tool and the activities. Students would read, then an activity would pop up, then maybe a quiz. It was all seamless. Our next results showed that more than 85% of instructors were using it. I think the same thing is true with games.

I get approached often by companies with games. They come with differing ideas of business models. Some look at us as a distribution channel. Pearson can just sell their game too, alongside our other curricula. No, we aren’t likely to do that. We have been burned with that before. If we have games, they are going to be integrated.

Game-based assessment
This is my personal area of research interest. There is so much potential to make games in which students’ actions tell us about their proficiency. In games, educators can observe a student’s sequence of actions, time spent on tasks, multiple attempts at activities, requests for help, communication process, and so on. In other words, games allow us to examine a student’s process of problem solving, not just the final product at the end. These observations can help educators make inferences regarding students’ mastery over skills, while offering new ways to assess factors not easily measured on multiple-choice tests, such as problem solving, critical thinking, collaboration, persistence, and creativity. The key here is not to re-create traditional looking tests within games but to design games such that the choices in a game tell us about what students know and can do. The assessment is invisible to the player.

Making a Good Game is Hard

I’ve learned over the past five years that making a good education game requires incredibly close collaboration between a game designer, a curriculum/learning expert, and an assessment expert. This doesn’t mean “throw it over the wall” to the game designer. It means nearly every major decision a player makes in the game is discussed. Does this choice help maintain engagement? Does it make sense in the narrative of the game? Do students at this level have the content knowledge to do this? Does it violate the way math works? Does this choice tell us something about what the student knows, depending on which option is chosen?

I have been in many conversations where a game designer or learning expert wants something in the game and the assessment expert argues that it totally changes the difficulty of the task and what we can infer about the student. Similarly, it seems often learning experts from traditional backgrounds want lots of scaffolding so students don’t make mistakes, but that can eliminate a key game element and assessment evidence. Assessment experts often argue for very constrained play so they can make observations of behavior in very controlled conditions, but that is again at odds with game play and the exploratory nature of learning.

So, to be done right, educational game development involves a lot of time, discussion, and iteration. Here are just a few key tasks required to make a learning game with assessment baked in:

  • Understanding how learners’ progress in the content area. Is there existing research about stages a student goes through when learning a concept? Common misconceptions? The nice thing about educational research about learning progressions is they often can lead directly to game levels. However, this research must be synthesized and understood.
  • Aligning a core loop of game play to the learning loop. This is a lesson I learned from Erin Hoffman and Michael John in work on Mars Generation One: Argubot Academy. A key loop of activity in all kinds of argumentation is: identifying evidence, linking evidence to claims, and then identifying problems in an opposing person’s argument. Game play was designed to follow that same loop.
  • Creating evidence models. It is necessary to determine what choices in the game tell us about a players’ knowledge and skill. These choices must be designed into the game so the options available differentiate between people who are proficient in a skill of interest and those who are not. In other words, someone who is proficient is likely to make one choice and someone who is not is more likely to make the other.

All of this takes, time, expertise, collaboration skill, and a million other little things to go right.

Curriculum Development

When a company like Pearson sets out to create a curriculum, they are creating anything from a semester (e.g., a higher education course) to multiple years worth of content (e.g., a K-5 math curriculum). There are always very tight budgets and very tight timelines to get content created. To have any hope of getting this much content created on time and on budget, there have to be clear, replicable processes in place. These processes likely have been refined over time to eliminate any wasted time and activity.

Right now, to make a series of conceptual games that have good learning and assessment design in no way fits into the budgets or the timelines. I say a series of games, it could also be one long, threaded game. Regardless, it takes months and months and tens of thousands of dollars to create a game that covers one narrow slice of curriculum. A recent project I was involved in used a learning progression to align curricular content and design assessment and took six months and about $250K to make a game that covered just geometric area for 3rd graders.

Of course there are cheaper options. We’ve done them. We make mini-games. We don’t include assessment or reporting. We don’t do a thorough review of the research on learning. We have just one boring game mechanic. These aren’t amazing games.

I can’t tell you how many conversations I have been in where it was suggested we just take the 5–10 most difficult topics and make games for those. That is still going to be a big chunk of money and either a lot of time, a huge staff (if working in parallel), or likely both.

Here is the kicker: we have not found that schools and teachers are willing to pay much more for a curriculum with games versus one without. They like it if we can check the box hat says, “yes, there are games included,” but we have not found that to be a deal maker or breaker in the world of curriculum buying. This isn’t even talking about whether those games are actually good. That means there isn’t any extra revenue for those game companies who want to partner with us in a shared revenue model.

Where does that leave us?

I have seen some beautiful learning games, but realistically they are individual games, not integrated into curricula, and without assessment.

To really get a conceptual game that has what game designers would recognize as quality game design, integrated into curricula, with assessment baked in just does not fit into the world of large scale curriculum development.

So, should we abandon hope? I have only one game among my team’s current research projects and it is a stand-alone mobile game for Syrian refugee children in Jordan. It is part of our social responsibility efforts. What I do have is a number of simulation-based assessment project. We are working out the process for both learning progression and evidence definitions. We are working on creating templates that can be easily modified to create tasks across courses and grades. These are much cheaper to make and build than games and are a step up in engagement from the typical classroom experience. I hope we can use this work as a building block for future game work, but we aren’t there yet.

My work on games and game-based assessment as a researcher at a big education company has shown me that even with great distribution, we really need new processes and business models to get large scale creation and integration.