Player Clustering: A Recipe for Great Game Design
At Field Day, we make learning games, but we also study them. In one of our latest projects, we used data clustering to understand how different types of players use our games and how we can customize our game design to support them. The paper isn’t published yet, but we’re sharing a sneak peek! Keep reading to find out what we learned.
Player clustering is an amazing tool for designing better games, but not everyone knows how to use it. We want to help demystify the process.
In case you haven’t come across the term before, “player clustering” means separating gameplay sessions into specific, thoughtfully chosen categories and using those categories to understand different players’ stories— the unique ways they experience our games.
We believe this method is literally a game-changer. It allows researchers and designers to learn from and iterate on games after they go out into the world.
“The amount of emergence that happens in a game is just incredible,” said David Gagnon, director at Field Day. “You get to author the little rules that work with the player to co-create the experience. That means you have to test when you’re building a game. You have to see how your players work with it and see what emerges, and use that to iterate on design.”
The first stage of play-testing starts out small, when the game is still in early stages. We connect with local classrooms through our teacher fellowships. The teacher fellows use our games with their kids, and we end up with feedback that informs our design.
But what about later, when our games leave the nest? During this stage, our games get played by hundreds of thousands of kids. We end up with massive data sets. That’s where player clustering comes in. It provides a powerful, research-based method to understand how people are actually using our games and react accordingly.
“The game is part of the equation,” David said. “The player is part of the equation. And the game is the magic between the two.”
The research team on this project included David Gagnon, John McCloskey, Luke Swanson, and Jenn Scianna from Field Day here at UW-Madison, and Erik Harpstead from Carnegie Mellon University. I sat down with David, John, and Luke (via socially distanced Zoom chat) to talk about what they learned.
John, research intern at Field Day, said the paper will be geared toward researchers or designers who are interested in player clustering but don’t have a lot of experience with it.
“We really tried to illuminate a more clear process,” John explained. “It’s not a recipe where you can follow everything through. But like when you’re making bread, you can add more flour or water depending on if the dough is too wet or too dry. It’s a method that will give researchers a basis.”
Case Studies: When Data Becomes Story
Our research team started with a long exploration phase. Then they developed the step-by-step method and applied it to two of our games: Lakeland and the Wave Combinator. When I talked with John and Luke, they told me about some of the surprises they uncovered.
Luke, who worked with the Wave Combinator game, said the team discovered something unexpected about how some players used the game interface.
“Players can use sliders to change the shape of a wave,” Luke said. “They can also click little buttons to really fine-tune their changes. When we did the clustering, we found a big group of players who used way, way more arrow clicks than anybody else.”
According to Luke, these players were spending a lot of time clicking the arrows, when they could’ve easily adjusted the slider instead. These players might not have been struggling with the content, but the tool was getting in their way. We can support these players, who might be getting frustrated or discouraged, by fixing the tool to support them.
“It’s a great concrete example,” Luke said. “We can look at the clusters, notice something about the players in that cluster, and think, oh, maybe we need to do something to help people realize there’s a better way to play this game than to repeatedly click this little arrow.”
John brought up another example with powerful ramifications for design. In Lakeland, one of the key learning goals is for players to understand how dairy farming can cause phosphorus pollution — and eventually destroy the lakes.
“We found that the players who invest in dairy farms succeed in having more of an intended experience,” John told me. “From a designer standpoint, we think players will make a farm, and they’ll start having algae blooms, and they’ll realize that manure runoff creates algae blooms. But players can’t have that experience if they’re not buying dairy farms or putting down enough manure.”
These examples demonstrate how player clustering can reveal important insights. So how does the method work? How can other designers and researchers use it?
Recipe for Player Clustering
- Preheat your oven. Just kidding. In step one, you’ll need to identify specific events from the game data log to focus on. Our research team divided events into three categories: player actions (i.e. buying a farm or hovering over a tile on the map), system feedback (i.e. a chiming sound or a health gauge running low), and progress (i.e. earning a badge or unlocking a new part of the game story)
- Aggregate events into features. In this step, take individual events (for example, players buying farms) and consolidate the events into features (i.e. a player’s total money spent in level one).
- Prepare the data. This is sort of like kneading your dough before you put it in the oven. Decide which gameplay sessions you’ll include in your analysis. We suggest focusing on “typical” player experiences (95% of gameplay sessions) and filtering out the outliers.
- Start clustering! Use best-practice methods to find mathematically significant groups within the data. (If you’re looking for the gritty details, see our published paper for more info.)
- Visualize and evaluate the results. Use data visualization techniques that make it easier to interpret the data. Our team used radar charts with an axis for each feature. Then inspect each category and work to describe and name each cluster based on its characteristics. For example, our Lakeland progress clusters included Food-centric players (an average number of farm achievements, but fewer achievements in other categories) and Caution Ahead players (average money and population achievements, with fewer algae blooms).
If the categories tell a meaningful story, our work is a success! If not, we need to go back and try out the process with different events, aggregate features, etc. That’s where the whole adding-flour-and-water-to-make-bread thing comes in. Don’t be afraid to experiment.
“The fun thing about this analysis is that it’s surprising,” David said. “During production, you do all the hard work to imagine what players might do, and those theories become the design. But because data mining is just raw data, the results always surprise you . . . It gives you the opportunity to answer the questions you were never asking.”
We’re excited about how player clustering will allow us — and other designers out there — to embrace the emergence and co-creation that happens in great game design.
If you’re a game designer or researcher and want to hear more about what we’re learning, feel free to reach to David at djgagnon@wisc.edu!
