Games that Spark Curiosity: Where Does VR Stack Up?
We designed a Virtual Reality experience that transports players to the IceCube Neutrino Observatory in the South Pole. Here’s what we discovered.
At Field Day Lab, we love geeking out over Virtual Reality. And not just because it’s fun and exciting and feels a little futuristic. (Although it definitely does.) We make games to share cutting-edge research with the public, and we use machine learning and AI-powered data mining to understand how people learn. VR is a powerful tool to make that happen.
“VR is such an attractor,” said David Gagnon, director at Field Day. “When you have a VR experience out at a museum space, just the fact that it’s VR leads to a significant amount of interest.”
Case in point: Discover IceCube, our VR experience that transports players to the IceCube Neutrino Observatory in Antarctica, and then into deep space. There’s something awe-inspiring about leaping through space and staring down a black hole — even if it’s through a VR headset.
IceCube: Studying the Universe from the Most Remote Place on Earth
IceCube is an international collaboration of scientists based in the South Pole. The scientists at IceCube use a huge particle detector — made of light sensors and Antarctic ice — to study some of the most mysterious phenomena in the universe, like supernovas and black holes.
“I love that they see themselves as the scouts of the universe,” said Sarah Gagnon, Field Day’s creative director. “They’re out there doing science that paves the way for so much more.”
While the South Pole is a super cool setting for a game (pun totally intended), the abstract nature of IceCube’s work makes it challenging to communicate to the public. In an effort to explore new methods for outreach, the Wisconsin IceCube Astrophysics Center (WIPAC) teamed up with the Wisconsin Institute for Discovery (WID) and our team at Field Day.
“One of the biggest values of this work was the tight collaboration between researchers and designers,” David said.
The team set out to design an informal learning exhibit. The goal: to spread the curiosity and passion of the scientists at IceCube.
Comparing Tabletop Games and VR
The learning research component of this project compared the effectiveness of a multi-touch Tabletop game and a VR experience. Researcher and WID faculty member Kevin Ponto, who had previously partnered with IceCube, was the project’s Principal Investigator and one of its guiding forces. (Want to learn more about Kevin’s research? Check out this great article about his work with VR!)
Field Day created the VR experience, while the Virtual Environments Group at WID designed the Tabletop game (available here). Both experiences are located side-by-side on the main floor of the Discovery building, where anyone can play them.
To evaluate our work, we invited players to take a post-exhibit survey, with 154 responses. We also enlisted a group of 31 players to take both pre- and post-exhibit surveys.
The research team included a bunch of awesome researchers and subject experts: James Madsen and Silvia Bravo-Gallart (WIPAC), Ross Tredinnick, Bryce Sprecher, and Kevin Ponto (WID), Rebecca Cors (WCER), and David from here at Field Day.
Now that the findings have been published, I sat down for a socially-distanced Zoom interview with David and exchanged a few emails with Rebecca to chat about what the research team discovered. Keep reading to get the inside scoop!
- Both the VR and Tabletop helped people learn.
The study: Our team hypothesized that players would gain understanding about IceCube, and the data bore that out. Players of both the VR and Tabletop reported an increase in their perceived basic understanding. This held true in both the post-exhibit surveys (which relied on players to look back and retrospectively rate their level of understanding before the exhibit) and the pre-post surveys (which compared actual levels of understanding before and after the exhibit).
The takeaway: Success! Both the VR and Tabletop experiences resulted in real learning. “People showed significant increases in basic knowledge,” David said. “There was new vocabulary learned. It proves that these kinds of experiences are obviously educational and can be designed to fit easily in public spaces.”
As Rebecca pointed out, this was also a rare chance to compare the value of retrospective and pre-post surveys in an informal learning setting. Researchers are still debating whether retrospective data can be as true to what participants really think as traditional pre-post data. Our team wanted to test that out.
“Our results provide strong, rigorous evidence that retrospective questions provide data that are as good as pre-post data,” Rebecca said. “Because retrospective questions costs less, this can really mean a lot for museum staff people trying to study their exhibits with small budgets and few other resources for research and evaluation.”
2. When it comes to practical learning, VR comes out ahead.
The study: Survey says . . . (ding ding ding!) VR comes out on top! This is another place where the survey results matched the research team’s hypothesis. Players of the VR reported significantly higher gains in understanding than players of the Tabletop.
The takeaway: The VR experience has a clear edge when it comes to learning. This could be for several reasons — a more immersive experience, a slightly longer play-time, and captions and voiceover.
On the other hand, as David pointed out, a two or three minute experience can only do so much. “You don’t want to put too much weight on whether someone knows a new word at the end, or whether they can answer multiple choice questions correctly,” David told me. “What we’re looking for is a really impactful, curiosity-providing experience.”
Which brings us to our next finding . . .
3. Both the VR and Tabletop sparked curiosity.
The study: Since VR is the newest tech on the block, you might think it would spark more interest than the Tabletop. That’s what the research team hypothesized. But as it turns out, both exhibits sparked curiosity. According to our surveys, VR players reported an average curiosity rating of 4.7 on a scale of 1 to 6, while Tabletop players reported a curiosity rating of 4.4 — a slight edge for VR, but not a statistically significant one.
The takeaway: Both the VR and Tabletop exhibits made players want to learn more about IceCube. According to David, this is the “bread and butter” of our work.
“What we’re looking for is an interest trigger,” David explained. “That’s when somebody isn’t interested in a topic, and then they have an experience at a museum, and now they’re very interested. That’s the holy grail of informal learning.”
Many studies show that interest gains from such a short experience fade over the following weeks. To become a dispositional, enduring interest, the situational interest players experience at the exhibit needs to be fed with more experiences about the same topic.
“I think it’s fascinating how studying factors like ‘curiosity’ and ‘sense of wonder’ is key to understanding how true, lasting interest develops,” said Rebecca.
4. Higher Scores ≠ Greater Understanding
The study: If we’re honest, this one surprised us. The research team hypothesized that players who achieved higher scores would gain more understanding. In reality, surveys showed that players’ understanding didn’t correlate with their scores. (Side-note: The VR experience didn’t include scoring, so this finding is based solely on the Tabletop game.)
The takeaway: Higher or lower scores didn’t correlate with higher or lower gains in understanding. This could suggest a problematic decoupling of gameplay elements and learning, but there’s a positive side, too: Players who weren’t skilled at gameplay were still able to learn.
In future projects, the research team will work to understand the exact moments where learning happens during gameplay.
The Best is Yet to Come: Future Work with IceCube
While our team at Field Day is excited about this research, we’re even more excited to continue to work with IceCube on public outreach. “The collaboration with IceCube and our initial, very positive findings have led us to write two proposals to look at new methods for educational data mining in VR,” David said.
One project will look at how we can make VR a better instrument of learning and interest. The other project will apply our findings from the pilot project to a VR experience about Antarctic research on penguins.
“We’re working to pair public outreach and data mining in all of our projects,” Sarah said. “That’s why we think it’s so important that we’re a research lab with a foot in data mining, as well as a design studio. Evaluation helps us understand how our games are used.”
If the new projects get funded (fingers crossed!), David is excited to build off the momentum of the pilot project and explore more sophisticated data mining approaches.
“Ultimately, the thing that we wanted to have happen happened,” David told me, grinning. “People who encounter the exhibit are moved by it.”
Want to learn more about our findings and how we conducted our research? Find the full paper here!
