Engineering Controlled Chaos
How to build a real-time game for classrooms
This spring, we released our first real-time game for classrooms, Quizlet Live. Our beta included over 1 million students and teachers and since launch a month ago, we’ve added another 3 million. Now, it’s grown to 1 million weekly users. These numbers are evidence that we built an engaging, educational game that really works for classroom environments. Our game design process, which involved visits to dozens of classrooms over several months of iteration, is a major reason for this success. Here’s how we did it.
Building Quizlet for the classroom
Quizlet has always had a social dimension — students share content and compete with each other. About half our traffic is during school hours when students are often competing in the same place in the same time in a classroom. But until now, Quizlet hasn’t had a real-time features. Quizlet can be more fun when you compete against your friends but it’s not essential to the experience.
As a team, we spend a lot of time at schools talking with students and teachers. Classrooms have unpredictable technology and bad wifi. But they’re are also incredible environments with complex interpersonal dynamics between students and odd desk configurations. If our game could harness that natural social energy and direct it towards learning, we’d create a learning experience that students will want to play again and again (and teachers will love that).
Finding the perfect game dynamic
Teachers and students have always asked us to build new games and we’ve even built prototypes of classroom games in the past. But none of them ever turned into something ready to launch. For this project, we wanted to explore what we could build before we settled on the perfect dynamic. And so, we built a configurable UI that could pull Quizlet data and allow us to try out different game dynamics.
As we played more, we saw potential in using teams. Competition could be less pressure and more fun with a team than when you were just on your own. But basic team-based concepts, such as adding up all teammates correct answers to a total score, weren’t enough. Yes, you were working with other people but it wasn’t truly collaborative. And what’s worse, if you didn’t want to work, you could easily check out of the process and not participate.
A prototype based on Spaceteam brought the team dynamic to the next level: you answered questions to gain points for your team but sometimes the correct answer would actually appear on another teammate’s screen. Now, you relied on your team for more than just cumulative points — you had to communicate and work together. However, the concept was also extremely chaotic and confusing.
In our next iteration, we found exactly what we were looking for: everyone the same prompt but only one had the correct answer. To answer the question, you had to work with the rest of your team. When we played it ourselves, we were learning and having an awesome time. As someone said, “If we build this, people are going to throw stuff.” While we didn’t literally want people to do that, we were definitely onto something with this game.
Refining the design principles with teachers
We had a game dynamic that we believed in but the real test of whether the game would be successful was getting teachers to adopt it. Before we finalized the UI and built an alpha to test with students, we refined our concept in paper prototypes with a dozen teachers.
Make learning collaborative
Quizlet Live’s emphasis on collaboration and communication was obviously a hit with teachers. And this was partially because they already use collaborative, team-based activities in their classrooms (albeit usually without technology). Think of how many class activities you did in school that required working with other students and think about how memorable those experiences were. Quizlet Live builds upon that concept and makes it easier and more accessible through technology.
Reward accuracy over speed
Race-based games inherently reward speed. This can lead students to act without thinking since they want to rack up as many points as possible (we did this too). That sort of snap decision making isn’t something that teachers want to encourage and research has shown that it doesn’t help them learn.
To help alleviate this problem, we decided to reward accuracy over speed. There’s a time constraint to keep the energy up but if you just answer quickly instead of thinking about it, you don’t get very far. Getting a wrong answer resets your team’s progress. Not communicating with your team had real consequences for your ability to win.
Embrace randomness
Our MVP was scoped to assign teams randomly not because of a conscious design decision but for ease of engineering. However, it had been our assumption that the game wouldn’t be successful without the ability to customize teams. As we talked to teachers, though, we found that random teams was actually a compelling component of the game. As one teacher told us, “This is a great lesson for my 12th graders. Life is random. You never know who you’ll have to end up working with.”
As we tested the game in classrooms, we found randomness also appealed to students. They were often teamed up with people they’d never even talked to before. In a class of 30+ students, it’s easy to go through a whole school year without knowing everyone in the class. And it was fun to get to know new people in the context of the game.
Make the game fun
Our first sketches themed teams to colors (gold, blue, magenta etc) to make your team clear through the UI.
One teacher suggested a far better theme: animals — particularly the animals that Quizlet gives students as profile pictures. They’re a huge hit with students and they identify very strongly with “their animal” on Quizlet. And so we swapped out colors for an array of unique animals from Alpacas to Oxen to Hedgehogs.
As we tested in classrooms, it was obvious this was a great idea — students from 5th grade to 12th grade get excited to see what animal they get and teams build connections immediately. We’ve even expanded the set of animals to 40 (based on students’ suggestions) and we might add more in the future :)
Scheduling a deadline for a real-world MVP
The next step was testing our concept in a real classroom with real students to see if it worked. A deadline would help us be ruthless in our prioritization of what needed work. And so we scheduled a class visit at Life Academy, a public high school in Oakland, in three weeks. We told the teacher we’d be demoing a new game with her 9th grade English class. Now, all we needed to do was build it.
When the morning of our visit came, the game infrastructure was in place, the basic UI was built, and we’d tested that it worked at the office the night before (although with significantly less people than the 25 students waiting for us at the school).
When we started the game, there was a deadening silence for the first few seconds. Finally a few students started talking but they were all saying “I’m confused.” or “Huh? I don’t get it.” Panic started to set in. Were they even going to be able to play this game? Had we overthought it?
Then one student yelled out, “Oh, I get it! Only one of us has the answer and we need to figure out who it is.” Quickly this information cascaded through the rest of the students and “What?” became “Ohhh!” And with that, they were off, talking to each other in order to figure out the answers.
An air of complete engagement permeated the classroom. No one was messing around elsewhere on the internet since everyone on the team needed to be on point to win the game. Anyone who’s ever been in a classroom of 25 students using laptops will be shocked by this. By the end of the first game, the students couldn’t wait to play again. And the teacher was thrilled to see how engaged and active her class was about learning. The core concept was validated — and now we just needed to refine, test, and iterate further.
Over the course of the next two months, we tested Quizlet Live in a dozen classrooms as we built out the fuller experience. And we focused on getting that initial confusion time down to nearly nothing through clearer UI and improved new user onboarding.
Beta testing with a million students and teachers
After we’d nailed down the final UI, we knew we needed to test it at a larger scale in order to get a greater diversity of feedback and find new bugs. Our initial beta group was 3,000 teachers and it grew to 50,000 teachers and 1,000,000 students, with hundreds of teachers requesting beta access every day.
If Quizlet Live turned out to be a hit (which it did), hundreds of thousands of students could be connected to games at any one time. In January, the max we’d ever connected was 35. During our beta period, we planned to scale up slowly from dozens to millions. Every week, we fielded thousands of incoming requests from teachers, asking to be added to the beta. They’d heard about the game from other teachers and couldn’t wait to give it a try with their students. This was the best validation that we were onto something with Quizlet Live. And with 50,000 teachers, we could also gather feedback at a scale far beyond the dozens of classrooms we visited around San Francisco.
Launching Quizlet Live
During our beta period, we worked through bugs, scaling problems, and made lots of great improvements to the overall experience. And most importantly, we were able to build an fun, educational experience that really worked for a classroom environment. Without all the testing we’d done with over a million teachers and students that wouldn’t have been possible.
Since our launch a month ago, we’ve added another 3 million teachers and students. And the response we’ve been getting from classrooms has been amazing.
Try Quizlet Live now!
If you’re a teacher, just click on “Live” on any study set on the Quizlet website to get started. You can also read more about how it works.
We’re also testing out some new classroom games inspired by all the great ideas and suggestions that came out of this process. If you’re a teacher in the San Francisco Bay Area and you’d like to test an alpha with your class, email us at live@quizlet.com.
This post was also published on the Quizlet blog.
