LEAPing onto Asteroids

Anushikha Sharma, Allan La, Jacob Mendelowitz, Morgan Muller

Introduction

Integrating motion controls into a game that people know and love is an easy way to make it a whole new experience. The question is, will people still enjoy it with the new controls? Will they enjoy it more? For the ‘Design for Fun’ sprint, we wanted to see if we could give the classic game of Asteroids a new breath of life using LEAP motion controls.

What did we build?

This demo video will help demonstrate the Leap controls that we mapped onto asteroids.

Demo Video: Demonstrating the controls to the game

For reference, below is the final set of controls that we use in our game:

• Turn Counterclockwise -> Tilt right hand (roll) to the right
• Turn Clockwise -> Tilt right hand (roll) to the left
• Move Forward -> Tilt right hand (pitch) downward
• Slow Down -> Tilt right hand (pitch) upward
• Fire -> Make a fist with left hand

Initial Ideation

We began by trying to understand the limitations of our sensors in order to narrow the scope of what controls we could possibly use. Once we started playing with around with LEAP, it seemed to us that the device was accurate with capturing the different hand motions. On further exploration, we realized that there could also be some inaccuracy if the hand position isn’t in the most optimal area for LEAP to capture. The real challenge was: How could we capitalize on the strengths of this platform to design fun controls while also keeping track of the limitations?

In the game of Asteroids, there is only one objective, to destroy all the asteroids and obstacles coming at the ship. The ship can propel forward, rotate clockwise and counterclockwise, as well as shoot. So, we started coming up with initial motion actions to map to each of these features of the game.

Figure 1: The ‘Asteroids’ Interface with the spaceship and the asteroids

The process of coming up with the initial motion started out with interviews. We surveyed some of our peers and asked them to perform gestures to accomplish certain tasks that they encounter in the game. For example, we asked questions such as “How would you make a gesture to rotate a stationary object? How would you make a gesture to shoot? How would you make a gesture to propel a stationary object forward? Would you have fun using these gestures to play a game?”

Figure 2: Asking some of our peers to demonstrate ‘intuitive’ controls for chosen tasks

Based off the initial interviews, we came up with the first list of actions:

• Turn Counterclockwise -> Rotate Right Hand Index Finger Counterclockwise
• Turn Clockwise -> Rotate Right Hand Index Finger Clockwise
• Move Forward -> Point using Left Index Finger
• Fire -> Punch Left Hand In A Fist Form Forward

We did a test run using the “Wizard of Oz” technique where we basically fake it until we make it. Initially it didn’t go as well because the user was dying way too fast in the game due to the difficulty of using the gestures. For example, the user got way too tired punching forward repeatedly in order to shoot continuously in the game. The same conflict occurred for when a user wanted to continuously propel forward by pointing over and over again.

Figure 3: Using the ‘Wizard of Oz’ technique to test out our initial controls

Before fixing these issues, we wanted to see how the other gestures worked. So we temporarily eliminated the “propelling forward” feature and only allowed the user to be stationary. As a result the game progressed a lot further. The rotations using the right index finger seemed to work naturally and seamlessly. We believed it just took the user a while to figure out the controls and the game itself. So naturally the user would be bad at the game in the beginning, then slowly get better.

After our magic simulation, we agreed that this seemed very promising to continue to the stage of developing the LEAP code to model these gestures. We also dug deeper into some of the moves that had come up in our surveys and used those, in combination with our ‘Wizard of Oz’ observations, to create a second list of actions:

• Turn Counterclockwise -> Rotate Right Hand Index Finger Counterclockwise
• Turn Clockwise -> Rotate Right Hand Index Finger Clockwise
• Move Forward -> Tilt Left Hand Downward (Will go forward as long as left hand is downward)
• Fire -> Make a Flat Left Hand = Not Fire, Make a Fist with Left Hand = Fire

We did a test run using the “Wizard of Oz” technique again and the results were definitely much better than last time. It was easier for the user to propel forward continuously and shoot continuously in the game. Originally, there had been too much punching and pointing and the user got tired and frustrated. We thought that the new gestures were simpler, more fun and reduced the amount of physical energy involved used to play this game. Thus, we began to develop the code in LEAP to use for this game.

Development and Testing

We had some challenges with the getting the jar and library files into our Java IDE’s. After several hours of struggling, we were able to connect our code to the leap motion and we had a working game with the mapped LEAP controls. The next step was to test this game with real users.

After we had mapped our leap motion controls onto the game, we asked a few students to try it for us. We had different roles in the testing process — one of us explained the rules, one of us asked direct questions to the user about the experience and a third person made some silent observations. The results were widely different from our ‘Wizard of Oz’ tests. The users didn’t enjoy the left hand switching from flat to fist and the rotation of the finger motions was unanimously marked as unintuitive and uncomfortable.

Figure 4: Morgan giving instructions to a user when we were testing our prototype

Hence, we switched up our controls a third time based on the user feedback:

• Turn Counterclockwise -> Tilt right hand (roll) to the right
• Turn Clockwise -> Tilt right hand (roll) to the left
• Move Forward -> Tilt right hand (pitch) downward
• Slow Down -> Tilt right hand (pitch) upward
• Fire -> Make a fist with left hand

Figure 5: Asking the same user to try again, once we tweaked the controls using their feedback

Now, most students enjoyed the game. There was consensus that using the fist pump on the left hand for shooting was a lot of fun but a little tiring and the swaying of the right hand to maneuver the spaceship was intuitive. We did realize that we had to be clearer with explaining the difference between the hand motion for propelling forward and and the hand motion for rotating the spaceship because our testers would at times get confused. The new setup made the controls more discoverable. If a player started to tilt left or right to rotate, they could quickly figure out how to move forward by tilting their hand in the other directions. There was still some frustration from the users when the leap motion didn’t fully catch their hand motions, but we just had to remind them to keep within the ‘catch space’ of the leap hardware.

Strengths and Weaknesses

We concluded from our user-testing that for anything to be “fun”, the controls needed to be easy to understand but the game also need to have an element of challenge to it, to keep the users engaged. The feedback we received from the class demo reaffirmed our conclusion. The popular opinion was that the controls were intuitive and simple. Users enjoyed the fact that they could use both hands during the game for maneuvering the spaceship and shooting at the asteroids. Though our version of the game was more challenging, the controls were smooth and our users were engaged. Familiarity with the original game also elicited some peals of nostalgic laughter that added to the experience.

Though overall the results were positive, there were some factors to improve upon. The version of asteroids we used was significantly more challenging than most. This extra challenge required fast movement which wasn’t always registered by our leap motion sensors. Hence, sometimes even though the player was making the correct moves, the spaceship would taper off in the wrong direction. In the feedback, a suggested way to counteract the limitations of the hardware was to have the controls for each movement be distinctively different so that the leap motion could register the significant change in hand-motion.

We wanted to make the controls as reliable as possible so people would enjoy the experience of the game and not be frustrated by performing actions they did not mean to. With motion controls, reliability is very challenging to get right. Some thresholds for movement felt good to us, but some users still needed more or less leeway in the gestures. If we had more time, we would want to spend more time trying to ensure that the thresholds for turning and staying stationary could be further apart and there was a better way to keep the ship still.

Conclusion

In conclusion, we learned a lot from this design sprint and are happy with our final product. We realized that user-testing with the actual product is very important because what users think they want versus what they actually want might vary significantly. The feedback from the initial surveys and the ‘Wizard of Oz’ results were vastly different from the feedback of our user-testing with the actual leap motion hardware and controls. We also learned to value the iterative repeat approach where once we got an idea of what a user would want, create it and test it again to refine the product even more. The challenge for this sprint was to create a fun but intuitive game and that combination isn’t as easy to create as might’ve initially thought. We struggled and it was a time crunch but at the end, we walked away with a fun LEAP-ified Asteroids game!