Overwatch: Off the Screen and Into Your Hands
Design Reflection
This Medium post is for HCI class taught by Professor Peck at Bucknell University. In this project we explored input methods to interact with game systems.
Leap motion Overwatch
Our team developed a version of Overwatch that could be played in 3D space using Leap Motion. Classic controls of the game were switched to motion controls. These controls come from hand movements. We wanted to make Overwatch seem more interactive and fun, and this led to some controls being more dramatic than they needed to be. A video demo can be found Here, and our code can be found Here.
Why Overwatch?
Our first concern in Design for Fun was achieving fun. That meant finding an enjoyable game that we wouldn’t ruin with motion controls. Overwatch was released 2016 and was instantly a big time hit and now has over 40 million players worldwide. Being a modern first person shooter (FPS) game, there are a plethora of different controls and inputs possible. This seemed a daunting task to tackle at first, but we made it work while keeping the complexity to a minimum.
To ensure we maintained the fun of the game, we followed Leap Motions own design principles found Here, and listed below.
- Tracking Consistency
- Ease of Detection
- Occlusion
- Ergonomics
- Transitions
- Feedback
We considered several other games before settling on Overwatch, doing Wizard-of-Oz tests and eliciting peer feedback on several. The various games we originally tested and the reasons we chose not to pursue these can be seen below:
- Mario Android
Nintendo does not make it easy to run a game on an android emulator. If we cannot easily connect the Leap Motion to the game, we don’t have a project.
- Classic Mario
Fine controls would have made this game a non-ideal candidate for the Leap Motion controller, which notoriously struggles with fast, precise motions.
- Doodle Jump
The interaction felt unresponsive to hand gestures in the Wizard of Oz test. This game requires very fast response speed. See above.
- Tetris
Requires too simple of interaction and only allows minimal error in the control. Tetris already borders on a masochistic exercise in frustration. Adding frequent errors in the input would make this game not fun to play.
- Bullet Bill 2
Not an especially fun game. Design for Fun fails when fun fails.
- Chrome dinosaur game
Funny, recognizable option that would have been more sarcastic than a real choice.
- Frogger
Classic game with only directions as controls. This was another strong contender.
- Fruit Ninja
Another team was already doing this.
Originally, PacMan seemed to be the best option, since it required reasonable amount of control, and it allowed some errors in the game, which will always happen when users control the game using Leap Motion. This game became fairly repetitive, so we wanted to switch to a game with multiple controls.
This led to the final choice of Overwatch. This game combined our two goals of having multiple motions, and having actions that match actions in real life. It also features walking and shooting, which are actions that seemed more fun to attempt in real life.
Stages of our Design Process:
- Exploring The Technology
- Exploring Game Options
- Wizard of Oz Testing
- Peer Feedback On Game Choices
- Implementation
- Testing
- (Peer Feedback On Testing)
- (Iteration)
- (Testing)
Exploring The Technology
We got the Leap Motion functioning on Elliot’s computer and saw first hand (pun intended) what the device’s interpretation of our movements looked like. Unfortunately a mistake was made installing the driver software for the device. We installed the Orion software when what we wanted was the V2 software. It meant that though we were using the latest tool available, nothing we tried to interact with was compatible. This cost us much of our design time.
Exploring Game Options
A rather fun class day, we scoured our own knowledge and the internet for games that might be fun to control by means of looking foolish on camera. Many of these proved to be poor fits rather quickly. Some others were a bit harder to let go of.
Wizard of Oz Testing
The fun continued as we learned just how ridiculous it is to play as the controller; it brought new nuance to the idea of cooperative play. The challenge of adapting someone else’s motions to keyboard or mouse inputs, while annoying, seemed like a good approximation for some of the accuracy issues we had seen in the Leap Motion controller.
We tested out various games using this method such as Pacman, doodle jump, the chrome t rex game, and frogger. While these were somewhat successful, they weren’t as fun as simply playing the games in the regular way. We wanted to make a game that seemed enhanced by doing it with motion. This helped us settle on the game Overwatch.
Peer Feedback On Game Choices
As much of our initial exploration occurred in class, we were able to get real time feedback on ideas we were tossing around, as well as toss out some ideas because we knew other teams were looking into them. The most actionable information we got from this process was the importance of brand recognition. The Leap Motion controller was going to be a beast to tackle, no matter what, and making very specific motions in the air was going to look and feel silly, no matter what. To be fun, then, we had to play to the joy of experiencing a game everyone knows in a new way.
Implementation
As mentioned above, we suffered compatibility issues that drained our time on this project. We did figure out the issue by consulting with our peers in class, but not before going down a lot of blind Google alleys. Once this software issue was resolved, implementation was a relatively straightforward affair of using the example tools given in the documentation for the project.
Testing
In this case, testing was a simple matter of troubleshooting code and everyone getting a chance to have some fun with Overwatch. We first had to choose motions that would be the most fun to implement, while keeping some accuracy. They also had to be distinctive enough, that the leap motion would not get confused over the various functions. We tested out motions in the air to see how it would feel with the game. We ended up settling on various motions to accomplish the tasks and then implemented this in the code.
Design Objectives
Challenges
To understand our objectives, we first sought to understand our needs. Using the Leap Motion interface, in order to deliver as fun an experience as possible, we’d have to contend with some of the following challenges of the medium:
- Imperfect recognition of the hands
- Imperfect recognition of hand gestures
- Imperfect hand position tracking
- Software challenges and driver issues
- Imprecision of unsupported hand motion in 3D space
- Little to no developed expectations for the medium
Needs
The repetition of imperfect in the above list sounds a little funny — of course it’s imperfect, nothing is perfect. But computers kind of are. Except for actual failures of hardware, keyboards, mice, even touch screens simply do what we tell them to do. We’ve come to expect that computers can be frustrating, counter intuitive, and involve nearly vertical learning curves, but one thing they don’t do is make mistakes. With this new generation of AI-based inputs such as voice and now hand recognition, we are interacting with technology that is deliberately fuzzy on the details. These tools offer incredible new powers of interaction, but also means our expectation of perfect translation of our inputs is no longer realistic. It will get better, but for now, our needs for the leap motion become:
- Motions that the computer can consistently recognize (Ease of Detection)
- Motions that people can comfortably perform (Ergonomics)
- Motions that people can consistently perform (Transitions)
Objectives
Based on the needs we defined in the above analysis, our team chose to follow these objectives in our design:
- Simple — Complex motions are difficult for players to perform quickly and difficult for the machine to understand in real time.
- Imprecise — The technology is simply not able to consistently interpret subtle differences in hand positions.
- Related — Most people have never played games in this way before. Learning these new controls will be easier if there is a thematic relation between the they take and the action the machine performs.
User Research
We approached our research in a number of ways. First and most informally, we loudly discussed the various games we were considering in class and evaluated the responses of our classmates, both for how intrigued they were and their knee-jerk reaction to how difficult it might be. Conversely, their silence meant that we might want to move on.
Among ourselves, we performed Wizard of Oz testing to evaluate different control schemes. We possessed limited skill as wizards, but the systems of control that showed the most promise we shared with others in and out of STEM. Responses were wildly inconsistent and not especially helpful for determining what people would and wouldn’t find intuitive.
By the time we had a working prototype on our final game, we were out of time for meaningful user testing. Fortunately, we were able to exploit some lessons learned from observing other teams using their leap motions as a useful proxy. Team Fruit Ninja demonstrated Leap Motion’s fragility if it is expecting one input but gets two. This consideration was part of the decision to pursue a two-handed interface.
How do we make our hand motions simple?
We couldn’t avoid having each hand need to make up to two motions at the same time. To accommodate this, we put most of the “doubling up” on the right hand, where one target was the position of the hand in space, while the other was the was the gesture the hand was making. These fundamentally distinct ideas help both the player and the computer successfully communicate intention.
How do we make our hand motions imprecise?
This consideration was layered throughout our design. A specific example is the hand tilt required for the “secondary fire” action. A hand tilt of 30 degrees proved to result in too many false positives. Sixty degrees mostly eliminated this problem. Also, to allow the Leap Motion to easily distinguish between both hands, the right hand was offset from the center, giving it a wide margin from the left.
How do we make our hand motions related?
Movement — Movement is straightforward. The direction the hand is moved or tilted corresponds to the direction the character on screen should move. (No inverted controls here!)
Shooting — Squeezing a trigger and squeezing a fist relate both in the physical motion and in the aggressive nature of the act. For this reason it was also decided to keep the right hand vertical instead of horizontal, further reinforcing the connection to holding and firing a gun.
Alternate Shoot — Tilting the hand for a secondary fire indicates a passing to an alternative.
Reloading — Circling with the finger is a common gesture for moving on to a new thing; in this case a new clip of ammo.
Emote — A wave is a wave.
Ultimate Attack — And the power fist gesture of the left hand evokes the magnitude of using the character’s Ultimate Attack.
Implementation
Much of the challenge this project faced was all Elliot’s fault for downloading the newer (and therefore wrong?) version of the Leap Motion software. This mistake cost us days of troubleshooting, and left us with little time for proper iteration of our prototypes.
When we finally did get our project operational, it was on Windows 10, with code written in Java using the Leap Motion SDK. We have to give appropriate credit to Zilin for taking the lead on this, essentially in one night.
References
http://blog.leapmotion.com/6-principles-of-interaction-design/
