Behavioral Prototype of a Cinematic Gesture Controlling System

Behavioral prototyping technique can be very effective in testing design assumptions for HCI applications when the actual technology is either not available or too expensive to develop during the design phase of a project.


Our group decided to design gestural controls for a home cinematic experience. We identified the core component controls of home cinema as being: play/pause, fast forward, and rewind. For each of these we developed a distinct gesture.

Sketch of 3 stock gestures


Play and pause were assigned the same gesture due to the toggle nature of their states. Play/pause was designated as a double overhead wave gesture, fast forward was designated as a vertical lift of the right arm where the previous play or pause state was resumed when the arm dropped, and rewind was designated as a vertical lift of the left arm where the previous play or pause state was resumed when the arm dropped. The gestures were designed to be grandiose so as to be explicitly distinguishable from common stretching behavior.

Room setup during testing


The user test was proctored in the actual home environment of our user. The first part of the test included four scenario questions which prompted the user for a gestural response. The second part of the test included the same four scenario questions, but was run after having the user “program” their own gesture controls. The final portion of the test included an exit interview, which consisted of questions regarding the user’s preference for either the standard gestures or their own “programmed” ones, whether they would use or recommend the gestural system, and whether or not they were surprised when it was revealed that the interactions were occurring via a remote keyboard rather than by their gestures.


The roles of our four members during the user test were as follows:

Jessica Bao (Scribe): Jessica was responsible for taking notes throughout the user test, capturing the user’s commentary and actions.

Jess Landquist (Moderator): Jess was responsible for proctoring the test and communicating with the user during the test.

Alec Martin (Operator): Alec was responsible for monitoring the user’s gestures and triggering the interactions from a remote keyboard during the user test.

Tristan Shi (Documentarian): Tristan was responsible for operating the camera over the user’s shoulder to capture the user’s actions and responses during the user test.


The process was documented using 3 cameras from different angles. An overview of the whole process was edited to a video.

Testing process video


The first portion of the test with the standard gestures that our team designed went well. The user understood and remembered the three gestures, and all of the test case questions were completed as intended.

The second portion of the test where we asked the user to “program” their own gestures seemed to be confusing. Initially, the user “programmed” the standard gestures, in which case we restated that they could design their own gestures. More concise and clear language in the user test could improve the testing of this system on future users. The user ended up designating gestures for play/pause as the same as the standard gesture, fast forward as the right arm extended horizontally to the right with the play or pause state resumed upon bringing the arm back toward their body, and rewind as the left arm extended horizontally to the left with the play or pause state being resumed upon bringing the arm back toward their body.

They indicated a preference for their own gesture controls for fast forward and rewind as being more natural and easier to remember, and stated that no modification of the play/pause control was necessary. The user said that they would prefer the gestural controls over their traditional remote, and that they would both use and recommend such a system.

When informed that the interactions were due to a remote keyboard rather than their gestures, the user did not seem surprised. They later indicated that they assumed the word “prototype” was synonymous with “fake,” however they felt that the overall setup of the system was believable. Again, changing the verbiage of the user test and referring to the system as something other than “prototype” could improve the test results and mitigate presumptions that users could have.

Show your support

Clapping shows how much you appreciated Tristan Shi’s story.