Getting Physical — Car Games Upgrade using Physical Computing!

Welcome to another edition of the 2021 MHCI 99P Labs Capstone Joymaker’s Journey! We are in the second half of our summer. In this post, we’ll do a quick recap of the journey we have taken thus far and talk about the fun and exciting things we had in the works during Sprint 3. Strap yourself in; there are a lot of turns on this ride!

Recap of our Journey Thus Far

If you have been following our journey so far, you would know that we were challenged to envision the future of social interactions in and around cars. After a couple of months of deep-dive user experience research, we identified a unique problem space that was particularly poignant to a dominant driving population — families. To read more about how we came to this problem, please see our prior post: Are We There Yet…?! With all of this, we came to our main UX Question:

Can play help us establish a shared presence between families in the car during short-term drives?

But what exactly is play? We determined that instead of digging our heels into one specific dynamics of play, exploring a number of interesting interaction touchpoints was a more effective way to give form to a potential play-based interaction system. Check out our last posting to learn more about our earlier interaction touchpoints and the dimensions we are curious about here: Interactions, Car Data, and Play Dynamics…Oh My!.

The last two months have challenged us to rapidly turn ideas into testable concepts. Up to this point, we had primarily focused on analog tests using a bunch of hacked-together solutions in a stationary foam-core buck. Although we got very valuable information, we were itching to really hit the road. It was time to get things in motion. It was time for a digital upgrade.

Define and Refine

But first thing’s first: sussing out what worked, what didn’t, and what we wanted to explore further from our Sprint 1 and 2 tests. In Sprint 1, we had focused on exploring how to leverage two different unique features of the car: that, when it is in motion, the environment around it is always changing, and that the car-space seems to be conducive to more personal conversations. In Sprint 2, we experimented with a broader range of items, including existing car games, buttons and lights, physical motion of a stationary motion, physical motion of a moving car, and music.

From these different game prototypes, we discovered:

• People want a personal or unexpected touch to the location-based triggers
• Self-disclosure fosters greater connectivity but is best when not so overt
• Physical collaboration fosters connection and generates a lot of excitement, but also demands focus and attention
• People like physical controls, such as buttons and lights, but the interaction falters without the right balance of structure and agency
• Fixing what isn’t broken may not be the best approach — I-Spy and Carpool Karaoke are already successful activities in the car. Learning why they are so popular is probably more effective than trying to enhance them with buttons and motion.

These findings provided orientation as we took steps towards increasing the technical fidelity of our game prototypes. Specifically, we synthesized our findings and questions to create three “version 2” game prototypes that would let us push our findings and play-based interaction exploration further.

1. Contextual-based Self-Disclosure

Our first of three was a contextual-based self-disclosure game. We brought back what worked from Sprint 1, adding the element of embodiment to the experiment to develop version 2 of the self-disclosure game. We discovered that often directly asking personal questions may seem unsettling or uncomfortable. For this experiment, where we used context-based cues outside of a pre-recorded drive to spark open-ended question prompts, we wanted to explore two key questions:

​​1. Does location-embedded prompts with distancing foster self-disclosure, especially for children answering questions?

2. What role can a CUI play? Is the kid more inclined to answer the CUI acting as a 3rd party (distancing) or parent?

2. Collaborative Buttons

Our second of three experiences continued centering around buttons but from a different angle. We learned in our motion game from Sprint 2 that people loved the collaborative nature of the marble maze. And although enhancing an existing car game with buttons was not successful, we wondered if adapting an existing game that was not currently playing in the car with the use of buttons would be more successful.

With this in mind, we build on the game Simon-Says, placing different colored buttons around the car and created a gameplay protocol that would require the different riders in the car to work together to press the buttons in each pattern. The questions we wished to explore were:

1. Do people like the interaction of pressing buttons at different locations in the car?

2. Can this kind of interaction foster collaboration?

3. Marble Motion Maker

Our final game experience was a digitized version of our analog marble maze game. We received the most positive reactions from our motion-making marble maze, but one question that we couldn’t shake was how conducive is such a high-attention game to a moving car?

We knew we were onto something powerful here, but that it required some tweaking. Taking a “divide and conquer” approach, we broke into two swim lanes: one group focused on turning our analog motion board into a digital game board, powered by Arduino and Raspberry Pi and a few “borrowed” smartphone features (excited to read about how we did this? See the section below!).

The other team rapidly iterated through a handful of ways that a motion board game could be played and how, physically, such a board could exist in the confines of the car. Some things we tried, including experimenting with clear and opaque board backs, trying different games like mazes or pachinko boards, and obstacle courses, are photographed below.

Ultimately, we settled on a game similar to the Simon-Says from game prototype 2 above, where each wall was labeled a color and players had to get the ball to the sides in the right order by tilting the board and leveraging the motion of the car. Here our key question was naturally:

1. How will the movement of the car impact an interaction based on the rider’s individual and collective movement?

Physical Computing — Bringing our Ideas to Digital Life

To begin to help us understand how the different interactions we were experimenting with could ultimately be incorporated into a data-driven system, we challenged ourselves to convert some of these experiences into digital modular versions. Since the motion maze was most successful and also one of our more unique interactions — in leveraging the motion of the rider’s body as the game controller — we chose to start with this game experience.

Physical computing was something new to the entire team, but by putting our heads together and leveraging our personal networks and the deep wealth of expertise among the CMU faculty (special shout-out to Nik Martelaro!), we were able to quickly draw up a blueprint for our iPhone-Arduino-Raspberry Pi-Servo Controller Board set-up.

Our inexperience ended up being an unexpected advantage, as it allowed us to more easily think of “hackable” ways that we could bring our ideas to life. For example, instead of trying to use weight sensors and other types of Arduino accelerometer-like sensors and needing to wire that up, we realized we could use the accelerometer in our iPhones, used in many places including the Level or Measurement app native to the iOS. Other hacks include finding ways to code in JS, a language familiar to the team, and cutting out the cloud, and routing everything directly on the Raspberry Pi.

All of this allowed us to take the concept of a body-motion-controlled game board and turn it into a real thing in just shy of two weeks! With some artful craftsmanship from the team to build the physical frame and board, and a touch of joy with the Lego game board made using Lego pieces found scattered in the lab, we had a truly modular digital game board. Moving two iPhones to the motion of your body, we could watch the board move before us.

Take a look at the video below to see for yourself! I guarantee it is hardly short of magical.

Games — check! Protocol — check! Rain…check.

This Sprint definitely leaned harder on the build and refinement. But with everything built and prepared, we were ready to hit the field and get some participants in our cars to try out our game experiences!

Unfortunately, a quickly discovered limitation of such an experiential method of prototyping is, although it is easy to pique the interest of potential participants, it becomes much harder when those potential participants are not actually out in the real world. It turned out that even if we were ready to get families in the car to try the games, Mother Nature had other plans. The week was filled with thick storm clouds, persistent rain, and a city of families less willing to prolong being outside for one second longer than needed.

After trying nine different parks, museums, gyms, and playgrounds across a handful of days, we had to admit that, for now, we had to call a rain check. But all of this driving around was not lost, we got creative and played the games ourselves during our recruiting escapades, providing us a valuable opportunity to body storm our games, tweak them as we went, so that they were even better for when the weather cooperated. We were also successful in garnering interest in future test runs when the weather conditions improved.

Looking Ahead on the Road

All in all, the sprint was a major push for our team, with more milestones met than not. Although we are still in the process of testing our latest iteration of the game experiences with families, the stall on this front has also allowed us to take a step back and start to think about how these disparate puzzle pieces can come together, give form to a system that can foster connection, create joy, re-envision the future of social interaction in cars.

Four short weeks stand before the end of this thrilling journey together. We know whatever that destination may hold, it is made richer by the amazing journey we have had to get there. See you next at our next rest stop!