How AR can connect people in person?
This blog post is related to the paper “Project IRL: Playful Co-Located Interactions with Mobile Augmented Reality” by Ella Dagan, Ana María Cárdenas Gasca, Ava Robinson, Anwar Noriega, Yu Jiang Tham, Rajan Vaish, and Andrés Monroy-Hernández presented at the 25th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW)
Time to focus on in-person tech
Picture yourself with a group of friends or in the company of your family. Everyone is sucked away, looking down into their mobile phones, focused on everything but the people around them. There’s even a term for this — Phubbing. Many of us have probably been in this situation.
Even before the pandemic, countless news media and scholars have debated how our technology disrupts in-person interactions. Regardless of its effect, the fact is that the vast majority of applications are for remote communication, and almost none are for in-person ones. We saw this as an opportunity to design for co-location.
In Project IRL, we took the opportunity to reimagine and reinvent ways technology can support, rather than detract from, co-located interactions. We learned how to turn people’s attention away from their phone’s content and towards each other.
What did we build?
We created a suite of five augmented reality apps that can be played on Snapchat. We identified opportunities to design for people who spend time in the same space:
Feeture Films allows parents to play with their kids by transforming feet into characters.
Fast Face is inspired by the popular toy “Bop It” and games like “Hot Potato,” where an object is getting passed around rapidly. Fast Face is for friends, roommates, or family members to play with fun AR face filters under some time pressure.
Treasure Treat is made for people and their dogs. People can play fetch with their dog while going through an AR treasure hunt! The dog’s role is to collect the virtual coins, and the person can explore and advance in the game levels.
Moo Mission is a competitive game and is an intergalactic AR twist on the game “Whack-a-Mole.”
Freezing Frenzy is an AR battle competitive game inspired by the classic “Laser Tag.” With this app, players’ entire bodies enable the AR experience. Both players trigger the game on their device, and then their bodies serve as the target that gets augmented. Players can then throw virtual snowballs at each other. Why AR can be useful when designing for co-location?
Why AR when designing for co-location?
AR makes sense for designing co-located experiences because, unlike other technologies, AR relies heavily on the physical world, i.e., the reality shared by people inhabiting the same place simultaneously.
Based on our survey of the literature, we identified five reasons why AR is capable of supporting playful co-located social experiences. AR experiences are grounded in the environment; they cultivate ambiance by stimulating various senses. AR experiences are embodied since they can support novel physical interactions. They can facilitate playfulness by being surprising, humorous, thrilling, or creating challenging content. Social interactions and relationships can also benefit from AR, and it can be memorable due to a well-crafted experience design that can be quickly recorded and shared.
Key takeaways for co-located AR experiences
From synthesizing our research-through-design process, user study, and literature review, we identified key elements to consider:
Device Arrangement– How are mobile phones distributed among participants? For example, does each person hold their phone? Do they pass one phone around? Or something else? In making this decision, designers should consider at least two approaches to shaping the experience:
- To encourage touch through proxemics. Designers can create a digital experience that gets people close to one another so they can touch, for example, by playfully bumping and pushing each other.
- To Nudge person-to-person communication via coordination, for example, by creating playful scenarios that require people to share information.
Affordances of Augmentation–AR experiences can visually alter the user’s view of the physical world. Designers can use these alterations to support co-location in at least two ways.
- To entertain both the player and others by utilizing the visual effects of the augmentations as comedic devices for disarming people with laughter. For example, by turning the user’s face into a potato.
- To foster opportunities for people to experiment together. For example, people move around and point their cameras at different parts of the physical space to allow them to explore their environment with others.
Co-located Play–Although Game Design is a field on its own, we identified three ways designers might want to optimize technologies for co-location:
- To Leverage play “in real-time,” whether the playful activity is competitive or cooperative, finding the right balance between fun and challenge also involves attention to physical agility.
- To enable any number of players by letting people try out the experience on their own first, but show them how adding more players would make the experience more fun.
- Building on familiar play means that you can draw inspiration from existing game experiences people might already be familiar with in the analog world, e.g., board games and toys.
The Roles of Enablers (and what are they?) — How are visual triggers used to foster social interaction? AR systems often rely on visual markers to function. Designers should explicitly consider the potential for markers to play a social role; we call these types of markers: Enablers.
Enablers are not just physical entities that trigger the AR experience–they also play an integral role as the central augmentation from which the entire experience unfolds, grounding the experiences in the physical reality of players.
The process of designing and studying the apps with participants led us to notice that there was extra value in focusing on enablers that augmented bodies (e.g., faces, feet, dogs, and the entire human body) to draw people to focus on each other.
Millions of people tried out these ideas, and so can you!
Our designed apps were polished and released to millions of people on Snapchat. You can try them out with friends, too, at letsplayirl.com.
We describe our complete design process, user study, and reflections in the paper “Project IRL: Playful Co-Located Interactions with Mobile Augmented Reality” (you can find it at [arxiv] or [ACM digital library]).
