Starpilot Heroes — An Experiment in Augmented Reality

In three weeks Mike and Paul, a dev and design pair, built Starpilot Heroes, a collectable card game prototype that integrates augmented reality as part of the core game concept. Here they share their experience.

The Experiment
Starpilot Heroes started as an in-house technology experiment to explore possibilities designing with AR. Players start by assembling a spaceship fleet with physical cards, then scanning that formation into the game. Your formation competes against an opposing enemy fleet in a colorful particle-filled battle scene to determine who is victorious, not unlike in other tower defense games. Cards have markers, hidden codes embedded in them for the computer to recognize, similar to a QR code or barcode. This AR technology allows players to can scan an entire card formation at once from some distance away.

The prototype decks contains a variety of hexagonal cards, including ships and power ups such as “ammo boost” that augment properties of ships attached to them. At the start, players can use maximum of 7 `cards to form their fleet, like so:

Picking a concept
The goal of this prototype was to integrate AR as a core component of the experience we were building. We started by paper prototyping various ideas that could be developed in within the 3 week constraint we were given.

Some concepts we played with included:

• Silly Snapchat-like filters for our photobooth
• Various software utilities
• A party app to scan your nearly depleted alcohol bottles, and get a list of cocktail recipes to make using what’s available to you.
• Scanning game cards representing building materials made of different types candy that can be stuck together. Build a Jenga-like tower and test the durability.
• Scan a unrecognizable street sign or car maintenance light, see what it means.

We eventually landed on a game concept in the vein of chess…in space!

Vuforia, Unity, and ARKit
We integrated the augmented reality framework Vuforia into the Unity game engine to recognize the cards we designed. It handles the computer vision to tell us in 3D space where the markers are and their orientation relative to each other. We took that data and put it into a Unity scene to play out the simulation battles. We’ve used Unity for our prior game projects (like Dungeon Highway and Exploding Kittens), so it seemed the natural platform for us given our time constraints.

ARKit became available to us midway through the project with Apple’s surprise announcement. After some late consideration to pivot into Apple’s framework, it became clear that it wouldn’t have worked for our particular design. ARKit finds empty space for you to display digital things in, and then acts like micro-GPS so you can move the camera around and look at it from different angles. It can’t recognize markers or 3D objects, which means it wasn’t right for our purposes. Still, it’s interesting tech that we’re excited to explore.

Designing in AR

The first week of development was focused on getting multiple markers to scan quickly and accurately. Vuforia has tools to help you determine what would be the most readable marker for print production. Once the image vectors are placed into Unity, we were able to test via webcam or phone. The only way to test was to hold a marker to the camera and hope the it would render. There were no systems in place for Vuforia to detect why Unity wasn’t recognizing the image. We defaulted to a few hypotheses: Is the paper material too glossy? Are there slight paper creases affecting image recognition? s the contrast not high enough? Is a finger overlapping the border? As we iterated our cards and markers, we narrowed those answers down and were eventually able to get prints that would scan almost as soon as the camera was hovering over the images. With that hurdle out of the way we could flesh out the rest of the prototype.

Playtest sessions

Once we got past our 3 week timeline to reach a playable prototype, we gathered a group of Substantialites for an internal playtest. We split up builds and card decks into two groups.

The card decks we used were cut by hand and printed in-house. It turned out that the lighting conditions of our playtest space had a drastic effect on the technology and its ability to accurately scan the cards. On the other hand, we discovered that the game was more engaging in social environments than we expected. People often shouted when a battle came to a close finish, and both tables had people from all over the company spontaneously playing together, strategizing on fleet configurations. In the excitement, nobody really minded our technical issues.

In a later iteration, we tested with a batch of laser cut cards that were professionally printed on higher quality paper. In reducing technical difficulty, we gained valuable feedback on how this prototype could be developed as a fully fledged game. We were extremely curious to know what players expected of the game was before seeing it, compared to what mechanics they wish they had and what ideas were brewing after observing the demo.

Post-prototype plans

We considered our prototype to be a bottom-up slice of functionality and design. We wanted to build something that could effectively sell the core vision of the game. Leaving parts of the game unresolved for testing allowed us to gain feedback in a way that was less biased than what we would have received from a more complete version. We were more interested in how the game could be pushed into different verticals of design, things that we didn’t explore, things that we should continue to progress. For now, we’re happy we learned more about what makes AR compelling as a medium. The game currently exists as a fun tech demo that allows us to approach the subject of augmented reality with slightly more answers than questions.

Want to try Starpilot Heroes for yourself? Get in touch.

games gaming augmented reality Vuforia Unity ARKit ar

Read more by Mike Judge & Paul Norris

Originally published at substantial.com on August 29, 2017. Interested in more about us? Visit our website or read your heart out here on Medium.