Dance Bot: a fun way to dance with your friends while staying physically apart 🎶

My experimental project at the 2020 Facebook AR hackathon

Shin Young (Lucia) Choi
Jul 17 · 4 min read

In May 2020, my friend, Andrea, and I joined a virtual hackathon hosted by Facebook. Using Facebook’s AR design tool (Spark AR studio), we and the other participants created World AR effects for Instagram. We explored applications of augmenting experiences on surfaces, people, and environments viewed on a world-facing camera.

presentation & demo video

Inspiration

COVID-19 impacted our lives — including our mental health, especially loneliness due to long periods of isolation. We wanted to create something that promotes people’s wellness and social interaction while supporting social distancing efforts to flatten the COVID-19 curve.

Our common love for dancing inspired our design. We saw that by combining physical movement and World Effect, we could create something that encourages us to exercise and engage with friends and other loved ones while remaining six feet apart.

How it works: the Heads Up! game with dancing

Our Dance Bot filter requires a minimum of two people. The first person reads out dance instructions as they record the second person dancing along to the Dance Bot according to those instructions. The players can then share the video online and encourage others to try our Dance Bot challenge.

Image for post
Image for post
snapshot of our demo

How we built it

Storyboarding to align the North Star

As we were working remotely, storyboarding ensured that we were on the same page regarding how Dance Bot would work and what we needed to build.

Image for post
Image for post
quick storyboard sketch

Character design and animation

We started a character design from our project name Dance Bot. We added elements like a disco ball as we went along. (Honestly, nothing could go wrong with it.) We continuously evolved our character based on 3D models I gathered and technical requirements by Mixamo.

Image for post
Image for post
Early sketches of Dance Bot

We animated our 3D model with Mixamo, an amazing tool that automatically rigs and offers numerous full-body character animations to apply on a 3d model.

Image for post
Image for post
Screenshot of Mixamo Animations library page

Choreography: putting together animation

I then arranged animation from Mixamo to create choreography.

Image for post
Image for post
Ideating choreography. For our MVP, we decided to have three themes: Happy, Funky, and Groovy

Art direction: ~ let’s get dancing ~

We gravitated towards 80s & 90s designs that use pop colors and fun shapes (aka Memphis). We applied these aesthetics to texturing and the overall look-and-feel of a filter.

Image for post
Image for post
Sample search result for “90s design pattern” on Pinterest. It screams “~ F U N ~”

Stitching together on Spark AR

Once I built and exported an animated character and instruction cards, Andrea stitched them together with patches on AR Spark. The patches determined the logic behind how Dance Bot would work.

Image for post
Image for post
snapshot of Spark AR patchwork

What we learned

Through this hackathon, we learned more about 3D and AR design workflow and tools. It was our first time using Spark AR, so we were able to practice the basic features of Spark AR and the usage of patches.

Sizes and textures matter…

In the later implementation phase, I learned about the size and texture import constraints. I had to tweak the character and its animations in order to meet file size requirements.

Image for post
Image for post
DanceBot created on Blender
Image for post
Image for post
Due to the file size limit, we had to take out assets (no cool Vans) and tweaked dance bot a bit in SparkAR

Closing Thoughts

Spark AR is a great tool that can easily create and share AR effects with a phone. I’m proud of our team being able to build and share our Dance Bot.

What’s next for Dance Bot

If we continued this project, we would love to add more dances and interactions. We talked about adding dances based on different cultures and regions. We also discussed making the dances more inclusive by taking into account different physical abilities (such as people in wheelchairs).

Image for post
Image for post
#DanceBotChallenge, anyone?

Project Links

Project Duration

May — June 2020

Built With

Blender, Figma, Facebook SparkAR

Credits

Andrea (Hella) Chen (Co-hacker)

3D models are from:

Shoutout to the creators of Blender, Mixamo, and SparkAR tutorials on YouTube ❤

Saper Vedere

Personal journal of learning “how to see” the world

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store