Google I/O 2016: Paper Planes
Paper Planes started as a simple thought - “What if you could throw a paper plane from one screen to another?” After gradual work and brainstorming, we shared the idea with our friends at Droga5, who helped us bring it to the biggest possible stage: Google I/O 2016.
The heart of our concept was to bring people together from all over the world, using the power of the web - an instant connection to one another by a simple visit to a website. Modern web technology, specifically JavaScript and WebGL, powered the experience on every screen. From the 50-foot screen on stage at I/O to the thousands of mobile devices in the crowd, one codebase was written which drastically reduced development time, allowed for more iteration, and increased time available for polish and animation.
Launched publicly on Peace Day 2016, Paper Planes is available on the web at https://paperplanes.world and can also be downloaded as an Android app on the Google Play.
Introduction
Paper Planes was featured at Google I/O 2016 on May 18th, 2016 as a pre-Keynote event, bringing together attendees and outside viewers, in the 30 minutes leading up to Sundar Pichai taking the stage.
Fans simply visited a URL on their mobile device and were prompted to create their own plane by adding a stamp that is pre-filled with their location.
Once a plane is created on mobile, a simple throwing gesture launches the plane into the virtual world. During the pre-Keynote, attendees would immediately see their planes fly into the 50-foot screen on stage. Users at home or an I/O viewing party would see their planes fly into a desktop screen, browsed to the same URL with no syncing required.
Later, users could come back to the mobile site and see where their planes were caught around the world. Each stamp on the plane read like a passport and a 3D earth highlighted flightpath and distance travelled.
Besides making their own planes, users could gesture their phone like a net to catch a plane that had been thrown from elsewhere and pinch to open it, revealing where it had visited.
The Communication Network
Underlying our installation for I/O was a network of servers on Google Cloud Platform. We used built-in geocoding headers to get approximate geographic locations for stamps and Socket.IO to connect all devices over WebSockets, funneling messages to our graphics machine at I/O rendering the experience.
I/O attendees were connected directly to the main server that fed into the graphics machine while those outside I/O were connected to the server nearest them, which relayed messages to the main server as well as to any desktop computers viewing the experience in that region.
An in-depth article on this complex network is available here.
Look & Feel
The aesthetic of paper planes aimed for a feeling of child-like playfulness and relaxation. We used a blend of pink, blue and teal pastel colors to create a sky gradient that constantly changed during the experience.
The earth was shaded with similar tones but we used specular lighting and a teal rim glow to help distinguish it from the background. The earth and sky were a constant mixing and changing pair that needed to look good regardless of the earth’s rotation or sky hue.
Taking advantage of the strong graphics card, we polished the big screen rendering with a few extra details such as gentle waves on the ocean by moving each vertex along the face of the Earth’s sphere using polar coordinate math.
Additionally, reflections on the ocean were added by rendering the entire scene from the center of the earth to a CubeCamera and passing in the resulting cube map into the ocean’s shader, generating real-time reflections.
Flocking
As planes were thrown into our world, they began to flock together and fly around the Earth. In order to animate a large number of planes at once, a technique called Instanced Buffer Geometries was used, which allowed us to render thousands of planes with a single GPU draw command, manipulating each plane’s position on the GPU in shaders.
Flocking is a complicated mathematical process that is tricky to performantly implement in code. Two techniques were tested: first, computing the entire flocking simulation in a number of shaders, running on the GPU, and second, creating a simulation on the CPU and sharing the load between several WebWorker threads.
We were surprised to find that using threads was almost equal in performance to the GPGPU method. Since writing simulation code is much simpler in JavaScript than inside shaders, we went with the second option.
We implemented the exact same simulation on mobile by just reducing the amount of threads and number of planes to be simulated
WebGL Text
Locations where planes originate cycled through during the event, creating an impactful connection for people around the world. We ran into a bug on Chrome for Windows, where no matter what we had tried, compositing CSS text animations onto WebGL incurred a severe performance hit, dropping our frame rate below 60 frames per second.
The solution involves generating a sprite sheet image with all of the characters upon it, along with a text file containing all of the font metrics, coordinating to each letter on the sprite sheet. Then, in the 3D scene, a quad for each letter was created with the corresponding section of the sprite drawn onto it. We were then able to animate each letter’s position and opacity as if it were any other Three.js object.
An added benefit was that text was entirely in the 3D environment, allowing planes to occlude it as they flew into the experience!
Optimizing for many devices
Paper Planes demonstrates how a single build using modern web technology can reach a wide range of devices - from a 50 foot screen at I/O to many types of mobile phones, desktop and laptops. As new features were added, adjustments for different devices were needed and implemented after beta.
A great example is the number of planes in the flock. While on the I/O screen thousands of planes could be distributed across 8 WebWorker threads, on mobile devices, the exact same system was used and scaled down to a few hundred planes across 2 threads.
On desktop, we took into account the type of graphics card a visitor has and calculated how long the CPU takes to calculate some basic trigonometry in a loop. We wrote code with these factors in mind and decided the right number of planes and threads to render the flock and get smooth performance.
Conclusion
With users participating from over 180 countries, the reception has been humbling. Extending an age old past time like paper plane throwing to connect people from all over the world through a simple gesture was an amazing feeling.
Paper Planes shows the power of the web through a simple experience that connected strangers from all around the world.
