The playful side of physics
How we built Simple Machines, an App Store Best of 2015
Sky Debreuil, Free Debreuil (twins!), and Yin Liu from Tinybop’s development team walk through some highlights and learnings from our iOS app Simple Machines. The app is an App Store Best of 2015 around the globe; it also won a Parents’ Choice Gold Award.
Every app in our Explorer’s Library series poses unique challenges. For the series’ fourth app, Simple Machines, we had to make a seemingly esoteric topic (simple machines are tools we use every day) engaging for our young audience. We had to create physics simulations that were scientifically accurate and also fun.
Here’s how we did it in Unity.
When building any interactive media, prototyping is key. Ideas on paper are quickly tested out with interactions at fingertips. This was especially true for Simple Machines. We didn’t have art assets when we started building, so we kicked off prototyping with test assets: simple squares and circles. This allowed us to focus purely on the physics and the interaction flow.
These quick prototypes gave us the opportunity to touch and play with the interactions almost every day early on in the development process. This provided two major benefits: one, we were able to put something in front of kids to play test and give crucial feedback; two, research could keep a close eye on the feel of the physics simulations to make sure we maintained scientific integrity.
We designed an input system with a focus on multi-touch and different types of gestures. We have a touch-and-zoom gesture, with support for an object “capturing” a touch. We wrapped the Unity Touch and created a spoofed version that works with a mouse. This provided us rich information like phase, number of taps, start time, and mean values.
To interact with a scene, we raycast the first frame, then switch to a trigger which is dragged around the scene with the touch, using physics events for input. This means the initial input is instant (physics events generally take one frame to process) but is lightweight — an additional trigger is fast while raycasts are extremely slow.
Events are sent to objects (the Collider2D) using SendMessage, although we have a layer of type safety on top of using interfaces. In most cases, we use a matching interface when using SendMessage.
As a side note, how these interfaces function and look is very similar to Unity’s canvas interface events. We try and go with the grain, and if possible, try and code with same mindset as Unity, as these features are generally better supported and it is simpler to have fewer styles.
We used a lot of what we call “effect graphics.” We combine a physics simulation with graphics effects. A good example is the pulley; the physics and the graphics are simulated separately.
Another example is the castle you can destroy with a lever. In this scene, the castle bends back and forth slightly. But the actual physics and objects do not move inside Unity. We use a custom shader to move the vertices right before they are rendered. Since objects aren’t moving inside Unity, this is very fast. Objects in Unity are made of points, and vertex shaders can animate these points directly in a basic way. So, we can do waving, bending, ripples, morphing, etc.
Automatic graphics sorting
We are always striving to improve our processes. We had some issues with sprites sorting in the app we released before Simple Machines, Homes. For instance, if you interacted with an item, it would show up in front of others when it was expected to be behind. So, for Simple Machines, we created a system that sorts graphic objects based on their order in the hierarchy of the scene, using editor scripts. Functionally, this works very similarly to the Photoshop or Unity canvas sorting system: the farther down objects are in the hierarchy, the higher their sorting layer. The sorting can be overridden if special sorting is needed. This system greatly reduced our production artists’ workload as they placed assets in the scene.
Dynamic mesh generation
In Simple Machines, you drag a slider to see an alternate view of the scene. When you interact with machines in this view, arrows illustrating force help you understand the mechanical advantage of the machine. To have these arrows stretch and bend dynamically with each machine and each touch, we created a system to generate meshes at run time so we could modify the geometry dynamically.
The editor tools we created also became useful in free-form deformation of meshes. For example, in the wheel and axle scene, a scarf worn by a cyclist is an animated mesh thats geometry is modified while it’s floating in the wind.
Illusion of life
We have some systems that add an illusion of life to objects to make them look more dynamic. For example, in the lever scene, a ball stretches and contracts when it collides with other objects — like a rubber ball.
We created this by moving the graphic to a child GameObject. We then added one extra layer in the GameObject structure that rotates in the direction the object is moving, and scales based on how fast the object is moving.
This allowed us to create illusion-of-life effects using only Transforms and Sprite Renderers, which are simple and very compatible.
Some more advanced effects were created with dynamic mesh generation, in which we use a somewhat complex base class (we call it Visual) that we pass points using IEnumerator<T>.
The triangular fulcrum below is an example of an “anticipation” effect, as your finger drags it left and right.
Mixing animation with physics
Simple Machines relies on physics for most interactions but we also used custom animations to give the app more character. We found combining animations with physics can be very effective. For example, in the wheel and axle scene, the cyclist’s body relies on physics while his facial expressions are a product of Unity frame animations activated through standard Unity Mecanim state machines. Instead of creating a “riding” animation, we moved the feet “targets” in code to follow the rotation of a kid’s finger; the cyclist’s calf and thigh naturally follow with inverse kinematics. In instances when the cyclist raises his hands up in the air, we animated the hand targets and the arms follow with a simple IK calculation. Ragdoll physics are employed when the cyclist falls off the bike, so the fall varies with the obstacles and the velocity of the bike.
Sprites atlas generation
We have found that working with sprite atlases in Unity can be painful. Setting sprite atlases manually is tedious and error prone. And removing a texture from a scene does not automatically remove it from the atlas.
We have a system that automatically sets sprite atlas names based on the level. Textures detected in the level are added to the sprite atlas using editor scripts.
In Simple Machines, this was automated for the most part. If a folder or texture name was prefixed with an underscore it was not placed in a sprite atlas.
We have scripts that run in Play Mode in the editor which throw errors if textures from other sprite atlases are detected in the scene. We used a lot of this soft state validation in the app.
When we want to opt out of this system, we can add [IGNORE] to a folder name.
Sound design process
For this app, we wanted to give sound design complete control over audio effects, making sure they could edit audio directly inside the Unity project.
We created a system to separate audio from the scene into a specific prefab, which could be fully assigned to Brian Jacobs, our master of sounds. Triggers are activated in the scene using what we call “global events,” which can be tweaked and recombined inside the Audio Prefab.
One exception was the generative audio in the inclined plane scene. The audio effects there are a result of collaboration between sound design and the dev team.
New apps, new learnings
While what we talked about here mostly applies to Simple Machines, it overlaps a bit with other apps in the Explorer’s Library series. But our Digital Toys apps have very different engineering problems and development processes.
Every new app we make gives us the opportunity to evolve our processes, solve new problems, and learn new things.