Introduction to spatial software

The Apple Vision Pro was announced back in June of 2023 and since then it has been a challenge to answer a series of essential questions like “When will it be launched?”, “Should we expect similar experiences to other devices?” and even “Is it the beginning of an end?”.

Motivation

Besides all the problematics created on the adaptation process of having another wearable device — nothing like a smartwatch — from Apple, the concept of Mixed Reality was clearly innovative. Until a year ago, we only had experienced softwares with Augmented Reality and Virtual Reality in separated scenarios:

• With augmented reality, we could design 3D models and project into our world with the help of camera software. Fascinating but is limited on immersion.
• Virtual reality was approached by Apple’s competition on this particularly race, e.g. Meta Quest from Meta, which provides full immersion taking the user alway from the real world.

What Apple came up with on WWDC ’23 is a device that could blend software with reality, make things feel alive, finding key moments to input immersion. Well, it drove me some interest to know better about how to build applications for Vision Pro, in the context of user interfaces and experiences.

So this article will comment on John Palmer’s “Exploring spatial software” talk on Config 2021 (available on: https://www.youtube.com/watch?v=H4LosXgKX4g).

Exploring spatial software

The way one might model the interaction of humans with the real world can be described as a system of bodies and spaces. These two concepts may seem a little abstract so we can also represent them as animals exploring and interacting with an environment.

Left: scheme representation for bodies (light blue circles) and spaces (light purple rounded squares). Right: scheme representation for the interaction between bodies and spaces — or animals and environment. Source: Author.

The bodies are supposed to be kinesthetic, move around and explore the spaces as they interact between them. An example of how well it can be applied is the mouse cursor inside a Figma project.

Figma multiple user cursors (bodies) interacting with an infinite canvas view (space). Source: Figma help. Available on: https://help.figma.com/hc/en-us/articles/1500004414622-Use-audio-to-chat-with-your-team.

When to invoke spatiality?

John’s take about spatiality is that it should only be invoked when helpful. Well, how to identify these helpful moments then?

• When another ways to present content would make it less user/task friendly (e.g. Adobe Illustrator).

Adobe Illustrator blank file centered canvas (space), with tools representing different types of bodies. Source: Adobe Community. Available on: https://community.adobe.com/t5/illustrator-discussions/full-screen-on-illustrator/td-p/10452732.

In general, creativity softwares feel right with a canvas space to join pieces together.

• When we have complex human dynamics (e.g. Zoom).

Zoom software default meeting interface, with user cards representing bodies and the collection of them composing a space. Source: Zoom developers. Available on: https://developers.zoom.us/docs/meeting-sdk/.

There is nothing like human meetings that could replace people’s own voice and figure.

• When we are trying to recreate real world scenarios (e.g. Gather)

Gather Town virtual office meeting, where the 8-bit users represent the bodies moving around the office itself (space). Source: Gather. Available on: https://pt-br.gather.town/blog/better-video-meetings-ultimate-guide.

Complementing the idea of complex human dynamics, one could build physical spaces inside the virtual world.

• When movement makes things fun (e.g. Google Maps)

Google maps dragging body for 3D view. Source: Google Maps. Available on: https://maps.google.com/.

Again, the cursor (body) moving through a map (space), but with this nice little touch of adding world’s physics to the 3D dummy.

General considerations

To achieve a more cohesive spatial product we should think on how to design its space. It is very powerful to remember that we can build on softwares what we could never build in the real world.

But also, one could remember that it is important to have a familiar and user-centered concept to create spaces that are user friendly. More on that may be found on my next article: Principles of Spatial Design.