Predict the future UX and UI of the VisionPro app 😎

Published in

Antaeus AR

6 min readFeb 18, 2024

Hello everyone!
I’m taka(@takataka530) from MESON, a company based in Tokyo, Japan that extends people’s gaze with spatial computing technology.

In this article, I discuss the future of UI and UX based on my impressions I gained while testing various apps using AppleVisionPro (AVP) almost every day for about 2 weeks 😎.

UX that may emerge in the future

Competition for space and an increase in experiences that target the desk area

As we enter a phase where many high-quality apps that take advantage of AVP features are launched, the competition will intensify.

And in addition to the traditional “competition for time,” “competition for space” will finally be in full swing.

For example, SunnyTune, which we launched, was designed based on the concept of “information goods” ,

SunnyTune, an app developed by our company that allows users to “feel” weather information

In order to use an application that blends into real space, “free space on physical space” is required. However, space is finite.

The intensity of competition and the value of space can both vary depending on the location within a room.

For example, around the desk in one’s room where one works, the value is likely to increase because the time spent by the consumer is longer = more likely to generate touch points with the application.

…

A device that gives a “casual” presence without letting the application close.

In such a situation, I wonder if an effective way to take space away from competing apps is to not create an opening for them to take it away from the competition, i.e., to not let them close the app.

Concept image of a notepad app (generated by midjourney). It would be interesting to have an experience where cute animals made of sticky notes are lined up in a row, and if you stare at them for a few seconds when you need them, they turn into notepads or used notepads turn into animals.

For example, it would be good to have cute 3D objects and a feature that plays healing sounds that soothe you while you work, except when you are looking at or manipulating them.

…

Support for converting an existing flat website into a three-dimensional website

First, please take a look at the video in the post below! This is a video of the Louis Vuitton website viewed with the AVP web browser.

The 3D model is utilized to give a sense of depth, and when the screen is spread out wide in front of you and operated, you feel a very deep sense of immersion!

Recently, more and more websites are incorporating 3D model designs. It would be a shame to show the 3D models created with Blender on a flat surface, and I think there is an issue of not being able to express the brand’s worldview on a flat surface.

Therefore, we think there may be demand for an application that can easily reconfigure existing PC screens into a three-dimensional form, for example, so that models and videos of products that are particularly appealing can float off the screen.

The target audience would be companies that have brands in the high price range, and it would be a good match with their brand website policies. It would also match the fact that buyers are likely to have high incomes and be able to afford AVP.

…

Increase in the number of applications that allow users to “feel” information

Now that developers have actual AVP equipment, we will see more and more applications with three-dimensional experiences and designs that are unique to SpatialComputing.

One concept that is likely to catch on is the “experience of information”. The weather application “SunnyTune” developed by our company is one such example.

Concept image of timer app (generated by midjourney). As time passes, candles melt, frogs and plants grow, sand from the hourglass falls…

For example, I have seen many timer apps for AVP, but they all have a similar UI with the time displayed on a flat surface. I think this could be made into an app that allows you to “feel the time” more easily….

…

Increase in sound-based healing applications

While Quest has many immersive meditation apps that focus on visual experiences, I believe AVP will increase the number of auditory-centered “casual healing experiences.”

It seems to me that AVP is more intended for business use cases than entertainment.
If working with AVP becomes a habit, I think the time spent using it per day will tend to increase. Then, I think that refreshing actions in the middle of work would be more effective to remove the headset than to open a dedicated application.

Light and sound meditation application “Endel” during the experience.
After using it for a long time at work, I had the impression that it was very visually demanding.

I myself, after actually using AVP for a long time, felt somewhat “relieved” when I removed it from my head, as my vision was clearer and the amount of information in the space was reduced.

On the other hand, I believe there is a need for a change of pace “while working”. This is the kind of behavior that I usually do with background sound.

Auditory information is hard to overflow and can take advantage of existing habits, so it is likely that healing applications centered on sound will appear quite often in the future.

…

UI that may appear in the future

UI-ization of “surfaces” such as real walls, desks, etc.

UIing “surfaces” such as real walls and desks
In X’s post, we often saw use cases where the screen was installed along the wall of one’s room or office.

https://x.com/darius/status/1753567662859538727?s=20

In fact, I tried it myself and felt very “comfortable” with it.
I thought about the reasons for this, and came up with three reasons.

1) Similarity to existing mental models
2) Clear tactile feedback during operation
3) Reduction of physical load during operation

details…

1) Similarity to existing mental models

In my room, I often have TVs, pictures, posters, etc. crawling on the wall, and when I installed a flat UI Youtube on the wall with AVP, it blended in naturally without obstructing the real space, giving the impression of a truly increased LCD TV.

Capture of how the photo application was set up along the office wall.

2) Clear tactile feedback during operation

In particular, since the website-based UI is similar to that of smartphones and iPads, I feel that people subconsciously seek familiar operations by touching the screen with their fingers. I felt that I could get a “sense of operation” by touching a real surface and feel at ease.

3) Reduction of physical load during operation

Although the high-performance hand-tracking function allows operation with less movement, the motion of picking and moving fingers in the air is still somewhat tiring. We felt that it is better to press the finger against a wall or desk surface, which allows operation with a certain degree of weakness.

…

AVP’s unique UI optimized for eye tracking

For example, I would like the decision button to be vertical or square….

In eye tracking, I often find that the up and down gaze alignment is not good enough. Several members besides myself feel the same issue, and I think it’s because we are unfamiliar with up-and-down eye movements in our daily lives because we don’t do them very often. The buttons are a little too horizontal while the left and right gazes are easily aligned….