Creating Collaborative 3D Media
This blog post summarizes a full paper, available here. The paper will be presented at the ACM Conference on Computer-Supported Cooperative Work and Social Computing on November 6th.
Augmented and virtual reality systems provide a new infrastructure for crafting rich end-user experiences. However, these systems can be tricky to build, let alone to design end-user features and interactions. In this post, we will share the lessons learned from our work crafting a collaborative authoring tool for people immersed in an augmented or virtual reality.
In complex cooperative situations, the task diversity can be very high. The main limitation to support this kind of situation with Augmented or Virtual Reality is that one device can be useful for some tasks, but less useful for other tasks. For this reason, we argue that one device is not enough to support complex cooperative situations. This is especially true for the manufacturing industry, where actors with different roles and trades are involved.
During the past two years, our team has developed a new programming standard for designing AR/VR experiences, that abstract away from specific devices to specific features, rather allowing developers to focus on design for user interactions rather than the limitations of a single device. We call this standard UMI3D. We are excited to announce, and present at the upcoming CSCW conference, our new work which focuses on the collaborative authoring tools for UMI3D and proposes a new user-oriented method for the design of collaborative 3D media. By working with the Unity3D game engine designers were able to construct a playful cooperative construction game in record time. Further, our process provided new opportunities for end-users to engage with the design process.
Context of our work
In September 2016, Gfi Informatique and The University of Strasbourg initiated the UMI3D project. Our early goal was to simplify the collaboration between multiple and potentially asymmetrical devices in Augmented and Virtual Realities, and to enable the design of collaborative applications without prior knowledge of user devices.
The existing approach to develop Virtual and Augmented Realities applications regardless of user devices is the one taken by some programming APIs like WebXR. This approach consists of creating a generic code API to access the main device’s features. It also allows to run the same code on all the devices that implement the standard API. The advantage of this approach is the possibility to write a unique code that will run on different, but similar devices (like concurrent HMDs). However, if we want to support very different devices, it becomes hard (if not impossible) to exploit the specificities of each device.
New collaborative design possibilities
Our previous work about UMI3D has shown that UMI3D reduces the design time of 3D media. However, it was required to ensure that the main requirements of collaborative applications are reachable using UMI3D. For this purpose, we have developed a package for the broadly used game engine Unity3D. This package makes the design of 3D media with UMI3D possible and provides designer tools which allow the creation of collaborative interactions. These tools let us demonstrate through examples the support of the cooperative applications’ requirements.
Moreover, the combination of UMI3D and Unity3D provides design features which have surpassed our own expectations. This makes it possible to design and configure the 3D media while it is being tested by several users. The result is a significant reduction in 3D media development time, which allows for shorter iterations during user testing. This conducted us to adapt the user-oriented design methods to take the best of those properties. The proposed design methods rely on run-time design sessions in which the designer co-creates the media with potential users depending on their needs and comments.
End-users are taking advantage of the devices’ specificities
To illustrate the usage of our design tools and of our design method, we used it to design a playful cooperative construction game. This game lets multiple users build a virtual town using different Virtual Reality devices. The run-time design sessions had a very important impact in the creative process of this media. This has even led to the design of an additional game mode that allows the users to play with the created village by reproducing social interactions. Finally, the user study we present shows how the users evaluate their cooperation after playing the game, emphasizing how some of them have intentionally chosen a task repartition which takes advantage of their respective device’s properties.
Using our system in real-life industrial use cases
The presented results let us conclude that UMI3D is now ready to be tested in real-life use cases. Our next objective will be to focus on industrial use-cases for which UMI3D seems to be well adapted for, like for example, the training of cooperative manufacturing processes or interactive and collaborative visualization tools for complex systems e.g. factories, manufactured products, Industrial Internet of Things.
If you have questions or comments about this study, email Julien Casarin at Julien.firstname.lastname@example.org
Paper Citation: Julien Casarin, Nicolas Pacqueriaud, and Dominique Bechmann. 2018. UMI3D: A Unity3D Toolbox to Support CSCW Systems Properties in Generic 3D User Interfaces. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 29 (November 2018), 20 pages. https://doi.org/10.1145/3274298