A new wave of AR Realism
Google ARCore Depth API Release
(completed September 30, 2020) The ARCore Depth API revolutionized the field of augmented reality by providing depth information without relying on LIDAR technology. Unlike LIDAR-based solutions, this API utilizes depth from motion, making it compatible with a wider range of mobile devices.
I collaborated with both designers from the Google Creative Lab team (to improve the user interface and conceptual aspects of the code samples) and the Augmented Perception team in Google Research to seamlessly integrate the Depth API into Unity Game Engine. Together, we achieved successful integration and enhancement of the API, bringing advanced functionality to game development.
Furthermore, we expanded the range of sample scenes available to AR developers and provided comprehensive documentation. The AR Depth Realism samples can be found here on GitHub. The “Lines of Play application can be found here on GitHub.
Integration into Various Platforms
The efforts invested in the development of the ARCore Depth API have resulted in its successful integration into Snap Lens Studio and Spark AR.
By leveraging the Depth API and, when available, LIDAR technology, realism in AR applications is significantly improved. The ability to navigate and avoid obstacles not only enhances the overall user experience but also enables the creation of more contextually aware and interactive AR content.
Lines of Play: AR Dominos
To demonstrate the capabilities of the Depth API, we developed an AR domino application that enables object interaction in real-world environments, thereby fostering dynamic gameplay creation. To facilitate rapid level editing in augmented reality, we implemented corollaries to vector points, lines, curves, and polygons. By utilizing these vector splines, users can easily create clones of any 3D object.
While we utilized dominos as the primary objects in this application, we made the entire application open-source, encouraging developers to replace dominos with diverse environmental game assets such as trees, mountains, and more. (Hint: If someone wanted to make an AR Minecraft? — it’s all there)
Addressing Precision and Efficiency
In AR level creation, striking the right balance between precision and efficiency is crucial. Recognizing this challenge, we aimed to mitigate it in our application. Before placing clones of objects in the real-world environment, we introduced ghost objects that visually inform users of the intended placement locations. This approach ensures that users can preview and refine their designs before finalizing them, saving time and effort while maintaining a satisfactory level of precision
By utilizing vector splines as paths, users can easily duplicate and distribute objects in a controlled and efficient manner. The vector path serves as a guide along which the clones of objects are replicated. This approach offers a streamlined workflow, as users can generate multiple instances of objects along the vector path without the need to individually place each clone.
The following glossary presents the iconography associated with the in-app functions for placing dominos, showcasing their respective visual representations and functionalities.
Enhancing Realism and Interactivity
One of the primary challenges faced in AR development is the lack of integration between virtual objects and the real-world environment. This limitation has hindered the widespread adoption of AR avatars and characters. However, with the introduction of the Depth API, we can overcome this obstacle and enhance realism by accurately incorporating virtual objects into real-world scenes. By integrating depth information, we enable natural movements and dynamic obstacle response, thus making the AR experience more immersive and lifelike.
Utilizing depth data from the Depth API, occlusion and collision can be seamlessly incorporated into augmented reality experiences. By analyzing the depth information of both real-world and virtual objects, occlusion can be achieved by accurately rendering virtual objects behind physical objects, creating a more realistic and immersive environment. Furthermore, collision detection becomes possible by leveraging depth data to detect physical obstacles and establish spatial boundaries.
In summary, the ARCore Depth API represents a significant advancement in augmented reality technology. Through collaboration with talented teams and integration into various platforms, we have successfully improved realism, interactivity, and integration in AR experiences. The AR domino art app serves as a practical example of how the Depth API enables object interaction and encourages developers to explore and expand the possibilities of AR content creation.
Read more in Ars Technica, Forbes, CHI Play 2020, and the Google Developer Blog!
