Immersive Data: The Future of UX Design

Since the first graphic user interfaces, we have relied on metaphors to make computers accessible. The desktop metaphor, rooted in the office with its files and folders, opened up computing to broader audiences and still dominates most operating systems today. The World Wide Web, with its pages and hyperlinks, is based on a library metaphor, referencing printed publications and citations or footnotes. Mobile device interfaces evolved from PDAs and are based on various metaphors from the world of personal organizers and phones, from the phone to the personal organizer, with an interaction paradigm where apps are modeled after physical cards.

What all of these metaphors have in common is that they provide familiar conceptual frameworks for understanding how to use intangible digital platforms. Increasingly, however, these metaphors limit us in a world where we generate zettabytes of data every day and where the emergence of AI is having a profound and disruptive impact on the world. There is untapped potential for new interfaces that break free of the old metaphors from a bygone era and that leverage AI to better empower exploration, understanding, and interpretation of very large and complex datasets.

The Rise of AI

Generative AI represents a massive step forward for computation with enormous implications for human-computer interaction. It will require rethinking fundamental assumptions that have shaped digital interfaces for decades.

A core capability fueling the rise of AI is its ability to synthesize and analyze information. AI facilitates a perpetual cycle of turning unstructured data into structured data, and vice versa. On the one hand, AI can turn data into stories, communicating ideas and information (synthesis). On the other, it can deconstruct narratives, analyzing them to create interoperable systems (analysis). This transformative dynamic between synthesis and analysis allows AI to continuously reconfigure information and ideas, enabling new possibilities for contextualization, sense-making, creativity, optimization, and personalization.

Realizing AI’s full potential requires a new approach to interface design that better facilitates knowledge sharing and creation. Data visualization offers a solution.

What started as a largely academic discipline at the intersection of science, technology, and art has evolved beyond static charts and graphs and interactive dashboards into rich, dynamic experiences that can bring stories to life and facilitate self-guided discovery. In this age of data, data visualization has emerged as the new interface paradigm, and, just as desktops and mobile devices transformed computing in previous eras, spatial computing is poised to revolutionize the way we interact with and understand data.

Leaving Flatland

Until now, our interactions with computers and mobile devices have largely been constrained to screens. Spatial computing is moving us beyond two-dimensional interfaces and enabling more natural and immersive interactions with information.

As science fiction authors like William Gibson and Neil Stephenson imagined, spatial computing is creating digital and hybrid digital/physical environments in which information is liberated from flatland and can interact with our real-world surroundings. Spatial interfaces make it possible to interact with information the way we interact with the physical world around us. Information is set free of its two-dimensional containers and made both scalable and tangible.

Spatial computing is the ideal medium for data visualization. Instead of looking at two dimensional charts and graphs, we will instead step inside dynamic 3D models and simulations. Imagine walking through a data landscape where market trends tower over you like skyscrapers, or diving into a holographic model of molecular structures with the ability to change parameters in real-time. Imagine exploring climate data on the surface of a three-dimensional globe, or health data mapped to a digital twin of a patient. These are the possibilities spatial computing brings to data visualization.

Immersive Data

Spatial computing is poised to transform the way we interact with and understand data by offering a more immersive, adaptive, collaborative, and integrated experience.

1. Immersive: Spatial computing provides an infinite canvas for data exploration, allowing users to interact with information in a three-dimensional, immersive environment. By leveraging natural interactions like touch, speech, and gesture, and utilizing our innate sense of spatial memory, spatial computing makes data exploration more intuitive, engaging, and efficient compared to traditional interfaces. This immersive experience enables users to uncover patterns, connections, and insights that may not have been apparent in two-dimensional interfaces.
2. Adaptive: Spatial computing enables users to manipulate the scale of data representations side by side without the constraints of screen boundaries. Visualizations can be dynamically scaled and viewed from any angle or distance, providing a flexible experience that caters to the user’s need for detail or overview. This adaptability allows users to tailor their data exploration experience to their specific needs and preferences, surpassing the limitations of traditional interfaces.
3. Collaborative: Spatial computing environments can be shared and experienced by multiple users simultaneously, fostering collaborative data exploration and decision-making. This capability goes beyond what traditional interfaces could facilitate, as users can now interact with data and each other in a shared, immersive space. Collaborative spatial computing enables teams to work together more effectively, discover insights collectively, and make better-informed decisions by leveraging the knowledge and perspectives of multiple individuals.
4. Integrated: Spatial computing also allows for the integration of real-world environments with virtual data overlays, creating a seamless blend of reality and virtual data representation. This hybrid approach enhances understanding and application of data insights by presenting information in its natural context. By integrating data with the physical world, spatial computing bridges the gap between abstract representations and real-world applications, making data more relevant, accessible, and actionable.

The New Frontier

As AI continues to advance and generate increasingly complex and growing data, spatial computing offers a powerful solution by providing a new approach to interface design that better facilitates knowledge sharing and creation. The immersive data experiences enabled by spatial computing will unlock the full potential of AI, paving the way for the next generation of tools that enable decision intelligence and empower individuals and organizations to make better-informed decisions.

The synergistic relationship between AI and spatial computing represents a critical step forward in our journey towards a more intelligent, data-driven future. Just as AI seeks to augment human intelligence and provide valuable insights, spatial computing seeks to enhance the way we interact with and understand data. By leveraging the strengths of both technologies, we can create a powerful symbiosis that amplifies the impact of AI and transforms the way we engage with data.

This convergence marks a paradigm shift and a new era for human-computer interaction and user interface design, leading us towards a future where data is not merely visualized, but experienced.

At Schema, we believe that the AI revolution requires designers to abandon conventional thinking and constraints to shape a future where AI and human creativity come together to augment and elevate digital experiences.

Connect with us.

Christian Marc Schmidt is Founder and Partner at Schema, a creative technology firm based in Seattle and New York City.