A glimpse of the future at Meta
Answering questions about the Quest 3, Mixed Reality, Artificial Intelligence, and what it might mean for StellarX
A few weeks ago, we were invited to Menlo Park, California, to attend the Meta Connect event as partners. 🤩 Oh, what Artificial Intelligence (AI) and Mixed Reality (MR) prowess we witnessed…
Following the recent release of the Quest 3, we noticed many people didn’t quite understand the hype. What’s it gonna bring to the table, exactly? Well, we assure you: there’s plenty to be excited about.
To show you what we mean, we thought we’d answer some of the recurring questions we’ve seen. So, without further ado… let’s dive right in!
What’s new at Meta?
There was so much happening all at once at Meta Connect — here are some of the highlights.
First, the open source large language model Llama 2 is definitely a valuable initiative. It is comprised of generative text models, which can be used for situations requiring natural language dialogue. Think digital assistants, or the model that OpenAI’s ChatGPT is built upon. Llama 2’s open source mode will drive innovation further; as a matter of fact, we’re already seeing a number of great projects emerge from it. For us, it means we’ll be able to incorporate more AI capabilities into our software in the long run.
It was also awesome to witness the latest Mixed Reality developments being moved over to WebXR. There is a great deal of potential there for interoperable projects, meaning a software like StellarX will be more and more able to exchange information with other software and apps.
Hasn’t Mixed Reality been around for a while? What’s different now?
Granted, Mixed Reality is far from a new thing. What is new with the Quest 3 though, is the capacity to access high quality MR content within a standalone, affordable, and fully mobile device. This generates more flexibility for developers and audiences alike. Many game developers are already taking advantage of MR features within their games, and coming up with new ways of interacting with both the virtual and the physical worlds. On our side, we are already leveraging that through StellarX, using MR as a gateway for people to create engaging remote and co-located learning and training experiences.
How can businesses leverage Mixed Reality?
Now, we know what you’re thinking: that sounds interesting, but also… rather abstract. What exactly can be done with MR? 😵
A good example of a use case would be an on-site collaborative training experience, where a group of people sharing a physical space could also share hands-on training involving both the physical and virtual worlds. The physical space could be augmented with overlays, from which trainees would be able to instantly extract knowledge; then, 3D objects could also be anchored to real-life physical counterparts (think digital twins), which would allow for safe manipulations, easy prototyping, endless trial and error, and more. Safety training or crisis management initiatives, for example, could benefit a lot from that concept.
Mixing both on-site and remote training experiences within a single session is another interesting possibility. By gaining access to the same Mixed Reality assets, in-person and remote teams could share their physical space with each other. Headset cameras could capture physical surroundings, and stream them as 3D models to remote users. Everyone could then undergo some kind of collaborative experience together, all while being in completely different locations.
That’s only a few examples that come to mind — but really, there are endless things to be done! ✨
What about Artificial Intelligence?
We can take a look at what’s new with AI at Meta by sharing some of what we will be using it for.
First, we aim to have AI agents help people create scenarios; more specifically, to have them recognize people’s intent through voice input, and then translate that intent into a fully functional interactive scenario. We have already developed and patented a codeless, easy-to-use scenario system within StellarX; but, we strive to make scenario creation even more straightforward for all users, and AI is an obvious way of reaching that goal.
Then, we are also exploring GenAI driven objects and scenes generation by leveraging photogrammetry, NeRFs, and other similar technologies to allow people to generate new content from a recognized intent.
Finally, we’re also looking into generative non-playable characters (NPCs). We want to allow creators to easily customize their NPCs by injecting knowledge into StellarX through generative AI-driven tools and features, so that they can then have natural conversations with those entities in a way that reflects their knowledge base and context.
Will there be changes to StellarX?
There are many optimizations and improvements that the Quest 3 capabilities will allow us to bring to StellarX.
We definitely aim to leverage the new MR features, such as scene mesh and dynamic occlusion. We’ve also already gone pretty in-depth about how we plan to put current AI advancements to good use. Hardware-wise, better memory management leaves a lot of room for optimizations that will allow users to create richer environments, more complex scenarios, and more intricate experiences. Then, there’s a lot of potential for improved rendering; real time lighting and real time shadows which will benefit from increased GPU performance. Finally, better CPU performance will support improvements regarding physics and simulation capabilities.
So, in a nutshell — there’s a lot to be done with what’s coming in terms of Mixed Reality, Artificial Intelligence, and new hardware. There is so much to explore still, and we are excited to see what will unfold in the near future. We can’t wait to show you the great things we are working on for StellarX, so stay tuned if you don’t want to miss a beat! 👀
This article was put together using statements from Pierre-Luc Lapointe, R&D Director here at StellarX.
Nous sommes une entreprise fièrement québécoise. Pour lire cet article en français, c’est par ici!
