Why We Are Building an SDK for Sandbox VR
At Sandbox VR, we are attempting to build the holodeck. We believe this to be the first new major medium since the motion picture. After all, though VR is fantastic today, there’s still a long road ahead.
Anyone devoted to building the future of VR knows this first hand. While the highs are extremely high and paint a clear picture of what’s to come, this was only made possible through the efforts of thousands of dedicated people all over the world — a large and diverse ecosystem of developers, operators, artists, and evangelists all working together.
To help make this possible, I’m excited to announce that we are building an SDK for our Sandbox holodecks that allows anyone to develop new immersive experiences for our guests. If you’re interested in becoming a Sandbox developer, please head over to our new developer page where you can sign up and be the first to learn of updates and join our developer community to build the future of VR.
Today, real people by the thousands are walking into Sandbox holodecks every day and enjoying truly delightful and immersive experiences with their friends. What would have sounded like sci-fi just a few years ago is now a reality in numerous cities worldwide.
The Sandbox holodeck experience is made possible today by technology and high-quality content that fully embodies you virtually with your friends, gives you complete freedom of motion, with unique high fidelity haptics. Users come into the store and have unforgettable experiences with their friends from start to finish. After joining Sandbox, I was not surprised to learn that our Net Promoter Scores are comparable to some of the most beloved products and brands of all time.
Sandbox is not just committed to ensuring this quality of experience, we live and die by it. What we provide in our holodecks has to not be just a little better than what users can get at home (which is a rapidly improving standard), it has to be an order of magnitude better. If we don’t hit that standard of quality, people will stop coming in at all.
This is why we don’t just want to be world-class in building the technology that powers our holodecks. We need to be. This is why our SDK will have capabilities like high-performance inverse kinematics, rigging, and motion capture capabilities. We’ll support popular commercial engines like Unity (which we already use for our own experiences), Unreal, and native for the true believers.
We’ll make it as easy as possible to build experiences that can take full advantage of the custom high-performance peripherals that we create for our holodecks and ensure compatibility with the HMDs and computing systems that we employ to power our holodecks.
Here are the components that we are building and making available to early developers later this year. We’d love your thoughts and feedback, so if you think we’re missing something, please reach out and let us know.
SBVR Runtime
The Sandbox VR Holodeck consists of numerous computing systems communicating on an extremely low-latency network (more on this later). All these computing systems need to operate as one, and the SBVR runtime is built to provide the interface and services to enable a consistent virtual environment.
Target Agnostic Build, Packaging, and Update Framework
Our Holodecks and technology are constantly improving and evolving, and we want experiences to constantly benefit from those gains. However, we don’t want to make this be a development burden. A core part of the Sandbox SDK is a target agnostic project generation and packing system. Generating a new project, developing a new component, or updating to the most recent version of the SDK should be a “one-button” operation.
Motion Capture Framework
Our holodecks are made possible by movie quality Motion Capture technology. These are relatively expensive systems that pick up wearable trackers which are converted into a point cloud which is in turn ingested by our IK system (more on this below).
Unless you’re a game or film studio, it’s pretty unlikely that you have one of these 14–16 camera setups lying around. However, you probably do have an Oculus Rift or Vive lying around if you’re building stuff for VR.
The motion capture framework is going to abstract away that component and put in placeholders so you can still build for VR without these expensive systems, with full confidence that things will translate correctly when deployed to our full-body motion-captured holodeck.
Further, motion capture technology is constantly evolving and we will be investing significant development efforts here moving forward. When things change for the better, your code shouldn’t break — things should just work.
Native High-Performance Inverse Kinematics
Inverse Kinematics (IK) allows motion capture data to be converted into rigging information for a given character. This is the technology that makes it possible for us to use a bunch of points in space to give you a full virtual body.
Doing IK well is not easy, and is still somewhat of a research topic (especially for robotics). At a minimum, we want this to be as optimized as possible because left to it’s own accord, IK can eat up a ton of CPU performance.
Also, there are exciting research vectors in terms of doing IK in more interesting ways, such as approaching it from more of a machine learning perspective. We want to be on the bleeding edge of this because the quality of IK at Sandbox is one of the most important things to ensuring users have a terrific experience.
Hardware IO Framework
We’ve already started building custom hardware peripherals to power our experiences. We have a handful of “4D” effects already, and over time we’ll continue to add more and more capabilities to improve our haptics, peripherals and holodeck environment.
From a development standpoint, however, the ability to utilize or manipulate these capabilities should be easy, low latency, and high performance. The Sandbox SDK will include a general hardware framework that will make those things just work.
Networking Framework for data and audio
As mentioned before, our holodecks are essentially a bunch of computers all wired up to work together. However, not all networking architectures are created equal. Our system needs to have lightning fast networking, and not just for game state. For example, audio is a big component in our holodecks — if you can’t talk to or hear your friends, the experience can become very isolating.
Not only do we need to build a lightning-fast networking solution for our holodecks, the system also needs to be something that’s easy to work with. For example, developing a Sandbox Experience should not require access to a Sandbox holodeck. The networking framework we are building will make it possible to create a mocked up multi-user development environment that’s easy to test and build in.
Help us build the future of VR
Most importantly, we’ll be thinking differently about how to break through the barriers of VR every single day. Our holodecks are dedicated spaces that we can fill with pretty much whatever we want. We already have some pretty cool stuff in there, and it’s a real shame if we’re the only ones that get to play with it.
With the Sandbox SDK, that will change. We want it to be possible for anyone with a laptop to be able to build an amazing immersive experience that people can have in our holodecks. If that were possible, would we discover entirely new paradigms of content for VR? I am excited to find out.
So if you’re a developer excited about VR, I hope that you’ll consider joining us on this journey. We would love your feedback as we bring this vision to fruition and build this future together.
Sign up here!
