Feb 25, 2021
Anything World — Natural Language Generated 3D Environments
The world of 3D asset creation is evolving more quickly than anyone expected. Open your mind to the possibilities of what an audio-only interface can achieve.
What if I told you there exists a Unity Plugin that allows you to speak into your microphone, and a dynamically generated world fully populated could just appear before you? You didn’t have to find assets and buy them. You didn’t have to model anything yourself. You didn’t have to rig anything, texture it, make it low poly, so it works in a game engine. You speak it, and it springs into existence. God-like powers right? Impossible you say?
The folks at Anything World are making the impossible possible. They are working on a system to make 3D content creation accessible for the masses. Using mainly your voice as an interaction mechanism, you can create entire environments and populate them with objects, animals, vehicles, etc. Once created, these objects can be assigned behaviors like a chase, follow, ride, attack, etc., that the developers have made available. They come to life without you having needed to create any animations yourself!
Still don’t believe me? I don’t blame you; I didn’t either until I saw it for myself. You can try it for yourself too! Here is a simple proof of concept prototype that was built in one day for an educational institution. https://storybook.anything.world/
There’s more, though. Models can be given voices of their own and respond to user voice commands. This functionality can create a chat-like interface and even leverages the system voice to speak the words the entity is responding with.
It’s magical, right? So what’s the catch? Why isn’t this everywhere? I met up with Gordon Midwood, CEO of Anything World, to discuss his amazing tech and hear his story.
“I started off as a game developer building indie games with an artist buddy of mine,” Gordon said. “We tried that for a while and then pivoted to built some voice computing plugins for Alexa. That was fine and all but a little boring. So when we returned to our passion, we thought — what if we combine the two?”
How It Works (Warning! Tech jargon ahead!)
To pull off a voice-controlled game asset generator, a LOT of other pieces had to fall into place. They wanted to be able to speak and have the system not only understand what they were looking for and serve it up, but it also had to be animated and alive, even interactable. This interaction design meant coming up with a machine learning algorithm that could peruse the metadata found on Google Poly and Sketchfab, looking for models that might match what the user requested.
Once found, another algorithm analyzes the model and compares it to open-source point-cloud data sets of similar objects to understand how to classify it. This would result in a model that would be classified as a biped, or quadruped, insect, snake, bird, or even artificial objects like a car, plane, spaceship, etc.
Once classified, it looks for the pieces of that classification like legs or wheels and attempts to interpret the separate components of the mesh that might move and cuts them apart. A procedural physics-based animation is applied based on the classification, and a special shader blends the pieces back together as it moves. The model then gets a few preset default behaviors like walking and at this point it becomes a “smart object” in the library.
The machine learning algorithms use supervised and unsupervised methods, on neural networks leveraging K-Nearest Neighbor algorithms. The entire process takes 1–2 hours currently, but once a smart object is in the library, it’s available for anyone instantly! For those models that have not yet passed through this process, a basic model is instantly served, but it will lack the rigging and behavioral animations that the smart objects contain which it is automatically queued up to become. The more users that use this system, the more models get pulled in from Poly and Sketchfab and made into smart objects. Due to the fact that the entire model is rebuilt, Anything World can save them to their own database, thus “saving” Google Poly models for future generations to enjoy!
What’s Next For Anything World?
Gordon Midwood and his team have lots of big plans for Anything World. They are closing on a round of funding this week to propel them forward to accomplish some great things and bring on a few more developers and key individuals. Currently his 8 person team is female-dominate (5) with 6 developers and 4 machine learning experts but it’ll soon be growing to tackle new endeavors. Among those great things are some of the following features they are actively pursuing:
- Moving to an automated traditional rigging system with bones, skinned meshes with automatic weighting based on the classifications
- Coming soon to Unreal Engine as a plugin
- Soon available for native web engines like Three.js and Babylon.js
- Bringing in high poly models
- Adding the ability to point to a specific custom model collection or provider
- Users can upload their own models to the site to become “smart objects”
- A built-in feedback mechanism so the machine learning algorithm can be told when something was done wrong so it can attempt to correct itself and learn from it’s mistakes
Gordon is also seeking partners and customers in a variety of settings, including:
- Creation games like Roblox
- Indie games like Spilt Milk Studios
- EdTech for schools and colleges like Koala
- Virtual conferencing solutions like RemotelyHQ
- Language learning in VR
- Hackathons and Game Jams like Global Game Jam
- AR Instagram filters
- Social networking meme generation
- And many, many more!!!
Brainstorming with Gordon for a few minutes, we came up with many use cases this tech could help solve or enhance. He wants to approach companies that will benefit from this technology and make it mainstream to reach younger audiences, like Roblox, or even older ones like colleges and universities. Ultimately this kind of technology could disrupt the entire 3D creation space, and we’re both excited to see that happen!
What This Means for the Future of 3D
As a UX Designer, I’m thrilled with the direction Anything World is going. For users in VR, sessions need to be reasonably brief before fatigue sets in. For younger users, it’s even shorter sessions. That means we should be empowering users to get more done faster, get them to the good stuff quicker. Audio interfaces provide that kind of flexibility. Tony Stark demonstrated this in the Iron Man and Avenger movies. There is minimal “interface” for him to interact with, it’s mostly audio commands that the AI Jarvis is interpreting for him, and he interacts mainly with the content itself.
With Anything World, the slow and tedious process of building the assets becomes irrelevant; they are served up to you at your whim for you to use in whatever way suits you. Combine that with the dynamic assignment of behaviors, classes that empower direct interactivity with VR controllers, and a visual scripting language, and you’re well on your way to building a product that anyone can use!
This tech has massive ramifications for the future of spatial computing and interactive 3D learning by making it accessible for everyone. It’s still in an early phase, but it’s well developed enough to begin playing with and building products in Unity. I was able to get it up and running in under half an hour with their tutorial videos. Their prefabs are very simple to use and the voice command interface is easy to extend to other use cases. They have a powerful API you can use too if you just want assets served. If you are a Unity developer, you should check it out at https://anything.world/ and have some fun with it!
If you’re a designer, educator, artist, or other non-engineer type but think you have a fit for natural language-generated 3D environments and assets in your project, reach out to Gordon and his team on their Anything World website and get a conversation started!
Be sure to follow Anything World on LinkedIn and Twitter for news and releases. I expect big things from this little company with a big vision and a big heart to make 3D creation accessible for everyone.
If you’d like to read more Medium articles about Anything World, here are a few links to whet your appetite:
Voice of Anything!
https://medium.com/@gordon_36444/voice-of-anything-ef6f2b9ee636
Everything! World
https://medium.com/@gordon_36444/everything-world-b8a7b5700cf7
Anything? World
https://medium.com/@gordon_36444/anything-world-becfa6f0afd0
***I am not in any way affiliated with Anything World or sponsored by them. My thoughts are my own. Their story fulfills a piece of a larger vision I have for making 3D content creation easy. If you’d like to read more about that vision here is a medium article I wrote on the topic. Thanks for reading!
3D Is Hard and It Shouldn’t Be
https://medium.com/inborn-experience/3d-is-hard-and-it-shouldnt-be-7843cb8b4357
