Creating an immersive 3D environment with Unity
The concepts of Metaverse and Virtual Interactive Environments are becoming more known and wanted all over the world given the current trends. In this article, we will go through extremely oversimplified steps, without explaining a lot of the under the hood stuff, to create a basic 3D environment in which we can walk and explore, all with Unity.
But first, what is Unity?
Unity is a Game Engine (also known as Graphics Engine), which itself it is software framework for the development of video games.
Game engines provide a suite of visual development tools in addition to reusable software components. These tools are great to enable simplified, rapid development of games. Game engines usually provide platform abstraction, allowing the same game to run on various platforms (PC, Consoles, Mobile, Virtual Reality, etc).
But video games are not the end of Game Engines. Over the last two decades, Game Engines have been used not only to make games, but also to make all kinds of 2D and 3D experiences, like animations (movies, shorts, etc), and they are also used in other fields like architecture, automotive, medicine, and many more!
Now, to the real thing.
We will be making a 3D environment like the one in the picture below
Setting up things.
Fist, we will need Unity Hub, the software that acts as a version control of all Unity releases. We can get it from here.
Once Unity Hub is installed, the process of installing a Unity version and creating a project is very straight forward. As stated earlier, we will go through extremely simplified steps.
Into Unity!
Once our project opens, we can begin by creating a giant mesh we can play with, this mesh is also known as a Terrain object.
Unity Terrain Tools gives us a varied set of tools to sculpt our terrain, among some of them are:
- Raise or Lower Terrain: paint the heightmap with a paintbrush tool.
- Paint Holes: hide portions of the Terrain.
- Paint Texture: apply surface textures.
- Set Height: adjust the heightmap toward a specific value.
- Smooth Height: smooth the heightmap to soften Terrain features.
- Stamp Terrain: stamp a brush shape on top of the current heightmap.
We are going to explore some of these tools next.
Playing with the rise and lower terrain tools that are given to us by Unity, we can transform that giant plane into something like this:
Now, if we play with the Paint Texture tool, we could get something like this:
Adding different types of water objects, we can achieve something like this:
Attaching a custom skybox to the scene, could look similarly to this:
Explaining Shaders
Effects like the water objects and the skyboxes are all achieved with something called shaders.
Shaders are code that control the colour of each pixel on the screen. These sets of instructions usually run on the graphics processor (GPU).
The position and colour (hue, saturation, brightness, and contrast) of all pixels, vertices, and/or textures used to construct a final rendered[1] image can be altered using algorithms defined in a shader and can be modified by external variables or textures introduced by the computer program calling the shader.
Here we can see how a shader can be used to move the vertices of a plane up and down, when we put all vertices into that motion, we can simulate different type of waves.
In this example, we do not move the vertices, but instead, we add a two or more textures (images) and make them scroll in a certain direction at different speed each, to simulate the flow of a river or a waterfall
In these last two images we can see how we can shift the colour of the skybox around the sun or moon (directional lights) while they are moving.
Not only that, we can also inject textures into the skybox shader to add clouds and/or stars!
Another notable example for the use of shaders is cel shading; people could be familiar with its looks, and it’s also known as toon shading.
Cel shading is an art style of non-photorealistic rendering designed to make 3D computer graphics or artworks look three dimensional by creating flat colours on top of a base colour, making the object look three-dimensional while keeping that 2D effect in it. Basically, make a 3D object look cartoonish.
Shaders have millions of possibilities, from a technical and artistical point of view it’s a great tool to achieve mind-blowing effects in any project.
Explaining Particles
A particle system is a technique in game physics, motion graphics, and computer graphics that uses many minute sprites, 3D models, or other graphic objects to simulate certain kinds of “fuzzy” phenomena, which are otherwise very hard to reproduce with conventional rendering techniques — usually highly chaotic systems, natural phenomena, or processes caused by chemical reactions.
The Particle System in Unity is a robust particle effect system where we can simulate moving liquids, smoke, clouds, flames, magic spells, and a whole slew of other effects.
Here is an example of what the waterfall will look like without particles.
And here with particles:
In this case, the crest of the waterfall is made up of various scrolling textures in motion, which is also called Texture Sheet Animation.
The waterfall base particles are made in the same way, but with other texture.
Adding Props
Now, we could get closer to the scene and start working on something a little different. We could start by adding props[2] like some buildings, floors, lamps, etc.
This PUB building is an excellent example, the sign in the bottom right is also consider a prop.
Getting more creative and placing a bunch of props together, we could end up creating a mini city island, like the following:
Here we can see the result of gathering many props together in a creative way to build a more complex scenario.
If we go even further and we add even more props, but this time something more environmental, like rocks, palms, bushes, etc., we will end up with a more natural scene, like the one in the picture below:
Adding Life
Right now, we have a beautiful scene, but it feels empty, there is nothing going on in it. Shall we make it livelier?
We are going to undertake the laborious task to add characters to the scene. For the purpose of this article, we are going to add only dummy characters with some animations, to make them feel more believable.
After getting the characters and the animations we want, we put them together (with some logic behind them) and bring them to life.
But let us add one more thing, dolphins!
The Final Touches
Now that we have a beautiful environment, people, and some dolphins, it is time to make the final touches to complete this adventure!
The final touches we need to apply are: some sort of character or controller so we can navigate throughout the scene, an interface (UI) so we can change some settings, and finally, some basic sounds, like wave sounds, background noise, etc. Also, we will be adding a day/night cycle, just to make things a little bit more interesting.
Here we have two videos showing the day/night cycle and a little exploration:
Bonus — Adding Virtual Reality Integration
At this point we have created a three-dimensional scenario full of life, which we can explore with our avatar, but to make this experience more immersive, we are going to add Virtual Reality (VR) support.
To achieve this, we are going to use the VR headset Meta Quest 2 (former Oculus Quest 2), since it is a powerful device recognized worldwide, and manufactured and distributed by Meta (former Facebook).
Unity provides us with a dedicated framework for XR[3] solutions. At the same time, Meta provides an official dedicated development package for its VR Headsets in Unity, which simplifies the development.
For a better immersion, we are going to make our app run natively on the device. Meta Quest 2 natively runs Android 6.0 applications onwards, so we are going to convert the target platform of our project to Android.
Then we download the official Oculus package from the Unity Asset Store, and integrate it into our project. We can download the package from here.
After configuring it, it is time to implement VR support in our project.
The package provides utility assets that simplify the integration. For example, special cameras that replace the default cameras in Unity, and allow the user/player to look to different sides with the headset, and have the camera follow user’s view (and giving a sensation of immersion).
So, we are replacing our 3rd person avatar with a traditional first-person controller but replacing the main camera with this new camera (a pair of cameras in fact, one for each eye for the sense of depth).
As an additional detail, we can include 3D models of our hands, and configure them to follow the controls that accompany the Meta Quest 2 helmet, thus giving the user the sensation that they have their own hands.
As a result, now the user can be completely immersed in our island, walk through it, and enjoy the landscape as if the user were inside it.
Conclusion
Game Engines are tremendously powerful tools for creating scenes and experiences like the one we went through.
Of course, there are many things to consider when making these types of experiences: lighting, shadows, materials, animations, shaders, coding, etc.
Making this type of environment is not an easy task. But with the right mindset, time and budget, many things can be achieved.
[1] Rendering is the process of generating an image or frame from a 2D or 3D model by means of a computer program.
[2] A good definition for a prop is anything that supports the scene but is not part of the level layout or character set.
[3] Term that encompasses AR (Augmented Reality), VR (Virtual Reality) and MR (Mixed Reality).
Emilio Pérez — Software Engineer @ intive
Lucas Emanuel Farías — Unity 3D Developer @ intive
