Mujoco Unity Plugin rendered with Intel Xe graphics
In this article, we will see how we implement the Unity Mujoco plugin to create a scene rendered with Intel Xe Graphics
What does the Mujoco plugin do?
Deepmind open-sourced Mujoco project. This plugin allows us to simulate (more discussed later).
It has got various environments to do simulations from such as
- Ant
- Hopper
- Cheetah
- Walker-2d
The objects are shown in the figure below.
The programming Platform
As now Mujoco is open-sourced and a Mujoco-Unity plugin is available that's the reason we can use C# to program and Interact with the objects through the scene we create(the prime focus of our article here with Intel Xe graphics).
How we use the Mujoco Plugin in Unity
The Unity plugin uses mujoco physics engine which in turn can be applied to unity assets implement game logic and use game object and have access to Mujoco Full API.
Unity plugin includes package.json and we add it from package manager which recognizes the package that imports the C# codebase to our project.
Unity needs the mujoco library to be added so that we can use the full effects of the Mujoco physics engine.
The rendered Pipeline flow
Getting to work with mujoco physics requires certain steps.
Step1:We will have to create a scene in Unity editor.
Step2:We need to go to the Unity editor asset menu.
Step3:Next we need to find out an option called import Mujoco Scene is available at Unity Editor.
Import Mujoco scene invokes the mujoco element object inside Unity editor.
There might be errors while adding the package file!
There might be an error message in the console popping up if we are not keeping the dll library in the right place. The same file structure needs to follow as shown below.
Step4:- Import Mujoco Scene allows us to add an XML file which is an object that contains all the capabilities of the Mujoco physics engine
Notes:- XML File
Mujoco loads the XML files that can be generated in two formats.
i)One would be the native MJCF format
ii)The other would be URDF format.
The XML file has a schema associated with the structure.
The general way a model file is recognized is shown below.
The XML file as a whole is a very important part of the integration of the Mujoco physics engine within Unity.
There are steps that govern the creation of the entire tree.
1st one is XML Schema
XML Schema forms the basis of how the XML file is organized. Hence it is very important to consider the 1st hierarchy in the tree. It consists of all the necessary information of what the structure looks like in reality.
2nd one XML Elements and Attributes
XML elements and attributes are the necessary way by which we can segregate the entire model file.
3rd one MJCF
MJCF file format is the heart of Mujoco physics behavior.
MJCF has all the things that are required to form a very important link to connect all the possible ways of how Mujoco behaves and this is known to be a unique top-level element named mujoco.
It is represented as mujoco — → (!)
The unique top-level element identifies the XML files as an MJCF files.
4th is mj_print schema
MJCF models are generated by function mj_print schema which prints internal XML schema as plain text or HTML, with style padding or  .
5th Custom Schema
From the function, we print the custom schema that acts as a bridge between XML elements and MJCF files.
6Th Parsers
The custom schema uses parsers. Parsers help in hashing through the right regular expression tree of the MJCF file into XML and vice-versa.
7th The model file
The parser in turn validates the model files.
Notes
Unity is heavily supported by the Nvidia Physics engine. In PhysX, every rigid body is a free body.
Mujoco requires explicit specification of joints for mobility.
Design principles with Intel Xe graphics.
As quick rendering is a very important requirement for fast simulation and visualization. Intel Xe graphics comes to the picture to render the output very fast.
Intel Iris Xe Max Graphics
Intel Iris Xe Max graphics improve the one-to-one mapping between MJCF elements and the Unity components.
We will now see how Intel XE Graphics improves the simulation in the unity scene
Scanning in the Mujoco Scene in Unity is a very important criterion as it goes through hierarchy and as rendering starts it uses Intel Xe graphics which enhances the scanning for game objects hierarchy for the Mujoco component as well as the other Physx components together. The entire combination of the results was very promising for the final render it was improved tremendously with Intel Xe Graphics and enhanced the entire simulation.
The Unity Game Engine and the scene
To get started with Unity we first have to create a 3D project.
Make sure the project resides in the root folder.
Before getting started with Mujoco we have to import the package.json file which can be downloaded from
We need to add the package.json file.
To go to the package.json file first we will have to find the package manager that’s there in the window option.
Exploring other XML files.
I used ant.xml that's there in the Open AI GYM package.
We will have to import the mujoco scene in unity
Then imported the ant.xml file inside unity
As the ant.xml files were available make sure the MjGeom(script)that has got plane extent is made to zero.
Add a 3d object plane from the Game object option. Delete the track camera from Torso.
Just extend the scale of the plane as we will be adding additional assets to it.
Drag and drop the ant object to assets folder such that it becomes a prefab.
Now we need to add a C# script for Unity prefab to work in runtime. To make things interesting I added the prefabs in a circular fashion.
The Script is as below.
using UnityEngine;
public class CircleFormation : MonoBehaviour
{
// Instantiates prefabs in a circle formation
public GameObject prefab;
public int numberOfObjects = 12;
public float radius = 5f;
void Start()
{
for (int i = 0; i < numberOfObjects; i++)
{
float angle = i * Mathf.PI * 2 / numberOfObjects;
float x = Mathf.Cos(angle) * radius;
float z = Mathf.Sin(angle) * radius;
Vector3 pos = transform.position + new Vector3(x, 0, z);
float angleDegrees = -angle*Mathf.Rad2Deg;
Quaternion rot = Quaternion.Euler(0, angleDegrees, 0);
Instantiate(prefab, pos, rot);
}
}
}The prefab is the ant prefab we created. The results are as shown below.
As I was going through the scene I captured a snippet of both the GPUs in my ASUS Vivo book. The results are
Conclusion.
Intel Iris Xe Max graphics helped in my simulation results as it produced more stunning visuals as well output rendering was faster.
It was able to bring in both the Nvidia PhysX as well as Mujoco Physics applied together in tandem.
More Work on Unity ML to follow with Intel Xe Graphics. Stay tuned.