WebXR — The new web

The web has grown from a read-only web (Web 1.0) to interactive web (Web 2.0) and then moved into the semantic web which is more connected and more immersive. 3D content on the web is there for quite some time, the most web-browsers support rendering 3D content natively. The rise for XR technologies and advancements in devices and the internet is leading the demand for Web-based XR. It will be a norm for web-based development.

We are going through the Covid19 time, and most of us are working from home. These days, one message is quite popular on social media, asking to keep your kids engaged, and let them bring Google 3D animals in your bedroom.

My son Lakshya with his pet Simbu (Google 3D animals)

However, this feature was there for some time now. Google search pages have provision to show models in 3D in your space, you need a compatible phone for making it work.

This is WebXR. Experiencing XR directly on a website without a need for special apps, or downloads. Generally, XR application development needs a good amount of effort and cost on platforms such as Unity 3D, Vuforia, Wikitude. The content is delivered in the form of an app for XR enabled smartphones or head-mounted devices such as Hololens, Oculus, NReal, or similar. With WebXR technology, we can build applications directly with our favorite web technology (HTML, JavaScript) and deliver the content on our website.

There are few WebXR libraries popular these days such as babylon.js, three.js, and A-frame. WebVR is quite stable but WebXR is experimental at this stage in most of the libraries. In addition to Google and Mozilla, organizations like 8th Wall, XR+ are also putting great effort into making the WebXR stable. At ThoughtWorks, we have built some cool projects for our clients utilizing available WebXR libraries.

In this post, we will say hello to WebXR, and also takes you through the classic XR development in Unity 3D. The first part of the article is for people who are new to Unity 3D, so they need to say hello to the Unity 3D platform, then we will go to the WebXR world, and understand how we can co-relate both the world.

Note: — We will discuss the WebAR part for android here, you may relate it to WebVR and for other devices, the concept is more or less the same. Follow the respective libraries documentation if you are more interested in the WebVR part or for other devices.

Hello XR World!

If you haven’t yet gone through Unity 3D, please read below a cool article by my colleague Neelarghya Mondal.

Let’s get started with Unity…

Then set up the first XR project using AR Core — the google guidelines, Import the ARCore SDK package into Unity.

1. Remove the main camera, light and the add prefabs “ARCore Device” and “Environment Light” available in ARCore SDK.
2. Create a prefab of Cube with rotating script

public class RotateCube : MonoBehaviour
{
    private Vector3 rotation = new Vector3(20, 30, 0);
    void Update()
    {
      transform.Rotate(rotation * Time.deltaTime);
    }
}

3. Create an InputManager class that instantiates the cube prefab at the touch position.

public class InputManager : MonoBehaviour
{
    public Camera camera;
    public GameObject cubePrefab;
    // Update is called once per frame
    void Update()
    {
      Touch touch;
      if (Input.touchCount > 0 && (touch = Input.GetTouch(0)).phase == TouchPhase.Began)
      {
        CreateCube(touch.position);
      }
      else if (Input.GetMouseButtonDown(0))
      {
        CreateCube(Input.mousePosition);
      }
    }
    void CreateCube(Vector2 position)
    {
      Ray ray = camera.ScreenPointToRay(position);
      Instantiate(cubePrefab, (ray.origin + ray.direction), Quaternion.identity);
    }
}

4. You can test it directly on an Android Phone (ARCore compatible). Connect the phone via USB, and test it with ARCore Instant preview. It needs some extra services running on the device. Alternatively, you can just build and run the project for android, and it will install the app to the device, and you can play with your XR app.

This completes the first part, and now we know how a typical AR app is built and delivered.

Find the source code here

Hello Web XR!

Let’s say hello to Web XR. We will try to implement a similar example that we have implemented in the last section, but this time without the Unity 3D platform and without building an app for the device. We will implement a simple web page that can do the same, using WebXR library https://threejs.org.

Create a web page

1. Create a work folder and open it in the editor — I use Intellij but you may use any text editor. Create an HTML file and host it in web-server — index.html, Intellij has built it web-server which is just a click away, but you may use other easy ways
2. You may download three.js dependencies on your own or just download via some package manager — npm install three
3. Follow the steps described by three.js and update the index.html page — https://threejs.org/docs/#manual/en/introduction/Creating-a-scene
4. For XR enabled scene keep renderer.xr.enabled = true;
5. Now, index.html looks like below with an On Touch logic which adds a cube on touching the screen.

<script type="module">
  import * as THREE from './node_modules/three/build/three.module.js';
  import { ARButton } from './node_modules/three/examples/jsm/webxr/ARButton.js';
  var container;
  var camera, scene, renderer, lastmesh;
  var controller;
  init();
  animate();
  function init() {
    container = document.createElement( 'div' );
    document.body.appendChild( container );
    scene = new THREE.Scene();
    camera = new THREE.PerspectiveCamera( 70, window.innerWidth / window.innerHeight, 0.01, 20 );
    var light = new THREE.HemisphereLight( 0xffffff, 0xbbbbff, 1 );
    light.position.set( 0.5, 1, 0.25 );
    scene.add( light );

    renderer = new THREE.WebGLRenderer( { antialias: true, alpha: true } );
    renderer.setPixelRatio( window.devicePixelRatio );
    renderer.setSize( window.innerWidth, window.innerHeight );
    renderer.xr.enabled = true;
    container.appendChild( renderer.domElement );
    document.body.appendChild( ARButton.createButton( renderer ) );
    var geometry = new THREE.BoxGeometry( 0.1, 0.1, 0.1 );
    function onSelect() {
      var material = new THREE.MeshNormalMaterial();
      var mesh = new THREE.Mesh( geometry, material );
      mesh.position.set( 0, 0, - 0.3 ).applyMatrix4( controller.matrixWorld );
      mesh.quaternion.setFromRotationMatrix( controller.matrixWorld );
      scene.add(mesh);
      lastmesh = mesh;
    }
    controller = renderer.xr.getController( 0 );
    controller.addEventListener( 'select', onSelect );
    scene.add( controller );
    window.addEventListener( 'resize', onWindowResize, false );
  }
  function onWindowResize() {
    camera.aspect = window.innerWidth / window.innerHeight;
    camera.updateProjectionMatrix();
    renderer.setSize( window.innerWidth, window.innerHeight );
  }
  function animate() {
    renderer.setAnimationLoop( render );
  }
  function render() {
    if(lastmesh){
      lastmesh.rotation.x += 0.1;
      lastmesh.rotation.y += 0.2;
    }
    renderer.render(scene, camera );
  }
</script>

Host and test the web page

Publish the page in the web-server and test it. The page is now hosted in the inbuilt web-server of IntelliJ, make sure external access is enabled. http://localhost:8001/hello-webxr/index.html. It is still not ready to be used on browsers, we need to do some customizations on the browsers to make it work.

On PC, we need to install WebXR emulators in the browser to make it work. VR emulator is great but the AR emulator still doesn’t work well.

On Mobile, open chrome browser and open chrome://flags page. Search for XR and enable all the experimental features. Make sure The device must have ARCore support.

Now we want to test the page which is running on PC/Laptop in the chrome browser on the android phone. There are multiple ways to do that (port forwarding, hotspot, etc). Simplest is, come on the same wifi network and access the page via IP instead of localhost. Type in http://192.168.0.6:8001/hello-webxr/index.html

Once the web page is loaded, it will ask to start XR experience, and also ask for the user consent for accessing the camera.

Once approved, it scans the area and you can start placing the cubes.

Source code is available here

Try this link in your android phone right now, and experience the WebXR.

https://thinkuldeep.github.io/hello-webxr/

I tried, a similar implementation can be done with Babylon JS but “immersive-ar” mode is not stable there when we host webpage, however, “immersive-vr” mode works great.

So these webpages can be opened in any supported browser on the XR devices, you get all advantage of Web, and server the XR content simply from your website.

This completes our introduction with WebXR.

Conclusion

We have got an introduction to XR as well as WebXR. There are some features in XR which may not be achieved by WebXR but it is still a great option to take the technology to a wider audience. There are already great examples at the respective sites of these libraries, the community is also submitting some good projects samples and games. Try those, and stay tuned to WebXR, it is coming.