Integrating Augmented Reality Objects into the Real World with Light and Shadows
Using the WebXR Light Estimation API we can make 3D objects appear to be physical parts of our real environment by having real lights affect virtual objects and virtual objects casting shadows onto real surfaces.
This article will explore how to set it up in AFrame to make your own AR look really good.
Starting from the standard AFrame Boilerplate:
<html>
<head>
<script src="<https://cdn.jsdelivr.net/gh/aframevr/aframe@0d23f9b21c33ab6821046ce95835492cb84996c5/dist/aframe-master.min.js>"></script>
</head>
<body>
<a-scene>
<a-box position="-1 0.5 -3" rotation="0 45 0" color="#4CC3D9"></a-box>
<a-sphere position="0 1.25 -5" radius="1.25" color="#EF2D5E"></a-sphere>
<a-cylinder position="1 0.75 -3" radius="0.5" height="1.5" color="#FFC65D"></a-cylinder>
<a-plane position="0 0 -4" rotation="-90 0 0" width="4" height="4" color="#7BC8A4"></a-plane>
<a-sky color="#ECECEC"></a-sky>
</a-scene>
</body>
</html>First we are going to replace the AFrame script with the latest testing one from the AFrame github. Any numbered version after 1.3.0 would also work but this has just landed in the code so a numbered release hasn’t been made yet. You can copy the latest version URL from here:
<script src="<https://cdn.jsdelivr.net/gh/aframevr/aframe@0d23f9b21c33ab6821046ce95835492cb84996c5/dist/aframe-master.min.js>"></script>Next because we are working with Augmented Reality we need to shrink the various shapes down from over 2 meters to something more reasonable to fit inside an indoor space. We will also group them all up with a parent entity to make it easier to keep them together in AR.
What I did here was to shrink the objects to 0.2 their previous scale and also added a replacement camera at a lower height so it still looks good.
<a-camera position="0 0.4 0" wasd-controls="acceleration:10;"></a-camera><a-entity id="objects" scale="0.2 0.2 0.2" position="0 0 -1">
<a-box position="-1 0.5 1" rotation="0 45 0" color="#4CC3D9"></a-box>
<a-sphere position="0 1.25 -1" radius="1.25" color="#EF2D5E"></a-sphere>
<a-cylinder position="1 0.75 1" radius="0.5" height="1.5" color="#FFC65D"></a-cylinder>
</a-entity>
We will use the ar-hit-test component to position our objects. It looks at the rays cast by the user tapping on the screen or pointing in an AR headset to find real world locations to place objects at the real world location the user selected. The AR hit test component is configured on the <a-scene> element.
<a-scene
ar-hit-test="target:#objects;"
webxr
>This now allows us to place the objects into the real world but still they stand out as clearly fake. We need to add lights and shadows to make it fit in. By having them be affected by real world lights and cast shadows on real world objects it really ties it together to increase their believability.
The reflection component on <a-scene> handles the lighting estimation API and letting objects reflect their environment. It has two parts the component and a directional light it controls, we will add both to the scene.
<a-scene
ar-hit-test="target:#objects;"
webxr
reflection="directionalLight:#real-light"
>
<a-light id="real-light" type="directional" position="1 1 1" intensity="0.5"></a-light>In AR the light now matches the color, intensity and direction of the main light source in the real world. But it doesn’t cast any shadow on the floor. To set this up we we need to make the light itself cast a shadow and an element for the shadow to fall on.
NB: There is a bug in THREE which prevents the reflection map provided by the WebXR lighting estimation API showing details on the standard material. So if you would like to have a shiny material which reflects the real world you should make it a phong material instead.
To make the light cast a shadow add light="castShadow:true; shadowCameraAutomatic:#objects;" to the <a-light> . In this configuration the shadowCameraAutomatic property is set to the objects you want the shadow to follow. This ensures high quality shadows that fit your object even as the objects, and the light moves around, ensuring shadows continue to work well in dynamic conditions like Augmented Reality.
<a-light id="real-light" type="directional" light="castShadow:true;shadowCameraAutomatic:#objects;" position="1 1 1" intensity="0.5"></a-light>You still won’t see a shadow because we need to get our objects to cast a shadow. We will add the shadow component to the #objects so they can cast a shadow on each other.
To cast it onto the floor we need to make a special plane to receive the shadow. There is a special shadow shader in the material which can receive a shadow but is otherwise invisible. We will use this to see the shadow. We will also add the shadow component so it can recieve shadows but we will prevent it from casting shadows itself.
<a-plane id="shadow-plane" material="shader:shadow" shadow="cast:false;" rotation="-90 0 0" width="2" height="2"></a-plane>Unfortunately this shadow plane will be stuck in place at the starting position so we need to make a small AFrame component to allow it to follow our objects to display their shadow.
<script>
AFRAME.registerComponent('follow-shadow', {
schema: {type: 'selector'},
init() {this.el.object3D.renderOrder = -1;},
tick() {
if (this.data) {
this.el.object3D.position.copy(this.data.object3D.position);
this.el.object3D.position.y-=0.001; // stop z-fighting
}
}
});
</script>An object that uses this will sit slightly below a sibling object. We will add it to our plane so that it always sits under them receiving their shadow.
<a-plane id="shadow-plane" follow-shadow="#objects" material="shader:shadow" shadow="cast:false;" rotation="-90 0 0" width="2" height="2"></a-plane>We now have the lighting set up nicely but there is one last thing we can set up to increase the realism and that is to setup the tone mapping and to tell AFrame to use the physical model for lighting, set renderer with this configuration on the <a-scene>
<a-scene
reflection="directionalLight:#real-light"
ar-hit-test="target:#objects;"
webxr
renderer="physicallyCorrectLights:true; colorManagement:true; exposure:1; toneMapping:ACESFilmic;"
>The renderer.exposure property can be tweaked in real time to make the scene lighter and darker as needed.
const scene = document.querySelector('a-scene');
const data = scene.getAttribute('renderer');
data.exposure = 0.5;
scene.setAttribute('renderer', data);I hope this helps you build more realistic AR scenes. The renderer properties are especially useful for any AR, VR or 2D scene and the automatic shadow camera tracking is very useful when you have objects with shadows moving around.
