Breaking Down the Portal Effect: How to Create an Immersive AR Experience
In this post, I’ll dive into the powerful and exciting world of augmented reality (AR) portals. I’ll show you a step-by-step process of how to create a virtual gateway to another world.
This tutorial expects you to have some general knowledge about computer generated graphics and have basic understanding of the Three.js library.
Please note that this post will only focus on the technical implementation of the portal effect using Three.js. While the techniques and frameworks for WebAR are constantly evolving, the principles of creating a portal remain the same. Once you have a solid understanding of how to create a portal, it should be relatively easy to integrate it with the popular WebAR frameworks like 8thWall.
What is an Augmented Reality Portal?
An augmented reality (AR) portal is a virtual doorway or gateway that allows users to access and explore a virtual world or environment.
AR portals use the camera and display of a device, such as a smartphone or tablet, to create an immersive, three-dimensional experience that seamlessly blends the virtual and real worlds. Users can move around within the virtual space and interact with objects, similar to how they would in the real world.
This technology can be used for a variety of purposes such as e-commerce, gaming, entertainment, education, training and tourism.
An overview of the steps
In the next section I will help you with all the required steps to build a portal.
- Setting up the scene, camera, renderer, portal and ground planes
- Creating a render target for the portal
- Attaching the render target texture to the portal material
- Creating the render pipeline
- Clipping the virtual world inside the portal
- Adding controls
- Creating a check to see if the user stepped through the portal
- Setting up local clipping planes for objects that move through the portal
Let’s build the Portal
The result
Let’s start by showing what you will be able to create after you’ve completed all the steps.
1. Initial setup
Here we go! Let’s start by creating a scene, camera, renderer, portal and some ground meshes.
In this tutorial we build two virtual worlds; an outside world and an inside world. When building the actual augmented reality effect you would most likely use the camera-feed as background texture for the outside world.
// Create the scene
const scene = new THREE.Scene();
// Create the camera
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
camera.position.z = 5;
// Create the renderer
const renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
// Create the portal mesh
const portalGeometry = new THREE.PlaneGeometry(1, 1);
const portalMaterial = new THREE.MeshBasicMaterial();
const portal = new THREE.Mesh(portalGeometry, portalMaterial);
scene.add(portal);
function createPlane(width, height, color) {
const geometry = new THREE.PlaneGeometry(width, height, 1, 1);
const material = new THREE.MeshPhysicalMaterial({ color });
return new THREE.Mesh(geometry, material);
}
const groundOutsideMesh = createPlane(4, 4, 0x00FF00);
groundOutsideMesh.rotation.set(-Math.PI * 0.5, 0, 0);
scene.add(groundOutsideMesh);
const groundInsideMesh = createPlane(4, 4, 0x0000FF);
groundInsideMesh.rotation.set(-Math.PI * 0.5, 0, 0);
scene.add(groundInsideMesh);2. Create a render target
Next, create a render target for the world shown inside the portal. In the next step we will use the texture of the render target on the portal material.
// Create the render target for the "inside" of the portal
const renderTarget = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight);3. Update Portal Material
Set the render target texture as its map and slightly modify the fragment shader to draw the texture based on the screen resolutions rather than the default generated UVs.
const resolution = new THREE.Vector2();
const portalGeometry = new THREE.PlaneGeometry(1, 1);
const portalMaterial = new THREE.MeshBasicMaterial({
map: renderTarget.texture
});
portalMaterial.onBeforeCompile = (shader) => {
shader.uniforms.uResolution = new THREE.Uniform(resolution);
shader.fragmentShader =
`
uniform vec2 uResolution;
` + shader.fragmentShader;
shader.fragmentShader = shader.fragmentShader.replace(
"#include <map_fragment>",
`
vec2 pos = gl_FragCoord.xy/uResolution;
vec4 sampledDiffuseColor = texture2D( map, pos );
diffuseColor *= sampledDiffuseColor;
`
);
};
const portal = new THREE.Mesh(portalGeometry, portalMaterial);
function resize() {
const width = window.innerWidth;
const height = window.innerHeight;
resolution.set(width, height);
}
window.addEventListener("resize", resize);4. Create render pipeline
We start with updating the visibility of all the objects in the scene. Only the objects that need to be visible for the ‘inside’ of the portal will be set visible, all the other objects including the portal itself need to be hidden.
In the second part of the render loop do the exact opposite. We show all objects representing the outside world including the portal and hide the rest.
It can be very tedious to keep track of all the object in the scene and manually set the visibility for each object. An alternative and cleaner approach is to use Three.js Layers.
Assign your objects to a specific layer and enable / disable the layers inside the render loop.
const mapLayers = new Map();
mapLayers.set("inside", 1);
mapLayers.set("outside", 2);
mapLayers.set("portal", 3);
// Helper function to set nested meshes to layers
function setLayer(object, layer) {
object.layers.set(layer);
object.traverse(function (child) {
child.layers.set(layer);
});
}
const portalGeometry = new THREE.PlaneGeometry(1, 1);
const portalMaterial = new THREE.MeshBasicMaterial({
map: renderTarget.texture
});
portalMaterial.onBeforeCompile = (shader) => {
// ...
};
const portal = new THREE.Mesh(portalGeometry, portalMaterial);
setLayer(portal, mapLayers.get("portal"));
scene.add(portal);
function createPlane(width, height, color) {
const geometry = new THREE.PlaneGeometry(width, height, 1, 1);
const material = new THREE.MeshPhysicalMaterial({ color });
return new THREE.Mesh(geometry, material);
}
const groundOutsideMesh = createPlane(4, 4, 0x00FF00);
groundOutsideMesh.rotation.set(-Math.PI * 0.5, 0, 0);
setLayer(groundOutsideMesh, mapLayers.get("outside"));
scene.add(groundOutsideMesh);
const groundInsideMesh = createPlane(4, 4, 0x0000FF);
groundInsideMesh.rotation.set(-Math.PI * 0.5, 0, 0);
setLayer(groundInsideMesh, mapLayers.get("inside"));
scene.add(groundInsideMesh);
function render() {
requestAnimationFrame(render);
// first render
// modify scene before rendering the inside
camera.layers.disable(mapLayers.get("portal"));
camera.layers.enable(mapLayers.get("inside"));
camera.layers.disable(mapLayers.get("outside"));
// render inside of portal
renderer.setRenderTarget(renderTarget);
renderer.render(scene, camera);
// second render
// modify scene before rendering the outside
camera.layers.enable(mapLayers.get("portal"));
camera.layers.disable(mapLayers.get("inside"));
camera.layers.enable(mapLayers.get("outside"));
// render outside of portal + portal
renderer.setRenderTarget(null);
renderer.render(scene, camera);
}
render();5. Add clipping
The next thing we need to do is clip all the parts of the world that shouldn’t be visible past the portal position.
For this we use clippingPlanes. First we create them.
// Setup Clipping planes
const globalPlaneInside = [new THREE.Plane(new THREE.Vector3(0, 0, 1), 0)];
const globalPlaneOutside = [new THREE.Plane(new THREE.Vector3(0, 0, -1), 0)];Then we add them to the render loop.
function render() {
requestAnimationFrame(render);
// first render
renderer.clippingPlanes = globalPlaneOutside;
// modify scene before rendering the inside
camera.layers.disable(mapLayers.get("portal"));
camera.layers.enable(mapLayers.get("inside"));
camera.layers.disable(mapLayers.get("outside"));
// render inside of portal
renderer.setRenderTarget(renderTarget);
renderer.render(scene, camera);
// second render
renderer.clippingPlanes = [];
// modify scene before rendering the outside
camera.layers.enable(mapLayers.get("portal"));
camera.layers.disable(mapLayers.get("inside"));
camera.layers.enable(mapLayers.get("outside"));
// render outside of portal + portal
renderer.setRenderTarget(null);
renderer.render(scene, camera);
}
render();6. Add Controls
Below you’ll find the code to add keys & mouse navigation to your world. Yes, this is not really part of how to set up the portal, however it’s a necessary step because we need to be able to navigate the worlds .
This part can be replaced entirely when hooking it up to an augmented reality framework like 8thWall.
const mapDirection = new Map();
mapDirection.set("isLeft", false);
mapDirection.set("isRight", false);
mapDirection.set("isUp", false);
mapDirection.set("isDown", false);
const mapKeys = new Map();
mapKeys.set("a", "isLeft");
mapKeys.set("ArrowLeft", "isLeft");
mapKeys.set("w", "isUp");
mapKeys.set("ArrowUp", "isUp");
mapKeys.set("s", "isDown");
mapKeys.set("ArrowDown", "isDown");
mapKeys.set("d", "isRight");
mapKeys.set("ArrowRight", "isRight");
// Setup Camera and Controls
const camera = new THREE.PerspectiveCamera(75, 1, 0.01, 100);
camera.position.set(-1.5, 0.5, 0.6);
const cameraLookAtTarget = new THREE.Vector3(0, 0.5, 0);
const controls = new OrbitControls(camera, renderer.domElement);
controls.enableZoom = false;
controls.enablePan = false;
controls.target = cameraLookAtTarget;
controls.update();
function render() {
requestAnimationFrame(render);
updateCameraPosition();
updateCameraTarget();
// ... render logic
}
render();
const speed = 0.05;
const directionVector = new THREE.Vector3();
function up() {
directionVector.setFromMatrixColumn(camera.matrix, 0);
directionVector.crossVectors(camera.up, directionVector);
camera.position.addScaledVector(directionVector, speed);
}
function down() {
directionVector.setFromMatrixColumn(camera.matrix, 0);
directionVector.crossVectors(camera.up, directionVector);
camera.position.addScaledVector(directionVector, -speed);
}
function left() {
directionVector.setFromMatrixColumn(camera.matrix, 0);
camera.position.addScaledVector(directionVector, -speed);
}
function right() {
directionVector.setFromMatrixColumn(camera.matrix, 0);
camera.position.addScaledVector(directionVector, speed);
}
function updateCameraPosition() {
if (mapDirection.get("isUp")) up();
if (mapDirection.get("isDown")) down();
if (mapDirection.get("isLeft")) left();
if (mapDirection.get("isRight")) right();
}
const worldDirection = new THREE.Vector3();
function updateCameraTarget() {
camera.getWorldDirection(worldDirection);
cameraLookAtTarget
.copy(camera.position)
.add(worldDirection.multiplyScalar(0.01));
}
function updateMovement(direction, isEnabled) {
mapDirection.set(direction, isEnabled);
}
function handleKeyDown(e) {
const direction = mapKeys.get(e.key);
if (direction) updateMovement(direction, true);
}
function handleKeyUp(e) {
const direction = mapKeys.get(e.key);
if (direction) updateMovement(direction, false);
}
window.addEventListener("keydown", handleKeyDown);
window.addEventListener("keyup", handleKeyUp);7. Walking through the portal
When moving through the portal we need to flip the visibility settings for all world objects.
This is required because after switching worlds we need to draw, on the first render pass, the outside world and then, on the second render pass, the inside world.
Below you’ll find the logic to test if the camera was moving through the portal.
let isInsidePortal = false;
let wasOutside = true;
const portalRadialBounds = 0.5; // relative to portal size
function testPortalBounds() {
const isOutside = camera.position.z > 0;
const distance = portalMesh.position.distanceTo(camera.position);
const withinPortalBounds = distance < portalRadialBounds;
if (wasOutside !== isOutside && withinPortalBounds) {
isInsidePortal = !isOutside;
}
wasOutside = isOutside;
}
function render() {
requestAnimationFrame(render);
testPortalBounds();
}
render();If ‘isInsidePortal’ is true, we switch the visibility in the render pipeline.
function render() {
requestAnimationFrame(render);
// first render
renderer.clippingPlanes = isInsidePortal
? globalPlaneInside
: globalPlaneOutside;
// modify scene before rendering the inside
camera.layers.disable(mapLayers.get("portal"));
if (isInsidePortal) {
camera.layers.disable(mapLayers.get("inside"));
camera.layers.enable(mapLayers.get("outside"));
} else {
camera.layers.disable(mapLayers.get("outside"));
camera.layers.enable(mapLayers.get("inside"));
}
// render inside of portal
renderer.setRenderTarget(renderTarget);
renderer.render(scene, camera);
// second render
renderer.clippingPlanes = [];
// modify scene before rendering the outside
camera.layers.enable(mapLayers.get("portal"));
if (isInsidePortal) {
camera.layers.disable(mapLayers.get("outside"));
camera.layers.enable(mapLayers.get("inside"));
} else {
camera.layers.disable(mapLayers.get("inside"));
camera.layers.enable(mapLayers.get("outside"));
}
// render outside of portal + portal
renderer.setRenderTarget(null);
renderer.render(scene, camera);
}
render();8. Objects moving through the portal
With local clipping enabled it’s possible to clip individual objects. We can use this to partially draw an object inside the portal and partially in the outside world.
First enable local clipping.
renderer.localClippingEnabled = true;Then add a new object that we like to use for this effect and add the clipping planes to the render pipeline.
function createTorus(color) {
const geometry = new THREE.TorusKnotGeometry(0.25, 0.03, 100, 16);
const material = new THREE.MeshPhysicalMaterial({
color,
side: THREE.DoubleSide
});
return new THREE.Mesh(geometry, material);
}
const torusMesh = createTorus(0xDDDDDD);
torusMesh.position.set(0, 0.5, 0);
scene.add(torusMesh);
function render() {
requestAnimationFrame(render);
// first render
torusMesh.material.clippingPlanes = isInsidePortal
? globalPlaneInside
: globalPlaneOutside;
// ... all other stuff ...
// second render
torusMesh.material.clippingPlanes = isInsidePortal
? globalPlaneOutside
: globalPlaneInside;
// ... all other stuff ...
}
render();That’s it! Now you know how to create your own portal effect.
Limitations and performance
When creating a portal there are different ways to go about it. I’ve found this setup to be both flexible and efficient. However, it’s important to keep in mind that depending on the unique requirements of your project, you may need to make adjustments to the setup.
Below a few points to take into account when using this setup.
1. Frustum Culling Near Plane
Although barely noticeable in a regular user flow, when standing in close proximity to the portal the near frustum may cut off the portal mesh, displaying the parts of both worlds partially.
2. ClippingPlane
A Three.js clipping plane is a two-dimensional surface that extends indefinitely in three-dimensional space. Due to this, it is not possible to create an object that exists inside the portal and ‘swirls’ back passing the clipping plane’s position. The clipping plane will cut off the shape.
3. Performance
This solution requires two render passes, making it computationally more heavy than alternative solutions that only require one render pass. Therefore, it’s essential to choose the solution that best fits your specific needs.
Thank you for reading.