Multiple Recursive Portals and AI In Unity Part 1: Basic portal rendering

20 min readMay 30, 2020

This tutorial is part of a multi-part series. See the table of contents here.

In this tutorial, we’ll be using render textures and screen-space shaders to create a single pair of portals that you can see through. We will then position these portals to create an illusion of a 3-corner room.

This tutorial is made with Unity 2019.3.7f1. It also uses the ProBuilder and ProGrids packages.

Repositories

You can find the repositories here: https://github.com/limdingwen/Portal-Tutorial-Repository

Download by clicking “Clone or Download”.

For this tutorial, these are the 2 relevant folders:

Part 1 Starter: Starter kit, includes level and player premade
Part 1 Final: Final project, includes Part 1 premade

If you just want the full scripts and shader code, you can find them at the very end of this page.

Preparation

If you don’t want to create the level and player yourself, you can grab it from the starter kit from the links above.

Creating the test chamber

Create a Unity project using the 3D built-in render pipeline template. From there, let’s create a sample level using ProBuilder. For this example, we are aiming to create a room that only has 3 corners. We can start by making a normal 4-corner room, then adding a pillar in the middle to hide the portal seams. We’ll then add in a simple hallway acting as the entrance into the room.

I highly recommend using ProGrids for this, so that your room’s scales will be snapped to the grid. This is important later, as we are placing our portals, as any discrepancies will be noticed by the player and will break the illusion.

Create a ProBuilder Cube by pressing Ctrl-K. To ensure that it stays on the grid, select the ProBuilder Cube, right-click Transform and then click Reset Position.

Now, click Open ProBuilder inside the inspector. To stretch the cube, switch to Face Mode, click on one of the faces, and then use the Move Tool (press W) to move it. Once again I highly recommend using ProGrids, so that you’ll be able to stretch the cube to the exact dimensions mentioned below.

Stretch the cube out to 9x4x9 (9 meters in width and length, and 4 meters in height), then selecting the entire cube and reversing the normals using Flip Normals in the ProBuilder window. You should have something that looks like a room.

Add another ProBuilder cube and stretch it out to 1x4x1. Place in the middle of the room to act as a pillar.

Next, select Edge Mode (right next to the Face Mode button) and select one of the horizontal edges of the room cube. Press the Insert Edge Loop button in the ProBuilder window to split the room into 2 parts. Select the other horizontal edge, and press the Insert Edge Loop button again. Now, your room should be in 4 parts.

Go back into Face Mode. Select one of the faces on the floor. Press the Vertex Colors button inside the ProBuilder Window, and click Apply on any of the colours. The part of the floor that you have selected should now be coloured.

Repeat the same step until 3 parts of the room have distinctive colours. We’ll leave the last part uncoloured, as that is the corner that we want to skip using our portals.

Select one of the wall faces (as long as it’s not the wall of the uncoloured portion of the room) and press the Extrude Faces button inside the ProBuilder Window. Use the Move Tool to move the extruded face out into a hallway. This will be the entrance to our room.

Creating the character

We’re just going to create a simple character controller to move around the scene with.

We’ll start off with a default Capsule. You can create one by pressing GameObject > 3D Object > Capsule. Move the capsule into somewhere within the room.

Rename it to Player, and parent the Main Camera to it by dragging the Main Camera onto the Player object. Next, place the Main Camera at position (0, 0.5, 0) relative to the Player (use the inspector for this).

Remove the default Capsule Collider component by right-clicking on the component and clicking Remove Component. Next, use the Add Component button to add in a Character Controller component. It should automatically fit the form of the capsule.

Next, create a C# script named PlayerController. As this isn’t a tutorial about character controllers, I won’t go into much detail here. This is the code in PlayerController.cs:

using UnityEngine;public class PlayerController : MonoBehaviour
{
    private CharacterController characterController;    public float moveSpeed = 5;
    public float turnSpeed = 5;    private void Start()
    {
        characterController = GetComponent<CharacterController>();
    }    private void Update()
    {
        // Turn player        transform.Rotate(Vector3.up * Input.GetAxis("Mouse X") * turnSpeed);
        
        // Move player        characterController.SimpleMove(
            transform.forward * Input.GetAxis("Vertical") * moveSpeed +
            transform.right * Input.GetAxis("Horizontal") * moveSpeed);
    }
}

Attach PlayerController to the Player. If you run the game now, you should be able to move and look around (left/right only)!

Let’s begin!

Portal theory

The diagram below shows, in a nutshell, how the portal system works. The Player is looking through Portal A. We use a Portal Camera to render the scene through Portal B, then impose that image onto Portal A.

To find the Portal Camera’s position, we first find the Player Camera’s local position relative to the visible side (green) of Portal A. We can then simply use that same local position and transform it into world space, but relative to the invisible side (purple) of Portal B.

For example, if the Player Camera is 5 meters away from Portal A relative to the visible side, the Portal Camera will also be 5 meters away from Portal B relative to the invisible side.

To find the rotation, we use the same principles. We first subtract Portal A’s visible side rotation from the rotation of the Player Camera. This gives us the local rotation of the Player Camera relative to Portal A’s visible side. We can then add Portal B’s invisible side rotation to the local rotation, to get the world rotation of the Portal Camera.

Creating the portal

Create a default Quad and fit it into the room as shown here. Scale the portal accordingly, though be careful not to scale the Z-axis. In my case, the portal has a size of 4x4x1. Once again, I highly recommend using ProGrids to make it as accurate as possible. Rename it to Portal.

Drag Portal into the Project tab to create a prefab. Double-click on the prefab to edit it. We’re going to add 3 children to Portal:

A transform named Visible. This will be a transform where the +Z axis (the blue arrow in the Move Tool) represents the normal of the visible side of the portal.

A transform named Invisible. This will be a transform where the +Z axis represents the normal of the invisible side of the portal.

Create a Portal Camera using GameObject > Camera. Remove the AudioListener component, and change the depth of the Camera component to -100. The position and rotation of the Portal Camera do not matter as we’ll be changing it in the script later.

Finally, let’s create the other Portal. Drag the Portal prefab you just created into the scene, and fit it into the other hole as seen below.

Scripting the portal

Create a C# script named Portal.cs and attach it to the Portal prefab. Let’s add the class declaration and the target portal variable:

using UnityEngine;public class Portal : MonoBehaviour
{
    public Portal targetPortal;

Adding the variables to reference the Transforms which represent the normals of the visible and invisible side of the portal:

public Transform normalVisible;
public Transform normalInvisible;

A reference to our Portal Camera, a reference to the renderer which renders the portal (the quad), a render texture which the Portal Camera will render to, and finally the material which will be used by our renderer.

public Camera portalCamera;
public Renderer viewthroughRenderer;
private RenderTexture viewthroughRenderTexture;
private Material viewthroughMaterial;

The final variable is a cache to our main camera since we’ll be needing to access the main camera every frame to calculate the PortalCamera’s position and rotation.

private Camera mainCamera;

We’ll add a helper function to transform positions from one portal to another.

We first use InverseTransformPoint, which converts positions from world space to local space, to convert the original world position to a local position relative to the sender portal’s visible side.

After that, we use TransformPoint, which converts positions from local space to world space. We use it on the target portal’s invisible side. This treats the local position we found as relative to the target portal and converts it into a world position that we can return.

public static Vector3 TransformPositionBetweenPortals(Portal sender, Portal target, Vector3 position)
{
    return
        target.normalInvisible.TransformPoint(
            sender.normalVisible.InverseTransformPoint(position));
}

Next, we’ll add a helper function to transform rotations from one portal to another.

In this function, we first take in a world rotation. Then, we multiply it by the inverse of the sender portal’s normalVisible rotation. Think of this as subtracting a rotation, so this finds the local rotation relative to it.

We can then multiply the target portal’s normalInvisible rotation. Think of this as adding a rotation, which will give us the final world position, relative to the target portal.

public static Quaternion TransformRotationBetweenPortals(Portal sender, Portal target, Quaternion rotation)
{
    return
        target.normalInvisible.rotation *
        Quaternion.Inverse(sender.normalVisible.rotation) *
        rotation;
}

For our Start function, we’ll first create a new RenderTexture and assign it to our render texture variable. It is the size of the screen, has a 24-bit depth, and uses the default HDR format (useful if you want effects like Bloom to show through portals, or if you’re just using an HDR workflow).

Calling viewthroughRenderTexture.Create() actually allocates and creates the render texture on the GPU. This is optional, as it’ll be created automatically on first use, but I prefer to call it there to prevent any unexpected lag caused by render texture creation later on in the game.

private void Start()
{
    // Create render texture
    
    viewthroughRenderTexture = new RenderTexture(Screen.width, Screen.height, 24, RenderTextureFormat.DefaultHDR);
 viewthroughRenderTexture.Create();

Next, we’ll clone a Material from our renderer by calling viewthroughRenderer.material. This gives us a copy of the material assigned to our portal so that we can modify it for this portal while leaving other portal’s materials alone. This allows us to have different textures per portal.

We’ll assign the render texture from our previous step into our new material. We’ll also assign it as the target texture of our Portal Camera. This tells the Portal Camera to render onto the render texture instead of the screen.

// Assign render texture to portal material (cloned)

viewthroughMaterial = viewthroughRenderer.material;
viewthroughMaterial.mainTexture = viewthroughRenderTexture;// Assign render texture to portal camera

portalCamera.targetTexture = viewthroughRenderTexture;

Finally, we’ll cache a reference to the main camera.

// Cache the main camera

mainCamera = Camera.main;

We’ll use LateUpdate to render our portal every frame. This ensures that all of our objects will have moved (e.g. the player) before updating the portals. If we do it in Update instead, there’s a chance the portal will lag behind the player’s movement, causing the portal view to lag behind by 1 frame.

First, we’ll use the functions we created earlier to find the positions and rotations of the Portal Camera. We pass in ourselves as the sender, our target portal as the destination, and the main camera’s positions and rotations to be transformed.

private void LateUpdate()
{
    // Calculate portal camera position and rotation    var virtualPosition = TransformPositionBetweenPortals(this, targetPortal, mainCamera.transform.position);
    var virtualRotation = TransformRotationBetweenPortals(this, targetPortal, mainCamera.transform.rotation);

Next, we position and orientate the Portal Camera. As the Camera component on the Portal Camera is enabled, it’ll render to the render texture by itself at the end of the frame. Mission accomplished!

// Position camera
    
portalCamera.transform.SetPositionAndRotation(virtualPosition, virtualRotation);

We shouldn’t forget to clean up after ourselves. In Unity, you have to Destroy() cloned materials and render textures that we created, or there’ll be a memory leak. We also call viewthroughRenderTexture.Release() to release the render texture on the GPU before destroying it. Just in case.

private void OnDestroy()
{
    // Release render texture from GPU
    
    viewthroughRenderTexture.Release();

    // Destroy cloned material and render texture
    
    Destroy(viewthroughMaterial);
    Destroy(viewthroughRenderTexture);
}

Go back to Unity, and double-click on the Portal prefab. If you haven’t attached the Portal script, do it now. You should see a few fields we have to fill out.

Leave the Target Portal field black for now. Drag the Visible, Invisible and Portal Camera objects into their respective fields. For the Viewthrough Renderer field, drag the MeshRenderer component on the Portal object in it.

Exit out of the Portal prefab by clicking on the back arrow at the top of the Hierarchy tab. Select Portal and drag in Portal (1) into the Target Portal field.

Do the same for Portal (1), but reversed; drag Portal into its Target Portal field instead.

Click play!

If you see that the portal is rendering the view from the other portal, and moves around whenever you do, that’s great! That means that the Portal Camera is doing its job. However, right now, it looks more like a TV screen than a portal. This is because we actually need to show the render texture as a screen-space texture.

Screen-space textures

(Credit to Ronja Böhringer for this shader. Tutorial here.)

The difference between screen-space textures and normal UV-based textures is that while normal textures follow the shape of the object using a UV map, screen-space textures look the same regardless of how the object looks or how the camera is facing. In a way, you can think of the screen-space texture as “filling the screen”.

This is exactly the kind of effect we need for the portal to work correctly. I won’t go into detail here on how the shader works, but it will produce the result as seen above. If you need more information, feel free to check out this tutorial.

Create an Unlit Shader by going to Assets > Create > Shader > Unlit Shader. Rename the shader to Portal Viewthrough. Copy this code:

Shader "Custom/Portal Viewthrough"{
    //show values to edit in inspector
    Properties{
        _Color("Tint", Color) = (1, 1, 1, 1)
        _MainTex("Texture", 2D) = "white" {}
    }

        SubShader{
        //the material is completely non-transparent and is rendered at the same time as the other opaque geometry
        Tags{ "RenderType" = "Opaque" "Queue" = "Geometry"}

        Pass{
            CGPROGRAM
            
            //include useful shader functions
            #include "UnityCG.cginc"

            //define vertex and fragment shader
            #pragma vertex vert
            #pragma fragment frag

            //texture and transforms of the texture
            sampler2D _MainTex;
            float4 _MainTex_ST;

            //tint of the texture
            fixed4 _Color;

            //the object data that's put into the vertex shader
            struct appdata {
                float4 vertex : POSITION;
            };

            //the data that's used to generate fragments and can be read by the fragment shader
            struct v2f {
                float4 position : SV_POSITION;
                float4 screenPosition : TEXCOORD0;
            };
            
            struct fragOut
            {
                fixed4 color : SV_TARGET;
            };

            //the vertex shader
            v2f vert(appdata v) {
                v2f o;
                //convert the vertex positions from object space to clip space so they can be rendered
                o.position = UnityObjectToClipPos(v.vertex);
                o.screenPosition = ComputeScreenPos(o.position);
                return o;
            }

            //the fragment shader
            fragOut frag(v2f i) {
                fragOut o;
                float2 textureCoordinate = i.screenPosition.xy / i.screenPosition.w;
                
                fixed4 col = tex2D(_MainTex, textureCoordinate);                
                col *= _Color;
                o.color = col;

                return o;
            }

            ENDCG
        }
    }
}

Once the shader is in, create a new material called Portal Viewthrough using Assets > Create > Material. Change the shader to our new Portal Viewthrough shader.

Double-click on the Portal prefab again. Drag and drop the Portal Viewthrough onto the Portal to apply it. Save the prefab and click play.

The portal looks and behaves like a portal! That’s great, but why does our portal become blacked out when we aren’t standing in the correct square?

Clipping the Portal Camera

(Credit to Daniel Ilett for his tutorial on Matrix Clipping, which helped me get through this problem.)

When the player is standing in some positions such as the one demonstrated in the gif above, the Portal Camera is in a weird position. In this case, the Portal Camera is blocked by its own portal, which blackens the portal. This is not what we want.

How do we solve this problem? The player is standing in a valid position, and the camera is also in the correct position for the correct perspective. However, it is blocked by its own portal! Artefacts can also appear if an object is between the Portal Camera and the target portal.

We solve this problem by using the near clipping plane. However, we’ll have to use a near clipping plane that is on the same plane as the target portal. This means using a clipping plane that is non-parallel, which requires us to calculate an oblique projection matrix.

How the oblique projection matrix looks like in 2D.

We need to first calculate the mathematical plane of the portal. Let’s create a variable to store it:

private Vector4 vectorPlane;

And calculate it at Start:

private void Start()
{
    ...
    
    // Cache the main camera
    
    mainCamera = Camera.main;
    
    // Generate bounding plane

    var plane = new Plane(normalVisible.forward, transform.position);
    vectorPlane = new Vector4(plane.normal.x, plane.normal.y, plane.normal.z, plane.distance);
}

We’ll be creating an oblique matrix which uses our target portal’s plane as the near clip plane.

We’ll need to use the main camera to calculate the oblique matrix. This is because the CalculateObliqueMatrix function requires an existing projection matrix to work off of. Since we’re overwriting our Portal Camera’s projection matrix with the oblique matrix, it cannot be used for the calculation.

This comes with the side effect that the Portal Camera will inherit the near/far clip plane and FOV settings from the main camera. This might be desirable, however, since any differences in FOV might cause the portal illusion to break down.

Calculating the oblique matrix is honestly black magic to me, so I’ll just provide the code that helped me. If you want to learn more about how it works, I suggest checking out Daniel’s tutorial.

private void LateUpdate()
{
    ...
    
    // Position camera
    
    portalCamera.transform.SetPositionAndRotation(virtualPosition, virtualRotation);
    
    // Calculate projection matrix
    
    var clipThroughSpace = Matrix4x4.Transpose(Matrix4x4.Inverse(portalCamera.worldToCameraMatrix)) * targetPortal.vectorPlane;
    
    // Set portal camera projection matrix to clip walls between target portal and portal camera
    // Inherits main camera near/far clip plane and FOV settings
    
    var obliqueProjectionMatrix = mainCamera.CalculateObliqueMatrix(clipThroughSpace);
    portalCamera.projectionMatrix = obliqueProjectionMatrix;
    
    // Render camera
    
    portalCamera.Render();
}

Click play!

It’s seamless! Well, almost. Did you notice any problems with lighting in the gif shown?

The lighting is inconsistent between portals, as well as the shadows being cut off. This might not seem like much in this simple scene, but it’ll get worse on complex scenes. More importantly, it’ll break the “no portals” illusion that we are going for here.

Lighting limitations

If you’re going for a Portal (the game) approach where portals are obvious and known to the player, feel free to skip this section.

However, if you’re going for a “non-euclidean” approach where portals must not be seen by the player, you will have to apply heavy limits on your use of lighting and shadows.

Lighting solution 1: Don’t use any lights

The first way of doing it is to simply not use any lights or shadows. This technique has been used in other non-euclidean games before, such as the critically-acclaimed Antichamber.

To achieve this, delete the Directional Light from our scene. This will cast your scene into darkness.

Next, go to the Lighting tab (Window > Rendering > Lighting Settings) and change the Environment Settings. Change the Source to Color, and then change the Ambient Color. Setting it to fully white (#FFFFFF) will let your scene have “pure colour”. If an object is green, it’ll appear as green, exactly. However, feel free to play around with the Ambient Color setting to your liking.

Lighting solution 2: Only use vertical directional lights

However, for a game like Quantum Tournament v0.1.0, I decided to use realistic materials, which looked terrible without lighting. Even a little bit of lighting would give depth. As you can see in the screenshot below, there is a slight bit of lighting on the player models and the gun models.

To achieve this, select Directional Light and make it point exactly downwards.

However, this makes the walls black. In other games, we would solve this problem by baking the lighting. However, as the Unity default lightmapper does not support portals (if there are any that do, please tell me, that would be amazing), baking lights are off the list.

To solve this problem, we turn to ambient lighting once again. For this approach, we shall set the ambient lighting as something lower than pure white. This gives the directional lighting a chance to give surfaces facing upwards a brighter colour, giving the scene depth.

You can use shadows on the directional light. Just be careful when making indoor levels; the outside of the level roof doesn’t cast a shadow. If you have geometry inside the level that casts shadows but the roof itself doesn’t, players may ask questions.

There is a limitation to this, however. This only works if your portals don’t change the orientation of the player, such as a wall-to-ceiling portal, as the lighting will change between them. Floor-to-ceiling portals work fine though, as do usual wall-to-wall portals.

Lighting solution 3: Cheat point lights and spotlights!

If you want to use point lights or spotlights, you’ll have to cheat! This effect was used in Quantum Tournament v0.1.0’s main menu for its explosion effect.

I’m sorry if the bullet holes give you the heebie-jeebies.

When you add a point light or spotlight, use the Range property to make sure they never cross into a portal.

Next, set the Render Mode of the light to Important, to make sure pixel lighting is used. If vertex lighting is used, it is possible a vertex beyond the portal is used to apply the light, stretching the lighting across the portal and breaking the illusion.

It should be said that there are 3 major limitations to this approach:

If you use Forward rendering, each pixel light adds an additional pass to the rendering. In other words, if you have too many pixel lights, you slow the game down. Since this approach requires you to use pixel lighting (or some clever vertex placement) to ensure the illusion is not broken, this limits the number of point lights and spotlights you can place in the scene.
Your lights can’t cross a portal, so they have to be small and localised unless you use some clever placement.
Since you have to ensure point lights and spotlights never cross into a portal, you can’t have player-spawned lights without limits. For example, a muzzle flash. Dynamic lights are still fine though, as long as they never cross a portal.

Also, you can use shadows, no problem.

Final result

After implementing lighting solution 2, changing the directional light to be fully horizontal and adding ambient lighting, here is the result.

Looks great, no artefacts! The player wouldn’t even know about the portal.

Closing

In this tutorial, you have learnt how to create a basic portal which the player can see through without visual artefacts, and you’ve also learnt a few limitations that we have to live with if we want to create preserve the “no portals” illusion.

In the next tutorial, we will discuss how to allow the player to walk through the portals seamlessly.

Author’s note

Did you enjoy this tutorial? Did it help you make a game with portal-like mechanics? If so, please consider buying me a cup of coffee. Thank you! ❤

Full scripts

PlayerController.cs

using UnityEngine;

public class PlayerController : MonoBehaviour
{
    private CharacterController characterController;

    public float moveSpeed = 5;
    public float turnSpeed = 5;

    private void Start()
    {
        characterController = GetComponent<CharacterController>();
    }

    private void Update()
    {
        // Turn player
        
        transform.Rotate(Vector3.up * Input.GetAxis("Mouse X") * turnSpeed);
        
        // Move player
        
        characterController.SimpleMove(
            transform.forward * Input.GetAxis("Vertical") * moveSpeed +
            transform.right * Input.GetAxis("Horizontal") * moveSpeed);
    }
}

Portal.cs

using UnityEngine;

public class Portal : MonoBehaviour
{
    public Portal targetPortal;
    
    public Transform normalVisible;
    public Transform normalInvisible;

    public Camera portalCamera;
    public Renderer viewthroughRenderer;
    private RenderTexture viewthroughRenderTexture;
    private Material viewthroughMaterial;

    private Camera mainCamera;

    private Vector4 vectorPlane;

    public static Vector3 TransformPositionBetweenPortals(Portal sender, Portal target, Vector3 position)
    {
        return
            target.normalInvisible.TransformPoint(
                sender.normalVisible.InverseTransformPoint(position));
    }

    public static Quaternion TransformRotationBetweenPortals(Portal sender, Portal target, Quaternion rotation)
    {
        return
            target.normalInvisible.rotation *
            Quaternion.Inverse(sender.normalVisible.rotation) *
            rotation;
    }

    private void Start()
    {
        // Create render texture
        
        viewthroughRenderTexture = new RenderTexture(Screen.width, Screen.height, 24, RenderTextureFormat.DefaultHDR);
        viewthroughRenderTexture.Create();
        
        // Assign render texture to portal material (cloned)
        
        viewthroughMaterial = viewthroughRenderer.material;
        viewthroughMaterial.mainTexture = viewthroughRenderTexture;
        
        // Assign render texture to portal camera
        
        portalCamera.targetTexture = viewthroughRenderTexture;
        
        // Cache the main camera
        
        mainCamera = Camera.main;
        
        // Generate bounding plane

        var plane = new Plane(normalVisible.forward, transform.position);
        vectorPlane = new Vector4(plane.normal.x, plane.normal.y, plane.normal.z, plane.distance);
    }

    private void LateUpdate()
    {
        // Calculate portal camera position and rotation

        var virtualPosition = TransformPositionBetweenPortals(this, targetPortal, mainCamera.transform.position);
        var virtualRotation = TransformRotationBetweenPortals(this, targetPortal, mainCamera.transform.rotation);
        
        // Position camera
        
        portalCamera.transform.SetPositionAndRotation(virtualPosition, virtualRotation);
        
        // Calculate projection matrix
        
        var clipThroughSpace =
            Matrix4x4.Transpose(Matrix4x4.Inverse(portalCamera.worldToCameraMatrix))
            * targetPortal.vectorPlane;
        
        // Set portal camera projection matrix to clip walls between target portal and portal camera
        // Inherits main camera near/far clip plane and FOV settings
        
        var obliqueProjectionMatrix = mainCamera.CalculateObliqueMatrix(clipThroughSpace);
        portalCamera.projectionMatrix = obliqueProjectionMatrix;
        
        // Render camera
        
        portalCamera.Render();
    }

    private void OnDestroy()
    {
        // Release render texture from GPU
        
        viewthroughRenderTexture.Release();
   
        // Destroy cloned material and render texture
        
        Destroy(viewthroughMaterial);
        Destroy(viewthroughRenderTexture);
    }
}

Portal Viewthrough.shader