Multiple Recursive Portals and AI In Unity Part 3: Raycasting through portals
This tutorial is part of a multi-part series. See the table of contents here.
In the last tutorial, we finished building a non-euclidean room that can be walked around seamlessly. In this tutorial, we’re going to create a cube that we can shoot through portals.
This tutorial is made with Unity 2019.3.7f1. It relies on the final project from the previous tutorial, Part 2. If you don’t have it, feel free to download the previous project or the starter kit from the links below.
Repositories
You can find the repositories here: https://github.com/limdingwen/Portal-Tutorial-Repository
Download by clicking “Clone or Download”.
For this tutorial, these are the 3 relevant folders:
- Part 2 Final: The previous project, includes Part 2 premade
- Part 3 Starter: Starter kit, includes Part 2, the new player controller, and the shootable cube premade
- Part 3 Final: Final project, includes Part 3 premade
If you just want the full scripts, you can find them at the very end of this page.
Preparation
If you don’t want to modify the player and create the shootable cube yourself, you can grab it from the starter kit from the links above.
Adding true freelook
If we’re going to be able to shoot things, we should be able to look up and down as well. Go to PlayerController.cs and add in a reference to the camera, and a variable to store our current up/down rotation:
public Transform playerCamera;
private float verticalRotationAbsolute;In Update(), modify the vertical rotation based on user input and clamp it:
private void Update()
{
turnRotation += Input.GetAxis("Mouse X");
verticalRotationAbsolute += Input.GetAxis("Mouse Y") * -turnSpeed;
verticalRotationAbsolute = Mathf.Clamp(verticalRotationAbsolute, -89, 89);
}Finally, in FixedUpdate(), we rotate the player camera based on the stored and clamped vertical rotation:
// Turn player
transform.Rotate(Vector3.up * turnRotation * turnSpeed);
turnRotation = 0; // Consume variable
// Turn player (up/down)
playerCamera.localRotation = Quaternion.Euler(verticalRotationAbsolute, 0, 0);If you play the game now, you’ll be able to move the view up and down. Let’s add a crosshair for us to aim with, and do away with the annoying mouse cursor in the way.
Now that I’m able to look down, I see that my shadow gets cut off whenever I go through a portal. Why is that?
When going through a portal, your body is not cloned to the other side of the portal. Basically, your body gets cut off whenever you go through a portal, which causes your shadow to be cut off too. We’ll fix this in a later tutorial (Part 5, Object Cloning) but if you’re bothered by it, you can turn off shadows on the directional light.
Adding a crosshair
Let’s first hide the cursor. Modify Start() in PlayerController.cs like so:
private void Start()
{
characterController = GetComponent<CharacterController>();
portalableObject = GetComponent<PortalableObject>();
portalableObject.HasTeleported += PortalableObjectOnHasTeleported;
Cursor.visible = false;
Cursor.lockState = CursorLockMode.Locked;
}Next, create a UI image by pressing GameObject > UI > Image. The UI canvas should be automatically created for you. You should see a blank square in the middle of your screen.
Let’s select a default Unity sprite, “Knob”, to be our crosshair.
Finally, rename the Image to a helpful name, and resize its width and height to 10 pixels each.
You should now have a helpful crosshair in the middle of the screen!
Adding a shootable cube
Create a cube by pressing GameObject > 3D Object > Cube. Move it such that it’s somewhere within the level.
Create a new C# script named ShootableCube.cs and attach it to the cube. When the cube is shot, using SendMessage(“OnShoot”), it will toggle its colour between green and white.
using UnityEngine;
public class ShootableCube : MonoBehaviour
{
private new Renderer renderer;
private bool isGreen;
private void Start()
{
renderer = GetComponent<Renderer>();
}
public void OnShoot()
{
isGreen = !isGreen;
renderer.material.color = isGreen ? Color.green : Color.white;
}
}We now have to allow the player to shoot by using raycasts, but first, we have to give the player’s character controller a separate layer to prevent the player from shooting himself, since the raycast will begin from inside the player.
Click on the Player object and then click on Layer. Click “Add Layer…” and add a new layer named “Player”. Go back to the Player object, and click on Layer again. This time, select the new layer you just created.
Select “No, this object only” since we don’t want to change the layer of our camera.
Go to the PlayerController.cs script. We’ll first add in some variables to define the layer mask our raycast will use, and a boolean to store our left-click mouse input.
public LayerMask shootingMask;
private bool shoot;In Update(), every frame, check for left click. If the mouse has been left-clicked that frame, we’ll set the shoot boolean.
private void Update()
{
turnRotation += Input.GetAxis("Mouse X");
verticalRotationAbsolute += Input.GetAxis("Mouse Y") * -turnSpeed;
verticalRotationAbsolute = Mathf.Clamp(verticalRotationAbsolute, -89, 89);
if (Input.GetButtonDown("Fire1")) shoot = true;
}In FixedUpdate(), every tick, we’ll check if we have to shoot. If we do, we’ll perform a raycast and send a OnShoot message to any collider that we hit.
// Move player
characterController.SimpleMove(
transform.forward * Input.GetAxis("Vertical") * moveSpeed +
transform.right * Input.GetAxis("Horizontal") * moveSpeed);
// Shoot
if (shoot)
{
if (Physics.Raycast(playerCamera.position,
playerCamera.forward,
out var hitInfo,
Mathf.Infinity,
shootingMask.value))
{
hitInfo.collider.SendMessage("OnShoot", SendMessageOptions.DontRequireReceiver);
}
shoot = false;
}Finally, go back to the Player object in the editor. Set the Shooting Mask variable as shown below.
Click play!
As you can see, when we left-click on the cube, it’ll toggle between green and white. But what happens if we try to do the same through a portal?
Nothing happens. Don’t worry, that’s what the rest of the tutorial is about!
Let’s begin!
Portal theory
When the raycast hits a portal, we don’t stop there. Instead, we spawn a new raycast at the target portal, with the direction and position transformed using the same helper functions that calculate Portal Camera and teleportation.
We repeat this until we either run out of recursions, hit something that is not a portal, or we don’t hit anything at all.
There is a gotcha to this, however. In some cases, it is possible that the newly spawned raycast begins from within the portal. Because of this, we also need to temporarily ignore the portal we are spawning the raycast from.
Scripting a generic recursive raycast
We can add the recursive raycast logic directly inside the Player’s shooting logic, but it’ll be better to add it to Portal.cs instead. In the future, other functionality like AI may use recursive raycasting to do non-shooting things, such as visibility checks.
First, we’ll need to be able to transform directions between portals, in addition to transforming positions and rotations. Let’s add a helper function to do so. It works the same way as TransformPositionBetweenPortals, just that it uses directions instead of points.
public static Vector3 TransformDirectionBetweenPortals(Portal sender, Portal target, Vector3 position)
{
return
target.normalInvisible.TransformDirection(
sender.normalVisible.InverseTransformDirection(position));
}Next, we’ll start on the recursive raycast function. Our function works a lot like the default Unity raycast. We accept an origin position and a direction, a layer mask to determine what surfaces can be hit, and output for hit information. We’ll also accept a parameter that determines how many recursions we should limit ourselves to.
public static bool RaycastRecursive(
Vector3 position,
Vector3 direction,
LayerMask layerMask,
int maxRecursions,
out RaycastHit hitInfo)
{
return RaycastRecursiveInternal(position,
direction,
layerMask,
maxRecursions,
out hitInfo,
0,
null);
}For the actually recursive function, let’s start off by defining the function. The signature is mostly the same, except for 2 new parameters. currentRecursion tracks which recursion we are in, and we can pass in an ignoreObject for the raycast to ignore. This will be used to ignore portals that our raycast spawns out of, explained in the theory above.
private static bool RaycastRecursiveInternal(
Vector3 position,
Vector3 direction,
LayerMask layerMask,
int maxRecursions,
out RaycastHit hitInfo,
int currentRecursion,
GameObject ignoreObject)
{Before we shoot a raycast, let’s ignore the object passed into the function. First, we back up the layer the object is already on. Then, we set the object’s layer to 2, which is the Ignore Raycast layer.
// Ignore a specific object when raycasting.
// Useful for preventing a raycast through a portal from hitting the target portal from the back,
// which makes a raycast unable to go through a portal since it'll just be absorbed by the target portal's trigger.
var ignoreObjectOriginalLayer = 0;
if (ignoreObject)
{
ignoreObjectOriginalLayer = ignoreObject.layer;
ignoreObject.layer = 2; // Ignore raycast
}We’ll shoot a basic raycast and store the results.
// Shoot raycast
var raycastHitSomething = Physics.Raycast(
position,
direction,
out var hit,
Mathf.Infinity,
layerMask); // Clamp to max array lengthThen, we restore the original layer of the object we ignored.
// Reset ignore
if (ignoreObject)
ignoreObject.layer = ignoreObjectOriginalLayer;If we didn’t hit anything, we’ll stop the recursion here. Because C# requires us to assign something to all out parameters before ending the function, we have to create a dummy RaycastHit() struct before exiting.
// If no objects are hit, the recursion ends here, with no effect
if (!raycastHitSomething)
{
hitInfo = new RaycastHit(); // Dummy
return false;
}If we did hit something, check if what we hit was a Portal. If so, we shall first check if we are at our recursion limit.
If we are at the limit, we’ll also end the recursion here.
If we are not at the limit, we’ll recurse the function again. The new raycast origin and direction are transformed from our current portal to the target portal. We also increment the currentRecursion value by 1. We tell the new raycast to ignore the target portal, as per the theory.
// If the object hit is a portal, recurse, unless we are already at max recursions
var portal = hit.collider.GetComponent<Portal>();
if (portal)
{
if (currentRecursion >= maxRecursions)
{
hitInfo = new RaycastHit(); // Dummy
return false;
} // Continue going down the rabbit hole...
return RaycastRecursiveInternal(
TransformPositionBetweenPortals(portal, portal.targetPortal, hit.point),
TransformDirectionBetweenPortals(portal, portal.targetPortal, direction),
layerMask,
maxRecursions,
out hitInfo,
currentRecursion + 1,
portal.targetPortal.gameObject);
}
Finally, if the hit object was not a Portal, we’ll simply return the hit info and also return true, specifying that the raycast hit something.
// If the object hit is not a portal, then congrats! We stop here and report back that we hit something.
hitInfo = hit;
return true;Using the recursive raycast for shooting
We now need to use this function in the shooting logic in PlayerController.cs. For simplicity’s sake, I’ve hardcoded the max recursions to be 8, which is more than enough, but it can be converted into an inspector-editable value if needed.
// Shoot
if (shoot)
{
if (Portal.RaycastRecursive(playerCamera.position,
playerCamera.forward,
shootingMask.value,
8, // Max recursions
out var hitInfo))
{
hitInfo.collider.SendMessage("OnShoot", SendMessageOptions.DontRequireReceiver);
}
shoot = false;
}Final result
As you can see, we can shoot the cube through the portal, and it will change colour. Mission accomplished! The recursive raycast function will be very useful in later tutorials, as well as for implementing your own functionality.
Closing
In this tutorial, you have learnt how to raycast through portals using recursion. You have also learnt to use recursive raycasting to let the player shoot a cube through portals.
In the next tutorial, we’ll discuss how to convert our current basic portal rendering system into a recursive one, allowing us to make portals visible through other portals.
Author’s note
Did you enjoy this tutorial? Did it help you make a game with portal-like mechanics? If so, please consider buying me a cup of coffee. Thank you! ❤
Full scripts
PlayerController.cs
using UnityEngine;
public class PlayerController : MonoBehaviour
{
private CharacterController characterController;
private PortalableObject portalableObject;
public float moveSpeed = 5;
public float turnSpeed = 5;
private float turnRotation;
public Transform playerCamera;
private float verticalRotationAbsolute;
public LayerMask shootingMask;
private bool shoot;
private void Start()
{
characterController = GetComponent<CharacterController>();
portalableObject = GetComponent<PortalableObject>();
portalableObject.HasTeleported += PortalableObjectOnHasTeleported;
Cursor.visible = false;
Cursor.lockState = CursorLockMode.Locked;
}
private void PortalableObjectOnHasTeleported(Portal sender, Portal destination, Vector3 newposition, Quaternion newrotation)
{
// For character controller to update
Physics.SyncTransforms();
}
private void FixedUpdate()
{
// Turn player
transform.Rotate(Vector3.up * turnRotation * turnSpeed);
turnRotation = 0; // Consume variable
// Turn player (up/down)
playerCamera.localRotation = Quaternion.Euler(verticalRotationAbsolute, 0, 0);
// Move player
characterController.SimpleMove(
transform.forward * Input.GetAxis("Vertical") * moveSpeed +
transform.right * Input.GetAxis("Horizontal") * moveSpeed);
// Shoot
if (shoot)
{
if (Portal.RaycastRecursive(playerCamera.position,
playerCamera.forward,
shootingMask.value,
8,
out var hitInfo))
{
hitInfo.collider.SendMessage("OnShoot", SendMessageOptions.DontRequireReceiver);
}
shoot = false;
}
}
private void Update()
{
turnRotation += Input.GetAxis("Mouse X");
verticalRotationAbsolute += Input.GetAxis("Mouse Y") * -turnSpeed;
verticalRotationAbsolute = Mathf.Clamp(verticalRotationAbsolute, -89, 89);
if (Input.GetButtonDown("Fire1")) shoot = true;
}
private void OnDestroy()
{
portalableObject.HasTeleported -= PortalableObjectOnHasTeleported;
}
}Portal.cs
using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class Portal : MonoBehaviour
{
public Portal targetPortal;
public Transform normalVisible;
public Transform normalInvisible;
public Camera portalCamera;
public Renderer viewthroughRenderer;
private RenderTexture viewthroughRenderTexture;
private Material viewthroughMaterial;
private Camera mainCamera;
private Vector4 vectorPlane;
private HashSet<PortalableObject> objectsInPortal = new HashSet<PortalableObject>();
private HashSet<PortalableObject> objectsInPortalToRemove = new HashSet<PortalableObject>();
public static Vector3 TransformPositionBetweenPortals(Portal sender, Portal target, Vector3 position)
{
return
target.normalInvisible.TransformPoint(
sender.normalVisible.InverseTransformPoint(position));
}
public static Vector3 TransformDirectionBetweenPortals(Portal sender, Portal target, Vector3 position)
{
return
target.normalInvisible.TransformDirection(
sender.normalVisible.InverseTransformDirection(position));
}
public static Quaternion TransformRotationBetweenPortals(Portal sender, Portal target, Quaternion rotation)
{
return
target.normalInvisible.rotation *
Quaternion.Inverse(sender.normalVisible.rotation) *
rotation;
}
private void Start()
{
// Create render texture
viewthroughRenderTexture = new RenderTexture(Screen.width, Screen.height, 24, RenderTextureFormat.DefaultHDR);
viewthroughRenderTexture.Create();
// Assign render texture to portal material (cloned)
viewthroughMaterial = viewthroughRenderer.material;
viewthroughMaterial.mainTexture = viewthroughRenderTexture;
// Assign render texture to portal camera
portalCamera.targetTexture = viewthroughRenderTexture;
// Cache the main camera
mainCamera = Camera.main;
// Generate bounding plane
var plane = new Plane(normalVisible.forward, transform.position);
vectorPlane = new Vector4(plane.normal.x, plane.normal.y, plane.normal.z, plane.distance);
StartCoroutine(WaitForFixedUpdateLoop());
}
private IEnumerator WaitForFixedUpdateLoop()
{
var waitForFixedUpdate = new WaitForFixedUpdate();
while (true)
{
yield return waitForFixedUpdate;
try
{
CheckForPortalCrossing();
}
catch (Exception e)
{
// Catch exceptions so our loop doesn't die whenever there is an error
Debug.LogException(e);
}
}
}
private void CheckForPortalCrossing()
{
// Clear removal queue
objectsInPortalToRemove.Clear();
// Check every touching object
foreach (var portalableObject in objectsInPortal)
{
// If portalable object has been destroyed, remove it immediately
if (portalableObject == null)
{
objectsInPortalToRemove.Add(portalableObject);
continue;
}
// Check if portalable object is behind the portal using Vector3.Dot (dot product)
// If so, they have crossed through the portal.
var pivot = portalableObject.transform;
var directionToPivotFromTransform = pivot.position - transform.position;
directionToPivotFromTransform.Normalize();
var pivotToNormalDotProduct = Vector3.Dot(directionToPivotFromTransform, normalVisible.forward);
if (pivotToNormalDotProduct > 0) continue;
// Warp object
var newPosition = TransformPositionBetweenPortals(this, targetPortal, portalableObject.transform.position);
var newRotation = TransformRotationBetweenPortals(this, targetPortal, portalableObject.transform.rotation);
portalableObject.transform.SetPositionAndRotation(newPosition, newRotation);
portalableObject.OnHasTeleported(this, targetPortal, newPosition, newRotation);
// Object is no longer touching this side of the portal
objectsInPortalToRemove.Add(portalableObject);
}
// Remove all objects queued up for removal
foreach (var portalableObject in objectsInPortalToRemove)
{
objectsInPortal.Remove(portalableObject);
}
}
public static bool RaycastRecursive(
Vector3 position,
Vector3 direction,
LayerMask layerMask,
int maxRecursions,
out RaycastHit hitInfo)
{
return RaycastRecursiveInternal(position,
direction,
layerMask,
maxRecursions,
out hitInfo,
0,
null);
}
private static bool RaycastRecursiveInternal(
Vector3 position,
Vector3 direction,
LayerMask layerMask,
int maxRecursions,
out RaycastHit hitInfo,
int currentRecursion,
GameObject ignoreObject)
{
// Ignore a specific object when raycasting.
// Useful for preventing a raycast through a portal from hitting the target portal from the back,
// which makes a raycast unable to go through a portal since it'll just be absorbed by the target portal's trigger.
var ignoreObjectOriginalLayer = 0;
if (ignoreObject)
{
ignoreObjectOriginalLayer = ignoreObject.layer;
ignoreObject.layer = 2; // Ignore raycast
}
// Shoot raycast
var raycastHitSomething = Physics.Raycast(
position,
direction,
out var hit,
Mathf.Infinity,
layerMask); // Clamp to max array length
// Reset ignore
if (ignoreObject)
ignoreObject.layer = ignoreObjectOriginalLayer;
// If no objects are hit, the recursion ends here, with no effect
if (!raycastHitSomething)
{
hitInfo = new RaycastHit(); // Dummy
return false;
}
// If the object hit is a portal, recurse, unless we are already at max recursions
var portal = hit.collider.GetComponent<Portal>();
if (portal)
{
if (currentRecursion >= maxRecursions)
{
hitInfo = new RaycastHit(); // Dummy
return false;
}
// Continue going down the rabbit hole...
return RaycastRecursiveInternal(
TransformPositionBetweenPortals(portal, portal.targetPortal, hit.point),
TransformDirectionBetweenPortals(portal, portal.targetPortal, direction),
layerMask,
maxRecursions,
out hitInfo,
currentRecursion + 1,
portal.targetPortal.gameObject);
}
// If the object hit is not a portal, then congrats! We stop here and report back that we hit something.
hitInfo = hit;
return true;
}
private void LateUpdate()
{
// Calculate portal camera position and rotation
var virtualPosition = TransformPositionBetweenPortals(this, targetPortal, mainCamera.transform.position);
var virtualRotation = TransformRotationBetweenPortals(this, targetPortal, mainCamera.transform.rotation);
// Position camera
portalCamera.transform.SetPositionAndRotation(virtualPosition, virtualRotation);
// Calculate projection matrix
var clipThroughSpace =
Matrix4x4.Transpose(Matrix4x4.Inverse(portalCamera.worldToCameraMatrix))
* targetPortal.vectorPlane;
// Set portal camera projection matrix to clip walls between target portal and portal camera
// Inherits main camera near/far clip plane and FOV settings
var obliqueProjectionMatrix = mainCamera.CalculateObliqueMatrix(clipThroughSpace);
portalCamera.projectionMatrix = obliqueProjectionMatrix;
}
private void OnTriggerEnter(Collider other)
{
var portalableObject = other.GetComponent<PortalableObject>();
if (portalableObject)
{
objectsInPortal.Add(portalableObject);
}
}
private void OnTriggerExit(Collider other)
{
var portalableObject = other.GetComponent<PortalableObject>();
if (portalableObject)
{
objectsInPortal.Remove(portalableObject);
}
}
private void OnDestroy()
{
// Release render texture from GPU
viewthroughRenderTexture.Release();
// Destroy cloned material and render texture
Destroy(viewthroughMaterial);
Destroy(viewthroughRenderTexture);
}
}ShootableCube.cs
using UnityEngine;
public class ShootableCube : MonoBehaviour
{
private new Renderer renderer;
private bool isGreen;
private void Start()
{
renderer = GetComponent<Renderer>();
}
public void OnShoot()
{
isGreen = !isGreen;
renderer.material.color = isGreen ? Color.green : Color.white;
}
}