RealityKit 911 — Lighting and shadows
Neither a magnificent shape of geometry nor its exquisite texture makes a 3D model look realistically in AR/VR scene. It’s rather a light with its shadows that does a job. In AR, shadows is a key component of model’s integration, because shadows are grounding objects. Without shadows objects appear as if they are floating in the air. And without proper light intensity and light direction viewer never believe that a model belongs to a real-world scene.
RealityKit implicitly uses Light Estimation algorithm that helps us automatically light virtual models to match a real world lighting conditions. But also, we can explicitly add lights to our scene whenever we want to.
There are four light types in RealityKit that helps us light a scene: point light, spot light, directional light and image-based light (IBL). Unlike SceneKit that has only classes for lighting — SCNLight class with its corresponding LightType structure, — RealityKit, in order to propose a convenient and efficient programming interface for Swift developers, uses both — classes for inheritance (OOP paradigm) and protocols for composition (POP paradigm).
RealityKit 2.0 has four classes and three protocols to create a unique lighting conditions. You can use them together or separately, it’s up to you.
public class DirectionalLight: Entity, HasDirectionalLight
public class PointLight: Entity, HasPointLight
public class SpotLight: Entity, HasSpotLightpublic struct ImageBasedLight
An ImageBasedLight structure is nested inside Environment structure that is located inside RealityKit’s view. Image-based light is used for environmental lighting.
extension ARView { public struct Environment { public struct ImageBasedLight { public var resource: EnvironmentResource?
public var intensityExponent: Float
}
}
}
Hence, we can access environmental light properties using Swift’s dot notation:
arView.environment.lighting.resource
arView.environment.lighting.intensityExponentLet’s find out what RealityKit lights are and what shadow types they produce. We’ll test all light types in simple scene where only default RealityKit light is activated (default is IBL).
Directional Light
Directional Light simulates a parallel rays coming from a distant star — The Sun. The most significant aspect of this type of light is its orientation. A position of Directional Light is unimportant. This light produces depth-map shadows. The intensity of the directional light is measured in lumen per square meter.
Important: If there are no lights except directional light in a scene, surfaces that parallel to light rays’ direction will be black.
Here are DirectionalLight properties:
let directionalLight = DirectionalLight()directionalLight.light.color = .red
directionalLight.light.intensity = 20000
directionalLight.light.isRealWorldProxy = truedirectionalLight.shadow?.maximumDistance = 10.0
directionalLight.shadow?.depthBias = 5.0directionalLight.orientation = simd_quatf(angle: -.pi/1.5,
axis: [0,1,0])
Point Light
Point light is an omnidirectional light like an electric bulb. Its position is crucial but orientation is unimportant. This light type can’t generate any shadows at all at the moment, so there’s no instance property called shadow. The intensity of the point light is measured in lumen.
Here are PointLight properties:
let pointLight = PointLight()pointLight.light.color = .blue
pointLight.light.intensity = 15000000
pointLight.light.attenuationRadius = 7.0pointLight.position = [4,0,1]
Spotlight
As its name suggests, SpotLight simulates a cone-angled ray of light that you have seen many times in theatre, circus, or at a concert. This lighting cone produces hotspot, a region between inner and outer light cones (a.k.a. penumbra, that specifies the angle within the light cone at which the light begins to transition from full strength to no lighting), and soft shadow edges. If your device is equipped with A12 chipset or better, the type of a rendered shadows will be ray-traced (shadows of high quality), but earlier versions of Apple chipsets (A9, A10, A11) can generate only depth-mapped shadows (fake projections). The intensity of the spotlight measured in lumen.
Here are SpotLight properties:
let spotLight = SpotLight()spotLight.light.color = .green
spotLight.light.intensity = 2500000
spotLight.light.innerAngleInDegrees = 70
spotLight.light.outerAngleInDegrees = 120
spotLight.light.attenuationRadius = 9.0spotLight.shadow = SpotLightComponent.Shadow()spotLight.position.y = 5.0
spotLight.orientation = simd_quatf(angle: -.pi/1.5,
axis: [1,0,0])
Image-Based Lighting
In RealityKit your scene can be lit with a help of 32-bit High Dynamic Range Image that can be in HDR or OpenEXR file format. Image-based lighting works with positive and negative values. HDRI may have nine or more images at 1/3 stop increments, but you’ll need a minimum of three exposures to work with HDRI.
RealityKit has a default HDR lighting and we can easily increase an intensity of this light by stopping an exposure up like this:
arView.environment.lighting.intensityExponent = 2
And as I said earlier, HDR light’s intensity can be negative as well. We could reduce an intensity of environment light by stopping its exposure down:
arView.environment.lighting.intensityExponent = -2
Creating custom lights
Now, when we know enough about RealityKit light types, let’s create our custom rig of lights. Each class must inherit from Entity parent class and be conformed to a corresponding protocol.
At first we’ll create a custom directional light:
import ARKit
import RealityKitclass CustomDirectionalLight: Entity, HasDirectionalLight { required init() {
super.init() self.light = DirectionalLightComponent(color: .red,
intensity: 20000,
isRealWorldProxy: true) self.shadow = DirectionalLightComponent.Shadow(
maximumDistance: 10,
depthBias: 5.0) self.orientation = simd_quatf(angle: -.pi/1.5,
axis: [0,1,0])
}
}
Then a point light:
class CustomPointLight: Entity, HasPointLight { required init() {
super.init()
self.light = PointLightComponent(color: .blue,
intensity: 15000000,
attenuationRadius: 7.0) self.position = [4,0,1]
}
}
And, at last, a spot light:
class CustomSpotLight: Entity, HasSpotLight { required init() {
super.init() self.light = SpotLightComponent(color: .green,
intensity: 2500000,
innerAngleInDegrees: 70,
outerAngleInDegrees: 120,
attenuationRadius: 9.0) self.shadow = SpotLightComponent.Shadow() self.position.y = 5.0
self.orientation = simd_quatf(angle: -.pi/1.5,
axis: [1,0,0])
}
}
Now we can use these light types together in a ViewController.swift file:
class ViewController: UIViewController { @IBOutlet weak var arView: ARView! override func viewDidLoad() {
super.viewDidLoad() // Lights
let directLight = CustomDirectionalLight()
let pointLight = CustomPointLight()
let spotLight = CustomSpotLight() let lightAnchor = AnchorEntity(world: [0,0,-3])
lightAnchor.addChild(directLight)
lightAnchor.addChild(pointLight)
lightAnchor.addChild(spotLight) // Temple model from Reality Composer
let sceneAnchor = try! Experience.loadTempleScene().
sceneAnchor.templeModel!.position.y = -0.6
sceneAnchor.templeModel!.scale = [4,4,4]
// Plane primitive
let plane: MeshResource = .generatePlane(width: 7.0,
depth: 7.0) let material = SimpleMaterial(color: .darkGray,
isMetallic: false) let entity = ModelEntity(mesh: plane,
materials: [material]) entity.position.y = -0.6
sceneAnchor.addChild(entity) arView.scene.anchors.append(sceneAnchor)
arView.scene.anchors.append(lightAnchor)
}
}
Voila!
Often you need to light a scene with your own HDR file. For that just add an environment resource into your project — make a folder with a name “iskybox” and place an HDR image inside. The image should be an environment map of latitude-longitude projection. Drag the folder into your Xcode project file navigator. In the drop-down options pane that appears, choose Create folder references.
Then paste these lines of code into viewDidLoad() method:
override func viewDidLoad() {
super.viewDidLoad() arView.environment.lighting.resource = try! .load(named: "iskybox/city.hdr") arView.environment.lighting.intensityExponent = -1 // blah-blah...
}
However, if you do not need an environmental lighting, you can disable it using disableAREnvironmentLighting type property from RenderOptions structure that conforms to OptionSet protocol.
arView.renderOptions = [.disableAREnvironmentLighting,
.disableMotionBlur]P. S.
Undoubtedly, at the moment SceneKit has a richer arsenal of lighting types, to be precise, 2 times more. Here is their list:
• Ambient
• Area
• Directional
• Environment HDR
• Omni
• Photometric IES
• Probe
• SpotBut I’m sure that the list of RealityKit light fixtures will grow very soon too.
If you want to turn a lighting on, but do not want to overload the CPU, you should pay attention to texture-based shadows.
That’s all for now.
If this post is useful for you, please press the Clap button and hold it.
¡Hasta la vista!
