RealityKit 911 — Entity Component System (ECS)

Entity-Component-System is the core of the RealityKit’s Data-Oriented paradigm (opposed to Object-Oriented paradigm). I firmly believe that ECS is the most robust architectural pattern for AR/VR game engines. ECS follows the “composition over inheritance” principle. In RealityKit 2.0, announced at WWDC 2021, Apple engineers finally implemented a protocol called System — a missing element in ECS design. Systems are useful for implementing behavior that affects multiple entities in your scene, like the flocking behavior. Now the puzzle is completed.

@available(macOS 12.0, iOS 15.0, *)
public protocol System { }

Let’s demistify ECS pattern

There is a lot of speculation around Entity-Component-System. I feel I should demistify some of them.

• Entity is a node or, in other words, container for 3D character, or for anchor, or for camera, or for light, with two basic components — transform and synchronization. Thanks to these components we can translate/rotate/scale our entity as well as sync it over a network. Entity can have any number of additional components. To name a few — ModelComponent, PhysicsBodyComponent, PhysicsMotionComponent or custom components like FriendComponent and FoeComponent. The main idea of ECS is to have no game methods and data embedded in the entity.

Entity is a parent class for its eight subclasses

• Component is a data (or state) associated with our entity. That’s why this paradigm was called data-oriented. Not all entity’s necessary components are On by default. Every component conforms to its corresponding protocol. For example, AnchorEntity conforms to HasAnchoring protocol. The behaviour of any entity can be changed at runtime by Systems that mutate, add or remove components.

• System is a first-class element and a key piece of ECS. System is very useful for implementing behaviour that affects multiple entities in RealityKit’s scene. System transforms the state of component data. With system, RealityKit calls just single update method every frame per entire system, rather than call an update method every frame for every entity. The normal way to send data between Systems is to store the data in components.

Many game developers often call Entities as IDs, Components as data, and Systems as storage for code and logic. ECS pattern was implemented in many popular frameworks — Unity, Unreal, Lovetoys, Godex (Godot), Stingray, etc.

The following diagram shows ECS connections.

Entity Component System in RealityKit 2.0

Technically, in RealityKit, every entity has a list of its own components stored in Swift’s unordered Set. It’s easy to check this out.

let cameraEntity = PerspectiveCamera()
print(cameraEntity.components)
  //    ComponentSet(entity: ▿ '' : PerspectiveCamera
  //    ⟐ SynchronizationComponent
  //    ⟐ PerspectiveCameraComponent
  //    ⟐ Transform
  //    )

As we can see here, every entity has not only Transform and Synchronization components but also one ad hoc component.

Nonetheless, if a developer explicitly activates physics for Model Entity we’ll get two extra components in addition to the ModelComponent (turning physics On in Reality Composer is an implicit way):

let modelEntity = ModelEntity(mesh: .generateSphere(radius: 0.25))
modelEntity.components[PhysicsBodyComponent.self] = .init()
modelEntity.components[PhysicsMotionComponent.self] = .init()
print(modelEntity.components)
  //    ComponentSet(entity: ▿ '' : ModelEntity
  //    ⟐ Transform
  //    ⟐ PhysicsBodyComponent
  //    ⟐ PhysicsMotionComponent
  //    ⟐ SynchronizationComponent
  //    ⟐ ModelComponent
  //    )

Implementing Systems in RealityKit 2.0

We should start building ECS design programmatically unless you are tired. If it’s the case, pour yourself a large cup of latte and grab a chocolate cookie.

Ready? Steady! Go!

System is a protocol, so if any class conforms to it, we’ll have to add protocol stubs. When we need an initial setup for a System, we can place it in init(scene:). In RealityKit 2.0 update(context:) instance method is automatically called in every frame.

class SomeAbstractClass: RealityKit.System {
    required init(scene: RealityKit.Scene) { }
    func update(context: SceneUpdateContext) { }
}

It’s time to apply all the knowledge gained earlier. I suggest creating five 3D text elements (of white color) and then turn some of them (based on certain components) into orange balls. The first chunk of code looks like this:

import RealityKit
import UIKit
class ViewController: UIViewController {
    @IBOutlet var arView: ARView!
    var entity: ModelEntity?
    let anchor = AnchorEntity()
    override func viewDidLoad() {
        super.viewDidLoad()
        arView.backgroundColor = .black
        for index in -2...2 {
            let text = MeshResource.generateText("\(index + 3)", 
                          extrusionDepth: 0.01, 
                                    font: .systemFont(ofSize: 0.15), 
                          containerFrame: .zero, 
                               alignment: .center, 
                           lineBreakMode: .byCharWrapping)
 
            entity = ModelEntity(mesh: text, 
                            materials: [UnlitMaterial()])
            entity?.position.x = Float(index) * 0.75
            entity?.setParent(anchor)
            entity?.name = "Prim\(index + 3)"
            if entity?.name == "Prim2" {
                entity?.components[PhysicsBodyComponent.self] = 
                                                            .init()
            } else if entity?.name == "Prim4" {
                entity?.components[PhysicsBodyComponent.self] = 
                                                            .init()
            }
        }
        arView.scene.addAnchor(self.anchor)
    }
}

There are five text entities generated. Each entity has a ModelComponent. Second and fourth entities additionally have PhysicsBodyComponent.

5 text elements spaced 75 cm apart

Let’s change elements “2” and “4". This isn’t difficult to do. For that we have to create one more class and declare a static property of EntityQuery type in it. After that we must perform a query for each entity in a scene — whether entity has both components ModelComponent & PhysicsBodyComponent or not.

@available(iOS 15.0, *) 
class ChangeMeshAndColorSystem: System {
    static let query = EntityQuery(where: 
                                  .has(ModelComponent.self) && 
                                  .has(PhysicsBodyComponent.self))
    
    required init(scene: Scene) {
        print("RealityKit's first System is activated")
    }
    func update(context: SceneUpdateContext) {
        context.scene.performQuery(Self.query).forEach { entity in
            guard let entity = entity as? ModelEntity
            else { return }
            entity.model?.mesh = .generateSphere(radius: 0.1)
            var material = PhysicallyBasedMaterial()
            material.baseColor.tint = .systemOrange
            entity.model?.materials = [material]
        }
    }
}

The conclusive part of this small app seems quite logical — we have to register our new System. There’s no better place for this than AppDelegate.swift file.

import UIKit
@main class AppDelegate: UIResponder, UIApplicationDelegate {
    var window: UIWindow?
    func application(_ application: UIApplication, 
        didFinishLaunchingWithOptions launchOptions: 
            [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        ChangeMeshAndColorSystem.registerSystem()
        return true
    }
}

Time to compile! Those text objects that have both components will be replaced with orange spheres.

Only second and fourth text elements were substituted

// So we finally saw RealityKit's ECS in action.

That’s all for now.

If this post is useful for you, please press the Clap button and hold it.

¡Hasta la vista!