Rendering graphics content using the MetalKit framework

Photo by Steve Johnson on Unsplash

This article describes the basics of rendering graphics content with Metal. In the sample code, it creates a view that uses Metal to draw the contents of the view.

MetalKit View

First, you need to import MetalKit, which is a framework for building Metal apps faster and easier.

import MetalKit

TheMetalKit provides a class called MTKView, which is a subclass of UIView in iOS and tvOS (NSView in macOS). The MTKView class simplifies the effort required to create a Metal drawing application by providing a default implementation of a Metal-aware view. In the sample code, MTKView object is initialized from IB.

An MTKView needs a reference to a Metal device object in order to create resources internally, so your first step is to set the view’s device property to an existing MTLDevice. You can get a reference to the system default MTLDevice object with MTLCreateSystemDefaultDevice().

metalKitView.device = MTLCreateSystemDefaultDevice()

MetalKit view’s drawing

MTKView uses the delegate pattern to inform your app when it should draw. To receive delegate callbacks, set the view’s delegate property to an object that conforms to the MTKViewDelegate protocol, which provides methods for responding to a MetalKit view’s drawing and resizing events.

metalKitView.delegate = renderer

The functions provided by MTKViewDelegate are as follows:

• mtkView(_:drawableSizeWillChange:): The view calls this method whenever the size of the contents changes. Use this method to recompute any view or projection matrices, or to regenerate any buffers to be compatible with the view’s new size.
• draw(in:): This method is called on the delegate when it is asked to render into the view. By implementing drawing processing in this function, drawing to a view using Metal can be realized.

When is `draw(in:)` called?

The MTKView class supports three drawing modes and you need to change the properties before drawing.

• The default automatically invokes a redraw based on an internal timer. In this case, both isPaused and enableSetNeedsDisplay are automatically set to false.
• Draw by calling setNeedsDisplay(): In this case, both isPaused and enableSetNeedsDisplay must be set to true.
• Call draw() explicitly: This method manually asks the view to draw new contents. In this case, isPaused must be set to true and enableSetNeedsDisplay must be set to false.

Executes commands on a GPU

Metal performs various processes by sending commands to the GPU. Before explaining the actual drawing process, you need to understand the GPU and commands.

The relationship between the Metal app and the GPU can be compared with the relationship between the client-server pattern. The image below is described in the Apple documentation.

Client-server usage pattern when using Metal, Apple Developer Document by Apple Inc.

Your Metal app is the client and the GPU is the server.

You make requests by sending commands to the GPU. After processing the commands, the GPU can notify your app when it’s ready for more work.

A command performs the drawing, parallel computation, or resource management work your app requires. To send commands to the GPU, You need to understand the following protocols.

Command protocols

• MTLCommandQueue
• MTLCommandBuffer
• MTLCommandEncoder

Command Queue

The queue for managing the execution order of command buffers. Create from MTLDevice as follows:

commandQueue = device.makeCommandQueue()

Command queues are thread-safe and allow multiple outstanding command buffers to be encoded simultaneously. Typically, you create one or more command queues when your app is initialized and then keep those queues around throughout the lifetime of your app. In sample code:

private let queue: MTLCommandQueue
init?(device: MTLDevice) {
    guard let queue = device.makeCommandQueue() else { return nil }
    self.queue = queue
}

Command Buffer

A container that stores commands for the GPU to execute. Create command buffer objects by calling a MTLCommandQueue object’s makeCommandBuffer() method.

let commandBuffer = queue.makeCommandBuffer()

Below image shows the relationship between commands and their command buffer:

A command buffer’s relationship to the commands it contains, Apple Developer Document by Apple Inc.

To add commands to the buffer, create an encoder object, described below. When the command buffer calls the commit() method, the command buffer is added to the source command queue.

commandBuffer.commit()

The command queue schedules the execution of buffers in the order in which they are enqueued and the GPU executes the commands. Guaranteed to execute in the order in which the command buffers are queued.

The command buffer object is generated every time because it cannot be reused.

CommandEncoder

Create a command and add (encode) it to the command buffer. Command encoder objects are lightweight objects that you re-create every time you need to send commands to the GPU.

There are many different kinds of command encoders, each providing a different set of commands that can be encoded into the buffer.

• MTLRenderCommandEncoder: Graphics rendering
• MTLComputeCommandEncoder: Computation
• MTLBlitCommandEncoder: Memory management
• MTLParallelRenderCommandEncoder: Multiple graphics rendering tasks encoded in parallel.

Relationship between queue, buffer, and encoder

In summary, commands are sent to the GPU according to the following flow:

1. Command queue creates command buffer
2. Command encoder add commands to the command buffer
3. Command buffer is enqueued in the command queue
4. Command queue sends command buffer to GPU, then the GPU executes the command

Your app’s command queue, Apple Developer Document by Apple Inc.

Draw background-color to MetalKit view’s

In sample code, uses Metal to display the fill background color. To erase the contents of the view to a solid background color, you set its clearColor property.

metalKitView.clearColor = MTLClearColorMake(0.0, 0.5, 1.0, 1.0)

Next, render background-color in draw(in:). When graphic content is rendered, the GPU saves the result in a texture. texture ( render targets ) is the content of the drawable object. it’s blocks of memory that contain image data and are accessible to the GPU.

func draw(in view: MTKView) {
    // render graphic contents
}

The graphic content is rendered by following the steps described below.

1. Create a Render Pass
2. End a Render Pass
3. Present a Drawable to the Screen
4. Commit the Command Buffer

Create a Render Pass

First, to draw, you create a render pass, which is a sequence of rendering commands that draw into a set of textures. To create a render pass, you need an instance of MTLRenderPassDescriptor and aMTLRenderCommandEncoder object. You can create the render pass by encoding it into the command buffer.

let commandBuffer = queue.makeCommandBuffer()
let commandEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)

By default, at the start of the render pass, the render target is set to the current drawable texture of the view, and at the end of the render pass, all changes are stored back to the texture.

• render pass descriptor: It describes the set of render targets, and how they should be processed at the start and end of the render pass. In this sample, rather than configuring your own render pass descriptor, ask the MetalKit view to create one for you ( currentRenderPassDescriptor ). currentRenderPassDescriptor is render pass descriptor generated from the current drawable’s texture, the view’s buffers, and the view’s clear values. The color attachment at index 0 of the render pass descriptor points to the texture assigned to the current drawable.

func draw(in view: MTKView) {
    let renderPassDescriptor = view.currentRenderPassDescriptor
    print(renderPassDescriptor)
// output
Color Attachment 0
  ...
  clearColor = (0 0.5 1 1) // this property matches the view’s clearColor property,
  ...

End a Render Pass

In this sample, you don’t encode any drawing commands, so the only thing the render pass does is erase the texture. Therefore, the render pass is completed immediately.

let commandEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
// complete render pass (saved texture)
commandEncoder.endEncoding()

Present a Drawable to the Screen

Drawing to a texture doesn’t automatically display the new contents onscreen. In Metal, textures that can be displayed onscreen are managed by drawable objects, and to display the content, you present the drawable.

// Metal will display the given drawable on the screen.
commandBuffer.present(view.currentDrawable!)

You can get the object that owns the texture ( target of the render pass) by reading the currentDrawable property.

Commit the Command Buffer

Finally, commit the command buffer by calling commit(). After you call the commit() method, the MTLDevice schedules and executes the commands in the command buffer. You can only commit a command buffer once.

commandBuffer.commit()

Source code in draw(in:)

References

• Using Metal to Draw a View’s Contents
• Setting Up a Command Structure
• shoheiyokoyama/GraphicsRenderingSampleWithMetal