Core Image filters with Metal

We can create custom Core Image filters which is written in Metal Shading Language (MSL) in iOS 11 or greater.

Advances in Core Image: Filters, Metal, Vision, and More — WWDC 2017 — Videos — Apple Developer

It’s supported on A8 or newer devices.

It can achieve better performances because it’s pre-compiled at build-time. Existing way (GLSL or CIKernel Language) are compiled at run-time.

How to implement?

Followings are the new APIs added to CIKernel:

public convenience init(functionName name: String, fromMetalLibraryData data: Data) throws
public convenience init(functionName name: String, fromMetalLibraryData data: Data, outputPixelFormat format: CIFormat) throws

Here is a simple MSL implementation for CIColorKernel:

#include <metal_stdlib>
using namespace metal;
#include <CoreImage/CoreImage.h> // includes CIKernelMetalLib.h

extern "C" { namespace coreimage {

    float4 myColor(sample_t s) {

        return s.grba;
    }

}}

Let’s save it as kernel.metal or something (Any name is OK, but the extention should be .metal`).

A CIColorKernel which uses the shader function can be initialized using the new API:

let url = Bundle.main.url(forResource: "default", withExtension: "metallib")!
let data = try! Data(contentsOf: url)
let kernel = try! CIColorKernel(functionName: "myColor", fromMetalLibraryData: data)

So, a CIFilter subclass which has the custom kernel can be implemented like this:

import CoreImage

class MetalFilter: CIFilter {

    private let kernel: CIColorKernel

    var inputImage: CIImage?

    override init() {
        let url = Bundle.main.url(forResource: "default", withExtension: "metallib")!
        let data = try! Data(contentsOf: url)
        kernel = try! CIColorKernel(functionName: "myColor", fromMetalLibraryData: data)
        super.init()
    }

    required init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    func outputImage() -> CIImage? {
        guard let inputImage = inputImage else {return nil}
        return kernel.apply(extent: inputImage.extent, arguments: [inputImage])
    }
}

The custom filter can be used like this:

let filter = MetalFilter()
filter.inputImage = inputImage
let outputImage = filter.outputImage()

That’s it!

Build Settings

OK, now we can build the project. Let’s run it on your device!

+[CIKernel kernelWithFunctionName:fromMetalLibraryData:outputPixelFormat:error:] Function 'myColor' does not exist.

Hmm, but it says myColor does not exist. But it exists…

I found the solution here:

• kernelWithFunctionName:fromMetalLibraryData:error: — CIKernel | Apple Developer Documentation

We need to specify some options in the build settings:

• Specify -fcikernel flag in the Other Metal Compiler Flags option
• Add a user-defined setting with a key called MTLLINKER_FLAGS with a value of -cikernel

It works!

(Left: Original, Right: Filter Applied)