CoreImage 911 — Color Matrix 4x4
This is my second story about matrices. My previous story you can read here. This time I will tell you about CIColorMatrix filter, found in CIFilter class, that you should use to grade CIImages. This filter performs a matrix multiplication to transform the color vector, but as you guess it is able to do much more than trivial brightness increasing or decreasing.
Official documentation says: CIImage processing occurs in a CIContext object. Creating a CIContext is expensive, so you need create just one during your initial setup and reuse it throughout your app.
The following diagram shows you ins and outs of color matrix. It’s easy to imagine that if red data comes in, it can come out as green, blue or alpha data. So a Color Matrix 4x4 is an extremely powerful tool for many color ops.
↓ ↓ ↓ ↓
in in in in
R G B A
┌ ┐
| 1 0 0 0 | → out R
| 0 1 0 0 | → out G
| 0 0 1 0 | → out B
| 0 0 0 1 | → out A
└ ┘
0 1 2 3Identity Color Matrix
Let’s imagine we have an RGB image of woman with Alpha channel “tracing” her silhouette (RGBA pattern). Default values of such a matrix look like these — incoming red data comes out as red, green data comes out as green, blue — as blue, alpha — as alpha. So, default values do not affect the color of our image.
Decreasing Opacity
When you have a premultiplied image, where RGB is multiplied by Alpha, decreasing A value you decrease a whole opacity of RGB. Thus, any underlying layer becomes partially visible from under our translucent image.
Hidden image
If opacity of current RGBA layer is 0% (in normalized range values go from 0 to 1, where 1 is 100%), the underneath layer becomes fully visible.
Splitting channels
If you turn off green and blue components (set their values to 0), you’ll see a red register. That’s because grayscale channel data is internally multiplied by pure red color (and then by alpha, if image is premultiplied).
Color Inversion
Remember how a negative looks like on Kodak film? This effect is easily achieved by inverting the RGB values, in other words multiplying the R, G and B values by -100%.
I’m sure you’d like to know how to implement a color inversion, while operating with negative invisible RGB values. Consider CoreImage and UIKit’s regular normalised color values are in range 0 to 1.
Let’s extend CIImage class with colorMatrix method having a fifth parameter called “inputBiasVector". This parameter helps you offset invisible RGB values, that are below zero, to visible RGB spectrum.
import UIKit
import CoreImageextension CIImage { func colorMatrix(column0: CIVector,
column1: CIVector,
column2: CIVector,
column3: CIVector,
biasVec: CIVector) -> CIImage { return applyingFilter("CIColorMatrix", parameters: [
"inputRVector" : column0,
"inputGVector" : column1,
"inputBVector" : column2,
"inputAVector" : column3,
"inputBiasVector" : biasVec ])
}
}
Now apply colorMatrix method.
class ViewController: UIViewController { override func viewDidLoad() {
super.viewDidLoad() let image = UIImage(named: "imageFileName"), let imageView = UIImageView(frame: CGRect(origin: .zero,
size: CGSize(width: self.view.frame.width,
height: self.view.frame.height))) let ciImage = CIImage(image: image!) let corrected = ciImage!.colorMatrix(
column0: CIVector(x:-1, y: 0, z: 0, w: 0),
column1: CIVector(x: 0, y:-1, z: 0, w: 0),
column2: CIVector(x: 0, y: 0, z:-1, w: 0),
column3: CIVector(x: 0, y: 0, z: 0, w: 1),
biasVec: CIVector(x: 1, y: 1, z: 1, w: 0)) imageView.image = UIImage(ciImage: corrected)
self.view.addSubview(imageView)
}
}
Guess why parameters in convenience initializer of class CIVector are called X, Y, Z, W instead of R, G, B, A? That’s because X/Y/Z axes are always painted in red, green and blue in any 3D authoring app.
Red channel based values
If all three out values come from in red channel, a resulting image will not be a red register, it will be a grayscale.
Green channel based values
The same is true for the image based on the green channel values. Green color lives in the middle of the color spectrum (its wavelength is approximately 550 nm). Humans subjective perception of green is as best as we can imagine. Green pixels capture 59% of luminosity perceivable by humans. That’s why a grayscale image based on green channel looks so great.
Blue channel based values
Blue channel is the most noisy one, haven’t you heard? Most cameras and camcorders sensors are built to generate more green and red than blue.
Where sensitivity is low, we get a higher noise level.
Increasing brightness
Set multipliers greater than 1.0 and there will be a brightness boosting.
Decreasing brightness
Set multipliers lower than 1.0 and there will be a brightness decreasing. However, it should be noted that lowering the brightness of an image does not affect its transparency.
Color Grading
You can color grade and color correct images and videos in iOS and macOS. As you see, here’s a slight red tint based on data of green channel.
Let’s explore how to apply a color grading operation for video file. At first we should make an extension for CIFilter class.
import UIKit
import CoreImage
import AVKitextension CIFilter { func colorMatrix(column0: CIVector,
column1: CIVector,
column2: CIVector,
column3: CIVector) -> CIFilter { return CIFilter(name: "CIColorMatrix", parameters: [
"inputRVector" : column0,
"inputGVector" : column1,
"inputBVector" : column2,
"inputAVector" : column3 ])!
}
}
Then you’re ready to implement colorMatrix method to QuickTime movie.
class ViewController: AVPlayerViewController { override func viewDidLoad() {
super.viewDidLoad() let path = Bundle.main.path(forResource: "videoFile",
ofType: "mov") let url = URL(fileURLWithPath: path!)
let playerItem = AVPlayerItem(url: url)
let asset = AVAsset(url: url) let filter = CIFilter().colorMatrix(
column0: CIVector(x:1.0, y:0.5, z:0.0, w:0.0),
column1: CIVector(x:0.0, y:1.0, z:0.0, w:0.0),
column2: CIVector(x:0.0, y:0.0, z:1.0, w:0.0),
column3: CIVector(x:0.0, y:0.0, z:0.0, w:1.0)) let composition = AVVideoComposition(asset: asset,
applyingCIFiltersWithHandler: { request in let source = request.sourceImage.clampedToExtent()
filter.setValue(source, forKey: kCIInputImageKey) let output = filter.outputImage!.cropped(to:
request.sourceImage.extent) request.finish(with: output, context: nil)
}) playerItem.videoComposition = composition
self.showsPlaybackControls = false let player = AVPlayer(playerItem: playerItem)
self.player = player
self.player?.play()
}
}
Channel reordering
Sometimes you may need Avatar-like skin tone. No problem :). Just set channels values in reverse order.
Black silhouette
To generate a black silhouette of your image is possible thanks to setting RGB values to 0% and multiplying them by alpha with a value of 100%.
Semi-transparent shadows
Translucent shadow is a product of previous example but with alpha set to 20–80%. If you conposite such a shadow under our regular RGB image you’ll get a woman with shadow.
Desaturation
Have you ever wondered how Desaturate op works? For desaturation of an image, weighted values of color channels are required — R=30%, G=59% and B=11%. When developers look at these values just one question arises — why blue data is taken much less than green data?
The answer isn’t obvious: 30-59-11 rule better fits human perception of color.
White silhouette
In case you have to produce a white silhouette you need to supply data to the last column of the color matrix.
Masking with ID pass
Using a masking procedure based on ID pass is a common phenomenon in modern compositing (in The Foundry NUKE for instance). Rendered ID pass usually contains objects coloured with red, green, blue, cyan, magenta and yellow.
With the color matrix we are able to extract a blue circle and paint it with nice contemporary yellow.
However, applying ID masking for regular images is absolutely impractical.
All you have seen in this story is just a part of the capabilities that are available to iOS and macOS developers when working with a color matrix.
Unsurprisingly but color matrices can be found in such compositing apps as The Foundry Nuke, Apple Motion, and Blackmagic Fusion.
In Nuke we can find a node with a 3x3 color matrix.
In Motion resides a filter called Channel Mixer. It’s an analog of 4x4 color matrix, isn’t it?
In Fusion there’s a node allowing to manipulate 5x5 color matrix elements. The fifth channel in Fusion’s color matrix is ZDepth.
That’s all for now.
If this post is useful for you, please press the Clap button and hold it. On Medium you can clap up to 50 times per each post.
You can find more info on ARKit, RealityKit and SceneKit in my posts on StackOverflow.
¡Hasta la vista!
