A pre-processing filter for SSIM

In his paper Another Look at SSIM Image Quality Metric, Yuriy Reznik talks about a pre-processing filter that can improve SSIM score achieved by a video encoder.

[…] the design of a pre-processing filter that removes some of the low-frequency features and thus allows the encoder to encode the remaining high-frequency content with fewer errors. With proper tuning, such a filter may allow existing encoders to achieve improved SSIM scores.

Well that sounds interesting, let’s give it a try and see if we can improve some SSIM score. For this experiment, let’s use kodim04.png from the Kodak Lossless True Color Image Suite, the same image used by Reznik in his paper.

Similar to VMAF hacking, let’s use the good old unsharp mask to remove low frequencies. I say old because it actually comes from the era of darkroom photography. To apply an unsharp mask on our image, let’s use gimp on it’s default settings (radius=3, amount=0.5 threshold=0.0). Gimp turns out to be a good choice for this experiment since it implements the unsharp mask using a Gaussian filter which is also what SSIM uses internally.

We end up with the following images. One could argue that after the unsharp mask, the image actually looks better. It might be that the human visual systems favours higher frequencies over lower ones.pre-processing filter

Next, we encode both images. Similar to Reznik’s paper we will use H.264 with QP=42. This will produce noticeable compression artefacts

Success, Gimp’s unsharp mask filter using default values allowed us to improve SSIM from 0.9275 to 0.9390, not bad. Looking at the textured areas before and after the unsharp mask it looks like we over compensated for the loss of high frequencies in the image after compression.