Canvas-based blur detection with JavaScript

By Timo

Inspector Bokeh is an experimental JavaScript library to measure blur in images, created during the DaWandathon 2014. This post illustrates the chosen algorithm, based on a paper by Marziliano et al., and expands on some image processing basics.

Motivation

Shop owners on DaWanda’s marketplace upload around 40,000 product images of varying quality every day, ranging from snapshots taken with camera phones to professionally edited, studio-quality pictures.

banker’s lamp by Palazzo Int

Apart from requiring certain file formats and dimensions, we currently have no means of assuring the quality of these images. Some of our listed products are manually curated in certain sections of the website, but it wouldn’t be possible for us to classify every uploaded image this way.

The goal of this project was to experiment with automated quality checks in the browser. This could provide feedback to users and potentially prompt them to upload images of higher quality.

‘Why is this image bad?’

Initial research was conducted to answer this question by clustering images based on different types of degradations (I basically sat around for a day and bothered our backend with countless queries, looking for the worst images I could find).

Some quality problems are fairly easy to detect and solve (e.g. over- or underexposure, low contrast), while others would require complex analysis (e.g. cut off edges of objects, hard-to-read text on images). By far the most frequent cause was blur, so I focused on measuring this particular effect.

Measuring blur

There are several distinct types of blur, but for the sake of simplification I was looking for a general algorithm which only processes one given image. The orginal, unblurred version is assumed to not be available (no-reference blur detection). Researching the state of science on this topic, I compiled this chronologically sorted overview of papers:

Given the limited amount of time and working knowledge of statistical methods, I chose to implement the algorithm presented in the first paper from 2002 Marziliano et al.. It is relatively simple and based on measuring the width of vertical edges. Given a high enough contrast, it can be observed that edges get smoothened out and appear wider than in the original, if blur occurs.

Detecting vertical edges

As a preparation, the image is converted to greyscale. We only care about the overall contrast in light intensity, not about edges of a particular color. The main work is then done by a convolution filter. This algorithm examines the surrounding pixels of each individual pixel and calculates a new value with the weighted sum of its neighbors. Different kernels/matrices produce different results such as sharpening, blurring, and edge detection. The following 3×3 kernel is also called sobel operator and detects only vertical edges:

For each pixel C, we look at the surrounding pixels px1 to px8:

px1 px2 px3 px4 C px5 px6 px7 px8

The intensity values for all these pixels are multiplied with their respective weights and added up. This results in a new value for the center pixel C. To demonstrate this, assume the following actual values of a center pixel (87) and its eight neighbours in a greyscale image:

13 120 111 46 87 104 9 114 5

Now we plug the numbers into the sobel operator:

(px1 * -1) + (px2 * 0) + (px3 * 1) + (px4 * -2) + (C * 0) + (px5 * 2) + (px6 * -1) + (px7 * 0) + (px8 * 1)

After removing the zeros and filling in the actual pixel values, we get:

-13 + 111 - 92 + 208 - 9 + 5 = 210

The sobel operator turned the original center pixel with value 87 into 210.

The end result of edge detection, when it is done in both horizontal and vertical direction, combined with smoothing, is an image like the one below:

Measuring the width of vertical edges

The edges we detected are evident to a human seeing the greyscale image, but we still need to make them ‘machine visible’ — i.e. find out exactly where edges begin and end. Marziliano et al. used local minima and maxima to determine this, but didn’t fully reveal their process to get them, at least to my best knowledge and understanding. So I chose simpler criteria:

1. an edge begins when the intensity value is anything greater than zero (a black pixel, no edge found in the original image).
2. Given that we’re processing the pixels in a row from left to right, the edge continues as long as the intensity values keep increasing.
3. The edge ends as soon as the subsequent value decreases.

To not count edges caused by noise in the original image, we discard intensities below a certain threshold. Usually the clearly visible edges in an image have enough contrast around them to end up as 100% white pixels after edge detection.

Core algorithm in JavaScript

// For each row of pixels for (y = 0; y < height; y += 1) { // Reset edge marker edgeStart = -1; // For each pixel in the current row of pixels for (x = 0; x < width; x += 1) { // Intensity value of current center pixel value = pixels[y][x]; // Edge is still open if (edgeStart >= 0 && x > edgeStart) { // Intensity value of previous pixel oldValue = pixels[y][x - 1]; // Intensity is not increasing anymore, the edge ended if (value < oldValue) { // Only count edges above a certain intensity if (oldValue >= edgeIntensThresh) { // Calculate edge width edgeWidth = x - edgeStart - 1; // Count total number of detected edges numEdges += 1; // Add to sum of all edge widths sumEdgeWidths += edgeWidth; } // Reset edge marker edgeStart = -1; } } // Beginning of a new edge detected if (value == 0) { // Remember where the edge started edgeStart = x; } } }

The final blur score is the average width of a vertical edge, in relation to the width of the image:

avgEdgeWidth = sumEdgeWidths / numEdges; blurScore = avgEdgeWidth / width * 100;

A high blur score suggest a high amount of blur in the given image.

Inspector Bokeh at work

Running measure_blur.js as a Node.js module on an image and its blurred versions gives the following output:

1. Original image (378 × 528 pixels)
2. Gaussian blur with radius of 2 pixels applied
3. Gaussian blur with radius of 8 pixels applied

Conclusion

The algorithm still needs a lot of fine tuning and is far from anything production-ready. The major problem is that noise/grain is wrongly detected as edges. To ease this, the current implementation applies a low-radius gaussian blur on input images of lower resolution. Despite these limitations, Inspector Bokeh can be used to automatically detect extreme cases of blurred images when a threshold for the resulting blur score is carefully chosen.

Originally published at https://www.codeoclock.com/2015/06/05/blur-detection-javascript/ on June 5, 2015.