A Gentle Introduction To Convolution Filters

Applying a convolution filter is a common way to adjust an image and can produce a number of effects, including sharpening, blurring, and edge detection. In this article, we’ll discuss the basic process of applying a convolution filter, using a sharpening filter and a box blur filter as an example.

It’s worth noting that convolution is a general term that applies to many different signal processing algorithms. This article focuses on convolution in the spatial domain, as opposed to the frequency domain.

The most basic implementation is as follows:

Create a convolution kernel, representing a grid. (You’ll likely see this referred to as a matrix. A matrix is a one-dimensional representation of a two-dimensional object; in other words, an array representing a grid.)

Here are some examples of convolution kernels:

Using this “grid”, perform the following operation on each pixel in your image:

• Grab a sample of the surrounding image pixel values corresponding to the size of your convolution matrix. For a 3*3 matrix, this simply means grabbing all adjacent pixels ( up, down, and diagonal).
• Multiply the value of each pixel by the corresponding value in your matrix.
• Sum these values. If your kernel values summed to one, this is the value of your new pixel!
• If your kernel values didn’t sum to one, you have an extra step: divide this new value by the sum of the kernel values.

http://machinelearninguru.com/computer_vision/basics/convolution/image_convolution_1.html

In the example above, we are finding a new pixel value for 99.

Wait, what?

This process seems arbitrary at first, so let’s look at an example: blurring an image. Here’s the kernel for blurring:

[ 1/9, 1/9, 1/9,
 1/9, 1/9, 1/9,
 1/9, 1/9, 1/9 ]

When you run the filter with this matrix, each pixel will be the average of the surrounding pixels. There are nine pixel values; we’re dividing them all by nine, then adding them together. That’s mathematically the same as adding the 9 values together, then dividing by the number of values (9).

If I make each pixel the average of the values surrounding it, how will my picture look? Pretty similar to the pixels around it! If I have a pixel that is black, and the pixels around it are white, it will come out light grey. If it comes out grey, it’ll stand out less against the white background. And so, the overall effect of averaging the values is to make the individual features of an image less noticeable. In other words, it will make the image “blurry”.

From Wikipedia: https://en.wikipedia.org/wiki/Kernel_(image_processing)

Lets take a closer look at the sharpening matrix (also called a kernel). Applying a filter using that matrix results in the center pixel value being added 5 times, and the adjacent (top, bottom, and side) pixel values being subtracted. How else might we express this?

5*center - top - bottom - left - right // in other words:
5*center -(sum of four adjacent) 
// the sum of four numbers is four times the average number
5*center - 4*(average adjacent)  
center + 4*center - 4*(average adjacent)
center + 4*(center- average adjacent) 

In effect, we are finding the difference between the center pixel and the average of the pixels around it, multiplying that difference by 4, then adding that to our existing center pixel value.

So what’s the impact of that operation on our image? We are finding out how the pixel is different from the pixels around it, and then amplifying those differences, resulting in a more detailed image.

That’s just two examples of kernels, but there are a number of useful ones. Wikipedia has a list of the basics.

Now that we you understand the principles of image convolution, take a gander at this ‘HTML5 rocks’ tutorial, which shows you how to implement it.

The following code is an excerpt from that tutorial. I’ve added more comments.

Filters.convolute = function(pixels, weights, opaque){
  var side = Math.round(Math.sqrt(weights.length));
  var halfSide = Math.floor(side/2);
  var src = pixels.data;
  var sw = pixels.width;
  var sh = pixels.height;
 /*set the output width and height to match the image width and
  height*/
  var w = sw;
  var h = sh;
  var output = Filters.createImageData(w, h);
  var dst = output.data;
 
 // go through the destination image pixels
  var alphaFac = opaque ? 1 : 0;
  for (var y=0; y<h; y++) {
    for (var x=0; x<w; x++) {
      // current x coordinate = sx
      // current y coordinate = sy
      var sy = y;
      var sx = x;
      // position of current pixel in imageData Array
      var dstOff = (y*w+x)*4;
      //initialize variables for current pixel color
      var r=0, g=0, b=0, a=0;
      /* iterate through the surrounding pixels that fall under the
       the convolution matrix, setting their x and y coordinates*/
      for (var cy=0; cy<side; cy++) {
        for (var cx=0; cx<side; cx++) {
          var scy = sy + cy - halfSide;
          var scx = sx + cx - halfSide;
          /*check to make sure that the current surrounding pixel
           exists. For example if you are at the right edge of the
          image, then there is no pixel to the right/*
          if (scy >= 0 && scy < sh && scx >= 0 && scx < sw) {
            /* find where the current surrounding pixel is in the
              image pixels array*/
            var srcOff = (scy*sw+scx)*4;
           /*find the weight, that is, the corresponding value in
           the kernal, multiply the current surrounding pixel by it
           and add it to the running total.*/
            var wt = weights[cy*side+cx];
            r += src[srcOff] * wt;
            g += src[srcOff+1] * wt;
            b += src[srcOff+2] * wt;
            a += src[srcOff+3] * wt;
          }
        }
      }
      // assign the center pixel the value of the weighted sum
      dst[dstOff] = r;
      dst[dstOff+1] = g;
      dst[dstOff+2] = b;
      dst[dstOff+3] = a + alphaFac*(255-a);
    }
  }
  return output;
};

Image Filters with Canvas - HTML5 Rocks