A Brief Introduction to Computer Vision with MATLAB
Using the power of software to analyze the visual content of the world is as big of a revolution in the world as the light bulb of Edison was in the late 1870s’. With a camera, a computing machine (laptop, computer, etc.), and some knowledge of algorithms you can build some really cool products.
What is Computer Vision?
Computer vision is the process of using machines to understand and analyze visual content. While the algorithms have been around in various forms since the 1960s, recent advances in Machine Learning, as well as leaps forward in data storage, computing capabilities, and cheap high-quality input devices, have driven major improvements in how well our software can explore this kind of content.
In this post, I will be sharing how my first encounter with computer vision was, and also I will walk you through a basic program in MATLAB with which you can identify colors.
In the summers 2018, The Electronics Club, IIT Indore organized a 20-day workshop on "Beginner to Advanced course in Computer Vision Technology".Worried about the quality of the course, and my lack of skills in the domain thereof, I convinced myself to give it a try, and to my surprise, the things we're taught so proficiently that the concepts felt easy and natural, even though I had never read or heard anything about them before.
The basic element of an image: Pixel
The core concept used throughout is that the Image is made up of many pixels. For example, the screen you are using to read this story might be using 1920x1080 pixels to display if it's a 1080p screen. Each pixel on a screen is made of 3 subpixels, Red, Blue, and Green.
Some display technologies have up to 5 sub-pixels, the more they have, the greater the number of total colors can be displayed. So if you want a red image to be displayed all the red subpixels get to active state and other subpixels have a brightness of 0% each.
Pixel Based Filtering
Mathematically you can represent any color as a 3x1 matrix with columns being the value of red, green, and blue respectively. The value is generally from 0 to 255, so red can be represented by (255,0,0). Similarly green is (0,255,0) and blue is(0,0,255). You can make a guess at what black and white colors would be represented: Black=(0,0,0) White(255,255,255). Now using this basic concept we can filter out regions of a particular color out of a normal RGB image.
Let's start off by a simple image.
Suppose I want a region in the image with a blue color. The blue region is easy to identify in this image but there are many complex images where you need to analyze the image by color filtration and this technique would then be quite useful. I will be using MATLAB to filter out the required region.
The image is stored always as a matrix in MATLAB under the variable in this case. Now, if we apply a red filter to the image we get a filtered image as shown below.
(im(:,:,1) means filter to be applied is a red filter and the two colons depict that the filter needs to be applied to all rows and columns of the image matrix similarly im(:,:,2) and im(:,:,3) correspond to blue and green respectively)
If we apply a blue filter, we get an image in which all the blue regions are relatively lighter in shade compared to other regions. One thing to remember is by applying the filter we convert the image to a grayscale image.
Now we would like to have value for the blue region to be high (i.e. to highlight only bluer pixels in an image). We will convert image into black and white (B&W) and then for all bluer pixels we will have the value 1 and for all other regions, we will have 0. One important tip is that we will subtract this filtered grayscale image with normal grayscale image and something magical happens
Now if you want to highlight a region of your choice in the original image, you can use an inbuilt function of MATLAB ‘region props’. What region props allows you to do is plot centroid, have box bounding the area of interest, find the ratio of the area of the region of interest to the bounding box area, and many more.
So what we have just done is simple, we first turned all the pixels corresponding to the blue color white, drew a box around the white region, and then superimposed it on the original image by using the hold on function.
This is relatively a very easy example in image processing or computer vision technology in general where the region of interest has a uniform gradient throughout and shape is also quite well defined. The problems you'll face using real-life images will be more complex where you might need to properly set threshold, apply the median filter, and use many other functions to properly identify colors. However, using the same basic concept covered here you can start off and then extend it's functionality to more complex images.
We have been talking about processing an image. Now, what if we have a video?
We simply process every single frame in the video, plot a bounding box around the region of interest in every frame, and show them back at a certain frame rate depending on the computing power that is available. This marking of regions in-turn is known as Object Tracking and this again has wide applications
The video below shows what exactly I am trying to say.
What I did here is applied a filter such that all the pixels corresponding to pink color are filtered and a box is drawn around the pink region and in that way you can track the color. This is just an extension of the color detection of an image and when I first ran the code for this project it was a magical experience as it was like drawing in air, I continued playing with this for hours!
This is just a brief introduction to Computer Vision Technology. There is much mind-boggling application of this technology of the future, for example, Tesla cars using computer vision technology, ball tracking in many sports use recognition technology, image-recognition by google use the technology coupled with machine learning. The scope is very high currently in this field and I expect all readers to give a try towards this technology of the future.
