Image Segmentation Techniques using Digital Image Processing, Machine Learning and Deep Learning Methods. (Part 2)

Hi all and welcome back to part two of the three part series. In this part we will learn how image segmentation can be done by using machine learning and digital image processing. In the previous part we had seen that using only DIP technique for image segmentation, the problem of poor generalisation across different data sets of the same problem statement occurred.That is when the whole solution is customised to a particular one target use case and data set. So now to overcome this limitation we will take the help of machine learning. So lets jump in to the most famous technology right now i.e. Machine Leaning.

Figure 1

Table of Contents:

1. What is machine learning?
2. A brief introduction to different machine learning algorithms.
3. How can we use machine learning for image segmentation?
4. Advantages and disadvantages of using ML image segmentation methods.

What is machine learning?

Referring to one of the most famous book Machine Learning by Tom Mitchell, Machine learning (ML) is the study of computer algorithms that improve automatically through experience. It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as the “training data”, in order to make predictions or decisions without being explicitly programmed to do so. ML algorithms are used in a wide variety of applications, such as email filtering and computer vision, where it is difficult or infeasible to develop conventional algorithms to perform the needed prediction related tasks.

Figure 2: Different types of machine learning algorithms

Types of ML algorithms:

1. Supervised ML Algorithms: Supervised learning algorithms build a mathematical model of a set of data that contains both the inputs and the desired outputs. The data is known as training data, and consists of a set of training examples. The mathematical model can be as simple as a linear equation y=mx + c where y is the output value/dependent feature or labels, x is the input values/independent features and m and c is the weight and bias respectively. supervised machine learning is further divided into two types of problem statement i.e. regression and classification. Regression deals with problem statements in which a numerical continuous value has to be predicted whereas classification deals with those in which a class or categorical value has to be predicted.
2. Unsupervised ML Algorithms: Unsupervised learning algorithms take a set of data that contains only inputs, and find structure in the data, like grouping or clustering of data points. The algorithms, therefore, learn from the data that has not been labelled, classified or categorised like in supervised algorithms. Unsupervised learning algorithms identify commonalities in the data and react based on the presence or absence of such commonalities in each new piece of data. Unsupervised algorithms also used for dimensionality reduction i.e. when we want to reduce or group together the features or independent variables of the dataset so as to speed up the ml training process and also to remove multicollinearty.
3. Reinforcement Learning: Reinforcement learning is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. That is simple language, RL is a learning in which an agent learns how and what actions should be taken to go in what states that will give him the maximum reward. For example the problem of finding the shortest path in a given grid overcoming the given obstructions. Reinforcement learning algorithms are used in autonomous vehicles or in learning to play a game against a human opponent. To gain in-depth knowledge of RL you can read my one of the previous written blogs at this link.

A brief introduction to different machine learning algorithms

Figure 3. Some of the machine learning algorithms.

There are approximately hundreds of ml algorithms designed till today using different different mathematical and statistical models such as Bayesian based, tree based, gradient boosting algorithms, clustering algorithms, etc. It is very difficult to write about each algorithms in one blog as there is a whole bunch of mathematics and statistics behind these algorithms therefore I urge you to read about them or let me know if you want a separate blog on it.

Below is the list of commonly used machine learning algorithms:

1. Linear Regression: In statistics, linear regression is a linear approach to modeling the relationship between a scalar response (or dependent variable) and one or more explanatory variables (or independent variables).
2. Logistic Regression: In statistics, the logistic model is used to model the probability of a certain class or event existing such as pass/fail, win/lose, alive/dead or healthy/sick.
3. Support Vector Machines: An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on the side of the gap on which they fall.
4. Naive Bayes: These are a family of simple “probabilistic classifiers” based on applying Bayes’ theorem with strong (naïve) independence assumptions between the features.
5. K-Means clustering: k-means clustering is a method that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster.
6. Decision Tree: A decision tree is a flowchart-like structure in which each internal node represents a “test” on an attribute/feature (e.g. whether a coin flip comes up heads or tails), each branch represents the outcome of the test, and each leaf node represents a class label.
7. Random Forest: Random forests or random decision forests are an ensemble learning method for classification, regression and other tasks that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes or mean prediction of the individual trees.
8. Dimensionality Reduction Algorithms: is the process of reducing the number of random variables under consideration by obtaining a set of principal variables. Approaches can be divided into feature selection and feature extraction. Popular algorithms are PCA, LDA, and t-sne.
9. Gradient Boosting algorithms: Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. It builds the model in a stage-wise fashion like other boosting methods do, and it generalizes them by allowing optimization of an arbitrary differentiable loss function. Popular algorithms are GBM, XGBoost, LightGBM, CatBoost, Adaboost.

Figure 4. Commonly used machine learning algorithms

How can we use machine learning for image segmentation?

Image segmentation using ML mainly include the following steps:

1. Select a problem statement and labelled training data set.
2. For each of the training images, extract features by applying sobel filters, canny edge detection, gabor filters, cieLAB colour features, etc. Basically we can extract any and all image features which help us to properly define the image features.
3. Stack all the features in a 3-dimensional array and flatten it to obtain a 2-dimensional array. Similarly, flatten all the labelled/masked images and append it to the last column of the image 2-D array.
4. Now, we have successfully prepossessed our data set of images and we can simply apply machine learning algorithms for training.
5. To evaluate the performance of the ML model applied, we can use metrics like jaccard score/IOU value or simply an accuracy measure.

Figure 5. A screenshot iustrating the steps 2 and 3 or a view of the processed image data set. Here, columns names are the different filter names applied and rows are the values of these filters.

Figure 6. Architectural diagram to show the steps involved for image segmentation using machine learning.

To further enhance the accuracy or jaccard score, we can experiment with different ML algorithms and we can also apply SLIC or k-means clustering on the input image and then perform the feature extraction and modelling steps. This will first cluster the similar regions together which may increase the accuracy at the final step. To eliminate non or less contributing features, we can use the dimensionality reduction techniques such as PCA, LDA or can use feature importance curve generated by random forest algorithm and eliminate some of the features and hence can make the whole pipeling computationally less expensive.

Advantages and disadvantages of using ML image segmentation methods

Advantages: This method of image segmentation using machine learning will be fruitful in generalising the predictions well to another similar kind of data set on which they are not trained on, which is not in the case of DIP methods talked about in part 1 of this series. This method involves forward and backpropogation of weights called parameter learning which helps in identifying various hidden patterns and thus produce promising results. ML method are used when we have less number of non-complex images such as segmentation of parasitic cells with non-parasitic ones.

Disadvantages: Extraction of more features, stacking them and then flattening them again to be fed into the ML algorithm can be hugely computationally expensive. Although we can use dimensionality techniques to solve this problem but then also in parameter learning stage of training, a huge number of parameters updates will happen which again cause drainage of memory. Thus a more efficient technique is required which give prominent results both in memory efficiency and prediction accuracy such as convolutional neural networks or CNNs which will be discussed in the next and last part of this series. Second disadvantage of this method is that, one need to have a very good domain knowledge of the problem statement to select the features to be extracted for a better segmentation results.

Conclusion

In this blog post we have discussed what is machine learning and its types and a brief introduction to different machine learning algorithm. Further we have discussed show can we implement image segmentation using ML methods and what are the advantages and disadvantages of this method. In my next post I will be discussing the image segmentation techniques using Deep learning which produces much more prominent results in accuracy, memory efficiency in parameter learning and often generalises well than the ML methods.

Acknowledgement

I would like to thanks my ML course instructor Prof. Neelam Sinha and Prof G. Sirinivasa Raghavan , IIIT Bangalore for teaching me ML course and imparting valuable knowledge.

Code:

All the codes of the implemented algorithms shown in this blog is present on this link.