Machine Learning Cheatsheet
In this post, I review what machine learning is about starting from the data to the decision making. I think people should know how to use machine learning as a new programming paradigm.
Note that linear algebra is the basis of this area. For Reviewing linear algebra, check the following post.
Intuitional Linear Algebra Cheatsheet
This post reviews basic linear algebra for introductory machine learning.
It is a new programming paradigm endowing computers with the ability to learn from data. In better words, empowering computers to detect patterns in data and make sensible decisions based on what they have learned. The following figure shows the difference between traditional programming and supervised machine learning.
The following three example applications of machine learning in computer vision give out far better results than traditional programming:
- Character recognition: for detecting characters (on bank bills)
- Face recognition in most the modern digital cameras
- Pedestrian detection in driver assistant systems
From the highest level, we can divide the ML pipeline into the following steps that the following figure shows:
In this first step, the blood of the system (data) will be received from the physical environment through sensors. After digitalization, when data comes inside the memory, it is only bunches of zeros and ones, just numbers. From a level higher, they may mean the temperature of a specific motor, its rotation speed, image, etc. I mean we give them meaning by considering them in specific ways. In this phase, the quality of these input devices is important because if intolerable noise is added, the data will not be representative and the patterns in the data will not be detectable/ or something mistakenly will be detected. Let’s recall the famous saying: “Garbage-In Garbage-Out”.
In various ways, we can consider our data, for example, consider a taken photo in the following ways:
- 2D matrix of integers
2. 1D vector of integers
These methods exist in general machine learning approaches. This is because of the small amount of data and less compute/memory capability, also it can be because of the lack of deep neural networks. A good example is extracting the location of the center of the pupil of the eyes for using in models for predicting the place that they are gazing. In the vision, feature extraction is accomplished with the help of techniques like denoising, thresholding, eroding, edge detection, etc. (most of them are implemented in python OpenCV)
In this phase, based on the features and the goal task, the appropriate model is chosen. If we want the model to predict the price of a house or the location (x, y coordinates) of the eye gaze as this task should give us a value in a range, so, we should do regression. But, when it comes to detecting animals in images, as will give them classes, this problem is a classification. Pay attention that in the aforementioned problems, we had the answers for our training data and this kinda machine learning is called supervised machine learning. This can get expensive because labeling animal images can impose a lot of time for people to sit down and assign labels to images. Consider that we have 100 million images!
The following figure shows the machine learning domain overlook.
Regression is fitting a curve or line on the data that can be used as a predictor function for the new data that we pass to it. Classification is determining a curve or line that separates different classes from each other. In clustering, we don’t have labels for our data, so we want to make groups based on some features of the data. For this purpose, algorithms like K-Means are used (but there are a lot of algorithms for these kinda tasks). For dimension reduction, for example, principal component analysis (PCA), and singular vector decomposition (SVD) try to detect the most important portion of data that can be used for learning from and making decisions. Keep in mind that there are a lot of methods in each of these subdomains of machine learning. Neural networks, specifically deep neural networks, are a sub-domain of machine learning that relies on the abundance of data to detect trustable patterns in data. However, we should keep in mind that neural networks demand high compute/memory requirements and a lot of data to be effective, otherwise, we will deal with overfitting!
Evaluation and Optimization
Evaluation methods are used to measure and give a sense of how well a machine learning model predicts. They determine how trustable machine learning models are. For regression, mean-squared error (MSE), root-mean-squared error (RMSE), and mean absolute error (MAE) can be used as evaluation metrics (they are also called loss functions).
For classification, accuracy can be used as a metric but it is useful for applications in that we don't care about the mispredictions and they won’t cost us people’s lives and/or a lot of money. Because of the aforementioned concern, we have other evaluation metrics like Precision, Recall, F1 score, and cross-entropy loss.
The data set is usually divided into three parts: training, validation, and test. The training is used for tuning the model’s parameters, validation is used to test the metrics while the training phase and decide to keep training or put an end to the process. The test portion is used for getting the final evaluation details on the performance of a model.
There is an optimization function that optimizes the model by changing the model’s parameters for less loss (error), such as RMSprop, and Gradient Descent.
Finally, there will be a software layer attached to the model output to map the result to labels, specifically in classification. As in classification, the output is an array that gives the probability of belonging to each class. In regression, the data is regulated to a specific range, it should be scaled back.
Nowadays, Machine learning is a necessity for people to learn how to use it. It is the new programming paradigm based on data. The main drivers of machine learning progress are advancements in algorithms, the plethora of data, and powerful hardware.