Machine learning frontiers: what you must know before getting started
Shortly, an computer program is said to learn when its performance improves with experience. What this experience looks like?
In terms of artificial intelligence algorithms, this experience consists on processing data. The study field interested in making computer programs to learn is called machine learning.
> This article is part of the series Machine learning frontiers
Indeed, in the last two decades, AI has been strongly impacted by the advances in machine learning. Although the core algorithms used in ML that epoch were not new, researchers significantly improved the performance of computers when executing complex tasks in fields like natural language processing and computer vision.
> The most common machine learning task is pattern recognition (for example, finding a face in an image). However, there are other task types such as transcription and data generation. We would talk about these types of tasks in a forthcoming article.
Training computers to perform actions
The central paradigm in machine learning is using data to feed a training algorithm. The output of this training is a model which eventually is able to perform the task of interest. Therefore, instead of someone statically defining/coding a program to perform some action, a training algorithm teaches (fits) the model to execute that action.
The role of Neural Networks
One of the most well-succeed machine learning models (perhaps the most one) is artificial neural networks, or shortly, ANN’s. In despite of its success, until 2004 ANN’s were used only on toy apps, concepts or small self-contained scenarios. But why?
The computational cost of training traditional ANN’s with high-dimensional data (for example 640x480 images) is prohibitive, no feasible at all even using modern computers. This only changes after the advances in deep learning.
> To comprehend and use vanilla neural networks is still necessary. We shall discuss the details of classic ANN model in a forthcoming article
Deep learning takes the scene
The last decade eyewitnesses the raise of deep learning models. Deep learning is an umbrella of algorithms & techniques which makes use of spatial and temporal locality to reduce the required number of model parameters. This approach is usually called parameter sharing.
Actually, deep learning makes neural networks able/practical to perform complex high-dimensional-data tasks like processing VGA, HD or 4K images in real time. Indeed, without deep learning, we all would be locked up in the 90’s world.
> we shall cover deep learning models like convolutional neural networks, recurrent neural networks and generative adversarial networks in a forthcoming article
Before we finish this talk, there is something really important to mention.
Optimization and regularization
There are two central concerns in machine learning namely optimization and regularization.
When you use data to train a model, you want finding the optimal model which better fits the data at hands. It is optimization. The pitfall here is, if you go too far on this fitting, you probably will end up with a useless model that works only for THAT data, an overfitted model.
> models like this memorizes the training data instead of learn the regularities in the data.
Overfitting is a modelling error. Finding ways to avoid overfitting is called regularization.
It is highly recommended to read the next article covering the fundamentals of overfitting and regularization.
