What is Online Learning?

[ML0to100] — S1E9

In online learning, you train the system incrementally by feeding it data instances sequentially, either individually or in small groups called mini-batches. Each learning step is fast and cheap, so the system can learn about new data on the fly, as it arrives

In online learning, a model is trained and launched into production, and then it keeps learning as new data comes in

Online learning algorithms can also be used to train systems on huge datasets that cannot fit in one machine’s main memory (this is called out-of-core learning). The algorithm loads part of the data, runs a training step on that data, and repeats the process until it has run on all of the data

Using online learning to handle huge datasets

Out-of-core learning is usually done offline (i.e., not on the live system), so online learning can be a confusing name. Think of it as incremental learning

One important parameter of online learning systems is how fast they should adapt to changing data: this is called the learning rate.

High learning rate — Your system will rapidly adapt to new data, but it will also tend to quickly forget the old data (you don’t want a spam filter to flag only the latest kinds of spam it was shown).

Low learning rate — The system will learn more slowly, but it will also be less sensitive to noise in the new data or to sequences of nonrepresentative data points (outliers).

A big challenge with online learning is that if bad data is fed to the system, the system’s performance will gradually decline.

For example-

• Bad data could come from a malfunctioning sensor on a robot
• Someone spamming a search engine to try to rank high in search results.

To reduce this risk, you need to monitor your system closely and promptly switch learning off (and possibly revert to a previously working state) if you detect a drop in performance.

You may also want to monitor the input data and react to abnormal data (e.g., using an anomaly detection algorithm).

Read Next- What is Instance-Based and Model-Based Learning? [S1E10]

Summary Cheat Sheets, Notes, Flash Cards, Google Colab Notebooks, codes, etc will all be provided in further lessons as required.

Read through the whole ‘S1’ [ML0to100] series to learn about-

• What Machine Learning is, what problems it tries to solve, and the main categories and fundamental concepts of its systems.
• The steps in a typical Machine Learning project
• Learning by fitting a model to data
• Optimizing a cost function
• Handling, cleaning and preparing data
• Selecting and engineering features
• Selecting a model and tuning hyperparameters using cross-validation
• The challenges of Machine Learning, in particular, underfitting and overfitting (the bias/variance trade-off)
• The most common learning algorithms: Linear and Polynomial Regression, Logistic Regression, k-Nearest Neighbors, Support Vector Machines, Decision Trees, Random Forests, and Ensemble methods
• Reducing the dimensionality of the training data to fight the “curse of dimensionality”
• Other unsupervised learning techniques, including clustering, density estimation, and anomaly detection Part II, Neural Networks and Deep Learning, covers the following topics:
• What neural nets are and what they’re good for building and training neural nets using TensorFlow and Keras
• The most important neural net architectures: feedforward neural nets for tabular data, convolutional nets for computer vision, recurrent nets and long short-term memory (LSTM) nets for sequence processing, encoder/decoders, and Transformers for natural language processing, autoencoders and generative adversarial networks (GANs) for generative learning
• Techniques for training deep neural nets
• How to build an agent (e.g., a bot in a game) that can learn good strategies through trial and error, using Reinforcement Learning
• Loading and preprocessing large amounts of data efficiently
• Training and deploying TensorFlow models at scale

Disclaimer — This series is based on the notes that I created for myself based on various books I’ve read, so some of the text could be an exact quote from some book out there, I’d have mentioned the book but even I don’t know which book a paragraph belongs to as it’s a compilation. It’s best for the reader as they get the best out of the all promising books available in the market for ML compiled in one place.