Where next? After SVMs, CNNs and Word Embeddings

The plethora of knowledge involved in Machine Learning is the most fabulous thing about the subject. The theoretical and coding balance requires a steady and disciplined approach. In this five series tutorial, we saw CNNs, where we saw various approaches to different scenarios, and then worked on word embeddings, which was our gateway to Natural Language Processing, and finally ended with Support Vector Machines(SVMs) which were as powerful as Artificial Neural Networks, during the time of their inception.

Is this all there is to Machine Learning?

If we have this questions running, then we are absolutely right to think the same! The above mentioned topics are just a glimpse into the supervised learning models, that are available for various domains. CNNs capture deep learning, embeddings capture NLP & SVMs capture supervised ML algorithm, but broadly, all these fit the supervised learning approach. Thus, we should also realise that initially, when we talk about Machine Learning, we are told that there are three classes of ML.

  1. Supervised Learning
  2. Unsupervised Learning
  3. Reinforcement Learning

Unsupervised Learning:

It’s supervised learning revisited, but this time with no ground truth data. The dataset has no information about the correctness of data, and thus, the algorithms need to find patterns in the data. There are various types of unsupervised learning algorithms, ranging from clustering to auto-encoders in deep learning.

k-means clustering

The goal of clustering is to create groups of data points such that points in different clusters are dissimilar while points within a cluster are similar, k being the number of classes. In k-means the way these groups are defined is by creating a centroid for each group.

Hierarchical Clustering:

The challenge is here is unique as we intend to build a hierarchy of clusters, that is, get in a sense of granularity in various classes. The algorithm goes like this…

  1. Start with the initial k clusters
  2. Merge the two cluster that are closest to each other to obtain k-1 clusters
  3. Recompute the distances between the clusters
  4. repeat till a single cluster of k data points is obtained.

Dimensionality Reduction:

Principal component analysis (PCA)

You’re familiar with the coordinate plane with origin O(0,0) and basis vectors would be i(1,0) and j(0,1). The basis vectors tell us the span of the coordinate plane it covers. Hence we can have multiple vectors covering the same space.

This means we can change the basis of a space. Now imagine much higher-dimensional space. Like, 50K dimensions. You can select a basis for that space, and then select only the 200 most significant vectors of that basis. These basis vectors are called principal components, and the subset you select constitute a new space that is smaller in dimensionality than the original space but maintains as much of the complexity of the data as possible.

Another way of thinking about this is that PCA remaps the space in which our data exists to make it more compressible. The transformed dimension is smaller than the original dimension.

Singular Vector Decomposition:

from numpy import array
from scipy.linalg import svd
# define a matrix
A = array([[1, 2], [3, 4], [5, 6]])
U, s, VT = svd(A)

NumPy provides the function pinv() for calculating the pseudoinverse of a rectangular matrix.

from numpy import array
from numpy.linalg import pinv
A = array([[0.1, 0.2],[0.3, 0.4],[0.5, 0.6],[0.7, 0.8]])
# calculate pseudoinverse
B = pinv(A)

Application in Dimensionality Reduction

Data with a large number of features, such as more features than observations (rows) may be reduced to a smaller subset of features that are most relevant to the prediction problem. The result is a matrix with a lower rank that is said to approximate the original matrix. To do this we can perform an SVD operation on the original data and select the top k largest singular values in Sigma. These columns can be selected from Sigma and the rows selected from V^T.

What next?

There are energy based models under unsupervised learning methods, which are fascinating, for the reason that, these probabilistic models are being paralleled to energy distributions of particles in physics… Well certainly do explore these areas! Restricted Boltzmann Machines, Deep Belief Nets and Auto-Encoders.

Reinforcement Learning:

For readers interested in going deep into RL and the math behind it, I suggest you to go through David Silver’s Lectures and Richard Sutton’s book on RL. Some amazing math backs this beautiful subdomain of Machine Learning.


Markov Decision Process (MDPs): Inspired from finite state machines, these are models with finite states. The process is modelled probabilistically such that at each state, the future state is predicted based on the probabilities. Makes sense right? During the training of the agent, the rewards are associated with each transition that takes place. The definition if MDP is that the current state is independent of the previous states. Hence, it is memoryless.

Q-Learning: We have a function Q that takes a state and action as input and return the reward of that action at the current state. Before we explore the environment, Q gives the same (arbitrary) fixed value. But then, as we explore the environment more, Q gives us a better and better approximation of the value of an action a at a state s. We update our function Q as we go.

Discount factor helps in deciding important states and rewards, also an hyperparameter.

Coding Time:

Let’s have a look at Andrej Karpathy’s code of 130 lines, that implements Reinforcement Learning from scratch using numpy only, built on OpenAI gym’s ATARI 2600 Pong. I suggest you read his blog here to gain insights into RL.

Source: http://karpathy.github.io/2016/05/31/rl/

Well, OpenAI was founded to fasten the progress of RL and make it at par with supervised learning. We shall talk about using OpenAI gyms in another blog. Their documentation is understandable easily too. I suggest you go through Andrej Karpathy’s blog, the best article to get you started with RL. This was tutorial 5 in the 5 part series.

For any queries, suggestions or feedback, contact me at lalith@dataturks.com.




Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

DataTurks: Data Annotations Made Super Easy

Written by

Data Annotation Platform. Image Bounding, Document Annotation, NLP and Text Annotations. #HumanInTheLoop #AI, #TrainingData for #MachineLearning.


Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store