Faster AI: Lesson 0 — TL;DR version of Fast.ai Part 1
Let me start off by saying why I choose to write TL;DR version of Fast.ai Part 1 Course. Out of so many courses on deep learning on the internet, I honestly believe this course should be at the top. There are couple of reasons for that.
One major reason is its free and its not just free but it also provide all the materials required for you to follow along the course, including lesson notes, exercise files and a dynamic forum to ask anything if you ever got stuck.
Another reason is the way the materials are taught. Most of the courses, including the famous Coursera course on Deep Learning, it is taught using what i like to call, a more traditional approach, that is, it starts off by explaining elementary things in much details and gradually moves up to more larger pieces of the puzzle. I am not saying this approach is bad but certainly for me it was not an effective one.
In this course however, things are little more interesting, you start off by implementing an actual image classifier using deep neural networks. This direct exposure gives you overall picture of how things are moving in grand scale. Then slowly the instructor Jeremy Howard, he goes in detail of each pieces of the neural networks and explain things along the way. And at the end of the course you will learn the same amount of knowledge from any other highly acclaimed courses on deep learning but with more confidence on practicality of things with better implementation experience.
This series of TL;DR posts by no means are replacement to this amazing course, but instead what i like to achieve here is to reduce some shortcomings of this course, that is its length of each videos. Each video is around 2 hours long, which i think for new learner is bit overwhelming even before taking the course and many of you might have just drifted away because of that. What I wish is, you follow along these TL;DR versions and actually take the entire course after that. There is no hard and fast rule to actually learn deep learning and the length of those videos might put you off but they actually worth it and you will learn a lot, I promise.
In this Lesson 0 of Deep Learning for coders part 1, Jeremy starts off by explaining why deep learning is important and Its not just a fad but a major force of change and how it is changing the shape of the society. He gives his gentle introduction, his past experience with Kaggle as chief scientist there, his medical startup which he built around using deep learning and some ideas about what this course is going to be.
Neural networks in general are the models which are built with the idea of layers stacking up and with input provided at one end, for example: images, texts or any data, these inputs are passed through the layers and each layer learn something different about the same input. Using this concept, Jeremy tackles a famous image classifying approach using MNIST dataset and shows us how visually each layer of neural network learns the passed image of hand written digit from this dataset.
Generally dealing with Images, Convolutional Neural Network is used mostly because of its better accuracy results. What exactly a convolutional neural network is, that is covered in following lessons but for this one, he explains it with a more intuitive approach.
You have this image from dataset
And suppose you initialize a matrix with these values:
Which visually looks like a strip of black, white and grey. Now if you correlate these two entities, and correlate in simple terms means to multiply each element of that top matrix with that image ‘7’ matrix, this is what you get as a result:
This same approach is used in convolutional models to detect edges of the images, but instead of handcrafting the values of the matrix to multiply with the image, these values are generated randomly and there are many numbers of these matrices and each detect something different from the image, some detect edges, some corners and if you use models with many layers deep down, they can also detect part of objects as well.
To explain this visually, he tries out some other values for this top matrix basically to detect edges in different angles: horizontal, vertical, diagonal and shows us how each of those output looks like with different values of the matrix.
These matrices which is multiplied with images are called Kernel or Filters and in this article they are visually explained: http://setosa.io/ev/image-kernels.
You can read the Lesson notes as well for more details:
http://wiki.fast.ai/index.php/Lesson_0
While explaining the process behind this correlation, Jeremy goes through different inner steps of a convolutional network. And he explained how convolution is also very much similar to correlation with minor difference of rotating that filter by 180 degrees. He then goes through the concept of max pooling [Time: 29:45] and how they are used by the neural network to recognize the images.
In deep learning there are many number of filters and many layers in the model and to train a model simply means to optimize these values of filters with each iterations and make them able to recognize images with better accuracy. He then explained this process in a more simpler way with an example of a straight line, and uses famous Linear Regression approach but in deep learning way. [Time: 37:00]
He uses simple equation of the line: Y= A*X+B and calculates the values of Y with A,B and X given. Now imagine we have list of X values and Y values only and now the job of the model is to find the close to real values of these A and B. It does so by the process of Optimization. In each iteration it calculates a loss, which is difference in values of Y predicted by the model with random values of A and B with the real values of Y with accurate A and B. So with the help of this loss function and through the process of optimization, it recalculates the values of A and B in each iteration and by the end of certain number of iterations, the values are close to real A and B and so the loss is quite low.
As this lesson 0 is more of an overview session and only conceptual part is covered here but following lessons will be more technical and each lesson will deal with more complex ideas in deep learning.
Regarding this particular lesson 0, I highly recommend you to watch it as it is also very short one (around 45 minutes) compared to other lessons. It will give you basic idea what will be covered in whole course and how he is going to cover various concepts. Besides Lesson 0, there is also Lesson 1 Overview video which is generally about what the course is going to be, who is this course for and what are the differences between this course and others. This Overview video is also quite short (around 30 minutes), please go through that as well.
In this lesson he went through all these high level concepts and terminologies, the reason for that is to give brief glimpse of what this course is about. He went through image recognition using hand written digits dataset because later on in the course, we will be learning to tackle similar problem but with different dataset of cats and dogs and use that to classify between them, using different type of convolutional neural network.
The reason behind going through the concept of straight line and how a neural network learns is that, on higher intuitive level, if we can understand this simple phenomenon we can understand the workings of larger neural networks as well.
I hope you get the basic idea what I will be writing about in this blog series and I hope you will follow along with me. See you on next post.
