Resources for Starting Deep Learning
It is not too late to get into Machine Learning, but it can be quite daunting to decide how to start. There is no single best point to start your journey, and you will need to consider a few factors: the level of your existing knowledge and the practicality of your approach.
What is Deep Learning?
But first, let’s get the big picture right. There is a lot of confusion what constitutes as deep learning, machine learning and AI. Generally, the term AI is used when we talk about computer programs solving problems that they are not explicitly programmed to solve. A great example of something which is AI, but not machine learning is Deep Blue, the computer which defeated the chess champion Kasparov, which performed depth search to find an approximately optimal solution. In machine learning, probabilistic models are used to find these approximate solutions, and a specific area of machine learnig is deep learning where deep neural networks are used as the statistical model for learning. Deep neural networks are powerful because they can approximate any function, compared to the simplest form of linear regression which could only explain linear relationships.
Why would I use Deep Learning?
A common counterargument against Deep Learning is that it has no point, because humans can do the same thing. It might be true that humans can do the same thing sometimes, but there are a lot of cool examples where these machines learn play games like Go in a superhuman level, or detect diseases better than trained doctors. But besides these things, automation can make a lot of things in our society cheaper and more effective.
So now that you are convinced, let’s get started!
Having a minimal grounding in linear algebra and probability is certainly helpful, but taking these courses first is the easiest way to lose your motivation to learn. I advise to watch a video from Gilbert Strang’s Linear Algebra course as a supplementary to a machine learning course or project (see below) each day. The probabilistic component of machine learning will come a lot later, and I would recommend to consult Bishop’s Pattern Recognition book, which also touches machine learning topics. A lot of people advise to have prerequisite knowledge in optimisation, but to be honest there is no point in doing so, because the same two concepts (gradient descent and Lagrangian optimisation) come back which are already well explained.
From theoretical to practical,
- Hinton’s AI course. Takes a theoretical perspective, and needs all the prerequisites.
- Andrew Ng’s Machine Learning course. Very light on maths, but only covers the basics and it is a bit outdated.
- Andrew Ng’s Deep Learning specialisation. Still light on maths, but comprehensive.
- How to win a Data Science Competition. If you need a light course, which is very practical, but still gives you good intuition this is the way to go. Experts tell you about their experience who have already won competitions. From this course you also get to know decision-tree based algorithms, which are less pronounced in the AI courses above.
Projects to do
Doing a machine learning installation and getting the required hardware is not only a difficult, but can also be an expensive endeavour. Initially, I would advise to experiment with Kaggle competitions.
If you have a deep learning library on your computer, start doing a project. Download existing neural network example code from Github, and try modifying it so it does something you like. Can’t think of any applications? Try reimplementing an already existing papers from scratch or check out some search ideas on GitHub.
- GitHub repositories have very good collection of papers, which point out the most important papers on a given Machine Learning field. Try reading an older one, consult your books if you don’t understand something and try again until you understand. These are usually tagged with the word awesome, like this one.
- Bishop covers most of the things pre-2010 in Machine Learning and with a good mathematical level.
- Do some lighter machine learning reading sometimes to keep yourself motivated. KDNuggets and Machine Learning reddit offers most of these things. HackerNews is not strictly ML-related, but sometimes you can find relevant light readings there.
- If you need something more in-depth, you will have to read the actual publication. Sutton has a good Reinforcement Learning book, but I would avoid Goodfellow’s Deep Learning book.
If you are stuck, ASK FOR HELP, but (1) first try really hard to solve your problem, (2) then remember to learn the guidelines of these Q&A forums. You can go to MachineLearningLearn subreddit, Statistics StackOverflow (primarily for R), and Data Science StackOverflow.
Join or if you are really enthusiastic, organise a local meetup. There are many AI meetups advertised in the page Meetup.
Good luck on your journey!