Starting in Machine Learning
For starters in the world of AI. What is Machine Learning, what is Deep Learning, what is Data Science, what you need to know before start, sources of information, code and solutions.
Introduction
Hi, if you are new in the Machine Learning world or you are considering to start, this article is for you . I started over a year ago and it’s fascinating how many things you can do. In this publication i will present you some of the most basic concepts, possible application, what you need to know before start and what i think are the best sources for you to start learning. Let’s start with some basic concepts so you can know what we are talking about.
You will find the terms Artificial Intelligence, Deep Learning and Machine Learning are indistinctly used, but there is a difference. Machine Learning is a subset of AI and Deep Learning is a subset of Machine Learning, these two are the tools that made possible the current advances in AI.
What is Artificial Intelligence?
Artificial Intelligence is the branch of computer science that allows computers to perform tasks that normally require human intelligence. We can divide the concept of Artificial Intelligence into three types, Artificial General Intelligence, Artificial Narrow Intelligence and Artificial Super Intelligence.
- Artificial General Intelligence (AGI) refers to a machine that can do everything a human can. A machine that can learn by itself, think and behave as a human. This is the one most people imagine when we say the word AI, but we are still far from it.
- Artificial Narrow Intelligence (ANI) refers to AI designed to do specific tasks. It’s goal oriented, so this type will only work under certain defined parameters. It is also the one in which we have made progress. Some examples are voice recognition systems, image recognition systems, self driving cars, market predictions, online advertising, etc.
- Artificial Super Intelligence(ASI) is the AI that evolved from an AGI and became superior to humans. This is the kind on AI that slaves humans in movies or treat them as a plague, for example, the one from I-Robot or Ultron when evolves.
What is Machine Learning?
“Field of study that gives computers the ability to learn without being explicitly programmed.” — Arthur Samuel (1959)
And in a more scientific way
“A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” — Tom Mitchell (1997)
Imagine you want to predict if a person is jogging, walking, sitting or running based on acceleration data coming from your smartphone. Using conventional programming you would have to define many rules to determine if a person is doing one of those activities, and even so, you can run faster or slower, the person can be smaller or higher or move different when walking. There are many factors to consider so probably you will end up with a long list of rules and a complicated program, and that is because in conventional programming you take input data and define rules to get answers (if, then, else) but in Machine Learning you take data and define the desired answers for that input data to get rules.
So, to solve the previous situation, we can get a lot of acceleration data from your phone and label this data according to the corresponding activity (running, jogging, walking, etc), then the Machine Learning algorithm figures out the specific patterns in the data that determines what makes one activity different than the other. The difference is that the algorithm learn how to identify the activities through continuous exposure to the data. This is what we call Supervised Learning.
This new paradigm creates situations that were not feasible to do using conventional programming. An example is the classification of images, it would be very difficult to create a program, using conventional programming, capable of identifying if there is a cat or a dog in an image, but using Machine Learning this becomes a really easy task.
Types of Machine Learning Systems
Machine Learning systems can be classified according to the supervision they get during training.
Supervised Learning
In supervised learning you feed the algorithm with the data and the correct answer for that data. The previous situation was an example of a supervised learning problem where you tell your algorithm, hey, this is what walking and jogging acceleration data looks like.
Unsupervised Learning
In unsupervised learning we don’t have or we don’t know the labels, so the algorithm will try to learn by itself the patterns in the data. We can use this type of algorithms to separate classes in a group (Clustering) and to understand better our data. We can see a example of clustering in the next image. We started with unlabeled data and then used an unsupervised ML algorithm to cluster the data into the number of classes we suspect.
Another important use of unsupervised learning is anomaly detection. Sometimes we need to clean up our data before passing it to another algorithm, and as we saw earlier, ML algorithms are particularly good at detecting patterns in data, so they can also be used to detect when there is abnormal behavior. In the following image we can see four days of temperature data recorded by a router. In this case, our algorithm detected the abrupt drop in temperature and a high peak as abnormal.
Semi-supervised learning
These are algorithms that can handle partially labeled training data. Semi-supervised problems include some labeled data and mostly much more data with unknown labels, the goal is to classify some of the unlabeled data using the labeled information.
Reinforcement Learning
In this type, the learning system, now called the agent, interacts with the environment and receives rewards or penalties. The system learns by itself the best strategy to obtain the greatest amount of rewards possible. These are common learning algorithms used by robots to learn to move in an environment, stand and walk, so when you see that a robot is learning to walk, it is probably using these types of algorithms.
What is Deep Learning?
Deep Learning (DL) is a subset of Machine Learning that uses algorithms based artificial Neural Networks (NN). In Deep Learning we stack many layers of artificial neural networks to get our rules. Don’t worry if you don’t understand, later i will give you some sources from where you can learn what are artificial neural networks and the math behind.
The difference with ML algorithms is that we can build long and deep neural networks adding more and more data and the performance will increase. With traditional ML algorithms the performance will reach a saturation point, after that point it does not matter if the amount of data or the complexity of the algorithm increases.
Motivation for DL
To me one of the coolest ANN are Generative Adversarial Networks (GANs). These networks are the ones responsible for the Deep Fakes you have seen. If you check the site ThisPersonDoesNot Exist.com you will see images of people created by a GAN every time you refresh, although the images look very realistic, none of the persons shown there exists. But there is much more neural networks can do, check out this video to see how an algorithm can create art using GANs.
Another cool application today are Autonomous Vehicles. There is many research in this field and is definitely one of the most exciting technologies to be developed, this all thanks to the current advances in ML. Check out this video of Lex Fridman in MIT Autonomous Vehicle to see one in action while also doing activity recognition of the driver.
And if you want to know about profit one example is Netflix Recommendation Engine which saves the company about $1 billion per year, according to Netflix’s Chief Product Officer.
What is Data Science?
Data Science is the science of extracting knowledge from data. Data Science are techniques used to extract information from different sources, process this information and get meaningful conclusions.
For example, if we have the energy consumption of a company, the production estimate, the work hours and many other features a good Data Science situation would be extract insights of how to lower the energy consumption based in these information and how would this benefits the company. In contrast, a ML problem would be how much energy we will spend the next month, or how much the company will produce. Data Science has become very popular because now we can extract information of almost everything.
What to know before and how to start?
My recommendation is a code first approach. So if you have a basic understanding of calculus and algebra you can skip this part and use the links when you need to refresh some concepts. If you don’t have that background then you may want to take a look to at least one of the following suggestions.
Mathematics
You need to know basic algebra, what are vectors, matrices operations, gradients and some basic calculus concepts, like differentiating. Also, ML is all about probabilities so you need to the basics at least. Here are some links for you to check.
Andrew Ng Machine Learning Course from Coursera (first week). The first week is a good introduction to some ML concepts and the algebra you need. The entire course will be useful for you, it’s one of my recommendations to learn the mathematics behind the algorithms and how they work.
Mathematics for Machine Learning (Book). This is a free book that includes linear algebra, analytic geometry, matrix decomposition, vector calculus, optimization, probability and statistics. There is a lot of content there but just need the basics to start.
Linear Algebra from Khan Academy (Course): Free online course of algebra and Matrices operation, they also have one of calculus.
Seeing Theory (Interactive Site for learning Statistics):A visual introduction to probabilities and statistics.
There is a lot of content in this field and it will take you a long time to learn everything, so my advice is to start with just the basics, if you are in zero, and learn more when you need it.
Programming
The programming language is clearly Python. Python is the most used language in Machine Learning because of libraries like Numpy, Scikit-Learn, Pandas, Seaborn, Keras and many others that will make your life much more easier than any other language, also most of the documentation you will find is for Python. As i see it, you will be wasting your time trying to do ML in a different language.
Two good free courses to check out is Kaggle Python Course and Python for Everybody from freeCodeCamp.
Another really useful tool is Jupyter Notebooks because of the workflow it provides. Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations and visualizations. In ML you have to work with data and Notebooks are very good for this but not as good as others IDEs (like Visual Studio Code) for development, so my advice is that you use Jupyter to start and as you learn but consider doing the development in an IDE.
There are more advanced frameworks like TensorFlow for Deep Learning and deploy models, but you will see those as you go deeper into this world.
Where to learn Machine Learning:
Now, a list of resources for you to check out. Here are some places to start, to look for information and code examples. You don’t have to take the complete courses unless you want the certifications, but they contain important explanations and code examples you can check out if you need.
Learn by doing!! I started my own blog to share and create, and that is the best way to learn Machine Learning, by creating. So absorb the basic concepts, take one or two courses or read a book, check out some projects and start your own one project.
🎓Courses
Machine Learning by Standford (free). The course is teached by Andrew Ng which is an amazing teacher and one of the more important researcher of the field.
Kaggle free courses: Kaggle has a course for beginners in ML and an intermediate course, courses for visualizations, data cleaning and Deep Learning. The courses are free but you can even gain certifications of completions.
freeCodeCamp(free): Also a free course about ML where you can earn certifications. The is very practical so you will be seeing a screen with code in Python you can follow. The course is good and is very TensorFlow focused.
Full Stack Deep Learning(free): Many courses and tutorials you will see only covers the Machine Learning part, such as building the model and train it, but there is much more than that. This free course teaches you the rest of the process, how to approach a ML problem since the beginning, deploy your models and the most important tools you have to learn to complete the cycle. This is a more advanced course, but it is definitely one to check.
📚Books
Hands–On Machine Learning with Scikit–Learn and TensorFlow: This is one of the best practical books (emphasis in practical) for Machine Learning. The book guides you step by step through an and to end ML project, teaches how to get data, clean your data, visualize it, model the data, contains exercises and the most important algorithms you have to learn. Includes code examples and a GitHub repository.
Deep Learning by Ian Goodfellow: You can either buy the book or access it in its free HTML version. I think the next comment resumes how goo is the book. But notice that this book is only about Deep Learning.
“Written by three experts in the field, Deep Learning is the only comprehensive book on the subject.” — Elon Musk.
🌐Sites and Blogs
Medium: There are many people in Medium that share their work and thoughts about ML, you will find interesting explanations, code examples, more digestible versions of a scientific papers, tutorials. It is a quick way to obtain information on a topic or to get out of a doubt, and you don’t have to pay a subscription cause much of the best content is free.
Machine Learning Glossary: This is a dictionary with probably all the terms you will see in ML made by Google.
Machine Learning Mastery: This is a blog from Jason Brownlee dedicated to ML. There is a lot of content in the blog, many interesting publications, code examples, good practices and well explained concepts. It has helped me in many cases when i was looking for a quick example or a hands on about a specific topic.
Kaggle: Kaggle is a great place to learn ML, in there you can find courses, datasets and many notebooks with problem solutions, visualizations and interesting approaches to different problems. In Kaggle you can participate in competitions, that will help you test your abilities and your knowledge, also to share and exchange thoughts with others in the field. Datasets from Kaggle are normally cleaned, easy to download and work with, this makes your start less traumatic. I recommend you to start here and take the Titanic Competition which is the getting started in Kaggle, you can check my blog post about the competition.
Distill: A different way of present research. The web presents some algorithms and principles in an interactive, for example, you can check how Momentum works by setting different starting points and parameters.
💡State of Art
Arxiv Sanity Preserver: Allows you to keep track of recent papers and state of art in ML, sort papers by similarity to any paper, see recent popular papers, to add papers to a personal library, and to get personalized recommendations of (new or old) Arxiv papers.
Papers with Code: The name pretty much resumes what is. Papers With Code is a collection of open source papers about Machine Learning with the code and evaluation tables. You can check the latest progress in the field and see the code.
Made with ML: Is another web site to search the latest trends in ML, 100% free. In here you won’t just see scientific papers but also blog posts, tutorials, and work made by hobbyists. The platform can include the documentation about the project, videos and a link to the GitHub page or a Notebook with code for you to see. It’s a place to share practical projects.
References
Géron A. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. O’Reilly Media; 2019 Sep 5.
https://scikit-learn.org/stable/getting_started.html
https://codebots.com/artificial-intelligence/the-3-types-of-ai-is-the-third-even-possible
