1. On Machine Learning
Let’s talk about machine learning.
I would have liked to start this first part in a bit more of a creative way, but it is terribly hard to come up with a new and fun way to start an article about machine learning these days. So let’s leave the creative writing for later, and just focus on what it is all about: machines that can learn.
It is worth noting here that I should not be considered an expert in the field of artificial intelligence. Honestly, I just started really studying the topic a few months ago. I have never programmed a learning algorithm myself and my background lies in logic, mathematics and philosophy instead of computer science.
What I do have to offer consists of three things:
1. Some sort of outside perspective on the research field,
2. Support from a supervisor much more knowledgeable than myself and from whoever is interested enough to discuss with me,
3. All the resources that I have used so that you can check the work for yourself and see how much sense I am making of it.
If you are more familiar with machine learning already, the second part of this piece might not be worth your time to read attentively. This is more focused on those that are wondering what the fuss is all about and have the burning question what these learning machines are that I keep bringing up. So without further ado: this is a brief introduction to machine learning.
We encounter endless machines every day of our lives. The cars we drive to work, the computers we use at work, the coffee machine at the university or office where we procrastinate. Even the clock on which we watch the time tick away can be considered a machine. This is not a new concept of course, and we have learned to rely on machines for our way of life. Trusting machines like this, non-learning machines if you wish, has become quite a natural process. What is helpful for this trust is that we can see exactly what happens and why — or at least we can find an expert that can tell us what is happening when a computer or car malfunctions. In this way, all these applications are quite simple. That does not mean they are easy, very few people know all the ins and outs of a smartphone for example. However, we know what happens and why.
An example of this is — say — a toilet. We press the flush button and water floats into the bowl, after which the water reservoir is filled up again. An engineer knows what needs to happen and how to make that work, there is no rocket science at play. And even rocket science is simple in the same way, even if it is very difficult as well. The engineers still know what needs to happen and when, combustion engines are — however complicated — not ambiguous. We can consider the following question to understand this sort of simplicity: “Can we put into words exactly what needs to happen when?” Flushing a toilet ~ yes. Launching a rocket ~ well, in a lot of words we can. However, consider recognising whether a passerby is male or female ~ can we even do it in words?
At some point we have to face that some of our own skills cannot simply be learned by reading a manual. We cannot just put into words how we recognise the gender of a person on the street, even we do not always get it right. It is just something that we learn. So that is what we would have to automatise then: not just recognising, but learning to recognise. This is an example of where we enter the realm of machine learning.
Immediately when considering the act of learning, there is an important distinction to be made. We are not entirely sure how machines learn, and we are not entirely sure how people learn, but we are pretty sure they do it differently. The clearest difference is that humans are just much more efficient, a random example is a model built for the detection of pears in Japan. The model was trained on a data set of 1337 images to reach an accuracy of 98%, which is considered pretty impressive.
Now if it would take a toddler 1337 pears to start to be able to distinguish between them and other fruits, parents would start to get pretty concerned. This efficiency is also one of the main current research topics of machine learning, exemplary for this is the research agenda of Yann LeCun. LeCun is head of the AI research team of Meta (previously known as Facebook) and is a big voice in machine learning’s future. In articles like this one he shares his views and ideas about how machines learn and how he hopes they could learn in the future.
Machines do not learn exactly like we do. So, how do they learn?
In the training phase, a lot of data is fed to the model. A picture is shown, and the machine guesses its class. This is specific training data though, and this is why there is a correctly labeled class with each of these images. This could be an image of an orange, which is labeled “not pear”, as the model is just learning to distinguish whether or not something is a pear. At the start, our model will basically just give random results. At this point it really is as bad at pears as it is at penguins. This is where the learning comes in: Now the model can start to adjust itself to get better guesses for all the cases that it got wrong. If a specific part of the model is responsible for classifying many apples as pears, this will slowly change over the training period. Changing, tweaking, adjusting. Again, and again, and again. That is what we call learning in the way machines do it.
After enough iterations, the machine will have quite a good grasp of the specific subject matter that the training data is aimed at. What exactly happened to achieve this goal within the model depends on the framework and method used. However, something that more complicated — and therefore in many cases more accurate — models all share is the trait of non-linearity. To understand this better, consider the problem of determining cats from dogs. There is not one specific characteristic by which we can accurately distinguish the two. Cats and pears, there we would have a pretty good bet as the presence of paws should be a pretty good giveaway. Paws — cat, no paws — pear. This is a solution that we can consider to be linear, the two classes are separated by this line of the presence of paws. Now for cats and dogs, this gets a lot harder.
So the complicated models — that are amazingly proficient at telling even cats and dogs apart — start to take more than just this one feature into account. This is not always as easy or understandable, but it can be considered roughly similar to looking at features like paws, hair or fur, shape of eyes and more, and considering combinations of these too. Maybe this shape of eyes and that type of hair correspond to a specific species of cat: There is no clear line to distinguish the two anymore. We have to have a nonlinear solution, because the problem is just too complicated to solve accurately otherwise. If you are now thinking: “How hard can it be to tell cats from dogs?”, imagine an alien that has never seen either of the two and think about how you would explain to them what differentiates a cat from a dog. It is a pretty hard task.
Approaches that use non-linear techniques are accurate and powerful, even for very difficult tasks. This can be seen far beyond the extent of classifying cats or dogs. Take AlphaGo for example, an AI that plays the complicated Japanese game Go. This algorithm has only been beaten once by a human player and even this was one game out of five in a showdown. But what is important to realise, is that we still do not actually understand AlphaGo. The same holds for many of the YouTube-recommendation, finish-your-sentence, or even facial-recognition software that is used all around us. I have not explained the inner workings of all the interesting machine learning and specifically deep learning models (which are models working with hidden layers of complexity), but even after studying those we still do not understand exactly what is going on. Machines are just not that simple anymore, and that is precisely the next topic I will be talking about.
Hopefully you are saturated by this short and rather dry introduction on machine learning. This is nowhere near an exhaustive introduction to the topic, and I am not hoping to advocate it as such. It is merely meant to give you a first look at what I will be talking about when talking about the field. There are many more resources online, from interesting videos to full courses. But speaking from my perspective, what will follow is the more intriguing part of machine learning: How do we understand our models and what does that really mean?
