Deep Learning for Managers
Are you a manager struggling to make sense of all this machine learning, artificial intelligence and deep learning nonsense everyone is talking about? Perhaps you just want to know how these technologies may disrupt or influence your industry? Or, and ultimately, how can you advance your career and/or business using these technologies? Well, hopefully this blog series can help explain some of the concepts behind deep learning.
What is Deep Learning
Deep Learning is a branch of machine learning that uses a concept of connected networks that get fed information and essentially spit out the answer. We often use the human brain as an analog to deep learning and connected networks since that was the original inspiration. In fact, it is common to refer to these network models as a brain. Below is an image showing a comparison of a computer network beside a biological neural network.
Surprisingly, it is as simple as the diagram above and even that basic network shown can be quite powerful. Of course, more advanced networks can have millions of connections. For the human brain processing these millions of connections happens almost instantly and in parallel, but for a computer this is a different problem. Not to mention that training such networks requires a huge amount of computer processing power. So much so that this has been a significant detriment to deep learning, that is, until relatively recently. Within the last 7 years the advances in processing this type of data, which just so happens to match the same used in video games and CGI, has exploded. This paired with other advances have allowed us to build deep learning networks that can infer (believe) an answer almost as fast as a human. That record is expected to be broken within the next 5 years, if not sooner. This will certainly affect every industry and business, big or small, on this planet when that happens.
Training Deep Learning
Deep Learning brains are not just like program code where once they are constructed they can perform some function or task. Instead, deep learning networks need to be trained to a specific task. And this is where our analogy to the human brain works quite well. Any task that we may accomplish with some form of regular success takes practice. Be it throwing a basketball into a hoop or even just eating. Eating likely took you hundreds or perhaps thousands of attempts to get right. Where each time you ate you became slightly better and better. We often refer to this type of learning as trial and error and this principal is fundamental to deep learning especially. Since in a deep learning system we train the connections in the brain by feeding it examples of data and then often confirming how correct the answer is using a determination of error. From there we take just a small part of this error and adjust the brains weights accordingly. The reason we take just a small portion is more a function of how the error is pushed back through the system.
Backpropagation of Errors
The term we use to describe the process of pushing errors and adjusting connection strengths in a deep network is called backpropagation. All this fancy word means is essentially what we described with one specific clarification. In the case of deep learning we use the a rule of calculus called the chain rule to determine how much (mathematically) each weight in the network needs to be adjusted. In days of old, 10+ years ago, we used to do this by step by hand. As you may imagine this wasn’t trivial and it was a major limitation of training networks. Fortunately, through the advent of automatic differentiation, huge networks can now be constructed seamlessly and then trained. While training can still take hours to days or more and often requires massive amounts of data but this is quickly changing.
A New Age of Brain Power
The reason deep learning systems currently require so much data has to do with how we train them and fundamentally the network design itself. As it stands right now each connection can take thousands to millions of iterations to train in order to get acceptable levels of accuracy. Which is a big downside when you consider that while we ourselves train our own brain using trial and error, we definitely don’t need thousands or millions of iterations to learn tasks and especially not related tasks. Of course, research is working on reducing that need for training and this where we are again challenging the human analog. With new research from various institutions looking to reduce training sample size to less and in some cases only a few samples. If this is starting to sound scary you will be happy to know many others think the same way. In fact, some of these new brains have deemed to be too dangerous for public release.
Enter AI on the Cloud
If all this sounds incredibly complicated, yes it certainly can be and we are still in early days of this technology. As it turns out, Google, Amazon, Microsoft and others have realized this and now have turned to producing a whole set of various pre-trained networks for everything from image recognition, voice and text understanding, video indexing and automation through various other machine learning algorithms. Literally every day these online brains are becoming more sophisticated and it is expected that the bulk of AI/Machine Learning applications will use some form of pre-trained brain provided through a cloud service from these providers. In fact, if you are looking to get into AI/Machine Learning now as a compliment to your business, your best course of action is to look at these various services first. Your business may need to build something more specialized but it can be quite helpful to look at how the big boys built theirs first. Whatever the plan and direction of your business know that these AI techs are going to disrupt business and the more informed you are, the better you will be at making future decisions.
