Data upsurge, Modern Algorithms and Computing Power — Explainable AI Visualization (Part 4)
This article continues the background overview of the research ‘Explainable Deep Learning and Visual Interpretability.’
Machine learning now affects many aspects of our lives, from web searches, streaming services, recommendations on e-commerce websites, intelligent speakers, and stock predictions to criminal justice and healthcare systems. It is omnipotent in consumer products such as cameras, smartphones and personal assistants.
Businesses are incentivized to apply machine learning for enhanced decision-making, business forecast, cost reduction, risk management, productivity improvements, and new product and service development.
Machine learning is a branch of computer science that uses statistical techniques to give computer systems the ability to learn from data, find patterns in data and make highly accurate predictions. In other words, it's a set of techniques used to teach computers how to learn, reason, perceive, distinguish, infer, communicate and make decisions like humans do. It is a large part of modern artificial intelligence (AI). Here, data encompasses many things, such as words, numbers, images, videos and more.
Machine learning learns from experience, following nature's pathway millions of years ago. It’s a paradigm shift from programming where all the instructions must be given explicitly to the computer to ‘indirect’ programming. The computer learns patterns from the data without being explicitly programmed.
To say that computer vision could compete with visual abilities was unrealistic not long ago. It is no longer the case. Machine learning techniques can now recognize objects in an image. An adult can digitize handwritten characters, transcribe speech into text, classify wine types, support the diagnosis of severe diseases, match news items or products with the user’s interest and select results relevant to the search. Then there are self-driving cars on the road that can drive more safely than an average person.
What fuels these advances is the recent surge in data, the rise in computing power over the past few years and faster algorithms, which have led to breakthroughs in machine learning. These three are the essential ingredients that lead to the emergence of the deep learning revolution.
Data Upsurge
The exponential growth in data has fueled machine learning-based industries, technologies, and services in recent times. Nonetheless, humans and machines are generating more data today than ever before. Every day, humans alone produce a massive amount of data ranging from text, audio, video, sensory data and more. That number is expected to increase in the decades to come.
Another example is a car with more than 100 sensors to monitor multiple functions such as fuel level, radar sensors and ultra sensors for close-range detections.
Computing Power
Thanks to Moore’s law, processor chips continue to shrink in size while increasing computing power, which has risen three to four magnitude compared to the mid-1990s. The advanced computational power that has inevitably made all this possible derives from Moore’s law and the discovery of graphics processing units (GPU)s.
GPUs were first designed to boost speed and processing power in gaming industries and give gamers rich, high-speed 3D visual experiences \cite{Yosinski2015}. These were 20–50 times more efficient than traditional central processing units (CPUs) for machine learning and deep learning.
Open AI, a non-profit research organization that promotes AI safety, remarks:
Improvements in computing have been a key component of AI progress, so as long as this trend continues, it’s worth preparing for the implications of systems far outside today’s capabilities.
An example is speech recognition, where a computer has to perform millions of calculations per second for a system to learn and recognize patterns in the data. This task required tremendous computational power that, until quite recently, wasn’t available. In contrast, Marvin Minsky, a pioneering mathematician, a leading scientist and the founder of MIT Artificial Intelligence Laboratory (MIT AI Lab), worked on AI in 1957 when computers were billions of times slower than they are now. Those machines were costly and provided merely a fraction of the performance and computing speed.
Algorithmic Innovation
The data explosion and advances in computing power have made it possible for better and refined algorithms and enable more extensive datasets that algorithms can process at any given time for machine learning tasks.
Traditionally, algorithms were programmed explicitly by humans to perform various tasks. Modern algorithms have become sophisticated to the point where they can facilitate machine learning and allow computers to self-learn from the data. Until recently, there hasn’t been adequate structured or unstructured data to train computers to perform complex tasks independently, let alone develop sophisticated algorithms that allow machines to teach themselves.
An example is the autonomous vehicles car that relies on an enriched visual dataset to construct its map in real-time and navigate the roads. Each video frame collected by a self-driving car must be enriched with data to identify objects such as road signs, pedestrians, trees or sidewalks in every frame.
According to AI expert and MIT Sloan professor Erik Brynjolfsson, there have been some significant improvements to these algorithms in machine learning that has improved on the basic algorithms. Some of them were introduced 30 or 40 years ago, but they have now been tweaked and improved due to faster computing and vast volumes of data. This makes it more convenient to figure out what works and doesn’t work overtime.
These three things combined, computing power, volumes of data and better algorithms, could provide a million-fold improvement in some applications from image recognition and speech synthesis to self-driving cars.
The next article in this series covers the early revolution in deep learning and a single-layer neural network called the perception:
