Artificial Intelligence Techniques
What is artificial intelligence?
Artificial intelligence (AI) is the modeling and simulation of the way humans think and act. Ancient civilisations in Greece, Egypt and China have come up with the idea of mechanical men and automation. Philosophers as far back as Aristotle have tried to come up with different methods to describe human thought and knowledge. AI draws on the research and development from various disciplines including philosophy, mathematics, economics, neuroscience, psychology, linguistics and computer engineering. Through simulating the way humans think and act, we can use machines to help us solve many of the problems humans face.
The birth of artificial intelligence
The Turing Test, proposed by Alan Turing in 1950, is a method of inquiry in AI for determining whether or not a machine is capable of thinking like a human being. The test remains relevant today and in his proposal, six disciplines of AI were described:
- Natural language processing
- Knowledge representation
- Automated reasoning
- Machine learning
- Computer vision
- Robotics
In 1955, John McCarthy, officially coined the term “artificial intelligence”, with the following phrase:
“every aspect of learning or any other characteristic of intelligence can, in principle, be described with such precision that a machine can be made to simulate it. We will try to discover how to make machines use language, from abstractions and concepts, solve problems now reserved for humans, and improve themselves”.
Symbolic AI
Symbolic AI techniques are based on high-level “symbolic” (human-readable) representations of problems, logic and search. Symbolic AI was the dominant paradigm of AI research from the mid-1950s until the late 1980s. One popular form of symbolic AI is expert systems, which uses a network of production rules. Two main areas continue to be highly researched are robotics and computer vision dealing with image processing and spatial awareness.
Robotics
The Babylonians developed the clepsydra in around 600BC, a clock that measures time using the flow of water. It’s considered one of the first “robotic” devices in history. Subsequently, inventors like Aristotle, Leonardo da Vinci, Joseph Marie Jacquard have come up with various designs and implementations of robotics.
At the beginning of the 20th century science fiction works such as Wizard of Oz, Rossums Universal Robot, Metropolis, Runaround started to shape our idea of robots. In the latter part of the 20th century, real robots were being developed and in 2000, ASIMO became one of the first humanoid robots to walk, run and dance on two limbs.
Today, robotics are used in many fields such as manufacturing, farming, transportation, logistics, medical and military.
Computer Vision
Marvin Minksy was one of the first people to link digital vision to AI through a computer in the 60s. He instructed a graduate student to connect a camera to a computer and have it described what it sees. In the 80s, researchers started to explore different methods for image recognition. Kunihiko Fukushima built the ‘neocognitron’, which is the precursor of modern Convolutional Neural Networks. However, due to lack of processing power at the time and poor understanding of neural networks, development in this area was stunted as part of the AI winter period.
In the 90s, we began to see increased use of computer vision in the areas of surveillance. Going into the 21st century, research and commercial use of computer vision saw an exponential growth. Google played a big part in using its massive farms of computers to develop image recognition neural networks and other big players followed. Today, computer vision is part of our everyday lives from the apps we use on our phones to manufacturing, surveillance, art and transportation.
Machine Learning
Machine learning involves algorithms that enable software to improve its performance over time as it obtains more data. This is programming by input-output examples rather than just coding and is a form of pattern recognition. In this field, there is a need to feed machines a lot of data for the machine to learn and make predictions. Machines can learn in many dimensions and process large amounts of data.
Statistical learning
Statistical learning techniques are developed from the fields of statistics and functional analysis. Statistical learning deals with the problem of finding a predictive function based on data. Apart from allowing computers to beat humans at their own games, statistical learning has many practical applications in computer vision, speech recognition and natural language processing (NLP).
In 1997, IBM’s Deep Blue machine beat the then reigning world chess champion Garry Kasparov at the game of international chess. Deep Blue’s strategy is based on Bayesian statistics together with brute force calculations and pre-set rules.
In 2011, IBM’s Watson took on the TV game show, Jeopardy!. Operating like a search engine with impressive NLP and reasoning abilities, Watson proved that computers could not only master mathematical strategy games, but also knowledge-based and communication-based games, as well.
Deep Learning
Deep learning is an AI function concerned with algorithms inspired by the structure and function of the brain called neural networks (NN). Deep Learning uses layers of algorithms to process data where information is passed through each layer, with the output of the previous layer providing input for the next layer. The first layer in a network is called the input layer, while the last is called an output layer. All the layers between the two are referred to as hidden layers. Each layer is typically a simple, uniform algorithm containing one kind of activation function. Learning can be supervised, semi-supervised or unsupervised. The three main types of NN are artificial neural networks (ANN), convolutional neural networks (CNN) and recurrent neural networks (RNN).
History of artificial neural networks:
In 1943, neurophysiologist Warren McCulloch and mathematician Walter Pitts simulated how the neurons in the brain are connected with a simple electrical circuit. Donald Hebb then took the idea further in his book, The Organization of Behaviour (1949), proposing that neural pathways strengthen over each successive use, especially between neurons that tend to fire at the same time. In 1958, Frank Rosenblatt invented the Mark I Perceptron which was the first computer that could learn new skills by trial and error, using a type of neural network that simulated human thought processes.
Research into neural networks flourished until about the 80s, where the expected success and practical applications of AI was not seen. As investments and interest in AI dwindled during the AI winter period (~80s to mid-90s), development in this area was slow.
At the beginning of the 21st century, computers started to become faster at processing data and graphics processing units were developed, which are tremendously good at the calculations required for NN. Also around this time there was a resurgence of research into NN and discovery of better techniques using more layers. Another area which contributed to the explosive growth of NN and deep learning is big data — large databases of data/images started to appear and shared to researchers worldwide.
AlexNet is a NN design by Alex Krizhevsk and won in an image recognition competition in 2011 and several subsequent international competitions. AlexNet is based on convolutional neural networks(CNN) and showed the high performance achievable using such a network.
In 2012, Google conducted The Cat Experiment. The project explored the difficulties of “unsupervised learning.” Most deep learning projects previously used “supervised learning,” meaning the convolutional neural net is trained using labeled data (think images from ImageNet). For this project, unsupervised learning was adopted where a convolutional neural net is given unlabeled data, and is then asked to seek out recurring patterns. Basically the researchers “never told it during the training, ‘This is a cat,’ ”. The computer “basically invented the concept of a cat”.
Perhaps the most significant moment for neural networks recently, is when AlphaGo beat Ke Jie, the number one ranked Go player in the world. As simple as the rules may seem, Go is profoundly complex. There are 10 to the power of 170 possible board configurations — more than the number of atoms in the known universe, making is vastly more complex than chess. AlphaGo relies heavily on deep neural network technology in combination with a tree search algorithm.
Artificial neural networks (ANN):
ANN is a group of multiple perceptrons or neurons at each layer and is also called a feed-forward neural network as inputs are processed only in the forward direction. This type of neural network are one of the simplest variants of neural networks and they may or may not have hidden node layers, making their functioning more interpretable. ANN is used for solving problems relating to tabular data, image data or text data.
Recurrent neural networks (RNN):
In contrast to ANN, RNN have recurrent connections to the hidden state. This looping constraint ensures that sequential information is captured in the input data. This allows it to exhibit temporal dynamic behavior. RNN is used for solving problems relating to time series data, text data and audio data.
Convolution neural networks (CNN):
The building blocks of CNN are kernels (a.k.a. filters), which are used to extract the relevant features from the input using the convolution function. The convolution function is a mathematical operation on two functions that produces a third function that expresses how the shape of one is modified by the other. This function is key for the model to identify features from the dataset. For analysing images, this will help to different features such as shapes and edges. CNN models are the most popular form of NN at the moment and are being used across different applications and domains. They are especially strong in the fields of image and video processing, and are key in today’s facial recognition systems.
Sources:
