Emergence of the Artificial Neural Networks among other Models

Among all models of AI systems, Artificial Neural Networks (ANNs for short) is gaining prominence in learning and codifying complex behaviors. ANNs are modeled after the substrate structure of the brain, in the hope of getting closer to do things that the brain does well. Tasks such as speech and vision heuristics are done fairly effectively by the brain and hence ANNs should also do well with such tasks. In this logic, ANNs should also fall short in places where Brains are fairly weak, such as precise calculation of complex numbers.

Different #AI approach and philosophies

Before we get to ANNs, there are broadly about 5 differing branches through which AI can be modeled. (If you read the book “The Master Algorithm” by Pedro Domingos, he calls them as the 5 tribes.)

1. The Logic Based model: Logic programmers approach AI through inductive reasoning, predicates and propositional logic. Here, knowledge is represented based on Logic Theory. Logicians usually start from an existential set of background knowledge as against starting from scratch. The emphasis here is to establish truth of a statement through 0 and 1 (True, False). Pedro in his book call the practitioners as Symbolists.
2. Probabilistic Model: The practitioners of Probability Models are driven by the certainty (or uncertainty) of a event occurring, in order to derive the value for that event. Typically, the practitioners work with incomplete information or randomness and apply probability theory such as Bayesian inference, Hidden Markov, Decision Theory etc.. This approach is particularly good for noisy or conflicting observations. The practitioners are also called Bayesians.
3. Evolutionary Models: The practitioners of this model rely on goals, sub-goals, path analysis, mutations, search and mathematical optimizations etc. There is a heavier reliance on heuristics that mimics natural evolution and is quite Darwinian. The approach is to use Genetic Programming, Evolutionary Algorithms and Swarm Intelligence to solve problems. The practitioners are also called Evolutionaries.
4. Connectionism: The approach is based on creating models as a process of exhibiting complex behaviors through inter-connection of simpler units (neurons). ANN is the main model of Connectionism. The practitioners are also called Connectionists.
5. Classifiers: The practitioners rely mostly on Statistical Classification and Control systems. The main approach is inference and reasoning through similarities. Logic or Probit Regressions, Multivariate Analysis and Support Vector Machines are some of the main techniques used in this approach. The practitioners are also called Analogizers (or Frequentists) due to their reliance on analogies.

So why is ANN such a prominent approach among all?

Due to its connectionist approach where emergent processes for complex behavior is codified through interconnection of simpler units, ANNs are quite simple to implement. Also the main tool of ANN is the back-propogation, which is based on propagating information back into the interconnected simpler units to model correct behavior. Back-Propagation is quite intuitive to implement.

Another important reason why ANNs are prominent is that, ANNs can accommodate techniques and theories from all other models quite well into the Connectionist model. A full Bayesian model can be used for interpreting weight decays. Markov Chain Monte Carlo Methods can be used for better sampling methods. Genetic Algorithms can be used for efficient pre-training. Evolutionary Programming can be used in clever Activation Functions. Support Vector Machines is a variation on Neural Net model which can be easily codified as ANNs.

Given this, you get to hear a lot about ANNs in the form of Deep Neural Networks, Convolutional Neural Networks, Recurrent Neural Networks etc.

Its important to conclude that usage of a model is mainly based on the complexity of the problems, topology of information within the domain and accuracy of the solution expected. While ANNs does Pattern Recognition, Classification or Anomaly Detection tasks quite well, you may not want to use them for Regression based solutions.