The Birds and the Bees Reloaded
Pattern Seeking with Neural Networks and Representative Democracies
The conversion of raw inputs into meaningful output is the primary objective of the data acquisition and analysis process. It sounds simple enough, but the word “meaningful” can be difficult to define analytically. A close approximation may be that a particular arrangement of input values symbolizes a concept already stored or understood by the system. For example, medical students are trained to recognize the symptoms of known ailments in much the same way engineers are taught to trouble shoot machinery. In both cases, the “biological expert systems” must learn to identify patterns and associate them with known solutions.
With a desire to school machines into automated expert systems for uses such as handwriting or face recognition, Artificial Intelligence (AI) researchers have modeled the function of biological brain cells (neurons) and their networked linkages. Physiologically, each neuron accepts input from upstream neurons, calculates an output based on a weighted combination of the input values and passes it along to additional downstream neurons. Ultimately the values appearing at the output edge of the network of neurons symbolize a concept, a feeling, an emotion, urge, or any other thought associated with an intelligent organism.
Unlike the genetic algorithms described in last month’s column, neural networks (NNs) can be trained to recognize new patterns and solve problems by adjusting the interconnections between the neurons of an existing network rather than spawning entirely new ones. As in nature, after an appropriate NN size and structure is attained through genetics, each individual NN can be taught to behave intelligently. Although just what constitutes intelligent behavior is an excellent debate topic, it is safe to say the ability to produce an appropriate consequent (output) from a given antecedent (set of inputs) is intelligent. For example, the Inputs: “In middle of road” and “Car is approaching” producing the Output: “Run!” is intelligent while Output: “Smile!” is not a smart move. The example antecedent is a pattern that should be recognized and produce a consequent of the “evade” variety. Even though you may not have found yourself in that situation early in life, children are taught to recognize the pattern and to associate it with a proper response.
The antecedent/consequent is a principal process of formal logic and algorithmic or artificial neural network (ANN) tutorials often demonstrate the production of “truth tables” as initial applications. For this purpose, a single ANN neuron is designed to accept two inputs each having the value 0 or 1, and the process inside the neuron is simply to sum the inputs and compare them to a threshold value. To produce a logic “OR,” the sum of the inputs must be greater than or equal to 1. For a logic “AND,” the sum must be equal to 2. So why do we need the “network” in “neural network” if a single neuron can perform the logic necessary? The reason presents itself when an ANN is designed to produce a logical “exclusive OR” or “XOR.” Its truth table has an output of 1 when either input is 1, but not both. Exclusive OR situations arise in “sudden death” competitions. If both contestants of a spelling bee spell their words correctly or incorrectly, no winner is determined (conclusion “0”) and the bee is continued for another round until one contestant spells correctly (1) while the other misspells (0). In this case, the “sum and threshold” (SAT) ability of a single neuron cannot solve the problem. However, a network of multiple neurons can.
While each neuron continues to SAT, the higher-level XOR function requires two conditions: (1) Either, but (2) Not Both. The first and second conditions are met by adding a second “layer” to the network such that the first “input” layer calculates “Either” (the OR condition) and the second “output” layer satisfies “Not Both” (the exclusive condition). The input layer neurons calculate their “vote” for 1 or 0 based on the inputs and pass along their outputs to their representative neuron in the higher-level output layer. The output neuron then considers the “votes” of each and calculates the exclusive condition. Each layer of the multi-layer ANN makes a decision based on its configuration to recognize patterns and passes its consequent on to its higher-level representatives who in turn recognize more sophisticated patterns. The current flurry of ANN research revolves around effective methods of ANN training such that more advanced patterns can be processed to produce desirable behavior; much like our current election cycle switches out government representatives who are performing badly.
This material originally appeared as a Contributed Editorial in Scientific Computing and Instrumentation 21:4 March 2004, pg. 16.
William L. Weaver is an Associate Professor in the Department of Integrated Science, Business, and Technology at La Salle University in Philadelphia, PA USA. He holds a B.S. Degree with Double Majors in Chemistry and Physics and earned his Ph.D. in Analytical Chemistry with expertise in Ultrafast LASER Spectroscopy. He teaches, writes, and speaks on the application of Systems Thinking to the development of New Products and Innovation.
