Neural Networks

Neural networks have been around for years and have found a place not only in the business community, but also in the defense industry as pattern recognition software. Much of the terminology used in Neural Nets comes from the way the brain processes signals and passes these signals down to neurons in biological units. Neural Nets are basically nonlinear mathematical models that can sometimes outperform traditional econometric models if the relationships among the data elements are very nonlinear. The degree which nonlinearity is introduced into the model is determined by the user through the number of “hidden” layers specified. Actually, there is really nothing “hidden” in these types of layers — instead the are simply “activation functions” such as logit transformations.

In most neural nets systems, you have (1) Input Layer consisting of your explanatory variables and your prediction variable (2)a Hidden Layer (logit function), and (3) an Output Layer which is your prediction. Estimation is often done through a process of BackPropagation — a means of feeding back the errors into the model so better parameter estimates can be found. Usually, this process is computationally intensive and is solved through an iterative procedure rather than by inverting a matrix as is done with Ordinary Least Squares. Because of this rather brute force estimation technique, the development of faster computers over the last few years has allowed a greater proliferation of neural net software to enter the market.

One of the major drawbacks of neural networks is the perception that its a “black box”, any neural net software should be designed to demonstrate its result in a form that is readily explainable and easily understood.