Recurrent Neural Networks vs 1D Convolutional Networks
Find which architecture suits better your project.
1. Understanding the architectures
In the applied domain of Signal Processing, there are two main architectures that solves the job.
1.1. Recurrent Neural Networks
One of the most common architecture for this kind of problem are the Recurrent Neural Networks.
These kind of networks relay on a simple principle, feeding the output of a layer back to it’s self, thus creating a feedback loop between different layers of the network. Working with sequential data, the order in which information appears is important, by feeding back the previous input to the current layer, the network keeps a record of the input signal and can find patterns between different parts of the input. More advanced model follow this type of architecture such as LSTMs and GRUs.
1.2. 1D Convolutional Networks
The other main architecture used in this domain is the uni-dimensional convolutional neural network. Before speaking about how it functions, we should describe the core of it, the convolution operation.
The convolution of two signals is a mathematical operation, opposed to cross-correlation. It’s uses are vast in the engineering domain from designing filters, controlling dynamical systems and processing images, to new fields such as Deep Learning.
You are probably familiar to the normal Convolutional Neural Networks, those used in datasets that contains images. The principle is the same, images are just two-dimensional signals. Using such an architecture, the networks learn the best weights for the convolution layers(that acts as filters) in order to extract important components from a signal. For example if you are trying to detect a faulty motor, by measuring it’s rotation as a function of input voltage, you can extract the dominant frequency which differs if the motor is not in it’s proper state. The 1D Convolutional Networks does this extraction automatically choosing the best features to extract from the signal. Like it’s 2D counterpart, you can use complex models such as InceptionTime the counterpart of InceptionNet.
2. Which architecture to use ?
The question still remains, which architecture is best suited for your project.
If talking about performance and ease of training then the answer is simple, the 1D Convolutional Networks are the better option. Unfortunately the problem is much more complex. Throwing a 1D Convolutional Network on any dataset is not the best option. Knowing you dataset very well is the key for choosing the best architecture.
RNN are based on sequence-to-sequence processing of data, the length of the signal is meaningless, it will just process one point at a time, feeding back previous predictions. For a dataset where the signals are of variable length, the RNN architecture is the way to go. Such datasets can be found in applications like Human Voice Recognition and Text Analysis.
Like it’s imaging counterpart, the 1D CNN relays heavily on the dimensions of the dataset, for achieving the best performance from these networks, you have to use a dataset where signals are of fixes length. Such datasets can be found in applications such as Predictive Maintenance and Dynamical Systems Classification.
Thank you for your time, if you are interested in the domain of Machine Learning and looking for project ideas, feel free to follow me, as I will post new content soon.
