Convolution in NLP
Yes, it’s useful outside computer vision
In this blog post, we will talk about the convolution operation and how it has been used in natural language processing for the last 40 years.
1 Convolution
Convolution is a fundamental mathematical operation that combines two functions to generate a third function, representing the influence of one function on the other. In the realm of image processing, convolution involves sliding a filter or kernel across a signal or image, performing element-wise multiplication and summation operations at each position. This process, illustrated in Figure 1, enables the extraction of specific features from the input. Different filters or kernels can be employed to extract different features from the input data.
While convolution operation is widely used to capture local spatial information in image processing, it is also valuable for capturing temporal information in sequential data, such as language. One of the earliest applications of convolution in neural networks dates back to 1989 in the field of phoneme…
