Spectrograms and Scalograms: visualizing signal data
This is the 4th article in our MAFAT Radar competition series, where we take an in-depth look at the different aspects of the challenge and our approach to it.
If you want a recap, check out previous posts: the introduction, the dataset, and augmentations.
Our main approach to this competition was to treat the challenge as an image classification challenge and to use Convolutional Neural Networks to extract features that will help this sort of classification challenge. This approach helped us both in making the data more approachable for manual exploration, and for deploying data augmentation techniques that helped us overcome the small amount of data.
Spectrograms
One way of transforming radar signal data into images is by transforming the IQ matrices into spectrograms:
A spectrogram is a visual way of representing the signal strength, or “loudness”, of a signal over time at various frequencies present in a particular waveform.
The spectrogram visualization represents the amount of change in the returning signal from a transmission captured at different frequencies (Y-axis, also addressed as “fast time”) as a function of the time of the transmission (X-axis, also addressed as “slow time”). The intuition behind it is, that the movement of an object across some coordinates at the radar’s receptive fields will cause differences between the signal returned from that coordinate at different slow time points, the fast time points (frequencies) represents spatial distances from the transmission point. On top of the image, the doppler sweep burst vector is plotted which represents the center of mass of the moving object in the track. i.e, the largest shift in returned pulse frequency for each slow time point.
The intuition and instruction on how to produce spectrograms were given to us by the competition organizers. they could be found in this tutorial notebook:
Spectrogram processing:
Signal smoothing using Hann windowing (Wikipedia)
Discrete Fourier Transform (DFT — Wikipedia) which is a variant of Fast Fourier Transform (FFT — Wikipedia)
Magnitude for plotting — absolute value of complex value
Noise removal by setting the minimum value to be the median
Scalograms
Another way to visualize IQ matrices is by transforming those matrices into Scalograms, a scaled representation of a Continuous Wavelet Transformation (CWT) performed on the original signal. The CWT generates several values that correspond to the correlation between the radar signal and an example wave at different frequencies — also known as the “mother wavelet”. Further explanation and intuition about the use cases of wavelets and calculation can be found here:
Scalograms are a form of transforming CWT into a 3-d tensor, with a pre-determined number of layers, in which each layer corresponds to the convolutions of the signal data with a different scale mother wavelet for each layer. In our case, we chose to use 8 different scales for our mother calculation, since we couldn’t detect major differences when adding further channels, but the number of different channels and scale values may vary according to the data and the task.
The Scalogram calculation resulted in 3D tensors we later used as input to our CNN models, altering them to handle 3D input data and performing 3D convolutions. Unfortunately, in our experiments, the Scalograms did not show a significant improvement in results compared to the spectrograms, while imposing a much heavier computational burden. It might be that different architecture or a more delicate hyperparameter tuning — e.g. a different mother wavelet or the number of scales — might work better for this form of signal visualization.
