Have you ever wondered if your brain resonates with the music you listen to? This question is answered by Sebastian Stober, Thomas Prätzlich, Meinard Müller in their paper presented in 2015. Although it is dependent on a couple of factors, the answer is yes and there is concrete evidence to back the reasoning for this.
Now, think about it, we cannot extract an Electroencephalography (EEG) signal from a person and determine the beats and get the song, right? Think again, that is the goal in the big picture and this experiment is a huge step in that direction.
The Data
The data set consisted of 5 subjects listening to 12 musical classics. The songs for this setup comprised of “Mary had a little lamb”, “Jingle bells”, and the “Harry Potter” theme.
The above link is a sample of the dataset with the initial parts being the cue clicks which corresponding to the tempo of the song.
The Experiment
The subjects are made to listen to the song and the EEG signal from the brain is being recorded simultaneously. They needed to determine if the beats from the recorded EEG signal is similar to the beats present in the audio signal. How did they do that? Ideally, they could have directly compared the audio wave to the recorded EEG signal but unfortunately, it is not that easy. There is a lot of noise present in the EEG signal, the most basic noise coming from eye movements. This is eliminated from the EEG signal with the help of preprocessing techniques available in the MNE toolbox which is an open-source Python software for exploring, visualizing, and analyzing human neurophysiological data.
After performing some preprocessing techniques with the MNE toolbox, the authors found that there is still too much noise in the EEG signal. They used a novel problem to solve this issue. Each subject had to go through five trials for each song. Let us take two trials from subject 1 as A and B. Now let us consider a single trial from subject 2 as C. Since A and B are from the same subject, they will have similar features, A and C will have different features since they are from different subjects. In this method, they enforce the model to learn features that can help differentiate the subjects thus increasing the signal ratio compared to the noise. This method is called similarity constraint encoding.
Observations and Results
With the EEG signal and the audio signal ready for comparison, the authors converted both these signals into tempograms. Tempograms are graphical images that determine the frequency of each tempo in the signal. They plotted the histogram to determine which tempo occurred the most in the EEG and audio signal and this differed by a small value of 1 BPM. This was initially done for “Mary had a little lamb”.
The same procedure was repeated for all the subjects and stimuli. They implemented three strategies for comparison. Allowing a certain error in deviation of tempo, the authors first recorded the results of every single trial. The tempo average is taken for all trials to determine deviation in tempo for each subject. The third strategy implemented was taking the average of each subject’s tempo to determine the tempo deviation for each audio clip.
The strategies did work in some cases leading to interesting conclusions.
Conclusion
With the results they achieved, the authors concluded that they were able to stabilize the tempo and the quality of the tempo was highly determined by the music stimuli used. They were not able to conclude why the music stimuli affected their results and that is open to research.
With the advancements in technology since then, these findings will be helpful for anyone who wants to pursue research in this field. The foundation provided by the authors is strong.
About the authors
The authors mentioned in this paper are open and they have provided a detailed explanation of all the experiments that they conducted. They want the brain-computer interaction community to progress and they have been transparent in regards to their findings and data. Data is valuable these days and to keep the data set public just goes to show the commitment they have towards the field.
Stober, Sebastian et al. “Brain Beats: Tempo Extraction from EEG Data.” ISMIR (2016).
