The visual representation of musical repetition in popular music.
Music is mathematical.
Many scientists across the years also had interest in music and in its harmony. Indeed, many greek philosophers in the likes of Pythagoras and Aristotle considered music to be a part of mathematics. This developed into music theory. The breakout of technology in recent years helped the mathematicians of the third millennium to observe new mathematical patterns amongst music.
Companies, such as Pandora, already use algorithms to scan the different tones, rhythms, instruments and pitches in order to define if a user’s listening habits concord with certain songs. They are mathematically able to predict if the song will provide a pleasant experience to the user by comparing it to other liked songs. A similar process is also used in Spotify’s automatic and custom playlist creation. Also, different composers in the late 1990’s used the notions of golden ratio and fibonacci number in the composition of their songs.
Starting from these information, we believed such patterns could be represented visually. Many visual representation of music exist, the most common being the moving bands used in radio. However, these are often representing the wavelength and the frequency of the analyzed beat at the exact time in the music’s beat rather than depicting the entire partition. Indeed the majority of music visualizer found online look like those moving shapes in electro music YouTube videos. A popular YouTube video made by Cymatics also demonstrates how music could be visualized in real life.
VMR (Visual Music Representation), was programmed in order to depict the note repetition with tempo from its musical sheet. This is a new approach to musical visualization, as it focuses on the partition and pitch repetitions rather than the frequency of a song. When running, the project takes the form of a self-similarity matrix, or reccurence plot. This type of graphical representation depicts the sequence of similarities within a long series of data. This representation is often used for the recognition of different movements or for the segmentation of an audio file. The program was first inspired by Colin Morris’ project named Songsim, which represented lyrics repetition.
Before explaining how VMR functions, the following free programs are needed in order to run it properly.
-Processing (This was chosen for its ability to visualize a three dimensional project and to form the two dimensional recurrence plot quite easily.)
Now, the following steps have to be done in order to setup your musical sheet in VMR.
- Find partition on musescore and download it as a .XML file. (it can come from another source, but the sheet has to be in the .XML format)
2. Open the .XML file on Musescore
3. From the Musescore toolbar, select export (under the files tab) and name your file. Be sure to export it as a ‘’XML uncompressed (.musicXML)’’.
4. Go to your new uncompressed exported file. Select ‘’Open with‘’ and ‘’Choose another app‘’. There, you should select the XMLWriter application abovementioned.
5. Once the file is opened on XMLWriter, save it as a .XML documents in the XML folder placed in VMR.
Now that the file was converted properly, here is how the VMR project works:
One important thing to note is that the G4P library which is implemented in the sketch from the files manually. To do so, you must go into the libraries folder inside the VMR folder. Then, continue your path into the G4P file, where you will select the library folder where you will find the G4P file of the library. You must select the G4P.jar file and drag it into the sketch. The code will then be all set.
- In order to visualize the song, drag the desired file from the XML folder onto the VMR folder. Then, open the application and input the name of the file (with the .xml at the end).
- Select the desired partition and press the visualize file button at the bottom. The program does not compile or stack the instruments on top of the other, it simply shows pattern within the partition of a selected instrument.
- The music sheet in .XML file is read by the program, which determines the pitch and duration of every musical notes and pauses. It also determines if the notes are altered by sharp or flats in the entire measure.
- These notes are placed on the two axis and form a diagonal from top-left to bottom-right of squares with sizes relative to their duration. This diagonal is depicted in orange, due to the note being at the same moment in the song.
- For every pitch on both axis, a square is drawn on the array when it is repeated on the other axis. This forms a symmetry reflected by the diagonal of the array.
- For every additional squares, a shade of blue is assigned according to its pitch. The only exception being the pauses or silence repetitions drawn as white squares.
- The user can move (up and down arrow key)and zoom (W- Up, A-Left, S-Down, D-Right) within the recurrence plot formed.
Note: When a string is played on the musical sheet, VMR only acknowledges the bottommost note, since the chord takes the name of this note. Also, VMR reads each measure in order to determine if every notes are altered by sharps or flats.
VMR is not a perfect program and can be vastly improved. Indeed, its transformation into 3D processing would help to find new patterns as it would lead to the stacking of different instruments on top of the other. This stacking which would lead to all new relationships within the music sheet, since harmony and repetition of pitches amongst different instruments could be observed. The latter was tried, but partitions such as Despacito or Gimme Shelter were already difficult enough to render, that the computer used for the programing was not powerful enough to render the three dimensional version of the project. Therefore, an update that would be easier to run would be necessary.
Also, the program would need to be updated in order to be able to read two tabs at the same time and merge them into a single visualization in order to read piano sheets, since each tab represents the movement of one of the pianist’s hand. The same stands true for drum partitions. VMR also has problems determining if two notes of the same type have different octaves.
The next step surrounding this program would be to find different patterns from its dwo dimensional form. Once this is done and that VMR is transformed into a 3D processing program, patterns within different instruments of a same song will be found. The objective at this point would be to determine if a music sheet would give a popular music, before even playing it. To find ‘‘music rules’’ that would dictate if a song will be enjoyed. This would also lead to the ability to describe if a partition would be harmonious with another one mathematically and to, therefore, perfect songs that were already known and made.
Link towards the GitHub of the project, where it is possible to download VMR.
Feel free to participate and enhance the program on the proposed GitHub.
