Introduction to What We Are Going to Code in Google Summer of Code (Coding Period 1)

Introduction

Today we will be looking at what we are going to build in GSoC and what we have done from May 24 to May 29.

In this GSoC 2024 at SugarLabs, we are going to build Musical Ideation through LLM.

How We Are Going to Build This

Here is a rough system design to help you understand. We are going to build an ABC parser to MB code. Here’s how this works:

Students will create music, and to improve the harmony, we will send the MB code to LLMs for improvisation. However, training LLMs on MB code would be hard and time-consuming, requiring a lot of resources. Additionally, the output might not be great.

We thefore need a standard conversion like ABC, LilyPond, or MIDI. We initially thought of MIDI, but it has its challenges. MIDI has been easily accessible data due to its popularity in the music industry. However, to solve MIDI’s lengthy sequence challenges posed for transformer models with intensive training demands, the sequences are typically segmented into shorter fragments, which limits capturing a composition’s full continuity. Additionally, MIDI’s encoding of performance nuances can lead to quantization errors and unstable rhythms when being tokenized, as referenced in the ChatMusician paper (link here).

Therefore, we employ ABC notation, a score-oriented and plain text representation, for its notable advantages. Its high compression rate leads to shorter sequence lengths compared to MIDI, and it intrinsically encodes musical repetition and structure (e.g., by the use of repeated symbols), enhancing processing efficiency using language models. It also includes detailed musical symbols denoting performance techniques and avoids quantization issues, ensuring rhythmic precision in music generation. ABC notation’s compatibility…

Issues and Progress

But here is the issue: we don’t have the ABC to MB conversion yet!. This is where the ABC Parser comes to help. We will dive into how we are going to do this in next week’s Medium post regarding the ABC->MB parser.

So, this week we have built and accomplished the following things:

1. Connected with mentors and discussed the issues in the ABC to MB parser PR.
2. Below are the issues identified:

• Loop for repetition
• Naming convention to all capital letters
• Remove backslash
• Triplet -> 1/TimeSignature(denominator) * X

1. We were able to fix the backslash issue this week and are working on the triplet time signature. There are some issues while generalizing it not only for triplets but also for quadruplets and quintuplets.

With this, I hope you are very excited. Next week, we will look at the ABC parser implementation and how the MB code works. Very exciting!

Thank you for checking out this post.