LLM for Music Generation (Coding Period -VI)
Introduction
In week VI, we focused on several LLM models: ChatMusician, 2-bit ChatMusician, 4-bit ChatMusician, and LLama3.
ChatMusician
I installed ChatMusician, and it works greatly as it produces nice ABC notation. However, it is very resource-intensive, taking around 30 minutes to generate an ABC notation song. For current development, we could explore smaller models initially. Once we have sufficient resources, we could consider replacing the model with ChatMusician. You can find more details in the research paper, which I recommend you to go through: https://arxiv.org/abs/2402.16153. This paper explains why ABC notation is preferred over MIDI and discusses their training approach using ABC notation music.
2 & 4-bit LLM
I installed them on Colab, as shown in the screenshot, but they turned out to be not worth it. They were unable to generate meaningful music; initially, they responded in German, which is not useful.
LLama 3
I utilized LLama 3, which was able to create decent ABC notation music. I used it in the music block, and we could further refine its usage. Once we have the necessary resources, we could replace it with ChatMusician.
What’s Next
Create an API server using LLama3 at the backend to generate ABC notation songs. These responses can be used to create music in the MusicBlock using an ABC parser that we developed.
With this, thank you everyone, and see you next week!
