MusicVAE: A tool for creating music with neural networks

It’s hard to believe that my time at Flatiron School is rapidly coming to a close, but that means that it’s time for capstone projects! As such, I’ve been doing a lot of research for my own project, which I’m really excited about getting to tackle in the next four weeks. I don’t want to give away too much just yet, but it involves analyzing music and generating new data based on the music, which has led me to spend time learning about generative models, and specifically, Variational Autoencoders (VAEs). In this post, I’m going to present a high-level summary of this paper by the authors of the MusicVAE model, which is a product of the Magenta project, a collaborative project to bring powerful machine learning tools to the creative arts.

My interest in modeling music-related features has really highlighted the struggle of using current machine learning tools to generate sequences for something as complicated as music. Sequence data, or time-series data, has a flow to it, where each data point in the sequence is dependent in some way on the data points before it and potentially after it. In my previous blog post, I explored using Prophet to model time-series data, but in these style analyses, the independent factor is time and the goal is to understand and predict what happens in relation to time by decomposing the time-series into various attributes of time (e.g., seasonal trends, yearly trends, overall trends, etc.). For music however, decomposing it into various time-related trends wouldn’t make much sense…