SampleVAE - A Multi-Purpose AI Tool for Music Producers and Sound Designers

Revealing some of the tools of the MUTEK AI Music Lab

Max Frenzel, PhD
Nov 8, 2019 · 12 min read
Photo of one of our other AI music projects - AI DJ.
from tool_class import *
tool = SoundSampleTool(model_name, library_directory)

Generating Sounds

Random Sampling


Re-Generating a Sound

tool.generate(out_file='generated.wav', audio_files=[input_file])

Combining Multiple Sounds

tool.generate(out_file='generated.wav', audio_files=[input_file1, input_file2])

Sound Similarity Search

Finding similar files to a given snare sound.


Confusion matrix for a model trained to predict nine drum types.
probabilities, most_likely_class = tool.predict(input_file)
Confusion matrix for a model trained to classify sounds into 71 drum machines (e.g. Roland TR-808). This is not meant to show a good result, the dataset was tiny, with only a handful of examples per class. It is merely meant to show what’s possible.

Training Your Own Model

Preparing a Dataset

python --data_dir root_dir --dataset_name my_dataset

Training a Model

python --logdir model_name --dataset my_dataset

Improvements, Issues, and Ideas

Other Tools

Qosmo Lab

Cultivate the Chaos