Stanford CS 470 / MUS 356
Code in this drive folder: https://drive.google.com/drive/folders/12qmmzJJaqrab_lSykUo9B4qt4FsHnaDn?usp=share_link
Description: Wekinator is a tool that lets you quickly train ML models that operate on and output real time data streams. It’s incredibly easy to set up and transmits/receives messages using open sound control (OSC). In this assignment, we make and play with a couple Wekinator experiences. More details here: https://ccrma.stanford.edu/wiki/356-winter-2023/hw3
Moo Cow, Moo: I was driving past a herd of cattle while I read this assignment for the first time and the first thought that popped into my head was that I’m sure cows have different voices and it would be hilarious to have a model that listens to you moo and responds with a cow! Enter… moo cow, moo (MCM)… one man’s descent into insanity. MCM takes frames of mic input and sends MFCCs, RMS, and spectral centroid via OSC to Wekinator. I modulated my voice to come up with a variety of moos and found cows that matched the energy of each moo I came up with. Wekinator trained a classifier and, when running, sends the classification to a Processing program that displays your member of the herd. Enjoy.
Dancing Queen: DQ watches you dance and uses the openCV Python library to detect contours from the camera. Then we send information about the center of mass of those contours like its location and how quickly its moving to Weknator and train a model to detect dancing styles. Finally that classification is sent to a Chuck program that modulates which song is playing to fit your dancing style. You can easily imagine an implementation where instead of changing the song, you do something like bring down the cutoff on a lowpass and change the sound of the song rather than the song itself.
Reflection: Wekinating my world took a bit longer than expected… but it is super enjoyable to get into the small data mindset. I think I struggled with trying to make a “production ready” tool rather than just a toy where output errors are to be expected. But the small data mindset isn’t about super high performance. It’s about agility and playfulness. It’s about doing a lot of little things, and if you want it to add up to a big thing, that’s up to you.
Wekinator makes it easy to let go of the logic hangups I talked about in my last reading response. I think that’s because the interaction I was using happened outside the computer. Camera input has you dancing around. Mic input has you moo-ing into the abyss.
Overall I could 100% see myself using wekinator again in the future. It’s so fun. I could play with this all day.
Instructions to run MCM:
- Load Wekinator model in ./Models/
- Open and run chuck program moocowmoo.ck
- Open and run processing program cowcow.pde
- Run the trained Wekinator model
Instructions to run DQ:
- Load Wekinator model in ./Models
- Open and run python program figcap.py (need opencv)
- Open and run chuck program DancingQueeeen.ck
- Run the trained Wekinator model
