Machine Learning: As Simple as 1, 2, Up - A Postmortem
The past 2 weeks in our coding class, we have been working on projects that showcased the idea of machine learning. We were required to use the m15 JavaScript library for the input of the project and the p5 JavaScript library in some way for the output.
We decided to use ml5’s sound classifier for the input. This part of ml5 takes the noise from the machine’s microphone and well, classifies it. The sound classifier natively has 18 commands, the numbers 0–9, up, down, left, right, yes, and last but not least no. We struggled with deciding what to do with this tool, and finally, we decided to do something simple.
Our decision was to display an image depending on the command said. So, if the user spoke the word “two” the program would display an image of well, the number two. This was a little complicated to actually code in, but we were lucky because we chose something that we could do in the time-frame. Others either had to change their plans or just never finished what they wanted to accomplish.
An example of this was one group who wanted to use m15’s Pix 2 Pix for their input. Their plan was to have the user draw something, and the computer completes the image based on images of the Nintendo character, Kirby.
Our main problem with the project itself was corresponding each image to each command. We already pre-loaded every image and was able to display each one, but we took some time to make sure the correct image was displaying when the user said the correct word. To combat this, we assigned every image a different variable. This sounds like a lot, but the main variable was “img” and every image was assigned “img.zero, img.one” etc. It was tedious, but that’s to be expected with similar variables.
Our variables looked something like this:
And on and on. We had to make the main img var an array and the “subvariables” like “img.go” things in the main array itself. This meant we didn’t have to make a different variable for every one of the 18 images.
Now, to the main part of the code, the part where it compares what command you say and displays that corresponding image. This was a mess. A huge if statement with 18 different choices, one for each command. The draw loop was continuously running the function that decided the image, which was aptly named decideImage.
Bravo, the main JavaScript was done! We moved the canvas in the CSS, made the website look, well, like a website, and finally presented it to the class.
We learned that starting simple is very important in a project with so much room for creativity. We learned that machine learning is only restricted to the commands you give it, and the more leeway you give the code, the more the computer can “learn”. If you take away one thing from this, it is that you shouldn’t reinvent the wheel.
