Jack & Jill: MIDI Music Improvisor & Generative Sequencer
I recently completed an Arduino-based side project called “Jack & Jill” so I thought I’d share my experience.
Jack & Jill is a little box that generates MIDI notes quantized to a musical scale. It produces pseudo-random music which occasionally repeats.
You can hear and see it in action here:
How did the project come to be?
I write software for a living. Hardware was quite foreign to me so I wanted to challenge myself and learn something new. I also love musical gadgets and I’ve always wanted to build my own.
I started off simply wanting to experiment with MIDI on an Arduino so I purchased Sparkfun’s MIDI shield and started generating some random notes.
The moment I heard Ableton Live play sounds triggered by the Arduino I was hooked! I noticed the randomly generated notes were pretty boring to listen so I decided to build a random note generator that also quantized notes to musical scales. Jack & Jill was born.
How does it work?
There are two improvisors and a conductor. Jack outputs notes on MIDI ch. 1 and Jill on MIDI ch. 2. Generally Jack outputs bass notes while Jill outputs higher notes. This range can be shifted via the “Range” potentiometer.
The conductor makes sure that improvisers play in time but ultimately it is up to the improvisors to determine what notes to play and how often.
Each improvisor has two properties: Sparsity and Intensity. Sparsity determines how frequent and dense notes are. Intensity determines how light or hard a note is played.
Each improvisor follows a musical scale. There are 8 scales to choose from: Major, Harmonic Minor, Pentatonic Major, Pentatonic Minor, Whole Tone,
Arabian, Hungarian Gypsy and Oriental.
Elements of a good improvisation
This is a fascinating topic. The most challenging part of algorithmically generating music is making it sound, well, musical. But what makes something “musical”? What is music? What makes certain pieces of music interesting and others not? It’s almost impossible to answer because it’s so subjective.
That said, here are some guidelines I’ve found useful:
- Find a balance between change and repetition. People get bored when things repeat forever but they get uneasy when things never repeat.
- Vary timing and velocity in subtle amounts. Imperfection is what makes music human.
- Step out of the box appropriately. From time to time it’s ok to play accidentals (notes that don’t fall on the scale).
- Follow the ebb and flow of tension and release. Build up tension then release it. Rinse and repeat, but don’t overdo it.
I’ve only incorporated points 1 and 2. The improvisors will at times repeat motifs to keep things interesting. In addition, the velocity changes randomly from note to note.
Early stages
I began to sketch out ideas on paper. Initially I considered hardware and software at the same time. This was inefficient and frustrating. I was overwhelmed and crippled by decision paralysis.
I decided to step back and focus on one thing: Simplicity.
Simplicity was important to me for two reasons:
- Having to juggle less things in the early stages helped me stay focused.
- As a user, I prefer simple interfaces in which every parameter is directly controllable without menu diving.
The trouble with simplicity is that it’s hard. Really hard. Even though my gut was saying “keep it simple” my mind kept saying “add just one more thing!”. Part of the reason is that I got really excited and wanted to try all kinds of things. The hardest thing about keeping it simple is saying “no” to your own ideas.
My first design idea had no less than 16 parameters to control! It also had about 32 scales to choose from. There was also a full menu with an 128x64 OLED screen and the whole thing was built using an Arduino Mega. I was doing the opposite of simplifying.
I also started e̶x̶p̶e̶r̶i̶m̶e̶n̶t̶i̶n̶g̶ ̶w̶i̶t̶h̶ getting distracted by rotary push encoders and 8x8 LED matrices. Needless to say I was going down a rabbit hole and it wasn’t pretty. I learned quite a bit during this time but I knew I was going down the wrong path.
As I was approaching a functional piece of software I started to think about hardware design (which was pretty foreign to me, as a software engineer). I started going around Googling and YouTube-ing as much as I could. I emailed some of my favorite music gear companies and asked for advice.
(Thank you OTO Machines, BASTL and Critter and Guitar for your help!)
After hours of research I decided to go with a tabbed wood enclosure using a laser engraver (inspired by Amanda Ghassaei’s Arduino Waveform Generator).
At this time I took a laser etching and carving class at my local Techshop, started learning about PCB design and designed the circuit in Fritzing. The number of unknowns were starting to grow too large at this point. I told myself that if I didn’t simplify I would never complete this project. As they say: “done is better than perfect”.
This was a weeknight/weekend project so time was not a luxury I had. I decided to abandon PCBs (for now) and stick to manually building the circuit on Adafruit’s Perma-Proto board.
The next task I gave myself was to figure out how to move the Arduino onto a breadboard. I found this article and this video to be useful resources.
I only later realized that the Arduino Mega’s microcontroller was only available in SMD form. At this time I was new to soldering in general, let alone SMD. So I decided to simplify the project and move it onto an Uno.
The main reason I started on a Mega was to have multiple inputs. Remember at the time this project had no less than 16 analog inputs and multiple outputs, including an OLED.
Moving from the Mega to the Uno not only meant I’d have to reduce inputs but I’d also be giving up a lot of memory. This surfaced a few challenges.
In order to preserve my 16 inputs I discovered Sparkfun was selling these ready to use 16 channel multiplexers. So I decided to order one and give it a try. This worked surprisingly well. I put up some example code for anyone who’s interested.
The bigger challenge was the OLED. The amount of memory needed to store the buffer for the 128x64 pixel display was just too much for the Uno and left little room for anything else. So I ditched the OLED display. Often times simplifying requires cutting your losses.
I went back to the drawing board and decided to rethink this project with the Uno in mind. After stressing out about what to cut and what to leave I ended up with a new design which had only 6 pots and 5 buttons. This fit nicely on the UNO without the need for a multiplexer. I relied on simple LEDs to indicate state.
Software
I learned the hard way that some 3rd party libraries do a lot which makes them consume a lot of memory. I ended up writing my own and pulling out only the parts I needed (for the graphics library and for the MIDI library). I didn’t need the graphics library anyway.
This project relies heavily on the random number generator. Unfortunately, the random function in the Arduino repeats the same sequence each time you reboot it. The recommended trick is to seed it with a floating analog input pin. This works in theory, but in practice I still found the sequence of numbers repeating every 2–3 reboots. I needed something a little more random for my purposes.
I came up with the idea to store an integer in EEPROM and use that as the seed. Each time the project launched I would increment that number and put it back in EEPROM. This way, each time you launched the project it would seed the generator with a different integer. It would of course eventually wrap around at 16 bits but by then users won’t notice the pattern reappearing. If you’re interested I put up a short gist to show how this works.
Building an enclosure
Spoiler alert: I ended up not building an enclosure in the end.
Trying to build an enclosure for this project was surprisingly complex. I went through many failed attempts.
Attempt #1: “I’m going to build a custom case using a laser cutter!”
TechShop was inconvenient for me, I wanted to build something quickly at home so I can iterate on it. Plus, I just needed a simple rectangle, building a box from scratch was overkill.
Failed Attempt #2: “My box is pretty simple, I’ll just use a guitar pedal enclosure”
I used Affinity Designer to lay out the parts on the computer. I then transferred those dimensions manually to graph paper and stuck the paper to the enclosure. Finally, I used a center hole punch to punch holes through the paper to indicate where to drill. It turns out drilling 5/8" holes (for the MIDI jacks) through metal is hard without proper equipment which is why I gave up on this approach.
Failed Attempt #3: “Metal is hard. Let’s try wood.”
I couldn’t find a pre-built wooden box that was thin enough to slide the potentiometers and buttons through. I went through 2 or 3 boxes and all of them were too thick. On the plus side these were much easier to drill.
Failed Attempt #4: “Clearly I can’t drill, so let’s have other people do it.”
Finally I decided to pay someone else to drill the holes for me. I found a couple of places that do this: Mammoth Electronics and PedalPartsPlus. I went with PedalPartsPlus because Mammoth requires an order of a minimum of 10 pedals which increased cost quite a bit. The PedalPartsPlus enclosure was nicely drilled but I foolishly forgot to take into account the actual size of the components inside the case. The case was too small and left no room for pots and buttons. The turn around time for iterating on this was too slow, so I gave up on this approach.
Failed Attempt #5: “Metal doesn’t work. Wood doesn’t work. Outsourcing is too slow. Let’s try plastic!”
I found some ABS plastic enclosures on Amazon and bought a few. These were both large enough and easy to drill. So far so good right? Not quite. I quickly realized that the number of wires required to route all the components together turned into a massive rat’s nest. It was impossible to work it and looked really ugly.
F̶a̶i̶l̶e̶d̶ Attempt #6: “Perfboards and rat’s nests are a pain. Let’s just make a PCB and call it a day”
I decided to bite the bullet and learn how to make a PCB. I gave up on making an enclosure because it was turning into a massive rabbit hole. A PCB would allow me to have a clean standalone version of my project which for now was good enough for me.
I decided to use Eagle because it’s free for simple PCBs and has a large adoption in the Maker/DIY community. I picked up this book and watched Jeremy Blum’s Eagle series on YouTube and was up and running in no time. Sparkfun has thankfully open sourced its MIDI Shield schematic so I was able to borrow a lot of parts and components from there.
After a few days of laying out the PCB I ordered the first 3 boards from OSHPark and received them a couple of weeks later.
At this point it was as simple as soldering the components. Done!
Lessons learned
- Keep it simple. Start by simplifying. Continue simplifying. When you think you’ve made it too simple, simplify some more. Then back off just a little bit and you’re done!
- Cut your losses. Thinking “I’ve spent so much time and money I might as well just keep going” can lead down a slippery slope. I tend to fall for this trap a lot. I find it helpful to take breaks from my project (1–2 weeks) and come back with an objective mind.
- Order parts and try things quickly. A lot of hardware limitations will dictate the outcome of your project. I realized that the quicker I got to hardware implementation the quicker I discovered any unforeseen problems.
- Test your product by actually using it. This one sounds obvious but I made the mistake of spending too much time coding and not enough time testing the project in a real musical context. A lot of the functionality that I thought was cool turned out to not sound good at all. Code is useless unless it makes the product useful.
What’s next?
I’d like to eventually build a proper enclosure. Probably using something like front panel designer. I’m also interested in building eurorack modules so I’ve been toying around with that recently.
Final thoughts
All in all I’m pretty happy with the outcome. It’s nowhere near perfect, but at least it’s done (ish). I’ve learned a lot and I’ve still lots more to learn. Hopefully this post was useful to someone else out there. If you’re working on similar projects I’d love to hear about them!
Peace.
