How to Build a Collaborative MIDI App with Express.js & Socket.io

Andrew Bales
Jul 10, 2017 · 9 min read

In this tutorial I’ll show you how to build an app that allows you to use your midi keyboard to make music in real-time with friends! Here’s what we’re building:

Key light up pink for you, blue for other users.

Try out a live demo: https://midi-collaboration.herokuapp.com/

Check out the code: https://github.com/agbales/web-midi-collaboration

As you can see, playing a note lights up a key in pink and displays the input data in the list. If another user joins the session, their input will light up blue and their data will appear in the list as blue entries.

We’ll break the process into 5 parts:

  1. Creating an App with Express.js
  2. Connecting a Midi Controller
  3. Adding Socket.io for Collaboration
  4. Styling
  5. Deploying to Heroku

Step 1: Creating an Express.js App

We’ll get started by making a directory and initializing the project with these three terminal commands:

The NPM utility will ask for a bit of information to set up the package.json file. Feel free to provide that info or just hit enter to leave these fields blank for now.

Next, add Express:

In package.json, you will now see Express included as a dependency.

In the root folder, make server.js with:

In server.js, add the following:

This creates an instance of Express and sets the port that defaults to 8080. app.use instructs express to serve up files from the ‘public’ folder. If you ran the server now, you’d get an error. That’s because we don’t yet have a public folder!

Public folder

Let’s make that public folder:

Inside, we’ll make index.html, index.js, and a CSS folder containing style.css:

The ‘public’ folder should look like this:

Index.html should link to style.css and index.js. So let’s add the following:

This makes a simple header followed by an empty unordered list — this is where we’ll log our midi data. After the body, it references index.js. For now, let’s add a simple console.log to index.js so that we’re sure it’s working properly:

Update package.json

Finally, we want to update package.json so that we can run our server with the terminal command ‘npm start’. Add the following line to “scripts”:

Your package.json should look like this:

Make sure you’re in the midi-collaboration folder and start up your app:

Great! It’s now hosted at http://localhost:8080/

Step 2: Connecting your Midi Controller

We’ll use the Web Midi API to connect a USB midi controller to the app. If you’re seeing ‘index.js is connected!’ in your console, let’s replace the console.log with:

If your midi controller is hooked up, you should see the console.log for the MIDIAccess object:

Image for post
Image for post

In onMIDISuccess(), a for loop listens for midi messages. Right now it should be causing an error in the console. Why? Because we haven’t defined what to do when it receives a midi message.

Let’s create the onMIDImessage function referenced in the loop:

This function creates a new <li> element. It appends a text node with our midi data. It adds a css class of user-midi (this will be important later). Finally, it adds that new list item to the unordered list with the id “midi-data”.

Image for post
Image for post
Midi keyup and keydown events

Pretty cool, eh?

But what do these numbers mean? Also: where’s the sound?

MIDI Protocol

MIDI Protocol is a rabbit hole all its own, but for our purposes you can understand the numbers with a simple chart:

On / Off → 144 / 128

Pitch → 0–127

Velocity → 0–127

When working with this data, we’ll treat the first number like an on/off switch. 144 = on, 128 = off. The second number is the range of pitches. The final velocity input could also be understood as volume in our usage here.

If you’d like a more in-depth look at midi, here’s a good place to start.

Sound

The MIDI data is not the sound, but a set of directions: ON/OFF, PITCH, VELOCITY. We’ll need to make our own synth that can turn this information into musical tones.

First, let’s convert those value sets from an array into a ‘note’ object that we can pass to our sound player. In onMIDImessage function, add:

Above, the variable ‘d’ is assigned the incoming data: an array of three numbers. Those three values are accessed by their index and assigned as values for the object properties. The note object is then passed to the play function. Let’s write that function:

This function checks to see if the note is on (144) or off (128) and triggers the appropriate command. Both noteOn and noteOff reference two global variables that we established at the top of index.js to handle the starting and stopping of sound.

The frequency function is used to convert the midi note number into a hertz frequency. If you’re curious, you can read more about it here.

Your app should now play like a synth. Huzzah!

Step 3: Adding Socket.io for Collaboration

Now it’s time for the fun stuff: syncing multiple users in a session! Socket.io will help us do that by enabling ‘real-time bidirectional event-based communication.’ That means we can send and receive midi messages as they’re triggered on any browser as they occur. Add Socket.io to the project:

You’ll also need to add this inside the head of index.html:

Note: I’m using version 2.0.3 — be sure your versions of Socket.io match your package.json and html script. You might want this link to the CDN.

For Socket.io to connect users we need to assure that:

1) Users can both send and receive notes

2) The server listens for user input and broadcasts it to other users

User send/receive

First, let’s make sure the user emits a signal every time they play a note. This can be accomplished by opening index.js and adding a single line to the function onMIDImessage. Be sure to add this after the note object has been defined.

We also want to play any external notes that come in from other users. Inside index.js, add the following:

When we receive an ‘externalMidi’ message from the server, it triggers gotExternalMidiMessage. This function should look familiar — in fact, we could refactor this later, but for now we’ll repeat code for clarity. It displays the external note in the view in a manner that’s almost identical to how we treat midi input from our own keyboard. However, we’ve given the <li> a class name ‘external-midi’. This will be important in a moment when we add styles to differentiate between our midi input and that of outside users.

Finally, the note is passed to the player to trigger a sound.

Server

Now let’s make a bridge between users. We want to handle any incoming signals and pass them to any other sessions.

In server.js, require Socket.io and add the function newConnection. This will be triggered when the server gains a new connection.

The newConnection console.log will appear in the terminal every time a new connection is made. We also have socket.on listening for messages from any user and then triggering midiMsg, which broadcasts that data to every user except the original user.

With that, we’re all set!

Step 3: Styling

If you’re just interested in seeing your notes differentiated from other players, you can take a shortcut here and simply add these classes to your style.css:

That’s it! Now you will see your own input in green and any external signal in blue. Feel free to skip to the next step and deploy your app to Heroku!

If you’d like to create a keyboard interface, let’s keep rolling:

Build a keyboard

This keyboard we’ll make builds off of @baileyparker’s CodePen project. We’ll make some design and functionality changes to fit our usage. To start, open up this pen in a separate window:

HTML

The div with the class ‘piano’ houses all of our keys, which are labeled with a variety of selectors. Follow the html structure in the pen and paste the piano div and all its keys into your document (/public/index.html).

CSS

This is a big update for our styles. We’re importing a google font, assigning a background image to the body, and finally giving colors to differentiate between .user-midi (pink) and .external-midi (blue) signals. The ul and li elements have been styled so that they’ll slant back at a pleasing angle.

The keyboard styling takes up the remainder of the CSS. Worth noting here are the ‘active’ classes like ‘.piano-key-natural:active’ or ‘.piano-key-natural-external:active’. These are triggered by the Javascript when a note is played. If it matches the data number, the CSS will activate that key to be pink for your notes and blue for any external input.

When you copy the CSS from the pen into your project’s style sheet (/public/style.css), be sure to follow the notes included inside. Most importantly, you’ll need to update the path to the background.

JS

You’ll do the same thing for the Javascript: cut and paste it into /public/index.js below the code we’ve written. Again, it is important to read and follow the few comments within the code.

This code will manage midi controller connections. But you’ll want to focus in on two functions: onMidiMessage & updateKeyboard. The former handles local input and applies the appropriate class to light up the keys. The latter does the same thing (but with a different class) for external messages.

To get external notes to light up the keyboard, we need to return to the function gotExternalMidiMessage. After we call play(), add the following code:

The keyboard needs a specific kind of object data structure to update, so we create msg and fill it with data from our note. That msg object is passed to updateKeyboard which lights up the keys in blue for that external signal.

You should now see your keys light up pink for your own input! When we connect to Heroku add more users, you’ll see those external midi messages light up in blue!

Step 4: Deploy to Heroku

Instead of walking through each step here, I’ll direct you towards the comprehensive documentation at Heroku. They’ve done a great job reviewing options for deployment:

Summary

If all went well, you should now have an app that passes signals from one user to the next and plays notes from every user. The keyboard should light up with different colors for your input and that of external sources.

Image for post
Image for post
Completed App!

I’m excited about this collaborative app. It has been a real thrill hooking it up and inviting friends from other states to log on and play music! Looking back, I also see ways that the data could be passed more efficiently and areas where the design could be enhanced — it’s not friendly to smaller screens, for instance.

I’d love to hear how you’re using the app, and I hope it has given you a better understanding of how to combine Express.js, Socket.io, and the Web MIDI API!

HackerNoon.com

#BlackLivesMatter

Sign up for Get Better Tech Emails via HackerNoon.com

By HackerNoon.com

how hackers start their afternoons. the real shit is on hackernoon.com. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Andrew Bales

Written by

Writer & software developer. Talking code, art, music, literature, and education.

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Andrew Bales

Written by

Writer & software developer. Talking code, art, music, literature, and education.

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store