15 — Creating New Interfaces for Audio Expression

Unity Tutorial Series: Audio and Interfaces

Photo by Steve Harvey on Unsplash

Introduction

In this final part of the series, we’ll build our new interface for musical expression. I’ve included two options, one is a basic potentiometer (like we saw in the first part of the series) and the other combines the potentiometer with an LDR. In the videos I cover both options.

Warning: the second option requires a bit more data transformation than we are used to. This is because we have to separate our two values (ldr and potentiometer) by a comma, and then ‘store’ those in separate lists to access in Unity.

In both cases, we’ll be doing the following:

1. Wire out chosen interface
2. Set up our Arduino code.
3. Create some audio in Audacity
4. Create a basic Unity scene and add our audio.
5. Download and setup the Ardity package to enable serial communication with our Arduino.
6. Create our messageListener code in Unity and test it all out!

Step 1: Wire our Chosen Interface

Option 1: Basic (Potentiometer)

The basic option is exactly the same as we outlined in the first part of the tutorial:

• (Red wire) Connect a wire from the Arduino’s 3v pin to the + pin on your potentiometer (this is the clockwise side)
• (Blue wire) Connect a wire from the Arduino’s A0 pin (analog in/out 0) to the middle pin on your potetiometer.
• (Black wire) Connect a wire from the Arduino’s ground pin (GND) to the final pin on the potentiometer.

Option 2: Advanced (LDR + Potentiometer)

In this example, we’re wiring up both a potentiometer and and LDR. I’ve included a diagram below that shows (as best I could) one way to do this. We have the following:

• Red (3v) and Black (Ground) wires connecting to the side rails on the breadboard
• A potentiometer with one pin connected to power (red) another to ground (black) and the middle to the A0 input of the Arduino (blue)
• An LDR plus a resistor with one pin connected to power (red), and another pin wired first to the A1 input of the Arduino (yellow) and then to a resistor (10k, but you should try values between 4.7K — 100k to see what works best for you) and finally to ground (black).

Step 2: Set up our Arduino Code

Next, we need to test if our interface is outputting the values that we expect, or if it’s outputting any values at all. To do this, we need to open up the Arduino IDE and send some code to our device.

Option 1: Basic (Potentiometer)

The code below simply prints the analog value of our potentiometer connected to the A0 pin.

Option 2: Advanced (LDR + Potentiometer)

This example is a bit more advanced. We are reading both the A0 pin value and the A1 pin values, then printing those on the same line, separated by a comma, finally, the final Serial.println(potentiometerValue) is used to put our new set of values on a new line.

Once you have this code compiled and uploaded to your Arduino, go ahead and see which values you’re getting back by launching the Serial Monitor.

Here you see my LDR and potentiometer values separated by a comma

If things are working, then we’re all set to move on to the next step.

Step 3 and 4: Create some Audio in Audacity and Create a basic Unity scene and add our audio.

Make sure you have an audio file that you like, you’re ready to move on to the next step.

Refer back to our earlier tutorial Creating Audio in Unity. Make sure you’ve:

• Imported your audio.
• Placed your audio in the scene, and are happy with the placement of your Audio Listener (most likely your main camera)

Feel free to add other audio in the scene as well if you’d like to get a head start on this week’s project and Create a Interfaced Soundscape.

Step 5: Download and setup the Ardity package to enable serial communication with our Arduino.

Refer back to this article on how to connect your Arduino to Unity:

Arduino Tutorial Series: Connecting to Unity

Step 6: Create our messageListener code in Unity and test it all out!

Here is where you’ll deviate a bit depending on how you’ve chosen to set up your interface. In general, if you’re only dealing with a potentiometer, then you’ll be just fine using the code we’ve used for the past couple of weeks.

If you’ve chosen to make an interface that accepts two inputs, you’ll need to modify the example messageListener.cs script to store two values instead of one.

In both examples, I’ve included the full sample code after the videos.

Option 1: Basic (Potentiometer)

The code below simply adjusts the volume of whatever AudioSource we have assigned to our Message Listener GameObject in Unity:

Option 2: Advanced (LDR + Potentiometer)

Here is the MessageListener.cs script for multiple inputs. I’ll explain a bit about what line 27 does below the code, it’s magical.