Bridging the gap between design and code (and learn three.js in less than 4 days)

TLDR; This is how I created an audio-visualiser with Three.js, webaudio API in less then 4 days. Submitted it for an audiovisual contest at whitestone.io. You can see the results live at xenochiku.com and find the code here.

Update: I won the contest! Thanks for all the votes and love!

Hi my name is Lowen, I love music, I love coding and I love visuals. I decided to combine them, and create audiovisuals. I’ve been a front-end designer for 7 years. Now I use coding to bring the music alive. I use Processing, Unity and Max/MSP, but this is my first time trying Three.js.

I was traveling from Nepal to Bangladesh. While absorbing the culture shock, the pressure of participating in the contest would give me focus and pursue to finish my submission in 4 days of work.

First I broke down what my steps would be. Since I knew I had little time I wanted to keep things as simple as possible:

1. Find a three.js template that would kickstart my progress
2. Find an audio-analyser
3. Find the frequencies (bass/mid/high) with the audio-anaylser
4. Create live effects with these frequencies by using the values
5. Create the main object and play with styles

I gave each task a day. Whatever time I had left in the end I would use to play with the styles. Most days I could only work for around 4 hours.

1. Find a three.js template that would kickstart my progress

To save time, the first thing I needed was a boilerplate. I tried a couple of different ones, but ended up with superGuiGui’s three.js starter kit. His project is perfect for quick prototypes and included composer effects, a 3d object and a clear structure of code and files.

2. Find an audio-analyser

Audio consists of loudness for different frequencies. Better understood as bass, mids and highs. Last year I made some experiments. I didn’t want to use these experiments and I couldn’t find anything I liked. So I asked my first question in the creative-dev slack channel and tadaa:

later I found out that mattdesl is actually some crazy good coder with many contributions to the creative coding community :)

Mattdesl gave me a perfect native JS example and I started combining this with superGuiGui’s starter-kit.

3. Find the frequencies (bass/mid/high)

The next step was to define the frequency bands, I used this page for reference (see bottom). This was easy, but with more time I would create more effects for specific parts of the song.

4. Create effects with these frequencies

Then I used the values from those bands and assigned them to the light intensity.

simply assign the averages and multiply with arbitrary numbers until you’re happy

Suddenly I had a torus with some beautiful reflections and light changes!

The beautiful Wagner composer effects made by Spite

5. Create the main object and play with styles

The last step is the most fun but can be frustrating as well. I tried copying some of my processing sketches. But nothing showed up! WTF? I asked on the slack channel and found out I had to add faces to a mesh.

the friendly people of the creative-dev slack

What are faces anyway? Basically they consist of three points which form a triangle when connected. All these triangles add up to a mesh. It’s kind of the skeleton of the object, in triangles. On top of the mesh you can add materials, like fur, glass or stone. This way you can create realistic looking objects.

Now imposter syndrome kicked in. I wasn’t happy and got frustrated. So I decided to take a step back. I created a quick moodboard, finding whatever I could find about Mike Gao using Google images. I decided to use some neon-like colors for the background and the lights. Then I messed around with a bit of trigonometry and created an abstract object. I added movement and used some math to smooth out these movements.

Smooth the movement by using some easy math and ‘soften’ the values.

An updated version, still wasn’t happy but I was getting somewhere.

I didn’t have much time left, it was already March the 4th, but because of different time zones I could still work on it. So with my sleepy head I still had a few hours on March the 5th, in Bangladesh :)

I added a bunch of sphere’s, making them really big and show only it’s wireframes (the yellow lines you see in the gif below). I animated them a bit slower compared to the main object. This creates a bit of a movement contrast between the 3d objects.

I played around with the composer effects, which are effects which you can see in Hollywood movies. Such as Instagram-like color-filters or depth of field, etc.

the end result :)

And so time was up! I submitted the project to whitestone.io and uploaded the code at Github.

You can view all the participants their entries and vote for one of the submissions by entering the name in the popup. Unfortunately it doesn’t work on mobile.

crossing the street with a rickshaw in Dhaka

So all in all, I had a crazy adventure in Bangladesh while building something in three.js for the first time. What a crazy (rickshaw) ride!

Apparently emotional spikes help you remember things better, so hopefully this will stay with me after I hike for a a couple of weeks in the mountains of the Himalaya and continue to create more (with three.js) :)

Special thanks to superGuiGui for this three.js starterkit, which gave me a good boost and of course the amazing people in the create-dev Slack for answering my questions.

Follow me on Medium for more stories about my journey of combining code with art! Also see my Instagram if you like to see my experiments and work in audiovisuals :)

If you love what you’ve just read you could warm my heart 💚 by clapping for me or buy me a coffee ☕️