Bridging the gap between design and code (and learn three.js in less than 4 days)
TLDR; This is how I created an audio-visualiser with Three.js, webaudio API in less then 4 days. Submitted it for an audiovisual contest at whitestone.io. You can see the results live at xenochiku.com and find the code here.
Hi my name is Lowen, I love music, I love coding and I love visuals. I decided to combine them, and create audiovisuals. I’ve been a front-end designer for 7 years. Now I use coding to bring the music alive. I use Processing, Unity and Max/MSP, but this is my first time trying Three.js.
I was traveling from Nepal to Bangladesh. While absorbing the culture shock, the pressure of participating in the contest would give me focus and pursue to finish my submission in 4 days of work. The results can be viewed live at xenochiku.com.
First I broke down what my steps would be. Since I knew I had little time I wanted to keep things as simple as possible:
- Find a three.js template that would kickstart my progress
- Find an audio-analyser
- Find the frequencies (bass/mid/high) with the audio-anaylser
- Create live effects with these frequencies by using the values
- Create the main object and play with styles
I gave each task a day. Whatever time I had left in the end I would use to play with the styles. Most days I could only work for around 4 hours.
1. Find a three.js template that would kickstart my progress
To save time, the first thing I needed was a boilerplate. I tried a couple of different ones, but ended up with superGuiGui’s three.js starter kit. His project is perfect for quick prototypes and included composer effects, a 3d object and a clear structure of code and files.
2. Find an audio-analyser
Audio consists of loudness for different frequencies. Better understood as bass, mids and highs. Last year I made some experiments. I didn’t want to use these experiments and I couldn’t find anything I liked. So I asked my first question in the creative-dev slack channel and tadaa:
3. Find the frequencies (bass/mid/high)
The next step was to define the frequency bands, I used this page for reference (see bottom). This was easy, but with more time I would create more effects for specific parts of the song.
4. Create effects with these frequencies
Then I used the values from those bands and assigned them to the light intensity.
5. Create the main object and play with styles
The last step is the most fun but can be frustrating as well. I tried copying some of my processing sketches. But nothing showed up! WTF? I asked on the slack channel and found out I had to add faces to a mesh.
What are faces anyway? Basically they consist of three points which form a triangle when connected. All these triangles add up to a mesh. It’s kind of the skeleton of the object, in triangles. On top of the mesh you can add materials, like fur, glass or stone. This way you can create realistic looking objects.
Now imposter syndrome kicked in. I wasn’t happy and got frustrated. So I decided to take a step back. I created a quick moodboard, finding whatever I could find about Mike Gao using Google images. I decided to use some neon-like colors for the background and the lights. Then I messed around with a bit of trigonometry and created an abstract object. I added movement and used some math to smooth out these movements.
I didn’t have much time left, it was already March the 4th, but because of different time zones I could still work on it. So with my sleepy head I still had a few hours on March the 5th, in Bangladesh :)
I added a bunch of sphere’s, making them really big and show only it’s wireframes (the yellow lines you see in the gif below). I animated them a bit slower compared to the main object. This creates a bit of a movement contrast between the 3d objects.
I played around with the composer effects, which are effects which you can see in Hollywood movies. Such as Instagram-like color-filters or depth of field, etc.
And so time was up! I submitted the project to whitestone.io and uploaded the code at Github.
You can view all the participants their entries and vote for one of the submissions by entering the name in the popup. Unfortunately it doesn’t work on mobile.
So all in all, I had a crazy adventure in Bangladesh while building something in three.js for the first time. What a crazy (rickshaw) ride!
Apparently emotional spikes help you remember things better, so hopefully this will stay with me after I hike for a a couple of weeks in the mountains of the Himalaya and continue to create more (with three.js) :)