‘What I learn, creating a 100 AV daillies’-log

flowen
7 min readAug 2, 2017

--

So this will be like a diary for the next 100 audiovisuals.

So live happens and I never made it to a 100 😅 At this point I don’t any incentive to finish creating the rest. I still hope this might be helpful to someone out there.

All the code can be found on my github and all demo’s can be found by going to xenochiku.com/$numberofdailyhere. so /001 /002 etc. I also link them in this logticle (yes I just made up a new name, inspired by the ridiculous ‘listicle’).

Some topics I want to research combined with audioreactivity:

done: materials, models, importing 3d models, particle systsems
want2do: create math-generated models & pointclouds , more shaders exploration, vertex-deformations, ray casting, cellular automata, nature of code topics, morphing, collision detection

There’s still so much to learn.. I feel like I should specialise more after touching base with most subjects. Not sure what direction I want to go though 💥 mind exploding 💥 After this venture I also want to go back to VJ’ing and creating visuals with Processing and Unity.

day1 001/100

Pointcloud = array of vectors (points in worldspace(x/y/z))
Topology = the lines between these vectors -> edges -> faces/polys/prism
Thoughts:

  • Audio frequences are basically seeders, just like perlin noise. What about combining them?
  • create my own pointclouds w math → perlin noise, audiostream, etc

I found some old code from an example from the ‘Learning three.js book’. I combined it with a copy of my submission for the Whitestone contest and tada here we go. Day 1.

day 2 002/100

I really really hate building tools. But I love ES6. Spend 3+hours trying to fix a bug and post on Stack overflow. No answers. Did a quickie: copied yesterdays daily and changed some attributes, played around and within an hour finished. I’m addicted to Lorn.

day 3 003/100

After spending another hour on the bug, I ask in 3 different slack groups. One of my favorite slack’s: Creative-dev responds with some solutions and examples. I keep spending more time on it. Tried other examples, why don’t I understand it? Suddenly I realised this error came from the guigui module. I replaced the module with DAT-Gui and posted.

day 4 004/100

Feel disappointed, couldn’t finish on the 4th day already :/ I downloaded an STL model. Tried importing into threejs with three-stl-loader, didn’t work. Then tried converting in C4d to threejs, didnt work. Then tried converting in Blender to JSON, works!

NOTE: .STL, .OBJ can be huuuuuuuuGE

Then I faced an async challenge. I coded a wrapper around the threejs.JsonLoader so a promise is returned when the file has loaded.

Then I tried deleting vertices from the model, but that resulted in having a blank screen and Jeroen taught me that I needed auto remeshing/topology of my imported model. Sooooo here’s where I got stuck. Still looking for one.

These dailies are absorbing my whole day. Maybe it’s too ambitious of an idea?

try harder: Started with 004 again and spend about 3 hours fixing it. (actually day 5)

Found some resources how to draw dynamic models instead of finding an auto-remesher (still no clue if this exists). Stackoverflow was the answer (as usual) leading me to this fiddle: drawing a range of a buffer geometry helped me do it.

oops the JSON file is 120mb big… DOH, too lazy to change it now. Next daily!

day 5 005/100

Blender → object mode → Object modifiers (wrench icon) → add ‘decimate’ modifier → crank down ratio while keeping an eye on the quality + verts/Tris/etc in the top bar. From a 120mb → 12mb! (Though the effect is a lot less cooler)

Learned how models can be pushed in a group(array like structure) and then be iterated out of it for performance (is my guess).

day 6 006/100

My first shader. This one has cost me a few days at least. The pipeline of vertex and fragment shaders. Using frequency values and the mouse coordinates to send values into uniforms. For some reason couldn’t work with resolution… whatever. The different kind of variables you use in shaders. Vertex displacement. Other topics to check: noise, rays (casting/marching/whatever), fractals, domain warping, learn how to implement shaders from shadertoy into threejs.

day 7 007/100

Tried to add a texture to the custom shader.. hm didn’t work yet, but added as MeshPhong material and dealt with multiple promises (loading multiple files async). Tried adding it to the shader as sampled2D and texture2D() but got a weird “ERROR :GL_INVALID_OPERATION : glDrawArrays: Source and destination textures of the draw are the same.” error.

day 8 008/100

Aaaand added a texture to the custom shader.. hm not sure what to do with this now. I’m sure I can use certain coordinates and play with them. Can imagine some stuff with vertex displacement coordinates…

day 9 — /—

Tutorials from the mighty Spite: Shader basics talk, Mapping shader, vertex displacement with shaders. Was already 2am so decided to skip this day. What you’re skipping a daily? OMGWTFOHNOES

todo: check MIRA 2013 stuff

day 09 009/100

Created a bunch of random vectors with perlin noise driven shaders and inserted them in a geometry object. Added the faces and ended up with some weird ass geometry in a cube form. Used a buffered geo and drawRange as usual with audioreactivity. Pfft, what to do next time?

day 10 010/100

4x40k particle fields, generated by simplex noise dancing on the beat in their own directive flowfield. Whut can I say.

day 11 011/100

read about dot product

All day trying to port over Daniel’s Shiffman attractors system for particles.. but I’m probably missing some concepts. Working on a ‘big’ idea while keep trying to spit out these daillies. So I got ‘lazy’ and decided to modify 010’s visual. Ended up looking fkin awesome!

lowfield. Whut can I say.

day 12 012/100

Kept working on the attractors system.. got something working and created a funny accident which I put online. Special supercursor powers!

Plan is to create an ongoing particle cloud shooting at the mouse. Learned today that however many particles you have, without collision detection they can all end up at the same vector and basically represent a single point.

Could this be supercool with morphing? Morphing from vector(0,0,0) to the shape intended?

day 13 013/100

Tried to rework 004 because the model is so big. I ‘decimated’ the model in Blender to kill a lot of vertices and then use subdivision to regain more detail. This guy was so nice to create a package on npm for THREE.subdivisionModifier but there’s none for BufferSubdivisionModifier. So hopefully one will come (sorry no time to do it myself — no clue how anyway) or reprogram template and remove all the es6 stuff (which would be nice and minimal). Spend easily 3 hours on this :/

After this failure continued with the previous experiments. A particle system with attractors. I combined it with the attractor from 012 and have a funny particle system that follows the mouse with some splashy effects caused by audio.

day 14 014/100

Played with the DAT.gui control settings. with a listen() on the variable you can feed the values back, which not only looks cool but also gives you a bit of feeling about how your controls work in different combinations.

Still working with the particles and attractors. Getting some really cool and weird flashy effects with the way I’ve set it up. Actually am not feeding any audio data into the particle clouds.. hm

day 15 015/100

Particles+attractors can create wonderfully beautiful compositions of movements. Just playing around with the values I found what you can find online and honestly there’s not even any audio data I use.

day 16 016/100

Can’t stop w/ the particles! Although there’s still this flickering which I think occurs from forces switching really fast between positive and negative values.. I haven’t been able to figure out how to fix this.

Asked around at ‘the coding train’ Slack (which you can access by supporting Daniel Shiffman on Patreon, check out his amazing tutorials on youtube!) and got some tips.. though the flickering is still there, I got some beautiful shapes to play around with! This time also slightly transforming with audio data.

idea: write a file with FS with logs from audio utility. Maybe I could transform this in some sort of pointcloud? 🤔

day 17 017/100

Back on track after few days of no productivity.

Went back to 004 and added some spinning, microphone function and other calculations based on the drawRange of the buffer geometry. Some really cool effects like oscillating the min and max values. I’ve been experimenting more with this technique, but it has some factors that depends it’s look. Like the order of vertices (no clue how to change the order of a model), the amount of vertices (the more the cooler, but that will add to more size) and of course the model itself. I think it’s not possible to create different materials while filling up a geometry with points. But worth trying out! 018 is next.

If you love what you’ve just read you could warm my heart 💚 by clapping for me or buy me a coffee ☕️

--

--