Animation workflow and rendering in VVVV

Hey! In this blog post will be about my workflow in VVVV. Repitch recordings, a Berlin based experimental techno label asked me to do video for their next release — a compilation celebrating their first 5 years. Since it is compilation, the track that I was working with was very erratic in style and atmosphere, so I used a lot of fast changing styles to capture every segment (each takes around 10 seconds). We settled on 60 seconds long track, give it a view:

My first choice in creating the video was Blender, but I ran into few problems. Two biggest obstacles for me was rendering time, and workflow with music. That is why after some tinkering I decided to move onto my “home turf” — software VVVV (for those who never heard of it, check it out on vvvv.org, you can download it right away!).

VVVV has great superpower, that is perfect for working with music — the amount of direct control and visual feedback for you work is unmatched. It is node based realtime programming environment, with textual development capabilities in C# and HLSL. The power this software has under the hood is mostly unlocked by the community, which is essential part in its usability.

So for my soup of video creation, I needed these parts:

• Core of VVVV + addonpack, which is basic installation.
• DX11 implementation, including great node that takes care of exporting the video — Writer (DX11.Texture 2d NRT Advanced) — it makes animation smooth and adds antialiasing and motionblur. I would say it is essential node for getting good animations out of vvvv. All thanks to Vux.
• Emeshe2, rendering engine made by my friend Dávid Mórász. Awesome thing that does not work right now for most people, it is precursor to the emeshe3 which should see the light probably sometime this year. It takes care of lights, shadows, SSAO, PBR, all the good stuff. Saying it is clunky would be understatement, but it gives a lot of freedom and feedback in experimentation, which is what I want.
• Instance Noodles, by words of the developer, Kyle McLean: “modular patching system for Compute & Geometry shaders in DX11/vvvv”. Great for geometry effects, generating lot of stuff directly on GPU.
• vAudio, audio engine and unlikely ally, which does very simple yet essential part of my workflow: plays audio, but not just simply plays it, but skips through it as i drag my cursor on timeline. It plays little snippets of audio, Adobe Premiere style, so I can pinpoint any event in sound, where the kick comes in, etc. You can do this visually, but being able to hear it takes it closer to working in Premiere and cutting video, which I like! by tonfilm
• TimelinerSA — standalone timeline solution for vvvv, its terrible choice on my part and by far the weakest link in this project. However it is not dependent on vvvv, so if I occasionally crash, my timeline work is never lost. In this case it worked out, but I would not used this again.

Recording in VVVV:

As mentioned above, Writer (DX11.Texture 2d NRT Advanced) is really great tool in VVVV. You can render these basic style animations with motion blur extremely fast, and have direct visual feedback as you are patching your animation, this is the patch that I used for generating this animation:

You can see I have a player node connected to the last Renderer. That is my node which takes stack of images from Writer folder, so I can see rendered preview of animation. Useful if you are doing strange stuff with your art and want to see preview as fast as possible to make adjustments.

I had to make my own recording system for the timeline, essentially when I hit record, vvvv needs to move the timeline and position in audiofile, so it can grab correct volumes for my audio analysis. Associating attributes for generating animations (geometry effects, constantly moving objects etc.) with position of timeline makes constant results during rendering. I can switch my current timeline solution with ease in future project, and if I will ever be generating something from Vvvv I will for sure. This is driven with OSC protocol.

Big thing that made me use Vvvv instead of Blender, is the rendering time. I can export this 60 seconds animation in +/- 2 hours. In Blender it would took me at least 8 hours.

Workspace:

One thing that VVVV has in common with Blender is ability to customize your workspace however you want. I have basic scene components and engine controls on the left. Middle bottom is the realtime preview and left of course timeline. On the top left there is free space for sketching parts of the scene, that I later group into subpatches. I like the freedom that comes with vvvv canvas, it almost feels like drawing. Top middle is open for different expanded subpatches.

It could be much better, I miss something like layers on canvas, that would allow me to have 2 separate layer that interact together but I can switch between them, would help the “cable clutter” a lot.

Audio integration:

Interweaving visuals with sound is the best part of Vvvv. Using math with combination of audio analysis makes many things simpler. For example, adding energy from lower frequencies to light intensity on my main light makes everything pulsate to beat without me having to manually animate that stuff. I have to note here this is also possible in Blender, but it is way more tedious process compared to Vvvv.

You can see FFT audio output on the right top and bottom monitors. Energy of sound is moving vertex noise effect on torus, etc.. I did manually wrote kick drum triggers to the timeline, and they are triggering the animation on top left.

Art:

I have developed the art style used in this video year ago, for event I organized. To give little bit of context, it was series of events, I usually did hand drawn stuff in illustrator that was heavily inspired by traditional graphic printing techniques. In last iteration I wanted to replicate this look in 3d, but I dont like look of simple toon or post processes that emulates this.

So I opted for using combination of toon shader with normal maps made from noise stretched in one direction. This give impression of strokes in linocut technique. This technique is very quirky and I have to tinker a lot with setting and scales of normal maps but it generates nice results. I use a lot of generated normal maps, modifying simple noise texture to make the end result, that is why Blender Cycles was great for coming up with this technique, it gives a lot of freedom to experiment.

I like the way how Vvvv and Emeshe forces me to think, trying to get the most out of simplest scenes. I am using one main point light to play around with shadows and 4 spot lights to make some basic lights in the middle of the scene. This is how this specific light setup looks in emeshe:

This is the point where Vvvv interface is getting a bit chaotic. Vvvv is not designed with object oriented programming in mind, but devvvvs are working on new version of the language called VL, which should address these problems and give more power to user over which code is running and dealing with objects.

My next blog post will be about my work with real time visuals, my approach that combines game design with VJing, and my usage of steam controller, so stay tuned. I would love to get more blog posts like these out of vvvv community, so people would talk about their work. Thanks for reading!