A Solution for a modular MIDI world
The recent surge in AU plugin availability (for audio and MIDI) on iOS opens up new modular ways of stringing musical tools and instruments together. This modularity comes with a million new needs for creative MIDI routing: the plumbing between plugins and controllers.
Suddenly there is a lot of demand for MIDI handling and manipulation plugins. Most of which is very specific to one particular workflow or even one single project. While it’s fun to make such MIDI plugins, it’s not realistic to expect developers to address every esoteric use case with a dedicated MIDI plugin. There simply aren’t enough of us to create and maintain products for every specific need :)
The solution: make more AUv3 MIDI developers instead!
Introducing Mozaic
I’m creating a new product, codenamed Mozaic, to make it as easy as possible to roll your own AU MIDI plugins, ranging in complexity from simple/basic filters to moderately sophisticated apps and generative plugins. Mozaic will be able to cover a lot of plumbing ground.
If Super Mario has taught us anything about plumbing, it is this:
- It can be a lot of fun
- There’s always a learning curve, but a well-designed learning curve is gentle and contributes to the fun challenge
These two insights have been guiding principles for my design of Mozaic as a tool that seeks to balance power, accessibility and depth.
So what’s Mozaic?
Mozaic consists of two main ingredients: (A) a very readable and flexible scripting language and (B) a programmable AU MIDI plugin container
The MIDI plugin part is like a MIDI controller with lots of features (such as ready-to-use LFOs, AU Parameters, host communication, tempo syncing, timer events, scales, programmable metronome, etc.) which you can tap into using the scripting language.
This means all the heavy lifting is done for you by the plugin. As with all AUv3, each plugin instance can run a different script.
The scripts are powerful and designed for MIDI and music applications; you can set up an LFO using a single line of script and get usable MIDI values from the LFO with just one additional line.
Writing a funky MIDI filter (e.g. round-robin distribution of incoming notes over several MIDI output channels) would just take a handful of lines of script.
About the scripting language
I’ve put a lot of effort into designing a language that’s powerful for MIDI applications and at the same very readable for humans. This means that you can look at the preset scripts — or someone else’s code — and get a good idea of what’s going on. And make it easy to quickly modify a script for a project you’re working on.
The scripting language is pretty powerful, because I designed it specifically around MIDI plugin use cases.
- It’s fully event-based (e.g. you can make things happen when MIDI comes in, when the host starts playing, when the user tweaks a knob, and two dozen other useful events)
- It’s easy to respond to MIDI, or to generate MIDI of your own (e.g. for generative apps, CC generators, or oddball sequencing concepts)
- You get easy access to AUv3 functionality, such as the AU host’s status and transport information, AU Parameters, host synchronization, etc.
- You can freely use (and dynamically label) all knobs and buttons on the provided plugin GUI to let your users interact with your plugin as if it were a MIDI controller device
- built-in LFO support, pedal support, musical scales support, etc. makes it easy to do interesting stuff with just a few lines of code
- No need to know anything about the dreadfully undocumented internals of Apple’s AU standard or how to write realtime-safe code
I’m also writing a programming guide, which should be easy enough for anyone to dive into and get started quickly. It will teach you the syntax of Mozaic Script, but also explain how to effectively use conditionals, nested loops, variable arrays, and lots of other useful programming basics.
In summary, Mozaic will not be for everyone and does not offer the instant-gratification factor of the next synth plugin. But I’m putting a lot of depth and value in it, just under the surface, waiting to be unlocked by the creative community.
Answers to expected questions…
Q1: A programming language, really?
A: I started the project designing a modular MIDI UI, which let you ‘wire up’ your process. It looked interesting, but in the end it made it quite difficult to get an idea of what was happening by just looking at the resulting schematic. I realized that a screen full of tangled wires and modules could often be summarized in 5 lines of pseudo-code. And the code was easier to understand, faster to write, easier to modify and required just as much algorithmic thinking. My conclusion: writing a quick script is actually easier and more effective than doing things the graphical way.
Q2: Programming is hard, isn’t it?
A: I’m making a very conscious and rigorous effort to design a human-readable language that’s easy to understand for most people — even if you don’t have any programming experience. I also try to stick to fundamental patterns and mechanisms that are common to other languages, so many things you learn in Mozaic may also help you if you ever move on to another language or if you’re experienced in with existing scripting lingos.
Q3: So is it like JavaScript?
A: a bit. It’s a blend of Basic, C and Pascal. I’m still finetuning the syntax. The interesting bit is that I have to make the language userfriendly for humans, but also easy to run in realtime by the Virtual Machine (the “code engine”) that runs inside the plugin. The VM runs on the audio-thread to make sure that MIDI handling is done with supertight sample-accuracy, but that also means it has to be as efficient and fast as possible at running your code, all while being realtime-safe and non-locking. As I mentioned earlier, the heavy-heavy-lifting is all done for you; but some parts of the syntax are optimized for realtime interpretation :-)
Q4: Can I create my own GUIs for my plugins?
A: No. The standard UI is what you get to let users interact with your code. You can obviously dynamically change labels on the UI via code. Think of it as a cute little hardware MIDI controller which is fully programmable and has timers, LFOs, and MIDI IN/OUT/THRU.
Q5: Will it come with built-in scripts?
A: Yes, absolutely. It’s my aim to ship with a lot of interesting pre-made scripts which will make owning Mozaic worth it even if you never write your own. Obviously, there will also be tutorial scripts to show basic concepts of the language in action. And I am very open to hearing from users about things they’d like to do with scripts, so I can add them as useful factory presets to the built-in library.
Q6: When are you planning to launch?
A: 2019. Building this product is quite a bit more time-consuming than my average synth or effect. A lot of work goes into building my own Virtual Machine and optimizing it for running inside the realtime audio thread. Yeah, that’s complex, but ultimately a fun challenge. But it also makes it difficult to predict when I will be ready for beta (and obviously creating a programming language involves a lot of testing too).
