Odot, Part 2: Human Beings Who Made It

Summer of { Work | Writings }, 2017: Notes on Projects.

I was asked by a friend about why I chose blogging over writing things up formally and submitting a publication. The honest answer was, initially, because of the instant gratification: I’m trying to write more, and the goal isn’t to write well as much as it is to write a lot. But there’s another good reason to write informally: being able to declare your sincere love for people and work that you’ve done.

This is a sentence that I wish was the cornerstone of academic writing: I love my work and I love the labs and research centers that were kind enough to welcome me. The past 6 years, in spite of the usual graduate school turmoil, allowed me to rub shoulders with some of the most brilliant, passionate, compassionate, and inquisitive people — faculty, staff, and colleagues — I have ever met. In trying to define what odot is on Monday, I realized that the story is incomplete without talking about the people who made it happen, so I am fixing that today. This is the person-centric take on odot, and a love letter to two of my CNMAT friends.

Adrian Freed

Adrian Freed: teacher, mentor, friend.

This is Adrian. Adrian is a friend of mine. He was the research director at CNMAT from 1992 until 2016, an engineering counterpart to David Wessel, a scientist. (And I will write about David when the time is right.) Everything about Adrian is interesting. For starters, he is a Englishman Australian Frenchman who lives in California. He has the largest library of conductive fabrics in the world, a project he started in the 1990s, before “wearables” was a word people would recognize. He also worked at IRCAM, and at Bell Labs, and at LucasFilm games, and at Pixar before it was owned by Steve Jobs, and at a bunch of other places. He wrote one of the first DAWs, called MacMix, where you could splice sounds together using a graphical user interface. He also co-authored the OpenSoundControl specification with Matt Wright. A saying that one “knows how sausage is made” falls short of describing Adrian: Adrian knows how sausage factories are built, for many different kind of sausages.

Sort of like Forrest Gump’s journey through major events of American History, Adrian Freed journeyed through histories of computer science, design, and computer music, so just imagine that JFK was Ken Thompson, or Larry Tesler, or Xavier Rodet.

Adrian is a wizard. During my time at CNMAT, Adrian showed me how to do multitouch sensing on fabric, how to make a slider and a button out of an unassuming strip of conductive paper, introduced me to a fascinating niche of paper and cloth folding enthusiasts, and showed me how to draw (with a special pen), a fully functioning loudspeaker. I can keep going. Adrian’s contributions to CNMAT codebase included a magical routine for generating many sinewaves cheaply— he wrote that bit of code in 1980s for MacMix. He also taught me about clever approaches to polyphony, functional programming, and about many, many other technical things. And I was just one of many other students hitting him up for advice, while others asked him for help with machine learning, sensing decomposition of compost, instrument building, speaker building, hardware, etc. And, somehow, Adrian also found time to do some remarkable work with analog circuits and play guitar.

But, perhaps more importantly, Adrian taught me about working in research labs and about life in general. His first lesson, delivered to me when I was freaking out about presenting my work at a technical conference for the first time, was: nobody fails at CNMAT. It took a long time to figure out what this meant: that the lab itself must be structured so that audacious projects can be brought to fruition. His second lesson, which took several years to learn, was that good projects were a byproduct of having creative people work together. He credits David Wessel with that approach, but I learned it from Adrian: “put enthusiastic people under the same roof, and the rest will follow.” My first semester at Berkeley I asked both David and Adrian as to what I should be doing, and both of them asked me “well, what are you going to work on?” It’s hard to explain, after 6 years of coming up with various answers, what it felt like to have to answer that question for the first time.

I’m not a historian. I also have been at Berkeley since 2011, and working at CNMAT since about 2013. So treat the following with the same kind of skepticism as you would treat a folk-tale: overheard things as remembered by people. Language and memories fail us, and often! But I still think that it’s worth writing some of these things down. Plus, the timing is right: judging from the dates in accompanying documentation, the first iteration of odot happened in 2007, which means that odot is 10 years old this year.

Before It Began: The OSC Objects

CNMAT’s OpenSoundControl Max object source code lists 1996 as its earliest copyright year. The first publication of OSC was in winter of 1997 at the ICMC (International Computer Music Conference). OSC specification was co-authored by Matt Wright (currently teaching & rocking things at Stanford’s CCRMA) and Adrian Freed, coding up OpenSoundControl was Matt Wright’s doing. The object is still available in the current release of CNMAT Externals (as of May 2017), but today it’s history log lists 23 versions, which means that the number of revisions is at least a square of that. (Other authors of it are omitted here for brevity, but such things are made by a village.) The most surprising part to me was the fact FullPacket used for OSC bundles originates with OpenSoundControl (although the versions aren’t dated).

In 1999, Matt Wright and Michael Zbyszynski wrote the OSC-route external. An important feature in OSC-route was OSC pattern-matching using a Unix-shell-like pattern-matching syntax. This allowed one pattern to access multiple places in the address space hierarchy, becoming the first take of textual programming using OSC addresses.

Was OSC Enough?

CNMAT has many research interests, yet instrument building is an important focus for working with live electronics for music-making. Our collection of various controllers includes all sorts of things: Guitar Hero controllers, WiiMotes, Microsoft Kinect, 3D spatial controllers, systems of sensors attached to Arduinos, custom hardware, third party music instruments hardware, and weird, unorthodox, silly things we were excited about controlling sounds with.

The problem emerging from using all sorts of devices is that making things talk to each other occupies a rather large chunk of development time. The underlying principle is simple enough to be boring: some data gets sent over the wire or over the network. You need to take that data, parse that data, and use that data by feeding it into your system. Our systems are, generally speaking, somewhat questionable constructions of sound-producing technologies that we are constantly tweaking. So the second issue is that of mapping the data from your controller(s) to some sensible controls of the sound synthesizers. The two steps for achiving sanity while working on such projects are:

  • represent all controllers’ data as OSC bundles;
  • learn to program synthesizers using OSC bundles.

The order above is arbitrary — both problems need to be addressed for a programming system to be useful: if you can’t process the data, the universal OSC is useless to you, and no matter how adept you are at transforming and mapping data from OSC bundles, you still need controllers to speak OSC to strut your OSC programming skills.

How It Began: Max Sketches

The first mention of odot came out from a set of Max abstractions by Adrian Freed. In fact, o. was from a prefix used in Adrian’s library, created for the Summer Sensor Workshop. (CNMAT is known for running workshops every summer.) Since this was before my time, I’ll let the official README tell you what those were (emphasis mine):

odot: OSC wrappers and tools for Max/MSP
This is an exploration of building fruitful musical programming design patterns into a library of Max/MSP abstractions. Some of these patterns are ones I have developed in my own programming practices, many are codification of years of practice at CNMAT drawing often on David Wessel's ideas and compositions.

The goal of the library, thus, is to introduce a set of programming paradigms, using the named values of OSC (in fact, many o. objects wrap CNMAT’s old OSC objects). It is remarkable just how much odot there is in that first iteration: o.route provides a delegation outlet for outputting a bundle of unmatched OSC addresses, o.build is the old name for o.pack, and o.show provides the first example of textual representation of OSC bundles (using tables). More important is the idea that bundles can be sent around to various subsystems of your patch.

Along with the examples there is the document that (thanks to Adrian’s permission) I am reproducing here in full. The remarkable thing about this is how much of a beacon this set of values has been in the decade of work that followed, considering that I haven’t seen this actual document until I started poking around in old repositories while researching this article.

Adrian Freed Max/MSP/Jitter Programming Style Guide

Use Native Max/MSP/Jitter/javascript/java native constructs wherever possible (in that order)
Minimize state
Minimize hidden state (i.e. expose state)
Use long Specific names except for a few frequently used constructs
Avoid send/receive
use qmetro, defer etc. wherever possible
provide help files
build families of patches with a shared interface
Minimize the number of features in a single patch, create a suite of orthogonal patches
to avoid feature bloat.
Use OSC, javascript compatible names (i.e. _ and not dashes, leading letter).
Avoid message boxes where possible
Remove debugging from sub-patches

Fun aside about visual programming: after three version iterations of Max, Adrian’s examples look almost entirely unreadable. They weren’t particularly cleaned up in the first place (the goal wasn’t to teach how to implement mechanisms as much as to provide a library of objects to program with), but after every conceivable change in the visual side of Max, the current rendering of patches looks nothing like what Adrian actually composed, so I will not include these screenshots here.

By 2008 (sourced from an email timetag), odot became one of several approaches of enabling object-oriented programming in Max (a notable alternative being Jamoma). Odot examples from Adrian’s original library (that I can not track over time due to history-erasing migration to Git) included a method for smoothing sensor data using o.average, a way of polling data, a way to generate OSC from Max messages, named o.fromMaxMessages, and a lot of support for sensors and actuators, which used the o.i and o.o prefixes for input and output. But before describing next steps, I need to first introduce another good friend of mine.

John MacCallum

John MacCallum: teacher, mentor, friend.

This is John. John is also a friend of mine. Although we were acquaintances for the first couple of years at Berkeley, our friendship was forced by a rather difficult and involved project involving the spherical speaker array, Kronos Quartet, and the Santa Rosa symphony. We spent two weeks living in a hotel at Rohnert Park, and we carried a piece by Edmund Campion through rehearsals and several shows before returning, quite exhausted, back to Berkeley.

John had a hand in most of the tools that CNMAT currently has to offer. I think the reason for that was that he genuinely liked writing Max externals, and he enjoyed programming in C. He’s also a wizard when it comes to slow-cooking pork and grilling meats of various variety. John is also a composer of some very good music. And what John taught me, above all else, is how chipping away at a something is pretty much guaranteed to yield success, no matter how complex or insane that something is when you begin working. (To be fair, I think we both learned this from Adrian.)

John’s projects include a tempo-curving tool that allows multiple musicians to hear different click-tracks that diverge and converge to and from synchrony. His recent work deals with controlling music by using bodily sensors placed on dancers, and in that work the dancers learned that they can control their heart-rate (and therefore the music produced) with remarkable accuracy. He also at some point decided that it would be a good idea to roll out his own expression language called odot, but I’m getting ahead of myself.

The summer before I started at Berkeley, I attended the Max workshop where John taught. Knowing I would have to start teaching Max in my second year, I asked John to repeat his lectures and then remorselessly stole many of the things he showed, said, and did. His understanding of the under-the-hood aspects of Max was an inspiration to dig deeper into the libraries I was using, and the languages I was programming in. But before I lose track of things, let’s get back to the history of odot.

How it Began, Part 2: o.call & o.callpatch

How John convinced Adrian that implementing odot in C is necessary is an unknown, because Adrian’s default answer to implementing things in C was a hard “why?” It is therefore most likely that John just did it without asking, because people usually don’t tell you not to do something after you’ve done it, even if that something was a bad idea. John recalled once, over a beer, that the deciding factor was that o.route with pattern-matching was “a very gnarly” bit of Max. Apparently, that’s what it took to convince John that doing it as an external would be wise. However it happened, it happened, and then didn’t stop happening until John left CNMAT for greener pastures in Paris.

From the first odot paper: o.call was the language.

The first publication of odot happened in 2011. The API was still very much Adrian’s (examples in the paper use o.build), but since the library itself was mostly concerned with creation of bundles from data and delivering these bundles to sub-patches that needed them, the mechanism for operating on data in OSC bundles wasn’t the expression language we have now. Instead, it was o.call which called Max objects and wrapped the transcoding to-and-from OSC within a nice interface (see image).

To this day, o.callpatch is shipped in the /dev part of the odot library. And to this day, you can wrap some Max functionality to be used within odot. It’s still probably the easiest way to implement a random-number generator in your patch (since odot itself does not have a random number library to avoid issues of mutable state — every codebox must return the same output for the same input!). But looking at the image from the first odot paper, it’s easy to see that o.call is a language for transforming data: an address gets the result of a computation. And, with conditionals and loops, it gets real enough to want bigger and better things:

Computation!

The expression language that followed (and illustrated, briefly, in my previous write-up) was very much an attempt to do away with Max vs. OSC conflicts, enabling OSC data to be computed directly. The first object was o.expr — which could only handle one expression at a time (similar to o.call). It later became an o.expr.codebox, but important things like support for lambda map over lists and conditional execution using if were already sorted out.

o.io, Plug-and-Spray, and Why the Work is Incomplete

To conclude, I wanted to point out that the problem of heterogeneous controllers remained a motivation for odot development. Although we played around with the language, implementing traditional algorithms in a new language and seeing what we can break and why, the main reason to use odot was due to its ties with OSC. It meant that we could send computation contexts around, both inside our patches and between machines. It also meant that devices we had had to be taught to speak an odot API. The idea popularized by Adrian was plug-and-spray — plugging a device in and using an appropriate o.io object would result in a constant stream of bundles. These bundles were explicit and complete snapshots of the controllers’ state. All of the data was sent all of the time. This approach was optimized to minimize the development time of prototypes that mapped controllers onto sound-generating modules.

As authors and users of o.io and of the o., we wanted to eliminate the barrier of getting the device to send data to a Max patch. We wanted people who were really good at making synthesizers and sound-making systems to focus on making synthesizers and sound-making systems. We wanted control structures to be simple, and self-describing. Our library grew, and we learned a lot about different protocols and formats that many controllers used. But efforts to complete the work of making any device spray odot bundles is, sadly, incomplete. As a generation turns over at CNMAT, it’s unknown if our best practices for making bundles representative will survive, but digging around the old repositories and papers succeeded in eliciting hope.

Odot was a thing that grew naturally. It started with “let me show you how to do this” and grew into “let me show you how to do this quickly” and grew into “let us show you how to do it well” and grew into “let us try to make it effortless” and “let us try to support as many things as possible.” It grew from a library of Max abstractions into a collection of C abstractions, into an actual programming language, capable of various things (and, yes, admittedly, still missing a lot of features, some unimplemented, others not yet discovered). Odot turns 10 on July 23rd, which is the date on which Michael Zbyszynski committed Adrian’s work-in-progress into the SVN under /trunk/personal/adrian/odot/ noting:

I just checked in a pre-alpha version of Adrian’s sensor class patches to SVN.

When I arrived to CNMAT, I had a handful of medium-sized projects realized. But at the time, I thought that my work was, somehow, not real, because it didn’t feel like work. I just played around with things, writing some code, making some patches, writing, recording, and producing some music. And at Berkeley, I realized that it was real all along: that from humble beginnings, like a collection of “sensor class patches,” there may sprout a programming language…

…Provided, of course, that you share it with others, and will be willing to listen to them when they tell you that the thing you made is actually super cool.

Happy 10th Birthday, odot!

References

Coming Soon!