Oct 21, 2019
Science is from Mars, Art is from Venus?
How to Collaborate Without Killing Each Other
Okay, you clicked on the title, so you know we’re joking not joking. We’ve all been there. Different people with different disciplinary backgrounds who are all super talented. You all see the unique potential in working together, yet when you roll up your sleeves and to work, it’s like mixing oil and water.
Just getting to this moment is a privilege. In many organizations, folks are so siloed (or have siloed themselves) that it’s tough just to get here. If you are lucky enough to be here, you might be dealing with some of these…
- You might as well be speaking different languages and learning each other’s is an uphill battle or there is no guidance for how or a lack of interest/empathy from parties to do so
- Your processes are completely different and while you’re excited by what you can do and why you want to do it, the how is like a desert mirage where schedules and decision points smear away into the horizon
- Last and most deadly: you come into it with the most dangerous of all things in life and work — assumptions. These lead you off track or to spin out, toiling in isolation, pushing you further apart instead of drawing you together
Sparks make fire
My first brush with these things was early in my career at Last.fm. I was still pretty fresh out of design school and knew very little about the field of machine learning. I’ll never forget the day we hired our first PhD in ML/MIR. Our initial conversation went like this:
Hi! I’m Hannah, so great to meet you! I’m the designer here.
Hi! Nice to meet you too… but… why do we need a designer?
Looking back on this (at the time, terrifying) interchange now puts a smile on my face. That turned out to be one of the most fruitful collaborations and career-altering connections throwing me into a world I adore — designing and shipping products with ML. If it wasn’t for everything I learned during that collaboration I’d never be where I am today, thirteen years later.
Spoiler: not only did we learn to work together, but we did some excellent work. When we parted ways six years later it was with friendship and mutual respect. Today, I still think about how special the whole team at Last.fm was.
Cut to: first day of figuring out how we were going to make videos look better at TRASH. We’re excited by the vision, we’re going to take mobile-phone quality clips and edit them together with the beauty of ML! And sparkles! And dope time-based effects! And… and yet almost immediately, minor chaos was starting to unfold as we tried to talk about how we would do it.
Step one. Deep breath
Step two. Check your assumptions at the door to the meeting room. Can’t stress this enough! Your assumptions (which are thoughts) are going to muddy reality (the facts) if you don’t!
Step three. Build empathy. Ask about each other’s process, about their language. Favorite questions to ask in their discipline, favorite articles and quotes, and prominent people in their field to read.
Gen supplied me with EW Dijkstra quotes (sidebar: also the name of her dog!); we talked about the importance of the question “Why?”, as the most fundamental question in science; and she told me about her love for labs and reading groups. I told her about the importance of running a solid critique; made her a private Instagram of artists for the science group to follow, and shared my perspective on potential methods (having been through it!)
We were surprised to learn that labs (where scientists get together to talk critically about their research) and critique (where designers and artists gather to give constructive criticism on a project) sounded almost exactly the same. We both agreed these meetings were critical for the advancement of work, and at that moment, LabCrit was born.
Those initial beginnings felt awkward and bumpy at times and this is normal. Don’t let the discomfort throw you off. You don’t get fire without a few sparks. And that is exactly what you are aiming for: you’re not just trying to make an amalgam of your disciplines (ie. you figured out how to mix the oil and water in a suspension), you are trying to generate a chemical reaction to create something wholly new.
At TRASH, we believe our success lies not just in combining art and science but creating an environment where these reactions can happen. After a ton of iteration, lots of failures and the beginnings of modest successes, we’re sharing our process in three stages:
- Designing a “creative treatment”
- Research and development
- The feedback loop of “LabCrit”
1. The Creative Treatment
In the film industry, a “treatment” is a short document the creator writes to share their vision with their collaborators. It usually comes before the script and is presented in a visual format so people can quickly get the gist of the movie. As Brian Ward puts it:
[A treatment is] designed to grab the attention and excite the interest of a producer, commissioner, director or other collaborator, and to ‘sell’ the idea and it’s creative and commercial potential. But it’s also a useful tool for us, as writers, to help us find the essence of our story, our themes and our characters… A good treatment should give us a sense of the kind of film we’re intending to write, the way it will be told and how it will make us feel.
We invented the term “creative treatment” since we’re not making full movies nor are we writing full treatments. Our goal with the creative treatment is to quickly show the team the vibe we’re going for.
Vibes are what we call creative treatments in the app. Vibes are a combination of editing/post-production, visual fx, audio, and time-based fx.
A vibe is what you change to make your video look happy, sad, or more.
We use user feedback, data, market analysis, noteworthy examples from recent papers, and our own creative intuition to guide our creative treatments.
We use private Are.na boards to plan our vibes. Shouts to that team for creating a distraction-less collaborative tool for planning our visions!
2. Research & Development
Once we have the beginnings of a creative treatment, science provides a framework for what’s feasible right now and what’s still a dream. It is critical for art and science to have a tight feedback loop and not be a linear process. The best outcome is when the collision of these two disciplines results in unintended yet inspiring outcomes — or as Bob Ross says…
Here’s an example: we have a creative treatment called “love” which includes shallow depth-of-field as one of its elements. (This is that classic subject in the foreground, out of focus background effect). This was our first stab at using computational photography techniques to generate AR-like effects on common iPhones like the iPhone 6 to simulate depth without depth data. Our science group began by using an Image Segmentation Convolutional Neural Network that’s available via Apple’s Core ML. This neural network helped us skip the need for a depth cam because we could automatically make a good guess about where the foreground object would be. Dylan (science) and Ela (art) collaborated on how to use this foreground object mask and other fun techniques such as image dilation, Gaussian blur, and Poisson blending (shouts to where Gen learned these things at Brown CSCI 1950 G: Computational Photography with James Hays!) Below is the first draft of this effect that they think is looking good! What do you think?
3. LabCrit
Last, and most importantly: LabCrit. An artist doesn’t just make a creative treatment and hand it over to the science team to develop. We needed a regular forum for a critical discussion as one group. We created an internal cross-disciplinary forum for feedback that we call “LabCrit.”
Labcrit melds the principles of the laboratory meetings commonly conducted by scientific research groups (“Lab”) with the methods of art and design critique (“Crit”). Any member of our team can show work, and the rest of the team offers constructive criticism and ideas.
These meetings are a space where goal-oriented thinking can be constructive, but where sometimes the most fruitful conversations can stem from letting go of a specific objective and simply following the most interesting ideas. For example, a video editor could share a mood board of an aesthetic we’re interested in, and science can comment on how to create it. Or, a scientist could share a particular technique from the computer vision and ML research community, and art could consider how that fits with their creative vision.
Whew! It took some serious effort to get to this place. We hope this is useful to you on your efforts to collaborate with people of different backgrounds and you get some chemical reactions happening 💥
- Han, Gen (& Team TRASH)
