Montage: Open Source Video Annotation at Meedan

ed bice
5 min readNov 7, 2018

--

Montage at Meedan

It is with a great deal of enthusiasm we share the news that Meedan will be releasing our third major open source product, Montage, and with this our first formal partnership with the Good People behind the Bad Idea Factory. Those historians of open source video tagging and investigation software among our readers will recognize the project as one of the first efforts developed by the team at Jigsaw and will know it has been housed over the past few years at Storyful.

Tagging snippets and timestamped commenting in Montage

When Mandy Jenkins reached out to us late last year with word that Storyful was looking for a new home for Montage we were very interested and a bit terrified. Taking on a new code base you have not written and that has not been under active development for three years is like adopting a feral cat.

So, our initial reaction was: ¯\_(ツ)_/¯

Much of Meedan’s work over the past seven years has focused on our collaborative verification tool, Check. Check began as a tool for adding a verification layer onto the web, first deployed for the 2012 Egyptian Election and then refined over many years to serve as a collaborative workbench for digital verification efforts like Electionland, Documenting Hate, and, most recently, the OJA winning Verificado project in Mexico.

While Check excels in event based verification efforts, for the past five years Meedan has also deployed Check for human rights media annotation, working with some of the most talented open source intelligence (OSINT) researchers on the planet, including Bellingcat, Syrian Archive, Witness Media Lab, and the Berkeley Human Rights Center. Much of this human rights investigation work has focused on video annotation, tagging, and analysis.

While we have done some bespoke development on Check for video -most notably a partnership with Brave New Tech on Keep a tool integrated with Check that enables video archiving to a variety of endpoints (including archive.is, and archive.org) — our long-standing vision for developing a better, open source workbench for video annotation, transcription, and translation never managed to make its way from our dreams into our roadmap.

In the interest of turning this blog post into something more beneficial to the inter webs than thinly veiled PR, I will share a bit about the process we have gone through over the last several months as a so-you-want-to-adopt-an- unsupported-open-source-application primer.

Meedan’s Excellent Checklist for Evaluating Adoption of an OSS project

  1. Does the software serve a community of users you care about?
  2. Are the current users actively using the software to solve problems they cannot solve with other software?
  3. Is there a hero team who is willing to work directly with your team to refine the product and use cases?
  4. Does your team have the skills and capacity to maintain the software?
  5. Do your current users stand to benefit from this software?
  6. Does the software stack align with your current stack?
  7. Is there potential to add features to the product that will meet an observed market need?
  8. Is there a market/feature fit with your other software products?
  9. Is there or could there be strong differentiation from other open source or proprietary tools?
  10. Is there an active open source developer community around the tool?

Scoring:

8–10 = proceed to negotiations

4–7 = you probably shouldn’t adopt this kitten

1–3 = run, as fast as you can, away from this rabies infested hellcat

As we started to dig into the evaluation we were at something of an advantage in that we had advised Jigsaw early in the life of the product — back when it was graced with a name from this East Bay punk-lite soft-goth band — and were on the shortlist of companies to receive Montage. And, yes, it did feel a bit like a this song when we were told it was going to Storyful.

YouTube screencapture from Greenday’s Boulevard of Broken Dreams

We had the benefit of our initial feature and business planning from those early days. At that time, though we loved the UX, we had dreams (unbroken) for using this same segment selection and annotations UX for transcripts and translations, a major new feature direction for Montage, to be sure. However, our collaborative translation tool Bridge has given us a lot of UX insights (and technical/design assets) and having a way to express those in video is a very natural extension of that product. Video translation and tagging is a also a good market fit with our current user base (for human rights investigators and investigative journalists, the ability to organize a large set of evidentiary videos is a critical use case).

But, what of the market need and sustainability considerations? Well, increasing linguistic diversity and range on the web will be a major need for the next billion users, and with this growing diversity there will be increasing pressure to develop technologies that help move content and conversations more efficiently across languages. Validating this, media scholar Clay Shirky (NYU) recently shared his assertion that there is a significant, unmet need in global educational circles for better video translation, tagging, and annotation tools. NYU has a global footprint and needs a way to enable users/viewers/creators to move flexibly between source language audio, target language transcripts, target language audio, and third party annotations (of words/phrases/ sentences).

At the end of the evaluation period we scored eight checks, only missing question 6, and question 10. Stack incompatibility can be solved with clever engineers and resource to rewrite — an understood and solvable risk. Finding open source developer community to come behind the transcription and translation feature effort, though, was a bit of a sticking point. Until I started doing some web research and discovered that the amazing team at the open source transcription project HyperAudio was connected to the Bad Idea Factory and by extension to our favorite faux frenemy Dan Schultz.

So, we are thrilled that as we work to bring a major set of features into Montage we will do so in partnership with a few of the hugely talented and solidly self-deprecating open source developers from the Bad Idea Factory: Dan Schultz, Mark Boas, Laurian Gridinoc, and Piotr Fedorczyk.

Though there were some tumbleweeds and dry rot, we saw a ton of potential to add value to the community of users we care about and we saw a path to broadening the range of annotation forms we bring to our users and we saw a way to build bridges to our talented open source loving frenemies.

If we frame Meedan’s broader work as building tools to contextualize digital content, Montage fills an important gap in our stack and will be a valuable addition to the empower human rights investigators, journalists, and translators who are working to make sense of the global web. Have a look around at https://montage.meedan.com/ or, share your thoughts, questions, ideas on this product to us at montage@meedan.com.

--

--

ed bice

working every day to make the web a bit wider and more worldly with colleagues @meedan