Christopher Lemmer Webber on MediaGoblin and ActivityPub
In May of 2011, Chris Webber announced the GNU Mediagoblin project — a publishing platform focused on images, audio, and video. It can be compared to the likes of Flickr, Soundcloud, DeviantArt, or Instagram. As it developed, Chris became instrumental in pushing the W3C ActivityPub Protocol forward.
Thanks so much for joining me, Chris. In your own words, please introduce yourself!
I generally consider myself a “user freedom advocate”. This has a lot of permutations including general free/libre software and free/libre culture things (once upon a time I was software engineer and then tech lead for Creative Commons), though I suppose starting about 7 years ago my focus started to shift to network freedom things.
You probably best know me for starting GNU MediaGoblin, a decentralized media hosting platform (think decentralized YouTube or Flickr or Soundcloud, etc) and for being co-editor of ActivityPub, a W3C standard for federated social networks.
I guess the latter has even eclipsed everything else… recently ActivityPub was published as a W3C Recommendation, and a bunch of projects have picked it up, most famously Mastodon, which gets the most attention because of its wide userbase. I’m very happy that we’re starting to see a bunch of projects pick up ActivityPub… I think nobody, not even Mastodon’s development team, wants a monoculture. Happily I think things are moving in a good direction.
When did you initially become interested in federated networks?
Who here remembers StatusNet and its flagship server identi.ca when they were huge? Well I do, that’s how I got started here. The fediverse at that time was incredible; some of the most formative conversations I had on user freedom and licensing and all sorts of cool topics happened during that era.
Diaspora also appeared during the middle of that, and while we didn’t actually see interop between StatusNet and Diaspora, it was clear that interest and conversation about “oh shoot, we’re handing all our computing and data over to these large centralized systems… how do we fix that?” was a major topic of conversation. (The Franklin Street Statement, which came out around that time, also had a big influence on me.)
“At the time I was really loving using StatusNet for microblogging, but I was doing a lot of artwork which I wanted to share with friends. I was troubled by the growth of YouTube, Flickr, DeviantArt, etc as mega-hubs of media, and I thought… maybe I could build something here for myself, and others could use it too.”
I mentioned this on the fediverse and there was an immediate and large amount of interest. I had already registered mediagoblin.org for MediaGoblin as a pun on “gobbling media” and had written up a rough game plan, and having announced an IRC channel, way more people started showing up and expecting things and talking about things like they may happen. (I should note that I’m *co-founder* of GNU MediaGoblin… Will Kahn-Greene and Deb Nicholson were also co-founders, and early on Matt Lee hosted the web infrastructure for us, though we later moved on to our own stuff.)
So yeah, MediaGoblin grew fast, and at the time I always figured we were going to adopt OStatus because that’s what StatusNet was using, so I said, “MediaGoblin will be federated, and we’ll use OStatus”. OStatus wasn’t/isn’t a single standard though; it’s kind of a meta-standard, and so I printed out a whole bunch of documents and as passenger on a long car ride I was trying to make sense of it, but it was a lot of new ideas for me.
But after the StatusNet company shut down, Evan Prodromou announced that he was going to switch development from StatusNet to this new project of his called Pump.io. I started using it… the application wasn’t as polished as StatusNet from the perspective of the web UI, which was kind of a bummer. But I heard it had a new federation (and client-to-server) API, and Evan had written that up in API form.
It was immediately clear and made sense to me structurally… passing around json objects, actors have an inbox and an outbox, okay great. Plus, Evan introduced me to Jessica Tallon who had started working on a library called PyPump. Realizing that she understood things better than she did, I encouraged her to apply for Outreach Program for Women (now called Outreachy) and to start working on federation support.
Later we ran a crowdfunding campaign (well, a second one) to hire her to do this work. During the middle of that the standardization effort for ActivityPub started… but I’m getting ahead of myself.
Unlike quite a few federated platforms in the space, Mediagoblin is a GNU project. What was it like to create this system under the GNU umbrella?
GNU has been good for us generally. There are a lot of ups, and some constraints that can sometimes feel frustrating, but usually have good reasons behind them. One thing that’s good is the ideological clarity that GNU brings… software freedom all the way, and the user’s freedom in general is always front and center. I really appreciate that. Plus, being under the stewardship of the FSF meant we could use their fundraising infrastructure for the crowdfunding campaigns we ran.
“I think it was the right decision for MediaGoblin. I think MediaGoblin is philosophically aligned with GNU, and I believe in GNU’s history and core ideas, and being under the FSF’s stewardship has been great.”
GNU also has a lot of rules that are related to its history, such as the requirement to have info manual output and to have autotools like ./configure && make commands. Some of these have been harder for us to fullfill because they were designed for GNU as a system that was mainly centered around (GNU’s Not) Unix’y C (and sometimes Guile) original world. However I will say that GNU has made some attempts to update to see if these can be more easily accomodated.
What does the landscape of federated communication look like in 2018?
“Federated social web work tends to have peaks and valleys, but I think we’re at another peak.”
Well a lot happened between then, most famously that ActivityPub became a W3C Recommendation. Since then we’ve seen an increasing number of federated applications picking up ActivityPub.
This makes me fairly optimistic, because my primary reason for working on ActivityPub was concerns about a “fractured federation”, on which I have more to say below. We’re seeing real work on interoperability too, which has me optimistic.
Mastodon and Kroeg worked pretty hard together to test the standard against themselves, and nowadays we’re seeing Pleroma, Rustodon, and actually quite a few others picking up ActivityPub and getting real users. Yesterday I just learned about Funkwhale using ActivityPub, which you covered here.
Federated social web work tends to have peaks and valleys, but I think we’re at another peak. The last time I saw this amount of activity was in the StatusNet / Diaspora heydays of 2010–2012 (more happened with both codebases and communities since then, but I consider that timeframe a public interest peak).
For a while I was beginning to fear that the world was becoming complacent with the status-quo of Twitter/Facebook, but happily that seems to not be true. There’s a lot of reasons to be hopeful!
You are listed as one of the co-editors of the ActivityPub protocol standard. Could you tell us a little more about what that effort looks like, and how you got involved?
“There’s a lot to it… you have to reply to *every* issue that’s raised and either satisfy the person who raised it or clearly document why they weren’t satisfied, and any non-satisfied responses get reviewed.”
Sure! So where I last left off, we had hired Jessica Tallon to work on federation by implementing the Pump API. I saw Evan Prodromou make an announcement that he was co-chairing a new W3C Working Group on federation standards, and I messaged Evan asking if I could be involved since we were currently implementing federation support.
I figured we would just show up for an hour a week, since I didn’t consider myself an “expert” at the time… we would just be there to keep things on track. Well… of course that didn’t happen. The vast majority of my time for three years ended up being involved in ActivityPub standardization. However the first few months for me were spent around trying to get every federated social network to join the working group, because I was a big believer that this was our one opportunity to get things right. Not everyone did join (though some projects like Diaspora had representatives for a time but not for the entire process), but I do think we did manage to gather feedback from all the big federation projects.
There’s a lot to it… there are a lot of existing processes, and often times they seem annoying, but they do result in a better and more clear standard. You have to reply to *every* issue that’s raised and either satisfy the person who raised it or clearly document why they weren’t satisfied, and any non-satisfied responses get reviewed.
“The vast majority of my time for three years ended up being involved in ActivityPub standardization.”
There’s a multi-stage process from standards where editors work on their own Editors Draft, then things move on to a First Public Working Draft, a Working Draft, then when things get close finally a Candidate Recommendation, and then hopefully Recommendation! There’s a bunch of other steps in-between… most notably a test suite and gathering implementation reports. And that’s not to mention the couple of months we spent on use case analysis.
What projects and organizations are involved in AP’s development?
The Social Web Working Group wrapped up its standardization work, but a lot of work continues, and we are continuing that work in the Social Web Incubation Community Group, or SocialCG for short, of which I’m co-chair. So while ActivityPub reached standards recommendation, there’s plenty to do.
Here’s a shortlist of things I think are priorities for the SocialCG:
- Most importantly, the SocialCG is a space for implementors to talk and collaborate and to incubate new and interesting things.
- More work on anti-abuse and anti-spam tools.
- Decentralized identity. For instance, Hubzilla has “nomadic identity” in that posts are really tied to your public key fingerprint, and in a sense that’s more your username than anything else. But it would be good to be able to rotate your key, so Decentralized Identifiers are interesting work in that direction. (I actually do some work on this through my job at Digital Bazaar, though it isn’t yet connected to federated social networks.)
- Merging peer to peer networks and federated networks. I see that some people have started exploring running ActivityPub instances over Tor onion services. More stuff like that!
- I think there’s a ton to be learned from the object capability security community. Petnames, etc etc.
Why work on a new protocol?
Well, as I mentioned before, I was very worried about a “fractured federation”, the fact that we had many federated social networks but they couldn’t talk to each other. I hoped that we could help alleviate that through a standards process.
As for why not OStatus, it’s worth noting that ActivityPub’s core design also comes from Evan Prodromou’s work on the Pump API, and Evan was largely responsible for OStatus. Email-like addressing (this better enables private communication, which OStatus didn’t really support), a clear but extensible vocabulary, and a closer conceptual connection to the actor model (is that bit too academic?) I think are all solid reasons for moving to ActivityPub.
One thing that I think is a bit underexplored currently is ActivityPub’s client to server API, which is very very similar to its server to server (federation) API. If you’re starting from scratch it’s likely easy to implement one and get the other at little extra cost. If we see more applications integrate this, one cool thing is that you could borrow any application’s frontend or mobile clients for another application’s backend. That could be really powerful.
Within the last year, Mastodon made the decision from v1.6 onwards that they will support the AP protocol. Are there any other platforms on your radar that are implementing it as well?
The list has grown quite a bit! Aardwolf, Bridgy Fed, distbin.com, dokieli, Funkwhale, Hubzilla, Kroeg, Mastodon, Nextcloud, Numaverse, PeerTube, places.pub, Pubstrate, Smilodon, tags.pub, Pleroma, Pump.io, and Rustodon are all either implementors of or are currently implementing ActivityPub. Seems like a healthy list!
What’s your impression of all the new ActivityPub implementations coming into the fediverse?
I mean, I couldn’t be more excited. People are implementing and working together and I’m not even always involved, which is a good sign.
The first time I saw Peertube and Mastodon federating I was nearly hopping up and down with excitement. (That wasn’t the first interoperability test, but it’s the first time I didn’t even talk to one of the implementors at all… Peertube just moved forward with implementing on their own, and I think Mastodon’s lead developer Eugen Rochko was the one that convinced them and helped them to do it.)
What features of ActivityPub are you most excited about?
I mean, I’m most excited about seeing interoperability actually happen. Aside from that, I’ll say that I think the extensibility model is quite good, but most nerdily I think that ActivityPub being an implementation of the actor model and mostly being self aware of that fact is good.
Likewise I had no idea that Funkwhale was implementing ActivityPub until they were well on their way!
Will MediaGoblin support the AP protocol?
So! First off I want to say that none of this work would have happened without MediaGoblin. We originally joined the SocialWG because MediaGoblin needed and wanted to use a federation protocol that was going to last. And before that work even started, we actually *had* federation support, in a branch… and it even federated with Pump.IO! That was all Jessica Tallon’s work and she did a great job.
“MediaGoblin and the federation branch both diverged quite a bit, and work on reconciliation needs to happen to get that merged.”
Unfortunately the SocialWG took up way more of my time than I expected and I became absent from MediaGoblin. Boris Bobrov very kindly took up maintainership of MediaGoblin while I was gone and other people kept sending in patches. But I do think things slowed.
Unfortunately, there were two major divergences: MediaGoblin and the federation branch both diverged quite a bit, and work on reconciliation needs to happen to get that merged. Worse yet, the Pump API and ActivityPub also diverged, so even if we got Jessica’s branch merged as-is, there would still be a lot of porting to do.
“It’s hard for me to not be struck when looking at Peertube that maybe Peertube is in many ways a better MediaGoblin.”
On top of that, it’s hard for me to not be struck when looking at Peertube that maybe Peertube is in many ways a better MediaGoblin. That’s not condemnation of our work on the project, but rather that when MediaGoblin started there was no such thing as IPFS, no such thing as Webtorrent, and applications were written in a very different style in terms of concurrent programming than they are today.
But on a positive note, Peertube is managing to federate amongst itself and Mastodon using ActivityPub, which wouldn’t have made it all the way along the standards track without MediaGoblin’s use case. I’m not trying to claim credit for all of Peertube’s hard work… clearly, that’s on them… but it does make me feel a bit better about “oh right, we helped contribute to make this standards infrastructure exist so that others could build on it.”
Now that the ActivityPub spec work is done, what are you working on?
Well I’m chairing the SocialCG and I’m working at Digital Bazaar on some adjacent stuff on decentralized identity and verifiable credentials. But obviously what everyond really wants to know is… what’s the plan for MediaGoblin?
And the plan is, from my end… I’ve started a new project, which I’m currently calling “Spritely”. Mostly for now it’s fairly sparse… I’ve discussed it a bit on the fediverse, but I’ve avoided any sort of major announcement because it’s not really ready yet. I guess this is the first major public announcement. You heard it here first, folks!
So what is Spritely? At the moment, vaporware! What will be Spritely? I can’t prove anything about what it will be until it is built, but I can state its goals:
- An ActivityPub implementation, both server-to-server and
client-to-server - Written in the excellent Racket programming language
- Actor-model-centric and object-capability-aware
- A simple client will ship with it
- It will have media processing infrastructure, such as in MediaGoblin
- It will probably eventually have a distributed multiplayer game, which sounds crazy, but I have my reasons
- Focus on being easily and reproducibly packaged in fine free software distributions such as Debian, Guix, Fedora from the beginning. This means sacrificing integrating with more difficult-to-package tooling, such as anything that uses npm.
- An emphasis on a friendly and collaborative development community. Once I get enough written where I can get others to pitch in, that is. This is something we did right in MediaGoblin and I’d like to continue that here.
- I’m hoping to have a one-command-line-instruction set-up-and-go deployment for people self-hosting over something like a Tor onion service.
From a technical side, a lot of things about MediaGoblin’s architecture were difficult, partly because we started with ye olde request-response Python webdev infrastructure, because that’s what I knew. For instance, asynchronous task processing was supported by something called Celery, which is nice as far as things for Python which are available goes, but was always kind of a pain to set up and maintain and always felt like it existed as a separate entity from the web application… which is a strange thing when processing media is a major part of your design.
So the actor model approach matters. I did several experiments or previous revisions (XUDD, 8sync) that got me to where I understood enough to feel confident that this is the right route to go down.
So what happens with MediaGoblin? That’s a conversation I have to have with the MediaGoblin community. I’d be happy for this *to be* the future of MediaGoblin, but given that it’s a language rewrite I’m not sure if some people will be unhappy and will want to continue the existing codebase… which they’d be welcome to do.
One thing’s for sure, by the end of the year you should hear some more interesting things about it!
Is there anything you’d like readers to know about your work?
Did you know that I’m a humanities person by training? I thought there would be no way to ethically have a career doing only free software, so when I was in college I decided “I’ll become a writer and an artist to pay for my programming.”
Guess I was wrong about that one!
Thanks for reading this interview! If you’d like to learn more about Chris, ActivityPub, or GNU MediaGoblin, check out the links below.
- Personal Website: http://dustycloud.org/
- Fediverse Account: https://octodon.social/@cwebber
- MediaGoblin: https://mediagoblin.org/
- ActivityPub: http://activitypub.rocks/
- Federation and GNU, Libreplanet 2015 talk: https://media.libreplanet.org/u/libreplanet/m/christopher-webber-federation-and-gnu-2b47/
- Standardizing Network Federation, LibrePlanet 2018 talk: https://media.libreplanet.org/u/libreplanet/m/standardizing-network-freedom/
