Federation, Privacy, and User Experience

So Moxie Marlinspike has engaged in some self-justification about why he’s building yet another lock-in silo. He has to capture your IM metadata because, well, it’s for your own good. It probably hurts him as much as it hurts you. Well, obviously not, because you don’t reap the profits of the vendor lock-in, but you know what I mean.

Most of the supporting arguments for why he has to capture and store the details of every conversation you make, if not the content, are reasonably sound, but his conclusion is flawed, short sighted, and self-servingly unambitious.

He says that it’s impossible to build a federated system that is agile enough, and provides sufficiently high-quality user experience, to be competitive.

First, I agree it’s really hard, politically, to engage a bunch of developers to produce a coherent vision of a service if they’re not under one paycheck — that’s undoubtedly true.

Secondly, the average user’s desire to want to see who, of their contacts, is on the system they’ve just joined — and be visible to subsequent new joiners — means that we’re almost bound into sharing phone numbers — and worse, the searches for those of your contacts. There are research projects to prevent this, but they’re hard (and interestingly, don’t apply to silos).

And last but not least, users have an expectation of an almost thoughtless on-boarding process, making options such as “Pick from this list of servers” unattractive.

Moxie’s response is to declare these problems not simply hard, but “impossible”. He declares that if it were not impossible, organizations like the XSF would have the problem solved by now. But maybe the impossible just takes the XMPP community a little while longer to solve…

What could we do, right now, to address these complex sticking-points?

Let’s start at the end. Clients can make onboarding remarkably easy, and Conversations has demonstrated this to no small degree. We can probably make this easier by defaulting to jurisdictionally local servers, detecting servers on the local WiFi, and so on. Some of this requires supporting technical standards, but actually the bulk is achieved by client developers simply making nicer clients, and server developers and operators working to produce better servers (since you want to have the latest bells and whistles, right?).

That leads us neatly to the first point — how do we engage a bunch of disparate projects, many of which are Open Source (in the sense of “you may use our source code to build something else”, not as in Open Whisper Systems), to pull in roughly the same direction to achieve a common goal? Most industries of this nature have used a certification scheme. It doesn’t have to be amazingly complex or intricate, but I think we could easily enough string together a set of basic criteria for a trademarked label based on what specifications are supported. I say “easily” — it is of course fairly difficult, but it’s hardly impossible.

So what about that contacts thing? And here I despair, for of course we’ll need a central directory server that matches the searches to results. I mean, we could of course shard phone numbers by country, operating them independently such that the directory server operators were limited in what metadata they could collect, and while that mitigates the problem it doesn’t eliminate it. And of course, the user’s own server can mediate the searches, such that the directory services only know the domain of the request and not the user, limiting the resolution of the metadata gatherable. But the problem is this still has to be hosted somewhere and paid for. And the XSF could only, realistically, afford hosting for several years if we don’t get more sponsors — which we do, year on year.

Of course, the XSF would in principle be able to collect any metadata from all the servers, if it was footing the bill. In order to allay fears, we’d have to be able to demonstrate that the XSF had funded and supported an important service for years without intrusion. Like Jabber.org, for instance.

Out of this work, users would gain choice in who their service provider was. Choice in which client they use. Choice in whether they shared their phone number, or searched for contacts. Choice about whether they used Tor to connect. Choices, choices, choices.

Eliminating choice is certainly easier. Removing options simplifies the problems enormously. It yields the desirable property of the homogenous system, operated and under the full control of a single entity, which has its best interests at heart — and yours, too, if they happen to align.

But diversity is as good for computer networks as it is for human ones — a security flaw rarely affects every implementation of a standard (though it happens occasionally), and internal competition often yields some impressive results.

So while it’s not easy to go this route, it’s not impossible either. If you’re up for a challenge, come join us.