Hey Ben, this is Richard, the guy who wrote the blog post that kicked this all off. Thanks for taking the time to be thoughtful on this topic. I’d like to just contribute a few observations.
You’ve hit the nail on the head with the trade-off between the average and the weird. Mozilla and the other folks building tools that support big parts of the web have to think about how the web is used most often, and make it work for those cases. But obviously, having a web that enables permissionless innovation is how we got the awesome web we have, so we need to make sure we don’t break that property.
A lot of the conversation on this issue has been wrapped up in the way HTTPS tends to rely on “registries” (“CAs” in the jargon). People have tended to forget that the browser will trust whomever the user tells it to trust. If you want to use self-signed certificates without scary warnings, every browser on the market for the last 20 years has had the ability to do that — you just have to tell the browser what’s coming. There’s no inherent contradiction between HTTPS and the trust model of your choice. Indeed, WebRTC uses the same security layer with only self-signed certificates.
The deeper issue is that we don’t really have an intermediate point here between registries on the one hand, and on the other hand, having the user manually tell the browser what to trust in every instance. It’s a problem that’s pervasive in security — if you want a trust system that scales, there’s a need for centralized authorities. I would be delighted if there were a way to engineer around this obstacle. (I’ve heard some rumblings about blockchains being useful in this space; maybe there’s something there.) Hopefully, all of the negative energy in the discussions on this topic can get channelled toward developing solutions, so that we can get what the web needs in terms of both security and flexibility.
As I’ve said in some other threads on this topic, I’m under no illusion that HTTPS or the CA system is perfect. But to quote the great sage Mr. Rumsfeld, “you go to war with the army you have, not the army you might want or wish to have at a later time.” Our long experience with HTTPS shows that it’s strong enough to carry the web, and it looks like its weaknesses can be patched. Which is enough, at least for me, to get the movement started.