Holochain: Reinventing Applications
Holochain is a framework for writing fully distributed peer-to-peer applications. It is not based on blockchain technology. Nor is Holochain a single platform like Ethereum. It is rather like a decentralized Ruby on Rails, a toolkit that produces stand-alone programs but for serverless high-profile applications. Different from blockchain alternatives and as shown in this benchmark (based on the Holochain prototype), it actually works and performs better than Ethereum by several orders of magnitude.
Although it is technically correct to call the things you create with Holochain applications, it seems to limit the imagination of people starting to apply Holochain because this term comes with baggage. I see a possible near-term future for the upcoming Holochain ecosystem, and I would like to share my vision with you and invite your participation and feedback!
But before I start I want to clarify some things and introduce the philosophy behind the somewhat unique Holochain approach, as downloaded by the bright minds of Arthur Brock and Eric Harris-Braun. I don’t intend to deliver a full explanation of Holochain in this post and rather refer to our whitepaper for an in-depth and formal discussion and to already working examples like Clutter, our fully distributed Twitter clone.
Edit: For a quick (8min) explanation of Holochain, I’ve just uploaded this video:
But I do want to focus on the main paradigmatic shift that it represents and the benefits I see if we apply this new approach not only to the underlying architecture of the network itself and the means of providing a distributed data-integrity layer but also to building applications and UIs that fit the same model.
TL;DR: Holochain introduces a paradigm-shift concerning the architecture of distributed applications. This is a long section and the central and rather pragmatic message of this post comes after. But if you haven’t grokked yet what we mean by agent-centric computing and you’re wary about our scalability claims and keep comparing Holochain to blockchain, this philosophical ex-course is for you.
So the shift we are talking about is going from a data-centric to an agent-centric architecture. What does that mean?
The common approach when modelling any kind of complex system through software is to implicitly assume some kind of omniscient perspective. The machine that runs the software is supposed to model facts. When data gets fed into the system we assume to know how something really is. Object oriented programming also carries the notion of being objective.
As long as software runs only on one machine operated by one user, that makes perfect sense. You don’t want the machine to have its own perspective on things— let alone a perspective different from yours. You want the machine’s state to represent facts — correctly.
Now, if we add other users to the mix — think multi-user mainframe computer with many terminals — the machine’s objective perspective naturally serves as a surrogate for the objective physical reality we are used to. It is a space. If I put something in this space it stays there for you to find it. We can exchange data by writing to files located somewhere in the filesystem and could build applications based on this very principle of having a shared (cyber-) space that holds data. We assume the existence of data independent from our own existence as a user. In this scenario, data is the primitive thing that sits at the root of our ontology. If needed for certain use-cases, the concept of a user gets mapped onto this by having data represent users, i.e. a /etc/passwd file or a Users table.
If we then add multiple machines to the scene it gets really interesting. Every computer creates its own virtual reality in which facts are modeled. Without further complexity or added means of communication, these realities do not magically mirror each other. They are intrinsically monadic. There is no objective layer that forces every computer’s memory to be the same. Which actually makes it difficult to create distributed applications — software that runs on several machines and maintains its integrity and coherence of all parts in order to achieve its purpose.
This, by the way, is a problem that does not get introduced by software nor computer networks. Everything I’ve said above could have been said in almost the same way about how our human mind works. In order to enable us to interact with our environment in a meaningful and intelligent way, we have to model reality. We are taking in inputs as empirical experience and creating mental models that guide our actions. These (more or less) true hallucinations are what we (implicitly, unconsciously) use to construct the reality that we experience within the monadic space of our consciousness. We do not have a direct access to reality as it is. Nor are we able to experience the construct of somebody else’s mind directly. Although this is an insight that has been around for a while (described two millennia ago by Plato’s allegory of the cave) and was included into academic psychology under the term Constructivism, and comes up in every spiritual teaching that deserves this label, it remains quite counter-intuitive to the naive, unreflected mind. There is this quality of is-ness to our experience of the world. We think we know what is true since we’ve seen it. It takes a good deal of personal development to transcend this positivistic level of consciousness that is ignorant to the fact that one’s own experience is partial and biased by a world of previous partial experiences that act as the base-vectors with which we triangulate (interpolate) this new data-set. In other words, the experience that constitute our human lives is determined by us, the observer, the subject, more than it is by the object — and this is so in a radical sense. We become the self-fulfilling prophecy of our unreflected beliefs.
This may seem like a philosophical digression, but it is actually crucial to Holochain and distributed applications because: communication forms the fabric of our society. We, as a species, are so much more capable because we coordinate. Many act as if they were one. And the basis for that is communication. Exchange of thoughts. Telepathy through small mouth-noises. Language, written words, radio, television. The internet is not just a network of computers, it is the latest and most efficient way for achieving mainstream telepathy. Without it, you could not read my thoughts as you do right now. But, the way we used it up until now is data-centric. Even blockchain is stuck in that same old paradigm. We pretend to use websites and applications as places where something is the case. We imply objectivity where actually everything we see in our Facebook stream is what Facebook’s servers (and thus the company Facebook) communicates to the client software that runs on our phones. It feels like an open, public space — while it actually is Mark Zuckerberg’s living room in which we meet. A room that has this strange property that every time there is an interesting conversation going on between some folks, messages from advertisers are magically modulated over the sound waves traveling through the air and holographic images appear between people. Magic that is possible only because what we mistake for an open space actually is content communicated from Facebook to us. Facebook is saying something and we are listening. And even worse, Facebook is whispering something different into each of our ears — a fact that can and is exploited with severe consequences.
Mistaking one agent, one perspective, for the truth™ is a common correlate of an oversimplifying data-centric architecture and something that implies or promotes centralization of power.
The web, as it was introduced by Tim Berners-Lee a quarter of a century ago, was meant to be different. Sitting on top of a fully decentralized Internet, the web constituted of HTML and HTTP was implicitly agent-centric back then. The idea was that every scientist (which was the main target audience) would run their own HTTP server to distribute their written thoughts and research results to everyone who wanted to read them. Nobody else but the author has the power (read: the ability) to change/insert/censor their words. The space in which this happens is URLs/domain names or as a fallback even IP addresses. Because these addresses are what identifies a person and their machine, right? Err, well.. This assumption is actually part of the reason why this model could not reach mainstream adoption as those Web 2.0 services like Facebook (starting with MySpace and many others) did a decade later. Not everybody was a scientist and thus not everybody a) had a constant internet connection (at least not until recently) as universities did early on and b) was able to set up their own web server and code up their own app. So all these Myspaces, Facebooks, eBays and Airbnb’s could deliver on a need that most of those days’ Internet users had (and still have) by creating a space in which people could upload their data in order for others to be able to retrieve it — without the need to run a server farm with expensive always-on connection nor learn how to code a rich and easy-to-use UI. Of course this came with the cost of the aforementioned centralization and power imbalance. But pragmatically speaking, it is what had to happen.
In an agent-centric world, agents — users, humans with their machines — constitute the basic root layer of the ontology. Data, then, is correctly represented by something an agent either perceives as input or creates (says) as output. There is no absolute, nor objective truth — as in true outside of the context of an agent claiming or believing or denying the validity of an expression. Objectivity could be approached by inter-subjectivity — when and where needed — not as a central aspect of the architecture like with blockchains. Every agent is free to act as they deem useful and is sovereign to make sense of the world (=everything everybody else is saying and doing) as it makes sense for their unique perspective and use-case. If that entails trusting other parties’ output in the process it is a conscious decision and not implied by the architecture.
To go from data-centrism to agent-centrism means to understand that data intrinsically always is contextual. It inverts the ontological dependency between agents and data. Instead of modelling users as a certain kind of data, all data is an output of some agent. There is no data without at least one agent that acts as the source.
Truly Agent-Centric Applications
The UI of Holochain apps is loosely coupled in that the Holochain run-time after reading the application’s DNA and spawning a node just serves a directory containing a web UI. The application’s DNA defines certain functions that are made accessible to the UI.
Now this is fine and dandy and in conjunction with distributed hosting via Holo on an army of HoloPorts, we could basically rebuild every centralized Facebook or Amazon application in a distributed fashion and have the same UX as we do now. Because of the agent-centric architecture of Holochain that is working under the hood (and thus eliminating the need for either a centralized server or a huge blockchain that forces everybody to store, sync, run and validate everything) these distributed apps could actually scale to Facebook size while running a full client on a mobile device. That in itself is already a huge step forward. (Again, if this huge claim renders you doubtful please have a look at our whitepaper and benchmarks and try out Clutter now — and then give us feedback.)
But what really deeply excites me is to elevate this agent-centric paradigm-shift into high-level user interface semantics. Because here’s the thing:
The whole point of the Internet is to have agents/humans communicate with agents/humans.
See, those 2 billion Facebook users are not visiting facebook.com because they want to learn more about the company Facebook — they want to connect to their friends. Right now, you point your browser to medium.com to read this post of mine. It does look very different if you get my messages through Slack or WhatsApp, and I have to search many places for that link somebody sent me last week. Interoperability isn’t always compatible with business models of the big 4. But the intention for us to use these tools is the same. The Internet’s implicit central use case is telepathy — exchange of thoughts between humans. It is the technological embodiment of culture. And human culture is human-centric, not word- or book- or site- or app-centric.
Besides censorship, privacy and power concerns, the problem with monolithic, data-centric and centralized applications is that they are silos. Not only data but also user interface silos. One does not simply rip out Facebook’s timeline to show updates from friends merged with those coming in through Twitter, Google+, WhatsApp and email in one view. Even if you would have access to all data via APIs then you would need to write the UI from scratch and resembling the native UI would be hard work. And what if, after you did that, all your friends switch to Snapchat?
The way we think about Holochain apps is that they should be microservices, small units, modules with a clear purpose, inputs and outputs. Similar to text-based Unix/Linux command line apps that are combined in shell scripts or steered from a GUI to cater for a certain use-case.
With Holochain apps already being decoupled from their UI and using bridging, you could write an app that talks to all other apps running on the same device under the same agency (basically authorizing apps to access the same functions exposed to the UI) and presents you with an aggregated interface. But if every native app UI is its own standalone front-end (one built with React, the other with Angular, another one with native iOS widgets…), reusing UI components is almost impossible.
I would like to see the Holochain app ecosystem be smarter and make use of one quite significant fact:
Since Holochain is a space of distributed, agent-centric apps, it intrinsically is a human/social network.
Instead of reiterating the misplaced ontology of documents, websites and monolithic applications, why don’t we invert that dependency and put agents/users/humans at the center of how the Holochain app ecosystem works? Not only under the hood but consistently throughout all layers, including the user interface?
To be explicit, I propose to:
- build a general-purpose UI for Holochain apps, implying certain protocols that app developer could choose to follow. I am talking about a browser for the Holochain app ecosystem that is not opening HTML documents but instead is facilitating communication between the user and other users, groups, teams, organizations and the network as a whole. Think: decentralized WeChat. Basically a very generalized address book/team chat/social network/collaboration UI with no specialized functionality as that is to be filled-in by those many apps which are meant to work together as micro-services. It is an agent-centric UI in that it represents other agents and the relationships/teams/threads/social spaces the user is part of. And it is of course a container of Holochain instances. It displays entries of all Holochain apps using the new entry type semantics we are about to introduce with the next Holochain DNA iteration to automatically create a more fitting UI widget. Ultimately, it sports a UI component system that enables app developers and users to customize how entries are rendered. Qt’s QML is a super elegant UI component language that I see as the perfect fit for this. The native version of this browser could use native Qt. For Holo’s pure web based access to Holochain apps we could use this QML web port.
- define a human-centric/social API (the protocols/interface functions) this browser uses to query apps for entries. Besides UI, every Holochain app consists of a set of entry types it defines, their validation functions and semantics, and zome functions that are UI facing (see documentation). In order to support the browser’s ontology, an app needs to provide, for instance, a way to retrieve all entries that should be displayed when a user’s/agent’s profile is visited or rendered: get_entries_per_agent(visited_agent).
Or when displaying all entries within a social context (like a private message thread or a team collaboration space): get_entries_in_context(thread).
When entries can be read from social contexts they also need to be posted:
I’m just naming a few examples here to communicate the idea in principle. Maybe what we need is just all CRUD operations on all levels of the social holarchy — i.e. for agents, message threads, teams, organizations, …
- establish an inter-app URL schema to have fully qualified entry references. Within the context of a Holochain app, an entry’s address is just its SHA-256 hash which is used to do a DHT lookup within the network that is this app to retrieve the content (remember, each Holochain app is its own p2p network). In order to fully address content from an inter-app context, any link needs to include the app that is the space in which this entry lives. Luckily, every Holochain app is uniquely identified by its DNA. The current alpha version is already using the DNA’s SHA-256 as the unique identifier of that app when looking for and connecting to peers running the same app. Protocol Labs’ IPLD is built for the very use-case of linking entries across applications/networks/blockchains. Can we have every Holochain be an IPLD protocol specifier?
/<App DNA Hash>/<Entry Hash>
One central aspect here is these social contexts. Following the notion of Holochain apps being micro-services, like Unix commands, that can be aggregated within the context of an agent to implement more complex tasks, there should be a Holochain app for these social contexts that is:
- storing a list of friends in each agent’s local source chain
- introducing social context entries that are publicly shared in the DHT and that link to their members (think teams)
- allowing for cascading/fractal social contexts, so we can have teams of teams, i.e. organizations etc.
Social contexts is what the browser displays as locations. And there are implicit, transient locations like for instance all friends that are online right now:
This example shows entries from three different apps (Clutter, Holochess and HoloChat, and maybe even another app for the avatar: HoloVault). The red puzzle-piece shows that this is an entry of an app that is not installed locally. We are not part of that network so we can’t retrieve its contents, but since the browser knows which app it is (because it was referenced via the intra-app schema) it can offer an install button, which in-turn would use the Holochain of Holochains (HCHC), the distributed “app store” to download and install that app. This way of discovering new apps through the interaction with other users already using the app is an aspect that seems trivial but my feeling is that it would make a tremendous difference for the adaptability of this ecosystem. Especially since apps, in this alternative universe, are really a superposition of full-fledged application and plug-in.
Take my favorite example: a dislike button. If I want to dislike Philip’s Clutter post up there, I could just write my own new Holochain app that is basically just storing dislikes, with every dislike consisting of just one inter-app link. I could now dislike any entry of any Holochain app (or even any IPFS object, git commit, etc…). Now, with either a custom (QML) UI component or (even better) a few more traits in our social API, the browser could show those dislikes on every entry that has one. If we choose the latter implementation, adding functions to the inter-app API that know about inter-app links, those dislikes could now be shown to everybody as an entry from an uninstalled app, just as the HoloChat example above. Which means this functionality can spread through the ecosystem easily if users choose to install the app. And that would work without having to change the code of Clutter itself, and without having to get the whole community to fork into a different “consensus”. New ideas could easily be tried and if they prove useful they could be adapted by the ecosystem in large. Just as sub-cultures invent their own jargon which then sometimes becomes mainstream.
Now, if we zoom out of this online list, we would see several different social contexts that the user is part of:
For each of these, the browser receives lists of entries from all installed apps which are being rendered inside those areas respectively. Depending on the apps installed and used by the members of a context (and maybe the access rules of that group), a social context is like the power set of a chat room, a mailing list, a YouTube channel, a shared Google Drive directory, a ticket system, a git repository, a video chat or any real-time collaboration tool, and many more.
The point here is that being able to add any new functionality with just a click but maintaining the social structure in which the need for this new functionality arose makes a lot of sense for the user experience. This is how we think right now anyway when we cross app boundaries and build the stack we need to collaborate.
Social context, now as a general term, is the main thing we as social beings rely on to make sense of what is going on. It is what is always there for us to contextualize information, experiences and decisions — mostly implicitly. I believe it is crucial to build communication tools that make social contexts explicit.
And remember, since every Holochain app is its own sub-network, it is totally possible to spawn a cloned app instance (of say HoloChat) for each team, also with just the press of a button. It would create a fully distributed, encrypted by default, and, as described above, customizable virtual space that emerges just between the machines of the members of this team, no central servers, no miners, no peers that can’t cross the app’s membrane.
What I’m proposing here is actually reaching out of the scope of Holochain core itself. Though, it is true that these ideas could inform certain implementation details as we go through the next iterations of Holochain. But in order to create a thriving and synergistic ecosystem, it needs many agents working in coherence.
And the seeds for this are already planted.
Our anders.aamodt expressed very similar concepts for interoperability in his Wrex paper some weeks ago, which our Philip Beadle started prototyping on the current Holochain alpha version already, laying out an idea about Social DNA, a inter-app identity management called HoloVault, and details about one very central Holochain core app: DPKI (distributed public key infrastructure).
What I am describing here is touching and integrating the work of my dear friends and colleagues at Noomap in many ways. This Holochain browser feels like a light version of H4OME — the brainchild and big vision of Chris Larcombe and myself — but infused with the agent-centric paradigm-shift. I had many co-creative sessions with Andrea Harding about social architecture and social holarchies that prepared several of these ideas. Tan, I’m sure Synergy Space would benefit so much from this architecture. And I’d definitely love to see Chris coming up with a fractal, Noomap-like UI for this Holochain browser!
I am engaged in ongoing conversations with Harlan T Wood on this matter. We actually came up with a name for this browser: HoloScope! My feeling is that Harlan’s Trust Graph is also a very interesting candidate for one of the Holochain core apps. It could be used by HoloScope to represent these social contexts.
I know that Adam Apollo is also deeply involved in next-gen, decentralized Internet projects with extraterrestrial UI and surely would have some interesting thoughts on this. :)
I also want to reach out to Malek Jaber! Malek, long-time no see but I remember very well how our first half-night long conversation evolved around socially inspired, global software architecture. :) I think we need to talk again!
An interesting project that has many similarities to the Holochain approach and especially the social piece I am talking about here is Secure Scuttlebutt. I would love to get into touch with Dominic Tarr or mix irving and start thinking about an implementation of SSB on Holochain using the node-to-node communication capability as some middle-way between private source chains and public DHTs.
My gut feeling is that this thing wants to be co-created among several experienced architects, developers and designers. With this post I’m inviting discourse and participation across organizational boundaries, not only from those mentioned above but everybody who has feedback to share!
Next steps: hearing and incorporating your feedback and creating a more formally detailed spec.