To Cut Nature at Its Joints
On GUN, Perspectives and decomposing problems
To solve a big problem, reduce it to smaller problems. This must be one of the best-known problem-solving strategies, one we apply every day in myriads of situations. The big question being, of course: how to divide, before conquering? ‘[We] seek to carve nature at the joints’, Jerry Fodor wrote (The Appeal to Tacit Knowledge in Psychological Explanations, 1968), referring to so-called ‘natural kinds’. Wikipedia: ‘A natural kind labels a “real” structure of the “natural world”, not an “artificial” result of human reasoning’ (1).
A butcher carving up a carcass could use a chainsaw and produce humps of, say, a pound, rather easily. However, beyond any doubt, consumers will prefer the steaks and chops produced by a butcher who knows his stuff and cuts at the joints.
This is everyday stuff in computing. It is all about creating the right abstractions, hiding some data items while exposing others. Do it wrong and you’ll find yourself shoving information across borders endlessly, keeping double books, making programs that are hard to maintain.
I had reason to reflect on this issue while studying GUN (https://gun.eco) as a possible substrate for the Perspectives Distributed Stack. GUN is a fantastical system. It provides a small functional API to the application programmer, that allows him to build on well-known database concepts while not deploying one. Concisely: put a value away under some key, retrieve it using the same key. In other words, GUN functions as a wardrobe. The special thing about GUN is that there is no physical central location where the stuff that was put away, ends up. Rather, a value is stored on each computer that runs a program that has asked for it.
Actual storage is under the hood, however, as far as the application programmer is concerned. Conceptually, to him it is just a central place where he puts things and retrieves them from.
This reflects a problem subdivision programmers have lived with almost since the beginning of electronic computing. Persistence (storage of data) is separated from the logic of the application. For persistence, we use specialised solutions — databases.
Now, in the context of multi-user computer programs, a database functions as a bulletin board, too. People will put information up on it and others will take notice, using it if they are interested. Typically, a bulletin board will be located so that many people pass it in the course of their daily activities, e.g. in the hall of a building. It should be accessible. A bulletin board solves the problem of a group of people who need to communicate with each other. A bulletin board, like a database, is a central hub. Theoretically, communication among n people requires n*(n-1) connections. But a hub reduces this to just n connections.
In our time and age this translates to client-server internet architecture.
But a bulletin board lacks confidentiality. Some information is not for all eyes to be seen! This depends, of course, on the application, or rather, on the process that is supported by it. The application programmer can choose from various techniques to deliver information only to those who are interested and have a right to see it. Some of these techniques are supported on the persistence level (e.g. access list control), others depend on the logic of the application itself to hide information from some users and disclose it to others.
The point being, of course, that while a database solves one problem — n to n communication — it introduces another: a lack of confidentiality.
Let’s return to GUN. Divide all users of a particular program in two groups with respect to any given key: those who are interested in it, and those who are not. Now, when some user changes that keys’ value, how does GUN make sure that those who are interested in it, receive it (when they’ve asked for it)?
Interest (in a key) is defined at the application level. GUN has no insight in it; this information is not available to it. Nevertheless GUN has to take into account that any computer in the network may, at any point in the future, ask for that key — or has asked for it already. So it must make sure that when that happens, it can come up with the associated value. It does so by spreading an update message (a delta) through the entire network, to each computer. Those who are interested, will store it. Those who are not, just pass it on. Conversely, when a computer asks for the key, this request will ripple out in a wave from it through the network. Computers holding a stored value will respond; others will pass it on.
Much ingenuity has been invested in GUN’s algorithms to make this process as efficient as is possible. Let me mention just one that is too good to miss: a computer that knows an answer, still forwards the request to its peers (it’s private version of the value may be outdated!). However, it tacks its own answer to the request. Any computer that receives that enriched message and that has stored the same value, will not propagate it any further, neither respond. This might save a significant amount of effort; one imagines a ring of ‘computers in the know’ around the requesting computer, that will act like a containment field for the outspreading wave.
But no matter how clever GUN’s algorithms are, essentially, the entire network is involved in storing keys and values.
Now, let’s turn to Perspectives. Like GUN, only computers ‘interested’ in a particular piece of data will store it. As with GUN, there is no central server. So, like GUN, Perspectives has to figure out how to move a delta to the relevant computers in the network. But, in stark contrast to GUN, the distribution layer of Perspectives is very well informed about whom a particular piece of data is relevant to. This is because a Perspectives ‘program’ — we prefer the word ‘model’ — describes types of contexts, the roles that are active in these contexts and the actions available to each role (where consulting information is an action, too). Combining the model with an instance of a context, holding actual users filling the roles, Perspectives can work out precisely where to ship each delta. If it knows the IP address of the computer of a participant in a context, it will send the delta directly to that computer through https. So we see Perspectives needs to involve just a small number of nodes of the peer network for any delta.
Perspectives addresses the problem of n to n communication in a principled way — by modelling — and offers perfect confidentiality.
There is an important conclusion to be drawn from this comparison. Context, Role and Action are the main high level concepts offered by Perspectives. They allow the modeller to create an effective information system that supports co-operation of many people in whatever process. But we’ve seen that the information thus modelled plays a decisive role at a much lower level in the software stack, too: the level where information is stored and how it is to be exchanged between computers.
The time-honoured division of a computer program in a storage layer and application logic simply does not hold for Perspectives. It is a perfect example of how splitting a problem into smaller problems may lead to a suboptimal solution. In this case, were we to abstract the data layer away from the notion of the application itself, we would lose information crucial for targeted, efficient information delivery and storage — and lose confidentiality in the process, too.
Is this an extra burden on the Perspectives modeller (not being able to outsource storage to a specialised subprogram)? Not at all. Rather the inverse: storage and distribution is entirely taken care of by the software stack itself. The modeller need not think about it at all. He never even needs to store things, let alone send them off somewhere! He can safely assume that information is available magically wherever it is needed.
So even while Perspectives aims for similar non-functionals as GUN does, using GUN as its foundation would lead to a suboptimal software stack. This is not to say that Perspectives is superior to GUN. The designers of GUN assumed very little about the type of applications that would use it. Perspectives simply is more opinionated: it is created for co-operation. In its implementation, we capitalise on that specialisation. It also guarantees a non-functional that GUN lacks: to be private by design.
(1). The metaphor has a deep tradition. Thijs Ringelberg pointed out to me that Plato used it in Phaedrus (lines 265d up to 266a). It’s history may well be fathomless, given the hunter-gatherer nature of our ancestors.
This is the tenth column in a series. The previous one was: Homo Co-operans. Here is the series introduction.