Propositions Are Not Types: Naturalizing Information Content in Computing

Overview: “the hard problem of content” (Hutto & Myin 2013) is an argument that information as it exists in physical systems is only covariance information, which does not inherently have properties such as truth, reference, and implication required for it to work as propositional content. That means formal propositions implemented in computing systems can’t actually be carriers of content as information-processing theorists had hoped. But in order for software engineering to mature into reliable practice, we will need computing systems to participate in propositional activities on our behalf. Following Hutto & Myin’s solution that real-world content depends on shared scaffolded practices that augment basic agents, we explore socially situated computing as an approach to naturalizing information content in computing.

Part I: Propositions are Not Types

Over the last decade or so a research finding that was previously heard mostly in university math and computer science departments has become a popular slogan in applied statically typed functional programming circles. The message that inspires this movement: “propositions are types! programs are proofs!” (Baez & Stay 2009) It’s not a new idea that leading figures such as Philip Wadler and Simon Peyton Jones promote, it is a well-established finding that unfolded over a period from 1934 to 1969 in the work of mathematicians Haskell Curry and William Alvin Howard. From a purely mathematical standpoint there is nothing false about the Curry-Howard Correspondence as it is called. It is indeed useful, as it allows computer programs to be analyzed using the tools of formal logic, everything from proof assistants to compile-time type-checkers in general-purpose programming languages. Pretty neat.

Where we start to run into problems is that the emboldened typed functional language communities increasingly advocate for a radical perspective on logic and information processing that is reminiscent of the symbolic processing era of AI. Their hope or expectation is that the programs-as-proofs approach will scale to messier, more complex real-world applications of formal methods, transforming computing from the status of unreliable and irresponsible craft to robust engineering discipline. The goal of being able to declare what our systems must abide by and hold to is laudable and I share it, but the means to that goal brush over some thorny problems.

Lessons from Artificial Intelligence and Cognitive Science

Computationalism or “GOFAI” did not exactly die when it failed spectacularly in the mid-80s, but in the time since the AI winter there has been progress on understanding why such approaches are bound to fail. As new research on pre-AI cybernetics-era techniques such as neural networks began to show improvements, “symbol grounding” was identified as an overlooked problem with formal propositional systems. The symbol grounding problem is that symbols’ interpretations are not intrinsic to formal systems, but rather depend on users of a system to assign their meanings. Consider how any property of the world is encoded as data. Symbol grounding problems arise due to encoding of properties and events in ways that arbitrarily coincide with human interpretation, but do not intrinsically preserve lawful relationships to aspects of the world. Today computationalists in AI seem to be excited about combining the more grounded behaviors of neural networks with symbolic processing ideas, but symbol grounding remains a difficult problem for the information-processing paradigm generally.

The Hard Problem of Content

But there is an even deeper problem with information processing approaches to information content. Fred Dretske, the thinker perhaps most emblematic of the original computationalism, proposed that a “cognitive” system (meaning a system acting on information content) is one that converts analog information into a digital representation, and performs manipulations on that digital information, allowing it to perform actions targeting properties of its environment. Dretske’s notion of information is that data indicates or carries information about some property of the world if the two vary lawfully enough. Recently Dretske’s theory of content has met with a strong challenge from cognitive science philosophers Daniel Hutto and Erik Myin, that such accounts of content have no grounds on which to claim that lawfully covarying information “says” or “represents” anything about the world. This observation which they refer to as the “hard problem of content” or the “covariance-is-not-content principle” is that systems acting on covariance information, while acting on information, do not constitute content-bearing systems, because to bear content is to embody claims about how things stand, when in fact they merely embody capacities to affect the world.

So even a well-grounded symbol is not intrinsically about anything, it cannot be about something, it only lawfully varies in relation to some property of the world. The consequence of this for the design of computing systems is that we shouldn’t expect to find a way for computing systems much less biological agents to work with self-contained information-as-content when all we’ve given them is information-as-covariance lacking any way of participating in its larger social/pragmatic context. What we should look for instead are ways of scaffolding propositional interactions in effective world-involving ways. As Wittgenstein surmised after a career spent grappling with the basic nature of propositions, propositions themselves do not contain information about how things stand, rather they are like moves in games where the object is to know how things stand in the world.

Part II: Naturalizing Information Content

We’ve talked about covariance information, and how it is not the same thing as information content, but how can we naturalize content? In order to answer that, it is necessary to take a step back from a narrow focus on abstract information, and look at human-computer interaction, which is a special case of human-environment interaction. We will touch briefly on how basic agents use information effectively and produce it expressively, and then discuss how convention-based access to such information (via cooperative scaffolding of motivated attentional decisions) is leveraged in claim-making games– and how socially situated computing systems can participate in these “games”, reaping the benefits of participating in authentic information content.

Lawful and Conventional Access to Ecological Information

Embodied cognitive science research has shown that covariance information, while content-free and non-representational, can go a long way in explaining a host of complex cognitive behaviors. The most scientifically mature effort in embodied cognitive science is ecological psychology, founded by J. J. Gibson in the 1950s, 60s, and 70s. Ecological psychology has made as much progress as it has on naturalizing phenomenology by discovering that organisms acquire “ecological information”, information supporting perception/action, in the form of learning to exploit what are called “affordances” or physical properties of the environment that reliably specify directly realizable actions/outcomes to organisms. Ecological information is law-based covariance information embodied by organisms guided by affordances in their prospective control and navigation of environments.

But ecological information supporting the control of action can be accessed in a secondary way, by means of convention, as Sabrina Golonka published in 2015. Research has found that only law-based information can support direct perception, but conventions such as signs and gestures can be used to select or steer attention to a primary target of direct perception, and as such constitute a mode of access to ecological information. Importantly, such expressive conventions do not constitute content and representation. As José Medina (2013) puts it, basic convention-based expression “should not be understood on the Gricean model of conventional signs, that is, as involving or requiring fully formed communicative intentions and internal representations. Expressive behavior is not self-reflective intentional-referential behavior among rational agents who are representing each other’s minds and their contents.” Rather, conventional expression is a mechanism of directing attention.

Joint Attention, Scaffolding, and Claim-making

Conventionality does not in itself grant such sophisticated uses as representing content, rather it serves as the foundation for them. As Hutto & Myin (2017) write:

“content only arises when special sorts of sociocultural norms are in place. The norms in question depend on the development, maintenance, and stabilization of practices involving the use of public symbol systems through which the biologically inherited cognitive capacities can be scaffolded in particular ways. The practices in question are claim-making practices– and they are special because they require participants not only to respond to things but to do so by representing them as being thus and so independently of what might be said about them.” (italics theirs)

…

“Getting things wrong in a truly representational sense is not just a matter of being literally misguided in the way purely biological entities and creatures can be. It involves being subject to the censure of others– not just in the sense of being in or out of line with what is acceptable or not for some community, but being able to get things wrong in a game in which it is at least possible to be right according to how things are anyway. Only those in a position to play this sort of game can be said to have content-involving thoughts and speech.”

So then what does this socially situated “scaffolding” of basic direct and conventional access to ecological information require? The most empirically compelling answer to this question is to be found in Michael Tomasello’s research on primates and child development. Tomasello has found that primates use gestural conventions classified into two categories: “attention-movement” gestures, to get another agent to do a particular thing, and “attention-getter” gestures that call attention of another agent to something that they’ll respond to in some way. Humans in contrast (at around 9 months of age) develop a more powerful vocabulary of social attention control devices. Humans go through three stages of learning how to control attention. The first is “sharing” what they are attending to, not unlike the attention-getter techniques of apes. The second is “following into”, as in attending to what another agent is attending to. The third developmental stage is the skill of “directing” others to attend. Directing is the most impressive skill out of the three because unlike the attention-mover gestures of the great apes, directing attention occurs relative to a followed-into shared context of attending. It is worth noting that these social skills appear several years earlier in child development than the “theory of mind” skills. Because of their early development and marked divergence from other primates’ functionally similar abilities, Tomasello theorizes that they constitute an innate and evolved “infrastructure of shared intentionality” supporting cooperative communication that paves the way for complex tools of cooperation such as spoken language.

The development of joint attentional skills was a defining moment in becoming human as we know it. It made it possible for humans to construct attentional tools (paintings, glyphs, models etc) that augmented their gestural scaffolding of attention. Such scaffolding devices included the development of language itself, in which verbal constructions are literally used as tools. With joint attention, and its augmentation by scaffolding, we approach the aforementioned Gricean account of communication as prosocial, cooperative activity. This gives us the necessary ingredients for the social construction of claim-making scenarios, such that one might play the game and be successful or fail, with a given propositional move, at achieving socially defined objectives in a shared environment.

Scaffolding, Constructions, and Conceptual Metaphor

I would like to take a moment here to revisit the insufficiency of the formal propositional account of information content that is the focus of Part 1 of this article. There is a potential objection to our positive neo-Wittgensteinian account (that propositions are less like pictures or containers, and more like moves played in games), the objection being that any such game moves can only be smaller fragments of world models, and that the problem has simply shifted to a finer grain. We have already seen one way in which that is not the case, that is in pre-linguistic deictic social skills of joint attentional engagement. But that is not yet true claim-making, so what of the scenario of mature claim-making contexts? The fact is that the objection ignores that language even in its simplest cases does not consist in formal world modeling but in guiding and motivating flows of attention in a collaborative process of narrative sense-making. For consistency, I will assume the usage-based model of language constructions (Tomasello 2003, Goldberg 2006) as tools (which I also happen believe is true, given its elegance and empirical track record.) Humans are at base engaged in joint attention for the purpose of cooperating on activities, and language affords powerful leverage in those processes. As an example, around 2–3 years of age children begin to pick up on identification and possession constructions like “it is X”, “that is X”, and “that’s my X”. Using these constructions is to participate in engaged processes, supporting them by calling attention to something someone would presumably want to know. Further, it has long been well-established scientifically that conceptual metaphor (Ortony 1993), the practice of adapting familiar schemas from basic-level perception to make sense of or define more abstract or complex ideas. Conceptual metaphor is a an application of joint attentional scaffolding that human languages get a great deal of mileage out of. Again, this is a way of resolving sense-making to flows of motivated attention. So it is not a deflective move to take the pragmatic turn on propositionality, rather it is to embrace the reality that cooperative attentional scaffolding is the basis of sense-making and communication, and that the claim-making scenario (where what is expressed may win or lose, be correct or incorrect) is no exception.

Socially Situated Programming

The picture we have arrived at is of human culture as a large-scale stigmergic content ecosystem, consisting in human social transaction within contexts of content generation, testing, and upkeep. Individuals from a young age are confronted with the need to make myriad passive and active attentional decisions that become in some large part what is unique about the life experience, perspectives, and directions of any particular person. As social creatures we enter a world that offers us a wealth of pre-existing tools for directing attention in useful ways, and games or systems establishing utility of actions. Despite all of culture’s complexity, there is always one thing happening that makes content possible: scaffolded processes of attending in game-like social contexts.

For designers of interactive computing environments, including programming languages and other kinds of systems in need of open-ended declarative expression, the trillion-dollar question is: how can machines participate meaningfully and effectively in content? I will present two answers to this, one too general for it to be immediately obvious how it might be put to use, and one too specific to imply any sort of claim about the space of other possible solutions– just exploring one approach in some depth. It is left to the reader to experiment with other possibilities in this space.

The very broad answer can be summarized as “socially situated computing”. To define this I will start with defining the more encompassing “situated computing”. The idea of situated computing or situated programming is that computing is embedded more directly into situations than it traditionally has been, putting systems into shorter real-time feedback loops with events of interest to its users. Rich Hickey has emphasized the importance of this embeddedness and feedback for creating reliably effective software systems, whereas others such as Jelle van Dijk, M Eifler, and Bret Victor have focused on the implications of immersive technologies on embodied engagement with direct environments as we experiment with a plethora of new kinds of devices, sensors, and instruments. Others such as William J. Clancey and Rodney Brooks have focused on the importance of environment embeddedness for the intelligence of artificial agents. The common theme in all of these approaches is a reorientation of computing to be more ubiquitously agent-centered, context-sensitive, and feedback-oriented.

Phoebe Sengers (1996) coined the term “socially situated AI” to refer to approaches to AI that are not only aware of the agent’s relationship to its physical environment, but also its social environment. Expanding the scope of this idea a bit, social-world-involving software augmentation of experience is what I am calling socially situated computing. It is my contention that from an HCI perspective, an AI perspective, and a generally informatic perspective, embedding of computing into the contexts where content is maintained is a requirement in order to make systems content-aware.

Narrative Process Scaffolding

To answer the question posed earlier “how can machines participate meaningfully and effectively in content” with the broad brush of “socially situated computing” is appropriately non-committal given the current nascent state of the art, but at the same time it is unsatisfying because it doesn’t get into specifically how one might go about involving computing in contexts where content is maintained. As I see it, this problem has two sides to it: a causal decision science problem (Pearl 2000, 2018) and a human-computer interaction problem: on the human side, expression of causal motivation of attention, and on the machine side, causal inference by agents selecting actions that contribute helpfully within these contexts. Following Tomasello and Carpenter’s work on joint attention, the attentional skills of sharing, following-into, and directing are intuitively accessible to 9-to-12-month old babies. This research suggests that the skillful navigation of contexts of attending constitutes our ability to express and understand intentions. In other words, intending is decision-making about attending (and nothing else). If you know why an agent is attending, then you know what they are intending (limited to that specific context, of course). “Why attending” and “what intending” are different ways of expressing the same thing.

By establishing a protocol codifying joint attentional decisions into a set of simple gestures, we can make use of the basic joint attentional skills that we’ve leveraged for sharing motivation of attention since early childhood, to specify learnable contexts that software agents can participate in, extending our agency as users. To be a little more specific, there seem to be three kinds of attentional decisions: whether or not to enter into a context or center of attending, whether or not to attend to some other center in the context of the current one, and given the presence of an option to attend to another center, whether or not to exit. The causal questions that can be expressed using these primitives are respectively “does attending to or refraining from attending to this center cause this outcome”, “does attending to this other center influence positively or negatively the outcome of the present center”, and “does the availability of this other center indicates the outcome has been reached or is being maintained”. It’s important to note here that these causal models are propositional in our human-to-human description of them in talking about the approach, but in use they are modeling the causal properties of non-propositional capacities of expression. It is through the use of these capacities in explicitly claim-making processes that actual machine-accessible content emerges. I have called the approach of using in situ gestures to define causal models that scaffold intentional agent behaviors “narrative process scaffolding” (Levy 2018) because motivating joint attention in this way is theorized to be fundamental mechanism of narrative sense-making generally, including processes of claim-making that underpin propositional content.

The narrow prescription given here for “narrative process scaffolding” is still speculative and untested, but if it is developed into a robust practical computing paradigm, it is preferable over the formal propositional program that we have stated strong reasons to believe can’t work. NPS in contrast is guided by findings on how information content does appear to work. Further, if NPS turns out to be somehow irrecoverably flawed in its approach, it should be kept in mind that this is just one idea of how to design socially situated computing systems. The space is a wide open frontier for creative and scientific exploration.

From Metaverse to Noosphere: A Third Cybernetics?

Today we see the early signs of a third wave of cybernetics emerging. The first cybernetics tackled teleological mechanisms as physical control systems. Many of the most successful “AI” techniques today (neural networks, reinforcement learning, agents as dynamic control & feedback systems) began in that movement, which is ironic because AI and its attendant “cognitive revolution” arose in opposition to cybernetics. The second cybernetics sought an understanding of second-order or self-organizing autonomous systems. Such efforts eventually became part of 4E Cognitive Science: embodied, embedded (situated), enactive, and extended (scaffolded). The potential ramifications of socially situated computing and related trends suggest the third cybernetics will turn a microscope, and ultimately a telescope, on the 3rd-order processes by which autonomous self-organizing agents engage in joint attentional cooperation, and participate in claim-making processes constituting the narrative construction of the content of all of culture and conceptual thought. The third-order cybernetics is the one that would expose the workings of culture and personality to tractable computation.

Situated and socially situated computing is on the rise. There is renewed interest in and success of agent-based and agent-centered approaches in computing. Giant companies pour research money into nebulous efforts toward “cognitive computing” as a goal of agent-based computing approaches such as robotic process automation. There is new life being breathed into augmenting human agency, human-machine symbiosis, and intelligence amplification. The promise of “the metaverse” (the situated internet) is being cultivated in emerging web standards for augmented reality on the open internet. Self-sovereign identity, user ownership of data, and attentional sovereignty make the prospect of socially situated content-involving metaverse, an “electronic noosphere” more ethical and humane. Importantly securing sovereignty of identity and attention provides the base of necessary confidence required for participation in systems that inherently demand great depth and intimacy of integration of computing into personal and social experiences. Such deep integration of computing into human sense-making is likely to radically alter the human cognitive niche in ways that we as future individuals and networks will be in control of, but which appear impossible to form any detailed expectations about at present without actually beginning to undergo it.

References

• “Philosophical Investigations” Wittgenstein 1953 https://static1.squarespace.com/static/54889e73e4b0a2c1f9891289/t/564b61a4e4b04eca59c4d232/1447780772744/Ludwig.Wittgenstein.-.Philosophical.Investigations.pdf
• “The Ecological Approach to Visual Perception” Gibson 1979 http://b-ok.cc/book/864226/e0dd92
• “Knowledge and the Flow of Information” Dretske 1981 http://b-ok.cc/book/819455/a1f712
• “Intelligence without representation” Brooks 1991 https://people.csail.mit.edu/brooks/papers/representation.pdf
• “Metaphor and Thought” Ortony 1993 http://b-ok.cc/book/767226/2778f8
• “Socially Situated AI: What It Means and Why It Matters” Sengers 1996 https://pdfs.semanticscholar.org/1414/7b6006180a87c139a29f0cc1584b8437e915.pdf
• “Computation and Human Experience” Agre 1997 http://b-ok.cc/book/689039/b2182d
• “Situated Cognition: On Human Knowledge and Computer Representations” Clancey 1997 http://b-ok.cc/book/859418/09e510
• “Social cognition, joint attention, and communicative competence from 9 to 15 months of age.” Carpenter, Nagell, Tomasello 1998 http://booksc.xyz/book/15451904/9e3c39
• “Causality” Pearl 2000 http://b-ok.cc/book/907051/f3c24a
• “Constructing a Language: A Usage-Based Theory of Language Acquisition” Tomasello 2003 http://b-ok.cc/book/690493/92df60
• “Constructions at Work: The Nature of Generalization in Language” Goldberg 2006 http://b-ok.cc/book/1210861/e29e0d
• “Origins of Human Communication” Tomasello 2008 http://b-ok.cc/book/541274/39859f
• “Physics, Topology, Logic and Computation: A Rosetta Stone” Baez & Stay 2009 http://math.ucr.edu/home/baez/rosetta.pdf
• “Radicalizing Enactivism” Hutto & Myin 2013 http://b-ok.cc/book/2554656/1b7ea8
• ‘An Enactivist Approach to the Imagination: Embodied Enactments and “Fictional Emotions”’ José Medina 2013 https://www.jstor.org/stable/24475354?seq=1#page_scan_tab_contents
• “Laws and Conventions in Language-Related Behaviors” Golonka 2015 http://booksc.xyz/book/50082310/1e8631
• “Strange Tools: Art and Human Nature” Noë 2015 http://b-ok.cc/book/2640649/b1b44d
• “You’re Doing Mixed Reality Wrong” Eifler 2017 https://medium.com/@blinkpop/youre-doing-mixed-reality-wrong-d32aa54ae8af
• “Effective Programs” Hickey 2017 https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/EffectivePrograms.md
• “Evolving Enactivism” Hutto & Myin 2017 http://b-ok.cc/book/2947353/09d772
• “Steps to an Ecology of Bicycles for the Mind: A Situated Programming Manifesto” Levy 2018 http://senters.info/situated-programming
• “The Book of Why” Pearl 2018 http://b-ok.cc/book/3559945/bd1c4f
• “Designing for Embodied Being-in-the-World: A Critical Analysis of the Concept of Embodiment in the Design of Hybrids” van Dijk 2018 http://www.mdpi.com/2414-4088/2/1/7/pdf
• “Reflections on Dynamicland” Wittke 2018 https://medium.com/@wittkensis/reflections-on-dynamicland-65158b06196

This article is maintained at https://github.com/senters/pant/blob/master/pant.org

Visit http://senters.info to learn more about the senters project of implementing narrative process scaffolding.