How to coordinate despite our biases?
Let’s seek a strategy
TLDR:
(Short version here)
We need a clear map/graph of debates presenting the strongest version of each opinion, with references (wikidebat). And subjects-hyperlinks spatially organized in a semantic sameness gradient
We need to find a way towards a future strongly approximately preferred by more or less everyone (paretotopia/paretotropia, adversarial collaboration)
We need to select best arguments/solutions by using a convergence of agreement throughout the inner-groups/political spectrum (pol.is, community notes) rather than through sheer numbers, likes and virality
We need citizen science and gamification, to increase incentives and learning efficiency. Interactive design, aesthetics and narrations are proven functional in training practices (mental palace, flowstate, playfulness)
We need to leverage crowd wisdom; he average of a diverse crowd’s world modeling is (generally) more accurate than any of its constituents’ perspective, and strikingly close to scientific models.
Note:
Non-naive cooperation is provably optimal between rational decision makers,
We can do encrypted contracts based on such principle and the ones previously introduced.
We need ecosystemic synergy;
-> “Is this option really giving more options to the highest number/diversity of beings”
The inverse of alterity (otherness) is ipseity (selfness).
We’re talking about the in-between : “synergity”.
How to deal with our wrong models?
To break down the proposed strategy, we will go step by step, with detailed descriptions, references and quotes. Each time unknown vocabulary comes up, we will define it sooner or later.
Democracy, economy and fairness can be genuine if we have all the information necessary to make choices, aware of the context and consequences. Currently, this ideal is not respected.
Of course it’s an ideal, but we should do our best to reach it.
What to trust? How to allocate our time?
In the ambient war of signal: how to filter noise?
Hate and partisanship are worsening the lack of clarity, triggering destructive behaviors.
The more I dig these subjects, the more I find resources relating directly with the project.
Our ideas are very straightforward, and we often come to similar conclusions.
Ideal (synergity):
Agency through awareness about our real needs, in optimal synergy with the real needs of others.
We need knowledge: we should incentivize (unbiased) research
We need infrastructure: practical means to reinforce our lucidity and cooperation
To do that, we can use a paretotopian procedure (with adversarial collaboration, agnotology etc.).
“For Paretotopian Goal Alignment, a key concept is Pareto-preferred futures, meaning futures that would be strongly approximately preferred by more or less everyone.” Eric Dexler
“In science, adversarial collaboration is a modality of collaboration wherein opposing views work together in order to jointly advance knowledge of the area under dispute.” Wikipedia
“Within the sociology of knowledge, agnotology (formerly agnatology) is the study of deliberate, culturally induced ignorance or doubt, typically to sell a product, influence opinion, or win favour, particularly through the publication of inaccurate or misleading scientific data (disinformation). More generally, the term includes the condition where more knowledge of a subject creates greater uncertainty.” Wikipedia
Learn more:
4. SKIM THE MANUAL | Intelligent Voluntary Cooperation & Paretotropism
Adversarial Collaboration: The Next Science Reform
What is well-being? Often hard to define; a tractable way to formalize its reaching is ‘optionality’,
From which a moral system can be derived:
-> Is this option really giving more options to the highest number/diversity of beings?
It is an increase of options with respect to the options of others (which is why it isn’t rogue freedom). We will get back to this later, along the problem of our ever-biased models.
We need tools to nurture our intuition and leverage paretotopian optionality,
To do so, we can build a platform of co-learning, deliberation and citizen science.
Announcing the SoS Research Collective
Sympoiesis : “interfacing between different viewpoints (and in fact disciplines of science) to establish a principle of collaboration (synergy) by which the expertise of either standpoint can be combined, and enhanced, to bring out new understanding.”
The “steelmap”:
• An encyclopedic graph of “all” debates with the strongest version of each side’s argumentations.
Wikidebat & Wikidebates
+Wikidebates:Comparison of Existing Debate Encyclopedias
The Society Library
+https://www.societylibrary.org/connor-beff-debates
“Steelmanning. A steel man argument (or steelmanning) is the opposite of a straw man argument. Steelmanning is the practice of addressing the strongest form of the other person’s argument, even if it is not the one they presented.” Wikipedia
• Best arguments are not elected via popularity, but convergence across community’s inner-groups.
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
What do I think about Community Notes?
“But in the middle of these highly contentious actions, one new feature on Twitter grew rapidly in importance, and seems to be beloved by people across the political spectrum: Community Notes. […] It was originally called Birdwatch, and was first rolled out as a pilot project in January 2021.”
“Unlike simpler algorithms, which aim to simply calculate some kind of sum or average over users’ ratings and use that as the final result, the Community Notes rating algorithm explicitly attempts to prioritize notes that receive positive ratings from people across a diverse range of perspectives. That is, if people who usually disagree on how they rate notes end up agreeing on a particular note, that note is scored especially highly.”
Why diversity is the secret to solving complex problems
The aim is to contribute to the craft of a new meta space of interaction.
However; being exposed to opposed views can increase polarization (somewhat contrary to widespread intuitions). We are biased by the detailed knowledge we have of our own opinions, which leads us to think “this perspective is dumb compared to mine”.
We surveyed 150 experts on misinformation
“The most popular reason why people believe & share misinformation was partisanship. Identity, confirmation bias, motivated reasoning & low trust in institutions also received high levels of agreement, while education and access to reliable news did not.”
Tools enabling us to see the connections in our views are, among others, part of a solution. Not to completely erase our divergences; but to articulate common grounds, new perspectives and unfold each other’s position.
We can precise further our values, meaning and convergences (ie. with compatibility matching for constructive social synergies), through several methods involving AI, LLM, and latent spaces, such as Human universals, Moral graphs, Autonomy and Capacity approach to human welfare, as well as broad (worldwide/life/universal) empowerment.
OpenAI x DFT: The First Moral Graph
Governance of AI, with AI, through deliberative democracy
We need to see and share nuances, but since when are nuances viral and memetic? We’re wrapped in our prior beliefs; persuasion is hard or relies on affects. We aren’t that smart… but neither are we perfectly stupid. Intelligence is multifaceted. We are specialized by our causal past; in other words, we are not efficient at the same things, nor do we have the same clues… And smartness doesn’t always equal pertinence.
Other people (of course) are naive, formatted by their (biological, cultural) inheritance, manipulated by misinformation… Indeed large media, lobbies and governments can push their way up the hierarchy of effective misinformators, but truth is not always as self-evident as we would like it to be. Do we even really want truth at all? Our life is congested by doubts and contradictions. It’s a tentacular issue… We operate in constant back-and-forth of hypnosis and auto-hypnosis. From art to coffee, passing through love, ideology, tradition, property; everything steers our brain into bias.
Democracy and the Epistemic Commons
The Wisdom of Crowds and the Stupidity of Herds
If We Don’t Fix Sensemaking, We Won’t Survive
“[…] we can see that just like a smokestack billowing pollution into the air, most of what is being put into the information commons is pollution. If there are whole chunks of populations that you only have pejorative strawman versions of where you can’t explain why they think what they think without making them dumb or bad you should be dubious of your own modeling”
“All of the positions right now have some signal and a bunch of noise, so we want to be able to seek to understand so we can actually get the other humans that we need to be able to coordinate with, and that we can get the parts of signal. We can be able to see where there’s signal and noise and we can start to synthesize the signal across the space.”
At each point of a steelmap, you can zoom in on a concept, see disagreements precisely, their sub-parts, upvoted discourses with references, etc. Strawmaps can be sketched in parallel to spot fallacies. Everything is connected to everything else in a vast cartography of relations and hyperlinks, allowing free navigation through clearly phrased ideas, located adjacent to each other through similarity.
Due to the infinite amount of possible debates, texts can be procedurally generated by AI, then curated through votes. On top of that, links and references can be added, people can propose corrections, argumentations, etc… Mining solutions that are normally buried under a pyramid of predatory signals is one of the steelmap’s core functions.
The polarization lab — our tools
The wisdom of crowds | Karl Mattingly | TEDxBrighton
+https://www.dysruptlabs.com/
Verity
+https://www.improvethenews.org/controversy/ai-existential-threat
InfraNodus
+https://youtu.be/XMwa9q6wWkw
Encyclopedia of philosophy (with redacted arguments) such as :
Stanford Encyclopedia of Philosophy
Internet Encyclopedia of Philosophy
We can enable new social practices/media, moving out of classic debate forms/spaces where, as Jonathan Rowson says : “It’s still about verbal dexterity under pressure, it’s still about impressing the audience”.
Maven
“Maven is a new kind of social network–a serendipity network–that doesn’t work like the usual popularity-contest style of network you’re used to experiencing everywhere else. We don’t have likes (and therefore don’t count them) and you don’t follow other people’s accounts. Instead, you follow interests, and your feed is a reflection of the interests you follow.”
You may have heard of epoché, that beautiful concept meaning suspension of judgment? It’s closely related to the infamous “I know that I know nothing”, an adage that’s been turned into a catchy social banner, as a signal even to oneself that one is “wise” and skeptical… But a lack of commitment is impactful, it’s a choice. Doing nothing is action. Is it always the most careful one? Let alone the most rightful?
Steelmap allows to do a more active epoché. This is part of a foundational strategy coined as “gamification of epistemic carefulness”, teaching users to enjoy the careful but creative and thorough processes through which understanding is acquired.
Epistemic : relating to knowledge or to the degree of its validation.
Thinking Habits of Mind, Heart and Imagination for the 21st Century
We need reasons to be in and stay in this process-zone (stigmergy); mastering how to ask better questions. The key is to become skillful in the procedures and dynamics leading to legitimate predictions. Concretely, we can use game theory, interactive design and aesthetics to refine training and communication. The intent is to leverage (among other things) playfulness, awe, curiosity; many of which are salient qualities of intelligent animals’ learning strategy.
“Stigmergy is a mechanism of indirect coordination, through the environment, between agents or actions. The principle is that the trace left in the environment by an individual action stimulates the performance of a succeeding action by the same or different agent. Agents that respond to traces in the environment receive positive fitness benefits, reinforcing the likelihood of these behaviors becoming fixed within a population over time.” Wikipedia
-> Gamification of flowstate and learning
Playfulness is to system literacy what reading is to literacy.
= Efficiency + Longing
We can search for new entertaining dynamics and sensations to hack our urge for easiness and comfort, creating a culture that uses those very weaknesses to achieve proper decision making; so that our instinct and appetite correlates with sustainable bloom and rationality.
Analogy is at the heart of our reasoning mechanisms and memorization is more efficient in narrative contexts involving as many senses as possible: mind palace, myths, ideasthesia… Examples are numerous. This is a multiscale dynamic increasing our capacity to reach fruitful mental zones. There are many ways to achieve and refine flowstate through a savant fusion of embodiment, art and science. We yearn for aesthetic chills.
All this even relates to what many philosophies and spiritualities were trying to achieve through meditation, contemplation… Auto-hypnotic factors like choirs, introspection etc…
The brain’s ability to reach a realm of high, frictionless efficiency was progressively mastered, in person and online, as well as the transfer from individual to group flowstate. It took time… And the primal target was a ‘eureka of alterity’; grasping foreign concepts produces shifts in worldviews (not necessarily agreeing), leading to lucid dialogues, insights and growth.
“Ideasthesia (alternative spelling ideaesthesia) is a neuropsychological phenomenon in which activations of concepts (inducers) evoke perception-like sensory experiences (concurrents). […] While synesthesia meaning ‘union of senses’ implies the association of two sensory elements with little connection to the cognitive level, empirical evidence indicated that most phenomena linked to synesthesia are in fact induced by semantic representations. That is, the linguistic meaning of the stimulus is what is important rather than its sensory properties.” Wikipedia (+ see Bouba/Kiki)
The biology of fun and playfulness
Analogy as the Core of Cognition — Douglas Hofstadter
Subconsciousnetwork/noosphere
+https://www.youtube.com/watch?v=809FI-AlmLM&t=5s
The steelmap as a whole, its methods of deliberation (paretotopia, etc.) and its “proof of cooperation” (which we’ll see soon), are supposed to be clear and accessible. The protocol is meant to be as accessible and transparent as possible, from bottom up, with first-principled explanations.
Trust is earned not expected. And this is the point;
We have to test strategies to achieve paretotopia [in a complex environment with irrational players]
In a context with (AI, DIY biology etc.):
// Cheaper and cheaper means of destruction (homemade societal-scale impact)
/// Open-sourced traction (is a threat) [while] Locked-in authoritarianism (is a threat)
We need an adequate amount of openness and security.
= > How to make a global immune system faster than bad actors’ awareness?
The Collective Intelligence Project (see CIP Ecosystem Map v0 for other ‘democracy projects’)
A pro-social protocol for people and the planet
/Cliodynamics, computational sociology, evolutionary attractors
Wikipedia :
“Computational sociology is a branch of sociology that uses computationally intensive methods to analyze and model social phenomena.”
“Cliodynamics is a transdisciplinary area of research that integrates cultural evolution, economic history/cliometrics, macrosociology, the mathematical modeling of historical processes during the longue durée [long-term trends], and the construction and analysis of historical databases.”
Return of the oppressed
“My answer is that all of these trends are part of a complex and interlocking system. I don’t just mean that everything affects everything else; that would be vacuous. Rather, that cliodynamic theory can tell us specifically how demographic, economic and cultural variables relate to one another, and how their interactions generate social change. Cliodynamics also explains why historical reversals in such diverse areas as economics and culture happen at roughly similar times. The theory of secular cycles was developed using data from historical societies, but it looks like it can provide answers to questions about our own society.”
Climate change has toppled some civilizations but not others. Why?
We already have a lot of data and surveys enabling the constitution of a partotopian map, upon which we can bring alive deliberations.
Evolving linguistic divergence on polarizing social media
A framework for quantifying individual and collective common sense
Moral universals: A machine-reading analysis of 256 societies
Proof of cooperation:
We can construct our platforms through a formal system of coordination, agreeing on the rules our deliberative system is setting-up;
Non-naive cooperation is provably optimal with rational players
What The Prisoner’s Dilemma Reveals About Life, The Universe, and Everything
Report on modeling evidential cooperation in large worlds
Crowds are not always wise, so the dynamics leading to either good or bad ends have been studied. The ingredients of wisdom are in Surowiecki’s recipe: Independence, Diversity and Aggregation. People need to be able to think independently, have diverse origins (see Scott Page diversity theorem), and a system gathering perspectives (to enable an unfolding of the two other ingredients).
The optimality of cooperation has its own ingredients as well, based on reciprocity and altruism, we need to be: Nice, Forgiving, Retaliatory, Clear.
The notion of power asymmetry is central as well, because if you have so much power that you can deflect without retaliation, the balance is broken. Proof of cooperation and power of crowd are, I think, part of the solution to this issue.
Encrypted synergy contract -> You know that other players are not going to defect
(Within a deliberate margin of conflict/noise; necessary because of uncertainty/errors/consent)
We can use blockchain -> For its proof systems without necessarily using cryptocurrency although an adequate ‘complexity economics’ model is needed (see link below, not crypto related)
Eric Beinhocker & Diane Coyle on Rethinking Economics for A Sustainable & Prosperous World
We can establish a proof of cooperation with trustless code (ie. zk-stark).
What Does Trustless Mean in Crypto?
“A trustless system has a mechanism in place where all participants can reach a consensus on a single truth without any one overarching authority and without needing to know or trust each other, hence the name.”
“Single truth” can mean “a single meta-system” like the semantic map procedure, and diversity.
We agree to disagree but collaborate on searching for what is closer to truth.
We do have access to powerful methods.
I have a whole argument as to why cooperation is optimal in the long-term even in asymmetric contexts (so with weaker agents, see more here).
Such asymmetric cooperation uses superrationality, which can be leveraged in blockchain, as it has been proposed by vitalik buterin.
Furthermore an important question is privacy, and pseudonymity, to protect both the democratic voting process, and the people using steelmap platforms. It’s been long that we are able to guess who you are based on your in-app activities (many studies showed that); hegemony and predictive accuracy (on groups or individual behaviors) is threatening.
Private traits and attributes are predictable from digital records of human behavior
To Address Online Harms, We Must Consider Privacy First
My techno-optimism
“Zero knowledge proofs can be used for privacy, allowing users to prove things about themselves without revealing private information. For example, wrap a digital passport signature in a ZK-SNARK to prove that you are a unique citizen of a given country, without revealing which citizen you are. Technologies like this can let us maintain the benefits of privacy and anonymity — properties that are widely agreed as being necessary for applications like voting — while still getting security guarantees and fighting spam and bad actors.”
Lack of anonymity has heavy consequences: assigning value to concepts because of who holds them rather than what they mean is really dangerous.
Political communication on social media: A tale of hyperactive users and bias in recommender systems
Nadine Strossen — Free Speech: What Everyone Needs to Know
The limits of models:
Can we precise further the (meta) strategy/rules we need to agree on?
A model tries to describe a “thing” So it’s a set of useful constraints supposed to be as close as possible to reality:
-> Delimiting what is, and what is not
But we lack information, so our models have to be upgraded regularly.
In other words; a model will be incomplete and miss some portions of [what it is trying to define]. There are always things out of frame, a portion of unknown and uncertainty. We even gave names to issues of this kind: Map-Territory fallacy, Goodheart’s law, Gödel’s incompleteness…
Wikipedia:
“The map–territory relation is the relationship between an object and a representation of that object, as in the relation between a geographical territory and a map of it. Mistaking the map for the territory is a logical fallacy that occurs when someone confuses the semantics of a term with what it represents.”
“Goodhart’s law is an adage often stated as, “When a measure becomes a target, it ceases to be a good measure”. It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”
Study.com:
“Kurt Gödel published “On Formally Undecidable Propositions of Principia Mathematica and Related Systems” in 1931, where he proposed that any system of mathematical expression that could adequately describe mathematical phenomena was also incomplete. This meant that there existed, for all mathematical frameworks (such as arithmetic, geometry, or calculus), some phenomena that could not be proved using the basic building blocks of the framework.”
It follows that ethics and values are difficult to define properly, and that any model, if maximized, can be disastrous. For example if our model of happiness is “pleasure”, we risk “wireheading” which is: finding a way to provide a perpetual sensation of pleasure, and do nothing but that.
“Preference is a thing that expresses itself in environmentally contingent ways; there is no conceptualizable set of values that is truly invariant across environments. The things that people are able to consciously think of and verbalize as “values” are already far downstream of any fundamental motivation substrate — they are the extremely contingent and path-dependent products of experience, cultural conditioning, individual social strategies, and not a little trauma. Any consciousness that thinks it has its values fully understood will be surprised by its own behavior in a sufficiently new environment. Language severely under-describes conceptual space, and conceptual space severely under-describes actual possibility space. These are not problems to be transcended; they are simply facts of how abstraction works. Conceptualization is and must be Göedel-incomplete; descriptive power should grow as the information in the system grows, but the system should never be treated as though it has been, can be, or should be fully described.” Miya Perry
The specification of our well-being has to be sufficiently indirect to encompass unexpected contexts but precise enough to actually work. And we need something we can agree on.
What is the less harmful type or class of behavior that would:
Be as invariant as possible across contexts, and
Be better than letting things be as they would “naturally” without intervention?
-> Augmenting the number of options and the diversity of states in the universe might be judicious.
It’s a dynamic that shall be preferred in itself;
“Is this option really giving more options to the highest number/diversity of beings?”
So; preferring ‘optionality’ i.e. number of choices available to humans (and not just humans).
What I propose seems to be a conjunction of all three (or prior to all three) major morality frameworks*, and it’s not rogue unbounded freedom. With synergity, the default mode of action is enhancing the capacity of humans/wildlife etc. to achieve paretotopia.
*
Consequentialism: judging whether or not something is right by its consequences.
Virtue ethics: focus on your character, life and practice.
Deontology: strict rules you should obey.
Open-endedness and optionality can be non-linear ie. you sometimes need to restrict the number of options for one to not be frozen by possibilities. To increase options implies an increase of the capacity to choose. The limit is that those choices (ideally) have to not destroy meaningful choices of other agents/humans/wildlife… After which point choosers can opt for themselves however they fit. And always increasing this optionality, preempting lock-ins.
Open-ended evolution : “refers to the unbounded increase in complexity that seems to characterize evolution on multiple scales” (Corominas-Murtra, Seoane & Solé).
The aim is to provide affordance, access to adjacent possibles, expanding the action space and autonomy of a maximum of phenomenons.
This optionality approach could potentially solve lots of long-standing thorny problems in consequentialism, like wireheading or the fiendish difficulty of defining happiness/utility; and how even making the tiniest mistake in that definition can be precisely catastrophic.
For instance : local/myopic/short-timed happiness is regularly detrimental (egocentric interests, addictions, etc.), while local suffering can be beneficial (training, family dinner).
A non-myopic form of diversity (taking into account future and past states of the universe), leads to the necessity of ecosystems, and caring about it; because you can’t open the universe towards more states without ecosystems. Every action has to be carefully weighted, as it may destroy more states than it creates… It lets the time for ecosystems to grow and interact, so that they can bloom/mutate the unique/irreducible causal chains that cannot happen with higher (or slower) speed rate.
Computational Irreducibility — Wolfram MathWorld
“While many computations admit shortcuts that allow them to be performed more rapidly, others cannot be sped up. Computations that cannot be sped up by means of any shortcut are called computationally irreducible.”
Potentia — The Highest Moral Goal for Humanity | Dan Faggella
OMNI: Open-endedness via Models of human Notions of Interestingness
Autonomy in Moral and Political Philosophy
Then, after this fundamental layer, we can add more and more explicit ones to calibrate the triangulation of our preferences through data models such as: human universals (cultural convergences and invariants across culture).
Seven moral rules found all around the world
Profiles of an Ideal Society: The Utopian Visions of Ordinary People
Our aim is to find all the realistic ways to concretely agree on a decision-making method to achieve outputs as close as possible to the ideal we outlined. Imperfection doesn’t mean ‘complete inability’.
And things can be contextually optimal : we know we have uncertainties, but we may look for best approaches given what we know (and the constraints we deal with). The map-territory fallacy can become a fallacy in-itself (if it becomes an absolute measure… falling into Goodheart’s law).
See here (in the context of a specific dissension):
On the Map-Territory Fallacy Fallacy
For ‘synergity’, the more different and synergistic patterns are, the better it is; it’s a procedure to augment the number of ‘diversity dimensions’.
For example : 1010101010 has fewer diversity dimensions than 0123456789. In the continuous space of all possible different states (a ‘latent space’), the more distance there is between two states, the more qualitative is the difference; it’s like colors in a chromatic circle.
(Individual objects are a category, (ie. [my blue dog]), but very similar to other objects in the same meta-category; (ie. [blue dogs]), which with they share a low diversity “index”)
“Black” is more dissimilar to “white” than it is to “gray”.
A “cat” is closer to a “dog” than to a “chair”.
What we mean by ‘complexity’ or ‘sophistication’ is closely related to this qualitative diversity.
You simply combine it with the causal power of a system :
-> Number of qualitatively different external changes caused by the system/sub-system
In this frame (and more generally), one of the ways to minimize errors (especially irreversible ones) is to accept the limits of our calculation capacities. This uncertainty leads to an implicit degree of freedom which we can also explicitly increase further; to relax (utilitarian) mistakes through more tolerance for sub-optimality. Individuals and species need the power to make mistakes.
There is an irreducible halo of preference/consent/noise pivotal for optionality to function properly.
Then, to be more precise, the aim is to unfold the pareto-optimal state among the future possibilities, for you to choose your preferred states. Meaning that it optimizes for the points of convergences between the futures that would have more options.
All this relative to the options of other agents/humans/animals.
I precise further that because a key concern is to have enough sustainable non-myiopic vision, the ratio of a diversity index needs to be balanced with potential long-term growth.
There is a sweet-spot between too much local diversity and too little.
It is also closely connected to the complex fuzziness of happiness and consent, for example a human needs enough “space” (emotional, physical) to explore and avoid saturation of demands/people, but enough relation with others to flourish.
This reminds me of criticality and expander graphs.
“Phase transitions and critical phenomena are the changes of a system from one regime or state to another exhibiting very different properties, and the unusual effects that occur on the boundary between them. A change in the state of matter, such as from a solid to a liquid, is a classic example.” Nature Portfolio
“In graph theory, an expander graph is a sparse graph that has strong connectivity properties […] with several applications to complexity theory, design of robust computer networks, and the theory of error-correcting codes.” Wikipedia
New Proof Shows That ‘Expander’ Graphs Synchronize
Brain Criticality — Optimizing Neural Computations
The inverse of alterity (otherness) is ipseity (selfness).
We’re talking about the in-between : “synergity”.
-> This search for an adequate perspective is connected to the concrete proposals we’ve seen.
Citizen Science in Deliberative Systems: Participation, Epistemic Injustice, and Civic Empowerment
“It is possible to estimate the relative robustness of an insight — what we could call perspectival truth — by tracking its invariance across perspectives, while never forgetting that the conditions that make it true always depend on our own present circumstances (Massimi, 2018). Thus, multiple perspectives enhance robust insight, and a multiplicity of perspectives is what democratic citizen science provides. It is by comparing such perspectives that science provides trustworthy knowledge about the world — not absolutely true, but as true as it will ever get.”
The moral imperative to learn from diverse phenomenal experiences
Social Centralization and Semantic Collapse: Hyperbolic Embeddings of Networks and Text
“Here we explore the relationship between centralization in social networks and contraction or collapse in the diversity of semantic expressions such as ideas, opinions, and tastes. We advance formal examination of this relationship by introducing new methods of manifold learning that allow us to map social networks and semantic combinations into comparable hyperbolic spaces.”
We might already have the solution to our problems, but buried under the hierarchy of attention
-> Social enclaves (oligarchy, traditions, echo chambers, gate keeping, fashion etc.)
Thinking together
“While analyzing a global history databank spanning 10,000 years, Shin, et al found a disconcerting pattern. Civilizations scale until they are overwhelmed by the information environment they create. This is The Information Scaling Threshold.
[…]
When a society hits the information scaling threshold, it stalls out. It can’t function until it invents new ways of making sense that can cope with the complexity of the information environment. And societies that don’t pull off this transition? The paper posits they collapse.”
Scale and information-processing thresholds in Holocene social evolution
In part through the gamification of epistemic carefulness, the aim is to target an ‘Eureka of alterity’ -> Targeting shifts of understanding (not necessarily agreeing) of what was once a blindspot
Goal = Paretotopia -> Process (tropism/motion) -> Paretotropia
Citizen Science in Deliberative Systems: Participation, Epistemic Injustice, and Civic Empowerment
“This central insight also underlies a recently proposed multidimensional evaluation framework for citizen science projects, which makes a fundamental distinction between process-based and outcome-based aspects of assessment (Kieslinger et al., 2018; Schaefer et al., 2021). It identifies three core dimensions to citizen science: scientific, participant, and socio-ecological/economic. For each of these, it defines criteria of evaluation concerning both aspects of “process and feasibility” as well as “outcome and impact” ( Fig. 3). Such a framework can not only be applied to strategic planning, the selection of specific projects to be funded, and impact assessment after a project is finished, but also to monitor and, at the same time, to mentor participants and facilitate the progress of a project while it is running. Evaluation itself becomes a learning process — learning about learning — that supports participatory self-reflection and adaptive management practices (Schaefer et al., 2021).”
And aiming for optionality doesn’t mean we shouldn’t be (very) careful.
“For even the most innocuous-looking, straightforward, factual truth, if propagated rationally, can easily become a source of unpredictable changes. Take the idea that a certain fruit is poisonous. People who believe this idea will realise that it could save lives, and will want to remember it and to tell other people so that it can be put to that use. But because successive holders understand the idea, some of them may invent new uses for it. They may realise what would happen if their enemies ate some of the fruit, and so the art of poisoning would take its place among the society’s memes. Other people may discover that small doses of the poison cause pleasant sensations. Still others may find medical uses for it, and so on ad infinitum. The discovery of each new use constitutes a change in the society, and that change may cause further changes.” David Deutsch
My techno-optimism
“One frame to think about the macro consequences of technology is to look at the balance of defense vs offense. Some technologies make it easier to attack others, in the broad sense of the term: do things that go against their interests, that they feel the need to react to. Others make it easier to defend, and even defend without reliance on large centralized actors.
A defense-favoring world is a better world, for many reasons. First of course is the direct benefit of safety: fewer people die, less economic value gets destroyed, less time is wasted on conflict. What is less appreciated though is that a defense-favoring world makes it easier for healthier, more open and more freedom-respecting forms of governance to thrive.”
“There are inevitably going to be imperfections in classifying technologies as offensive, defensive or neutral. Like with “freedom”, where one can debate whether social-democratic government policies decrease freedom by levying heavy taxes and coercing employers or increase freedom by reducing average people’s need to worry about many kinds of risks, with “defense” too there are some technologies that could fall on both sides of the spectrum. Nuclear weapons are offense-favoring, but nuclear power is human-flourishing-favoring and offense-defense-neutral. Different technologies may play different roles at different time horizons. But much like with “freedom” (or “equality”, or “rule of law”), ambiguity at the edges is not so much an argument against the principle, as it is an opportunity to better understand its nuances.”
An eigenplan:
I think the next step should be to put together blueprints of all the possible ways to build what we have talked about, list the advantages, issues and alternatives (a sort of density matrix).
We need a common space where people interested in ‘synergity’ can see what can be done, what is currently tested or discarded, with explanations.
I would say that the first form of our steelmap should be a page of hyperlinks spatially organized relative to their semantic sameness. A straightforward wikipedia-cartography of debates.
────────────────────────
The next two sections are going to end this text.
The last one is on a subject I consider vital thus quite worrying.
I do think we should debate it urgently
Still, I write this warning in case you find yourself already too worried.
I.
If We Don’t Fix Sensemaking, We Won’t Survive
“If you feel a combination of outrage/scared, you know, emotional, and very certain with a strong kind of enemy hypothesis orientation; you have been captured by somebody’s narrative warfare and you think it’s your own thinking. Even if you win at a local battle whatever technologies you use to win, whatever social tech or info tech or whatever you use to win, the other side will reverse engineer and come back and you’re just escalating an arms race, you’re not moving towards real shared sense making and coordination”
-> The aim is to see the latent space of debates (semantic continuum and its possibilities), explore their inherent logic to the furthest, but also spot the points of convergences between inner groups.
Citizen Science in Deliberative Systems: Participation, Epistemic Injustice, and Civic Empowerment
“Sometimes, and especially in democratic citizen science, the goal is the journey. Democratic citizen science projects put a strong emphasis on facilitating their participants’ individual learning, and their inclusion in the process of inquiry at the level of the research community (see, for example, Wylie et al., 2014). Furthermore, the problems of how to manage collaborations, data sharing, and quality control are no longer peripheral nuisances, but themselves become a central part of the research focus of the project. Democratic citizen science is as much an inquiry into the natural world, as it is an inquiry into how to best cultivate and utilise humanity’s collective intelligence (see Nielsen, 2011).”
Benevolent AI Is a Bad Idea
“In every aspect of our lives, a million choices go unrecognized because we are trapped within the limited conceptual frames that steer us; human life is lived on autopilot and in accordance with inherited cultural scripts or default physiological functions to a far greater degree than most people understand. This is not to say that humans can or should be glitteringly conscious of every choice in every moment, the way people imagine they would be if they were spiritually enlightened. There is a simpler and more discrete kind of psychological expansion. The kind of psychological shift someone undergoes when, for example, realizing that they’ve been subconsciously seeking out harsh, judgmental friends as a method of trying to gain their parents’ love by proxy, and realizing that they actually have the option to bond with kind and accepting people, is the kind of subtle but profound broadening of scope that changes the option landscape, and changes the trajectory of a the person’s life.Whether we know it or not, our trajectories are currently determined by the way that the space of possible futures we can conceive of is narrowed by our conceptual baggage and limitations.”
“So it is for all of us: we are constantly surrounded by options and opportunities that we are conceptually blind to. The current space of imagined futures in AI is highly constrained by cultural imagination, with many people and projects pursuing a vision of AI in a way that is functionally identical to a traumatized individual pursuing an abusive relationship without realizing it, because their concepts of love and relationships have been formed and deformed by local experience and trauma.”
“But the possibilities for targeted and technologically aided increases in agency — not agency in the sense of having more doordash options or more ability to lobby government via apps, but agency in the sense of being able to think new thoughts and generate new possibilities — are likely to be extremely under-explored in the nascent field of AI, simply because this is not a common understanding of the meaning of agency.”
────────────────────────
II.
AI suggested 40,000 new possible chemical weapons in just six hours
‘For me, the concern was just how easy it was to do’
Terrorists will develop deadly bioweapons within years using AI, tech boss fears
“Dario Amodei, CEO of Anthropic, warned a Senate Judiciary subcommittee in the US on Monday about the grave risk of AI enabling people to create terrifying pathogens.
He said: “Over the last six months, Anthropic, in collaboration with world-class biosecurity experts, has conducted an intensive study on the potential for AI to contribute to the misuse of biology.
“Today, certain steps in bioweapons production involve knowledge that can’t be found on Google or in textbooks and requires a high level of specialised expertise — this being one of the things that currently keeps us safe from attacks.”
AI tools could already help fill in “some of these steps,” albeit “incompletely and unreliably”, Mr Amodei said.
However, he added: “A straightforward extrapolation of today’s systems to those we expect to see in two to three years suggests a substantial risk that AI systems will be able to fill in all the missing pieces, enabling many more actors to carry out large-scale biological attacks.”
AlphaGeometry: An Olympiad-level AI system for geometry
“In a paper published today in Nature, we introduce AlphaGeometry, an AI system that solves complex geometry problems at a level approaching a human Olympiad gold-medalist — a breakthrough in AI performance.”
If anyone has the capacity to use and test whatever they have in mind, the threat is everywhere.
But at the same time, giving control and data hegemony to only a few, sounds clearly dystopian.
<- Open-source // Data hegemony ->
Furthermore, even if a few huge corps alone have full control, it still doesn’t immune to accidents (eg. lab leaks). And open-source is useful in case of pandemics as well… We need clear headed debates to find a solution as close as possible to the best we can do.
Beside AI weaponry, a core aspect is AI irreducibility, which means that you cannot always know what a program will do before you start running it. In a context with such sensibility to hazard, mistakes can happen quickly, and make heavy casualties.
Irreducibility -> // <- Weaponry
Accidents and misuse are a huge concern.
UK less prepared for pandemic than pre-Covid, former vaccine chief warns
────────────────────────
Ressources
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
https://arxiv.org/abs/2306.11932
4. SKIM THE MANUAL | Intelligent Voluntary Cooperation & Paretotropism
https://foresightinstitute.substack.com/p/skim-the-manual
Citizen Science in Deliberative Systems: Participation, Epistemic Injustice, and Civic Empowerment
https://link.springer.com/article/10.1007/s11024-022-09467-8
Harnessing the Wisdom of the Crowd to Solve Complex Problems | Payam Aminpour | TEDxMSU
https://youtu.be/d5_E_0CIiBg
Computational Irreducibility
https://mathworld.wolfram.com/ComputationalIrreducibility.html
Report on modeling evidential cooperation in large worlds
https://arxiv.org/abs/2307.04879
Eric Beinhocker & Diane Coyle on Rethinking Economics for A Sustainable & Prosperous World
https://youtu.be/rB6CkhOglLI
Potentia — The Highest Moral Goal for Humanity | Dan Faggella
https://danfaggella.com/potentia/
OMNI: Open-endedness via Models of human Notions of Interestingness
https://arxiv.org/abs/2306.01711
What The Prisoner’s Dilemma Reveals About Life, The Universe, and Everything
https://youtu.be/mScpHTIi-kM
Benevolent AI Is a Bad Idea
https://www.palladiummag.com/2023/11/10/benevolent-ai-is-a-bad-idea/
If We Don’t Fix Sensemaking, We Won’t Survive
https://youtu.be/ya_p4RIorXw
Ground News
https://ground.news/about
How to Brainstorm better than ChatGPT with Knowledge Graphs and GPT 4 | InfraNodus Tutorial
https://youtu.be/XMwa9q6wWkw
Empowerment is (almost) All We Need
https://www.lesswrong.com/posts/JPHeENwRyXn9YFmXc/empowerment-is-almost-all-we-need
Frank Lantz | The Philosophy of Games | Win-Win Podcast
https://youtu.be/GTZRMKEHDu4
Tangle
https://www.readtangle.com/about/
The Goddess of Everything Else
https://youtu.be/Bbwp4PbWYzw
The consilience project
https://consilienceproject.org/
Jamie Wheal | Tackling the Meaning Crisis | Win-Win with Liv Boeree
https://youtu.be/vhJ3KkNlhaM
A pro-social protocol for people and the planet
https://docs.theos.io/introduction
Cris Moore on Algorithmic Justice & The Physics of Inference
https://youtu.be/jnNSWJSYXRo
Learning Altruistic Behaviours in Reinforcement Learning without External Rewards
https://arxiv.org/abs/2107.09598
Autonomy in Moral and Political Philosophy
https://plato.stanford.edu/entries/autonomy-moral/
Cultural universal
https://en.wikipedia.org/wiki/Cultural_universal
Open-Ended Evolution
https://alife.org/encyclopedia/introduction/open-ended-evolution/
Game B
https://www.gameb.wiki/index.php
Uninformed Individuals Promote Democratic Consensus in Animal Groups
https://www.researchgate.net/publication/51884215_Uninformed_Individuals_Promote_Democratic_Consensus_in_Animal_Groups
The Collective Intelligence Project
https://cip.org/whitepaper
CIP Ecosystem Map v0
https://docs.google.com/spreadsheets/d/18qCNZtoR_zh6Izpkx3N_YHCkCPdWgEBMguxh2QhCPJE
Harnessing Collective Intelligence for Dynamic AI Governance
https://www.ai4institutions.com/project-cards/blog-post-title-one-atyj7-p9zde-e4sa5-4lz4j-amdck-c2ak3
Tournesol
https://tournesol.app/
CaptainFact.
https://captainfact.io/
The Computational Democracy Project
https://compdemocracy.org/
Taiwan’s Digital Democracy
https://consilienceproject.org/taiwans-digital-democracy/
The diversity bonus in pooling local knowledge about complex problems
https://www.pnas.org/doi/10.1073/pnas.2016887118
Participedia
https://participedia.net/
MIT launches Center for Constructive Communication
https://news.mit.edu/2021/center-constructive-communication-0113
Computational Perspectives on Democracy
https://kilthub.cmu.edu/articles/thesis/Computational_Perspectives_on_Democracy/17041709
The moral imperative to learn from diverse phenomenal experiences
https://aeon.co/essays/the-moral-imperative-to-learn-from-diverse-phenomenal-experiences
Beyond Flatland: Machine learning and the end of the two-party binary
https://pdis.nat.gov.tw/en/blog/走出平面國-機器學習和二元黨派的終結/
Digital Democracy
https://www.nesta.org.uk/feature/future-curious/digital-democracy/
Ink & Switch
https://www.inkandswitch.com/
Dynamicland
https://dynamicland.org/
Pattern Catalogue
https://maggieappleton.com/patterns
Other Internet
https://otherinter.net/about/
Flow genome project
https://www.flowgenomeproject.com/
Using evolution science to build a prosocial world
https://www.bps.org.uk/psychologist/using-evolution-science-build-prosocial-world
A platform for connecting ideas and building knowledge.
https://www.are.na/
The Future of Digital Identity
https://dcbuilder.mirror.xyz/myIlus8pl6SbyuUR4ufGf9OYRps8hCGqeZHNYce3i94
Blockchains as a means to promote privacy protecting, access availing, incentive increasing, ELSI lessening DNA databases
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9871783/
Common Sense[Makers]
https://csensemakers.com/
Why we need an academic career path that combines science and art
https://youtu.be/c_rAeHPi23o
Operation outbreak
https://operationoutbreak.org/
Cooperative AI Foundation
https://www.cooperativeai.com/foundation
AI for institutions
https://www.ai4institutions.com/about
Democratic inputs to AI
https://openai.com/blog/democratic-inputs-to-ai
DemocracyNext
https://www.demnext.org/about
‘Generative CI’ through Collective Response Systems
https://arxiv.org/abs/2302.00672
New_ Public
https://newpublic.org/
Cortico
https://cortico.ai/
Fora
https://fora.io/
MIT Center for Constructive Communication
https://www.ccc.mit.edu/
How to fall in love with a system
https://aw.network/posts/how-to-fall-in-love-with-a-system
Playfulness and the meaningful life: an active inference perspective
https://academic.oup.com/nc/article/2023/1/niad024/7428705
Thinking together
https://subconscious.substack.com/p/thinking-together
Scale and information-processing thresholds in Holocene social evolution
https://www.nature.com/articles/s41467-020-16035-9
Social Centralization and Semantic Collapse: Hyperbolic Embeddings of Networks and Text
https://arxiv.org/abs/2001.09493
Input Crowd, Output Meaning
https://pol.is/home
DEF CON 31 — An Audacious Plan to Halt the Internet’s Ensh*ttification — Cory Doctorow
https://www.youtube.com/watch?v=rimtaSgGz_4
What do I think about Community Notes?
https://vitalik.eth.limo/general/2023/08/16/communitynotes.html
The Internet Could Be So Good. Really.
https://www.theatlantic.com/technology/archive/2023/10/social-media-platforms-business-models-dialogue-network/675655/
My techno-optimism
https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html
Adversarial Collaboration: The Next Science Reform
https://link.springer.com/chapter/10.1007/978-3-031-29148-7_32
How Platform Recommendation Systems Might Reduce Division and Strengthen Democracy
https://www.belfercenter.org/publication/bridging-based-ranking
The biology of fun and playfulness
https://www.sciencedaily.com/releases/2015/01/150105141659.htm
How we experience awe — and why it matters
https://www.ted.com/talks/beau_lotto_and_cirque_du_soleil_how_we_experience_awe_and_why_it_matters/transcript
A Thirst for Knowledge: Grounding Curiosity, Creativity, and Aesthetics in Memory and Reward Neural Systems
https://www.tandfonline.com/doi/full/10.1080/10400419.2023.2165748
How to Use Gameplay to Enhance Classroom Learning
https://www.edutopia.org/article/how-use-gameplay-enhance-classroom-learning/
The evolution of culture
https://takingchildrenseriously.com/the-evolution-of-culture/
All you need is links
https://subconscious.substack.com/p/all-you-need-is-links
Sunni Brown: Doodlers, unite!
https://www.youtube.com/watch?v=7fx0QcHyrFk
The Natural Laws of Children
https://www.celinealvarez.org/en/our-approach
Functions of Utopia: How Utopian Thinking Motivates Societal Engagement
https://journals.sagepub.com/doi/full/10.1177/0146167217748604
Analogy as the Core of Cognition
https://youtu.be/n8m7lFQ3njk
Daniel Schmachtenberger | Misalignment, AI & Moloch | Win-Win with Liv Boeree
https://youtu.be/KCSsKV5F4xc
Science Fiction and the Future of Science
https://council.science/podcast/science-fiction/
This Is How Power REALLY Works | Aaron Bastani Meets Grace Blakeley
https://youtu.be/Ru4rfzFasaI
Why Reasonableness Beats Rationality Every Time
https://youtu.be/Jd2yP6GC6hk
The Next Global Superpower Isn’t Who You Think | Ian Bremmer | TED
https://youtu.be/uiUPD-z9DTg
Blockchains as a means to promote privacy protecting, access availing, incentive increasing, ELSI lessening DNA databases
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9871783/
Evolutionary Psychology: Predictively Powerful or Riddled with Just-So Stories?
https://areomagazine.com/2020/10/20/evolutionary-psychology-predictively-powerful-or-riddled-with-just-so-stories/
Scale and information-processing thresholds in Holocene social evolution
https://www.nature.com/articles/s41467-020-16035-9
Cultural evolution is the change of culture over time.
https://culturalevolutionsociety.org/story/What_is_Cultural_Evolution
“There’s Nothing Mystical About the Idea that Ideas Change History”
https://quillette.com/2023/12/01/theres-nothing-mystical-about-the-idea-that-ideas-change-history/
Harry Potter and the Methods of Rationality
https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality#Reception
Climate change has toppled some civilizations but not others. Why?
https://grist.org/culture/climate-change-societal-collapse-explained/
Return of the oppressed
https://aeon.co/essays/history-tells-us-where-the-wealth-gap-leads
Cause, Effect, and the Structure of the Social World
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4445710
About seL4
https://sel4.systems/About/
From AI for people to AI for the world and the univese
https://gcrinstitute.org/papers/061_ai-world-universe.pdf
Unknown — Killer robot
https://netflix.fandom.com/wiki/Unknown:_Killer_Robots
The Capability Approach to Human Welfare
https://forum.effectivealtruism.org/posts/zy6jGPeFKHaoxKEfT/the-capability-approach-to-human-welfare
A psychological take on alignment
https://daystareld.com/psychology-agi-alignment/
Joseph Gordon-Levitt | AI and the Film Industry
https://youtu.be/Kcm51luS9J0
Daniel Schmachtenberger: Steering Civilization Away from Self-Destruction | Lex Fridman Podcast
https://youtu.be/hGRNUw559SE
The Hippocratic principle — Vanessa Kosoy
https://www.lesswrong.com/posts/dPmmuaz9szk26BkmD/shortform?commentId=jMitxvhFceaheD5zb
Resurrecting Gaia: harnessing the Free Energy Principle to preserve life as we know it
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10322209/
Paul Christiano — Preventing an AI Takeover
https://youtu.be/9AAhTLa0dT0
Participatory AI Risk Prioritization: Alignment Assembly Report
https://static1.squarespace.com/static/631d02b2dfa9482a32db47ec/t/6556228ccd929249f767a65c/1700143757657/Participatory+AI+Risk+Prioritization+%7C+CIP.pdf
Cyberbiosecurity Implications for the Laboratory of the Future
https://www.frontiersin.org/articles/10.3389/fbioe.2019.00182/full
AI suggested 40,000 new possible chemical weapons in just six hours
https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx
Cyberbiosecurity in Advanced Manufacturing Models
https://www.researchgate.net/publication/335614919_Cyberbiosecurity_in_Advanced_Manufacturing_Models
Synthetic Biology, Genome Editing, and the Risk of Bioterrorism
https://link.springer.com/article/10.1007/s11948-016-9868-9
How AI Is Supercharging Biosecurity
https://www.concentricbyginkgo.com/ai-and-biosecurity/
Here are a few belated takeaways on the DoD Biodefense Posture Review (BPR)
https://threadreaderapp.com/thread/1696114454130237516.html
We need a Science of Evals
https://www.apolloresearch.ai/blog/we-need-a-science-of-evals
Lessons from Rwanda: Building systems to protect against infectious diseases and biothreats
https://www.weforum.org/agenda/2024/01/systems-infectious-diseases-biothreats-rwanda/
Info hazard guidance for biosecurity discussions
https://docs.google.com/document/d/1VSfU3GiZumHDX2hoz3YY1PT2dQHtkbrfO8xLxI9BTGE/
Report launch: examining risks at the intersection of AI and bio
https://www.longtermresilience.org/post/report-launch-examining-risks-at-the-intersection-of-ai-and-bio
The International Biosecurity and Biosafety Initiative for Science (IBBIS)
https://ibbis.bio/
UK and U.S. announce new strategic partnership to tackle increased biological threats
https://www.gov.uk/government/news/uk-and-us-announce-new-strategic-partnership-to-tackle-increased-biological-threats
Coherent Extrapolated Volition
https://www.lesswrong.com/tag/coherent-extrapolated-volition
