Hangout Highlights — 14 May, 2021

Today's discussion spanned several loosely-connected topics: UI, Curation, the fundamental nature of Claims (as usual!), domain/OO modeling, to name a few. The discussion was also about, and simultaneously a perfect example of the misunderstandings we can have in these discussions that (hopefully unnecessarily) take up an inordinate amount of our time. (I have said more than once that if we could all spend a single week together in person, we'd be years ahead on this project).

Misunderstandings

After the hangout on May 12, Steven and I started a loooong thread in our Slack Team regarding the proper way to handle what Steven was calling argumentText. In the end, it became clear to me that about 75% of our discussion was spent on arguing against what we had imagined the other person was proposing, rather than the real deal. The disagreement was at times so absurd that both of us remarked on how shocked we were at the supposedly ridiculous proposals of the other.

Of course, both of us were actually being very reasonable, and that for all that, there were only a couple very minor actual disagreements (discussed below). Here are the lessons learned there:

1. When you're talking to someone (especially when they know their stuff) and they seem to be saying something ridiculous or idiotic, it's probably you that misunderstood them
2. In general, but especially in these cases, it's a good idea to adopt the strategy of repeating back to them what you understood, in your own words, to make sure you really got their point
3. Diagrams, when possible, can really help to clear things up quickly (see examples below)

Of course, I learned these lessons after the written argument, and yet we repeated the same errors in the hangout. Knowing is only half the battle.

What is argumentText?

In our prior activities through the "bottom-up" process that I was running, we had managed to agree on a very simplified class diagram for our base layer.

Simplified simplified model

The relevant concepts here are:

• A Claim is the true/false proposition that one can make and debate (e.g. "The table is made of wood")
• A Claim can be the negation of another Claim, such that if one is true, the negation must be false (e.g. "The table is not made of wood" — note that that "The table is made of iron" is not a negation, but rather a mutually-exclusive Claim, not modeled here)
• Arguments for or against a Claim are represented by the Support class, which has a Conclusion (Claim) that it supports, and a Premise (Claim) upon which it's based (e.g. "The table is brown" supports "The table is made of wood" — note here that "The table is brown" is itself a separate Claim, and the Supports instance connects it to the Conclusion)
• Support extends Claim, which means that Support is itself a Claim. This means it can have negations ("The table is brown" does not support "The table is made of wood"), and can itself be the Conclusion of other Supports (e.g. "Wood is brown" supports ("The table is brown" supports "The table is made of wood"))

There are many other interesting aspects of this model that have been discussed to exhaustion in previous hangouts. I will document them as they come up again in future discussions.

The confusion between Steven and I arose out of a simple disagreement:

• In Socratrees, Steven does not support user-entered text for the Support class — only Claims have user-entered text
• In the defunct Gruff, I have always supported optional user-entered text for the Support (e.g. "It's brown" instead of "The table is brown") for the purpose of readability

Given the model presented above, anyone that can read UML should understand there's a problem in each approach: If Claim has a Statement (the text entered by the user), and it is required for Claims (which everyone in the CDL has more or less agreed on), the model requires Support to have a required user-entered Statement, which conflicts with both of our objectives.

Stephen's fundamental question to me was "How do you handle argumentText?", which, after much confusion, I came to understand as this: "What is your solution for user-entered text in Supports?" The question isn't as simple as it sounds, because it touches on three different realms of focus: the model, the user interface, and the process of Curation.

Modeling argumentText

The solution to the problem of user-entered text (I'll call it Statement, as per the model, but if you watch the video, you'll see that even that generated a lot of confusion) is actually very obvious and simple. If text is to be required in one class, and optional in the other, you can model this using an abstract parent class:

Statement is required in Claim, and optional in Support

Likewise, if you don't want user-entered text at all for the Support class, you can just change the Support model:

No Statement…

One other point discussed was the possibility of auto-generating the text, which Socratrees does when showing a Support.

On the contrary

The example above actually represents the case of a Support supporting the Negation of the Claim, but in either case, the text is autogenerate using the formula:

"Assuming that " + Premise.Statement + " does it oppose that " + Conclusion.Statement + "?"

This isn't the only case of auto-generated text possible. We discuss in the hangout, and elsewhere, that in the case where a Claim has been fully disambiguated (aka Context in the white paper), it should be possible to generate a reasonable Statement, even for the Claim. I'm sure we'll discuss that at another point in more depth, so I'll just leave it here with the concept of Statement as a method in the abstract class that can support a separate implementation in each subtype:

Make a statement

The Problem With User-entered Argument Text

There's a problem that is common to any argument map interface I've seen so far:

• Each Claim created by a user should be reusable in any debate, and should be debatable on its own terms
• Users usually create Claims as Supports for another Claim (mixing terms here, we often say “in the context of another Claim”)

The problem comes from the natural tendency to argue within the context of the debate, and, specifically to abbreviate with pronouns: "Paris is a great city" because (is supported by) "It is nice in the summer".

According to our model, you cannot have a Support without a Premise Claim. However, it's very common that as users argue their point, they require the use of Claims that don't previously exist. That is, users quite often are creating both a Support and a Claim at the same time. Given our model, there are many different ways to deal with this conflict:

1. Require the user to enter a text for the context-free (reusable) Claim: "Paris is nice in the summer"
2. Allow the user to enter context-specific (or, as Steven calls it, "indexical") text, and save the same text to both the Support and the Claim: "It is nice in the summer"
3. Require the user (or offer them option) to define a separate text for the Support and the Claim: "It is nice in the summer" / "Paris is nice in the summer"
4. (Sometime in the future, perhaps) Use NLP and/or disambiguation elements to suggest to the user replacements for pronouns they used in their statement: "It (did you mean Paris?) is nice in the summer"
5. Inherit disambiguation elements into the Claim (not represented yet in our model) to help explain the text: "It is nice in the summer [context: City-Paris, France]"

A solution with problems

Prevention vs. Curation

The first solution above is the one adopted in Socratrees. The problem is that when you allow the whole world into your system, there's no guaranteed way to programmatically ensure that users enter text according to your requirements. It's possible to help users understand the proper format, as with option 4 above, but it can never be fully guaranteed.

Socratrees solves this problem through prevention: specifically, a gating/publishing mechanism. When a Claim or Support is submitted, it doesn't appear to other users in the system until after an administrator has approved it. The approval process may include actually changing the text to fix grammatical issues, or fix problems like indexicality (e.g. pronouns). I believe Steven mentioned that one of the approaches is to teach users the proper use of the system, which means the process may involve asking the user to fix their own text.

Prevention or gating has a lot of advantages in terms of ensuring the quality and consistency of submissions. There are a few trade offs, however. First, there are scalability constraints related to the ratio of users with approval powers versus the number of Claims or Supports being added per day. Second, and possibly more serious, is the possibility for conflicts of interest, or at least the appearance of such. When there is a small group of users with control over the system, it opens the door to accusations of bias towards one viewpoint or another (e.g. accusations that fact-checking web site Snopes can't be trusted because they are funded by George Soros). Care would have to be taken (probably in the fundamental design!) to avoid that from happening.

On the other hand, solution 2 has its own problems. The system would have to tolerate the presence of very poorly-worded Claims, at least temporarily. There would need to be a process (e.g. Curation) for improving the Statement of a Claim over time. This is a very difficult design problem, and was briefly discussed during the hangouts, but I'm sure we will go into more depth in future meetings.

Immutability?

The two approaches also imply another fundamental disagreement with regards to the system: as a Claim is improved (either its text, or through incremental formalization, which we will have to go into another time), does the Claim change, or is a new version/copy made?

This will take us a long time to untangle yet, but here are some of the goals we have:

• The system should be fully transparent. If a Claim can be modified, the full history should still be accessible.
• Every time a Claim is changed, it could impact all previous voting, which means users may have to start from scratch. It's not clear if there are enough strategies to make this reasonable in practice.
• There is debate as to whether or not all versions of a Claim are "useful" and should still be debatable (i.e. receive new Supports, or be used as a Premise) after they have been modified.

When Indexical is not Indexical

Steven brought up a shocking revelation at the end of our hangout: "It is nice in the summer" is actually valid as a stand-alone claim, and he might actually accept such a statement were it made in support of "Paris is nice".

This is as complicated as it is surprising. He's right about this: "It's nice in the summer" is a valid Claim. However, if one were to happen upon the statement free of context, they would probably understand it to mean "The weather is nice in the summer (in general)", or something along those lines. They certainly wouldn't imagine that it applies specifically to Paris, France.

Most statements like this would not pose the same problem. None of the statements "It has a good subway system", "It has the Eiffel Tower" and "It has good restaurants" could stand on their own, for example.

When asked how it should be dealt with, Steven proposed that "It's nice in the summer" should be countered by "The fact that summer is nice everywhere has nothing to do with proving Paris is a great city", or something to the effect. Then, someone else could make the actual, relevant argument: "Paris is nice in the summer".

There were protests that this would only add to the confusion, but the hangout was over, and we can (possibly) revisit this another time.

Is Nice nice?