Will the Crowd Listen?

Improving language interpretation can guide us to practical truth, empower collective action, and yes, solve fake news

Poor language interpretation is holding our communities back. The built-in ambiguities of regular speech multiply into plagues of confusion in the huge, churning media world of today. We can turn back the tide by forging our different interpretations into a new tool for truth: the verifiability of language.

The internet makes communities more powerful than ever, but that power is hamstrung by our inability to agree on what things mean when we don’t share enough culture and context. Ask a friend or coworker what the best way to cook a steak is, and you’ll get a single, digestible answer. Ask the internet, and you get chaos: almost 6 million “best” ways to cook a steak? Something tastes off here.

To find the one really, truly best steak recipe, you’d want to find some hard evidence supporting each recipes’ claim to the throne. But here’s the (dry) rub: what kind of evidence proves a recipe is “best”? Most cooked with? Most eaten? Used by most top restaurants? It’s not clear because the definition of “best” isn’t clear. People don’t agree on what makes a thing best of its kind, and so claiming your recipe best is never truthful, always histrionic.¹

The term “fake news” works the same way: google “what is fake news?” and you get 6.2 million answers. A claim to have the correct definition of fake news or the best steak recipe isn’t verifiable, no matter your evidence.

Treating verifiability as a prerequisite to proof can upgrade our language interpretation, aiding our search for “the facts” we fetishize and upping the coherence of our collective actions. To begin with, individuals might start applying the idea to headlines, ads, and other diverse-audience communications. On the horizon are the possibilities of codifying it into truth-seeking ecosystems and AIs powered by user-submitted interpretations.

Interpretation’s broke

Meaning in language has always been ambiguous. Explaining why is a bit of an ouroborosian paradox, but this argument requires an attempt. I’ve got three bullets in the chamber:

First, words tend to have several meanings, as we can see flipping through a dictionary, and this opens the possibility of mistaking one definition for another. If I ask, “was Robert fair?”, I might be asking if he was just or if he was pale. “There’s smoke up there — are we under fire?” Are we being shot at or are we looking up a hill towards flames? These may not be the most realistic examples, but they illustrate the principle.

Second, words also undergo semantic change as time passes and as communities separate. Why this happens is another essay, but we have anecdotal and empirical evidence that it does occur. A perfect example of meaning shifting in time is “gay.” In 1950, it was typically paired with “happy”; today with “lesbian.” An example of difference in meaning between communities is “test,” associated with cricket in India and exams in the US.

Last, the mechanisms of human language interpretation are unknown and unique to each person. Computers use programs called interpreters (or their counterparts, compilers) to translate high level code humans write into machine code the computer can execute. English is also a high level code we write, and it too gets translated into lower level instructions, the kind that contract our muscles, fire our neurons, and so on. Our brains must be doing a similar conversion from English into cell/body signals, but there’s a big difference: we have a full understanding of computer interpreters that we lack for language. Every character in an interpreter program is known and standardized so everyone gets the same results from the same high-level instructions. Switching to English, if you ask 10 people to prepare the exact same steak recipe, you’re bound to get 10 different steaks. Since the instructions were identical, you might blame the cooks’ interpreters for the differences.

We should fix it

Improving our language interpretation is relevant to the search for truth and to the productivity of our communities. Truth and knowledge are strange subjects. Their natures have evaded epistemologists for millennia, while to layman they’re common sense. We’re not cracking the code here, but I do want to offer the possibility that our language interpretation troubles figure into our sloppy definition of “knowledge.” If you accept the inevitable ambiguity of language, especially its more abstract sides, the dearth of a precise “truth” concept is not a surprise. I’m not sure we need a better intellectual definition of “truth,” but there is at least a focused group of people deeply invested in epistemology, and perhaps this essay offers them something.

More directly, flawed language interpretation may be restricting the potential of large-scale collaboration. In culturally stable settings, ambiguity might actually be enhancing the efficiency of language. Unspoken context and various kinds of repetition let you say your piece quicker without much meaning lost, but only if you share much of your language interpretation style with your listeners. Greater differences in language interpretation equal less understanding. The way you interpret words and their context emerges from your own context: your work, your friends, your culture.

If language misinterpretation is proportional to cultural difference, communities with the greatest cultural divides will struggle most. Since a static community’s culture unifies in time, we should focus analysis on dynamic groups, where otherwise separate community-cultures are temporarily or permanently joined. I haven’t found any empirical research on this (if you know some, please link it in a reply!), but anecdotal examples are aplenty.

Politics is one. Elections, ballot measures, and town halls necessitate the gathering of normally separate districts, and these gatherings can highlight language interpretation differences. When our very own D.T. tweets that the U.S. Postal Service “LOSE A FORTUNE” working with Amazon, we see at least two distinct community responses. One side says he’s lying, that the USPS profits from Amazon’s business, and another agrees with him, arguing that the USPS isn’t charging Amazon enough (plus a bunch more off-topic replies, of course). This divide isn’t just opinion; it reflects an interpretational difference. The naysayers might be interpreting Trump’s statement as “the USPS nets a loss delivering Amazon packages,” in which case he is misinformed or lying, while supporters read “the USPS is missing a huge profit opportunity,” which could be true.

Journalism is a second example. Popular news outlets reach loads of different communities, opening their reporting up to easy misinterpretation. This NYT opinion piece is subtitled “Fascism poses a more serious threat now than at any time since the end of World War II,” and scrolling through its Twitter comments, there are claims that 1) no, it was actually the previous administration that was closer to Fascism; 2) yes, the current administration perfectly fits the “the bullet point list of what constitutes Fascism”; and 3) a further reply that “no such thing as ‘bullet point fascism’” exists. The first two commenters could conceivably be disagreeing only about the specific actions the two administrations took, while agreeing on what “Fascism” means, but that’s far fetched. The second two clearly interpret “Fascism” differently, and without a more shared definition, the status of the NYT piece as true or false is held in limbo.

Language is ubiquitous in human life, and so are its flaws. Precise measurement of this particular flaw’s impact on our lives is impossible, but you can sense it looming behind some of our most discussed online issues:

• Fake news — What it is exactly no one knows, but most agree it’s related to sifting fact from fiction, and interpretation is foundational to that process.
• Doublespeak — Language that pretends one side but plays another, like a politician claiming to support free speech as an excuse to gain votes from controversial constituents. Interpretation plays a role in understanding how one statement can play both sides.
• Echo chambers — Again, there’s no consensus on what these are, but a reasonable assumption might be: we read what makes sense to us, and posts or articles using others’ cultural context for meaning can appear incoherent or dishonest in our interpretation.
• Internet arguments — People get angry online, and they waste lots of keystrokes on arguments sparked by different interpretations rather than different values.

Cross-cultural gatherings may be more common today than ever before because of the bulk and radical accessibility of online communities. If solving these puzzles is important for those communities’ futures, so is improving language interpretation.

Improving the recipe

We can address this weakness by paying attention to the “verifiability” of word meanings. A word or passage’s verifiability is a theoretical rating proportional to the variance of what it is interpreted to physically represent in the world. That is to say, words are defined by the way we use them. The more similarly we use a word, the more communicative it is, and the higher its verifiability is. A word like “dog” tends toward high verifiability since people mostly agree on what it represents in the physical world. You can point at a dog, and most people will probably agree that’s what it is. “Justice,” on the other hand, likely has low verifiability since people often disagree on whether a trial’s outcome is just.² If a passage is highly verifiable, valid evidence indicates it’s true, for practical purposes.³ “I saw a dog today” is quite verifiable, so if I show you a picture I took today of me and a dog, that’s great evidence that my statement is true. “I saw a just trial today” has low verifiability, so even if I’ve got a video of the judge handing down the verdict, I haven’t proven anything if you don’t share my definition of a “just trial.”

No empirical measurement of verifiability exists (this may change, as we’ll discuss), but luckily it’s reasonable to intuit. As you read or listen, speak or write, ask yourself, “would most people interpret this the same way I would? How different would their interpretations be?” More technically, does the word or passage have more than one common meaning in this context? Does it have a very broad definition? Would most people agree with you about what real world examples fit your writing? The more varied the interpretations you can imagine others reasonably taking, the less verifiable a passage is, and the less effective it is at communicating to a diverse audience. Let’s look at some new examples through this lens:

• “Van drives into pedestrians in Germany” — Skateboarding shoe bumps into boring people in Germany, Indiana? I don’t think so. High Verifiability
• “Paris to turn a third of its green space into urban farms” — I rate Paris, turning, and urban farms as fairly verifiable, but what is green space? Parks? Anywhere in the city with grass? Medium Verifiability
• “Healthcare is a human right” — Healthcare can be everything from jogging and diet to injections and liver transplants. What is it here? Human rights: we declare these. What makes something a human right? Low Verifiability

And we revisit the earlier cases:

• “THEY LOSE A FORTUNE” — At least two common meanings are imaginable: did their total amount of money decrease, or did they miss an opportunity to make a fortune? Low Verifiability
• Fascism poses a more serious threat now than at any time since the end of World War II — “Fascism” is extraordinarily ambiguous: look how general and unverifiable Merriam-Webster’s definition is. Low Verifiability

These are headlines and excerpts, but the same evaluation process can be applied to every sentence and paragraph in the full articles. If a passage is highly verifiable, look for physical evidence — photos, video, peer-reviewed studies, eye-witness accounts. If you’ve got evidence for a verifiable claim, you have a usable “fact” that most of an audience will agree with. Evidence can’t prove an unverifiable claim, because there is no claim to prove when the audience has sufficient disagreement on its meaning.

Understanding verifiability doesn’t stop creators from being biased, telling one-sided stories, or lying. It doesn’t mean readers won’t judge content differently in terms of values, relevance, or interest. But it can help consistently identify practical truth in our media and conversations, which is powerful and needed. We can all use it to evaluate what we hear and say by changing a few habits, and if we do, our communities will sink less energy into arguments over unverifiable ideas.

Coding it in

The cynic in me points out that the just-explain-why-people-should-change technique is a pretty impotent culture shifter, so if verifiability is indeed a useful idea, we might end up implementing it in a more codified, automatic way. Two possibilities come to mind: 1) an ecosystem that crowdsources verifiability in popular media, and 2) a machine approach that learns to rate verifiability in real time using our huge corpus of online content.

First, let’s imagine an ecosystem that incentivizes readers to collaborate on real-time verifiability ratings of popular statements. A comparable system already exists: CAPTCHA and its descendants started as a way to discern humans from machines, but over the years they’ve been dual-purposed to provide training data for machine learning. We might use popular headlines, article excerpts, and social media posts as the input for a captcha-style system that asks users to output their interpretation — to rewrite it in their own words. (The content could be gated by the captcha, or users could be paid to do the captchas a la Mechanical Turk (yuck)).

The user-created interpretations would be fed into a distributional semantic analysis algorithm like the one used by Kulkarni, Perozzi, Skiena, and Al-Rfou. Their method approximates a word’s meaning through its most common neighbors:

“The distributional hypothesis states that words appearing in similar contexts are semantically similar. Distributional methods learn a semantic space that maps words to continuous vector space R [of] d, where d is the dimension of the vector space. Thus, vector representations of words appearing in similar contexts will be close to each other.” (Statistically Significant Detection of Linguistic Change, 3)

Instead of tracking semantic shift, we’d be seeking the dispersion of users’ interpretations, which is a good approximation of the passage’s verifiability. With a couple hundred user interpretations, a fairly accurate verifiability rating for any short piece of media could be within reach.⁴ From here, the same technique might be leveraged to crowdsource evidence for high verifiability passages. There’s probably a decentralized fact-checking ecosystem-on-a-blockchain idea in here somewhere…⁵

The next use case is a real-time, decentralized dictionary. If we periodically applied Kulkarni and co.’s distributional method to every word appearing in the publicly available corpus of social media, we could in theory create an automated dictionary that exactly reflects current online word usage. Each word’s entry would contain its verifiability (the dispersion of its context distribution) and most common usages (any distinct clusters of common partner words). This wouldn’t be a dictionary of traditional definitions, but we don’t really learn words from traditional definitions anyway. It would provide a valuable window into the big, changing picture of language and the verifiability of brazen claims plastering the walls of the internet.

Could machines learn to rate the verifiability of entire sentences or articles? Feed a deep learning system enough example interpretations from real people, and perhaps it could learn to predict the verifiability of brand new passages. In time, we might create a truly shared language interpreter program, compressing our unfathomable stigmergic semantics into a communication guide anyone can use. We are already attempting computer language definitions of historically human terms: “security,” “privacy,” “consensus,” and “reputation” have all been defined in code in one way or another. As the internet further infiltrates life, code’s power to effect worldly change will grow, and so more concepts will be reborn in programmed forms. Such coded definitions will never be perfect, but neither are our interior senses for words. Unlike our natural language processors, coded definitions are shared openly and interpreted identically everywhere.

This is a profound change leading us to a land we’ve never been. (If you interpret this essay the way I do, that is 🙂)

Let’s hope our collective conscience steers us well.

Footnotes

¹ “Recipe” suffers the same disease: How similar must two recipes be to be considered the same? Are all sous vide methods the same recipe? Is it even possible to repeat a recipe at all?! You can chase this metaphorical goose literally forever.

² Context is key. “He’s a real dog!” just as likely refers to a person, and you miss that focusing only on the word “dog.” The idea of verifiability is a simple one; actually measuring it is not.

³ “Practical truth” here means functional ideas, not the inexpressible, divine truth. We’re talking “the sky is blue” and “the speed limit is 65,” not the sublime beauty of art nor the complete universal theory of physics.

⁴ This verifiability rating would be valid only for communities adequately represented by the interpreting users!

⁵ Why not skip verifiability and go straight to crowdsourcing evidence? Because finding evidence is hard, crowdsourcing verifiability is comparably easy, and efficiency is central for internet-scale problems. A flow from verifiability to verification could be the most efficient way to reach practical truth about contentious topics on a mass scale.