The lexical-semantic hypothesis

Zombie, all too zombie

When GPT-3 came out in 2020, we were quite tickled by its ability to “continue” a piece of text in the prompted style. One musing thought was that “generating what’s next” is figuratively “telling us what’s to come”.

… if language describes the world, and text data embeds real-world knowledge, then linguistic plausibility could entail real-world plausibility

There you have it, the lexical-semantic hypothesis.

And to extrapolate from there:

… the machine has taken in such a vast amount of connections encoded in text — perhaps it knows something we don’t. Perhaps it could reveal ideas that are both novel and meaningful. You write down the past and it generates the future.

That is going a bit far. But we have a license if we pretend that we are thinking and talking about sci-fi.

The perceived quality of answers

Imagine we have a question-answering machine that is only capable of producing tautologies. You teach it “A ball is spherical”, then ask “What is a sphere?” It would answer: “A sphere is like a ball.”

At face value we might doubt the usefulness of such a thing, since it tells us nothing new. But what happens as you throw more data at it?

When lexicographers approach meaning, they collect pieces of evidence about how a word is used in a sentence, and use contextual examples to illuminate the semantic value of that word. Then, when you hold a very thick dictionary, you say this document tells us something about the meaning of various words, especially those unfamiliar to us.

AI cannot tell you about the current weather if it’s not connected to a weather monitoring system, neither can a human — because this information has to come from an external source. But what about information that is supposed to be entirely available from within the text? For example, it’s totally doable to give a weather report as input, then ask a machine or human to extract whether the report mentions raining.

The perceived usefulness of a question-answering machine, then, depends on two things. Firstly, where is the source of the requested information? Sometimes the answer is not in the input data, sometime it is — and that is the case for the lexical-semantic hypothesis. Secondly, you would perceive the answer as more useful, if the answer seems to present new information to you, and not just spewing out repetitive patterns.

The second part makes the concept of utility subjective. But subjective doesn’t mean it’s not sensible. It’s rather very intuitive. Human emotions are also motivated by gaining information. Imagine you have a missing package, you ask the bot about its status, the bot only tells you “the expected delivery time is X days, based on your input, it has been X+2 days, sorry to hear it’s delayed”. That would be an annoying non-answer, because it doesn’t give you any useful information to help with the problem’s resolution.

We have an intuition about the quality of information-presenting content. If a company hires a freelancer to write a so-called white paper, and the work is not very good, you could read it and instantly sense that it’s bland, overfilled with buzzwords, and without substance. Similar intuitions are at work when you see a presentation. You can sense how much new and solid information is being put forward. If for 30 minutes, the presenter has just rephrased what’s in the first paragraph of the Wikipedia page, you know it.

Is there a distinction between talking about a subject convincingly and knowing or understanding something about that subject? Maybe it depends on who you’re trying to convince. Does the audience know anything about the subject? Are they in a collective trance or paying attention? Very often, many of us can tell the difference.

At the current stage, machines have become very good at at spell-checking, enumerating, rephrasing, shoe-horning, and formulaic tabulation.

Internally extractable information can be helpful when presented in a different view. Information retrieval can be useful. We are basically talking about a search engine. If you ask the machine, “What is the Turing Test”, “What is Searle’s experiment”, it will happily supply details about such trivia. When your curiosity is not deemed trivial, you feel happy about getting the details. Even when there’s new knowledge being created, that information is new to you. But when you ask “How can one befriend trustworthy people”, apart from regurgitating prosaic non-answers, a machine can’t tell you much more, because it’s like solving an open problem. The information is not in there. Admittedly open questions are hard, even humans might not have any clue, but like machines many humans can generate a bunch of banal content.

In sum:

• There are questions that, given a bunch of input data, can be answered, cannot be answered, or can be answered but we don’t like the answer. The third category frustrates us. It’s also annoying to received non-answers to them, which we can often intuitively spot.
• The machine has a model of the world, but to appear that it understands your question, it also needs a model of your model of the world.

The ELIZA effect: the illusion of being understood

Chomsky was giving a lecture on the “cognitive revolution”. Amidst his rambling style, at one part he was talking about the definition of intelligence or being capable of language. For example, you may ask, what is flight, what is flying. Can a bee fly in stratosphere? In space? Then you really have to define “flying”. An eagle may be flying, a cannon ball may be flying, an aeroplane may be flying, but it’s a different kind of flying, you can even have different verbs in other languages to differentiate these.

Then Chomsky mocked the Turing Test, because now the definition would be, an aeroplane is flying, if it can fool a human into thinking it’s an eagle.

Descartes was hiding in the Netherlands and dissecting large animals to understand how the internal organs work. Scientists since then also built machines to understand mechanisms such as digestion, but they weren’t trying to fool anybody into thinking the machines were cows.

Credulousness of the beholder should not be a test of intelligence of the beheld. The idea of “can’t tell if it’s a person” invites a “not sure” meme.

Now what is happening when we get the feeling that we’re talking to a machine that is “talking” back?

Joseph Weizenbaum made a bot in the 1960s called ELIZA that had no intelligent mechanisms for conversation, but simply mirrored back keywords and used stocked phrases to invite the human to say more in the style of a Rogerian therapist. The urban lore was that many people started seriously talking to ELIZA, including Weizenbaum’s secretary, who was wholeheartedly pouring out her personal problems to the bot.

The premises of the Turing Test naturally comes with this ELIZA effect, where the perception of the existence of an interlocutor’s consciousness is hastened by the human’s willingness to participate in such beliefs.

And humans are often more than willing. People used to play text-based adventure games and were totally excited about that. VR rollercoaster simulation doesn’t need hi-res graphics to make you feel that everything is really moving. All you need is the blissful suspension of disbelief.

In dreams you can converse with characters, even have dialogues, but everything comes from your own solipsistic fabrication, from only memories and imaginations. In a dream, perception feels real, otherwise you become lucid.

If you replay a scene over and over from a film where a character is saying something, there is no interlocutor, there’s no speech, there’s no mind, the sequence of still images, when flashed in front of your eyes at 24 frames a second, creates a simulacrum of mental life in your brain.

Therefore, when someone claims “I just had a conversation with a machine, and it’s so good at it”, that says more about the delusional mental state of the human, rather than the capabilities of the machine, which of course had been in no illusory terms advancing.

When we say that the machine has passed many tests, or has gained incredible capabilities in language-based tasks, it only highlights how many tests we have set up are poor proxies and how many jobs humans do are bullshit jobs. GPT cannot heat homes, bake bread, collect garbage, fix plumbing, clean hospitals, or make trains not break down, meanwhile it can generate marketing content and pass some exams. We have not so much encountered machines that show intelligence but rather defined many tasks that are tests of bullshit intelligence.

Turning the lights on

One question naturally is, how is it different for a human? Not just because the circuitry is built in carbon, suddenly we have a soul. You may argue that an intelligence that can be turned out, turned off, then turned back on again is by definition not conscious; that somehow sentience is defined by fragility, and I would agree.

Many aspects of our being are like that of an automaton, we can breathe without thinking, so can it be with walking, riding a bicycle, driving to work, munching on popcorn while watching a movie. All these can be done mindlessly. We can also engage in smalltalk without thinking too much. We can follow certain formulae to spellcheck, compose sentences, paragraphs, even whole articles for “content marketing”.

In what sense are we not like the machines, when it comes to cognition?

My hypothesis is that, humans can generate more than tautologies, more than the aggregate of the information in the text. We can connect the dots, and not only that, through this process, get more information than plausible implicated facts from a pool of semantic memory. We can analyse and synthesise to arrive at more than what can be aggregated from the verbatim. In other words, humans have insights. We can read descriptions of the world, and reason about the world, then come up with novel and meaningful ideas that were not in the descriptions.

Of course there’s another plot twist, one suggestion is that the “next-word predictor” is a way of bootstrapping intelligence as you scale up the data, that intelligence is an “emergent phenomenon” (Dan Gilbert, maybe) in the way that a brain is more than a large number of cells. This claim is rather counter-intuitive, it’s like saying when Frankenstein zaps a corpse with 200 volts, it’s nothing more than slow barbecue, but when he ramps it up to 200 thousand volts, he creates life.

But then, it’s possible. If “memory is the mother of all intelligence”, machines are at least very good at memory. Unexpected capabilities emerge, internal models also emerge as the layers layer up. It’s like that talk by Jeff Hawkins, a few years before the first iPhone — intelligence is about making predictions. Being able to tell us something new, especially about something yet to come, would be very intelligent.