SkyNet Journeys [2]: The Turing Test

So here we are, in the second installment of the journey we started last time to investigate the idea of Intelligent Machines — and its time to take a tour and visit the Turing Test (wow, so many T’s!).

Imagine that you need to contact your cellular service provider to resolve some issue. You go to their website, and instead of calling the Customer Service number, you click on the “chat” icon. Within half a minute you start a chat conversation with a representative, who goes by the name of Geppetto. Yes, I know, a rather unusual name but that’s his name… he’s suffered enough from kids at school laughing at him, so give him a break, will ya? Anyway, Geppetto graciously helps you out. It takes a few minutes — he asks a few questions, you answer and ask a few in return, several clarifications are needed and made and eventually the matter for which you called is resolved.

And then, just you are about to “x” the browser window with the chat, a pop-up appears on the screen. “Click here for a $15 gift certificate to your favorite coffee shop in return for some feedback!” Eagerly you proceed, and then a single question appears on the screen:

“Thank you for agreeing to answer our survey. In your estimation, did you just speak with Geppetto, our kindly human representative, or with ChatGPT, our new advanced AI-assistant, nicknamed Geppetto?

I suppose you haven’t been in such a situation as of late, but if you ever are, it might interest you to know that someone is trying to run you through a version of the Turing Test — a test proposed by the mathematician Alan Turing in his 1950 article in the scientific journal MIND. He did not speak of cellular companies, and his life ended way before the Internet or Chatbots were ever conceived, but within the confines of what he knew then, he came very close to outlining this scenario. For him, the more computers were able to convince people via chat that they had spoken with the kindly old man Geppetto, the closer those computers were to machines that could actually think like humans.

Today we will dive into the famous article he wrote, in 1950, in which he made the case for the above claim. Personally, I find that one can learn much from touching these proto-sources, the places where ideas germinated and began to take shape. In these early stages of a field, the ideas are proposed and presented to the readers in a more straightforward manner, one that is not obscured by professional jargon. And indeed, in this sense Turing’s paper is excellent. Even though it is a paper with technological goals written by one of the brightest minds in computer science of that time, most of it is presented more as a kind of thought experiment, and in a language accessible to all, requiring no mathematical or computer science background to understand. So before we begin, I would like to recommend that you read Turing’s paper for yourself (click on the PDF icon in the linked page to download it). As long as you have a reasonable command of English, it is worth reading Sections 1, 2, and 6. Beyond the big questions he discusses, there are also some cool curiosities to be found there. For example, in Section 6 Turing shares his opinion on the existence of people with… ESP powers, such as mind-reading! If only for those curious trivia nuggets, I am glad I spent the time reviewing the paper :)

Let us begin with some biographical background— who was Alan Turing?

Alan Turing was born in 1912 in London, and was educated at King’s College, Cambridge, and Princeton University, where he wrote his doctoral dissertation. Today we would call Turing a computer scientist, but since the field known as “Computer Science” had not yet been invented, he was considered a mathematician. He invented some of the basic concepts and systems of thought that are accepted in Computer Science today, some of which I hope we touch upon in this series.

After completing his doctorate at Princeton in 1938, Turing joined the British government’s wartime intelligence service in WWII as part of a governmental codebreaking unit. His efforts led to the cracking of the German Enigma code, which the Germans used in their military, and this breakthrough contributed mightily to the Allied war effort. Turing’s experiences during the War led him to continue work on the design and construction of early computers, and in his work during those years he laid the foundations for several major areas of computer science, including artificial intelligence.

Unfortunately, what was sure to be a brilliant career was cut short. In 1952, Turing was arrested under the British laws against homosexual relations, and the authorities required him to undergo hormone treatment to treat his sexual orientation, and two years later he died of cyanide poisoning, apparently as part of a suicide attempt. Thus, beyond the terrible personal tragedy of his loss, the world was deprived of the influence of one of the most central figures responsible for the birthing of computer science in those formative years. (If you are interested, you might also wish to watch the Hollywood movie made about his life, entitled The Imitation Game. I cannot vouche for its historical accuracy — just sharing here.)

So that’s who Alan Turing was, and now we can finally get to our subject, the famous Turing Test. In 1950, Turing published an article entitled “Computing Machinery and Intelligence”. In this article he described a game that will remind you of the story with which we opened, the one with Geppetto: a man, let us call him the “interrogator”, sits down and chats via a chat program with two “entities”: a person and a computer. The computer has been programmed to do its best to converse like a normal person via the chat. The interrogator can chat freely with the two characters, ask each of them as many and any questions that he or she wants, and after a few minutes of this they guess which of the two characters is the real person, and which — the computer. This game he called “The Imitation Game” for the obvious reason that the computer is trying to imitate human conversation at a sufficiently high level of accuracy so that it can deceive the interrogator.

It is worth noting that the computer in this game has a harder time “faking humanity” than Geppetto did in the story we started with. In the Turing Test, the interrogator knows that he is in an experiment, and that one of the two players is a computer trying to deceive him, and so he can ask a lot of trick questions that you, in a conversation with a cellular representative, might not think to ask. Suppose, for example, you ask the interlocutor a difficult question in mathematics. If he answers it very quickly, you can use that signal to guess that he is a computer! On the other hand… perhaps the other party simply used a calculator? It can still be confusing, I guess.

So this is the Turing Test, and this game is, well — a game. That’s all. If someone were to invent it today it would probably be glossed over as it drowned in the sea of computer games that are released every day. There are also some “Turing games” online (such as here) that you are welcome to try your hand at if you feel like testing your ability to identify computers by their talent at small talk. But in his paper Turing proposed that it was more than a game — he proposed that if computers could win his Imitation Game often enough, then in his opinion this would be tantamount to saying that computers could think, i.e. that they had intelligence.

Now, this statement of Turing’s may sound simple to some of you, while for others it might cause their brain to explode with questions and challenges. And indeed, the more time I spend thinking about it lately, the more I realize how deep and far-reaching the implications of this statement are, specifically for the way we think about the whole issue of artificial intelligence. We will return to this statement again and again throughout the series, from different angles, and there will be things that I will present in general today that we may sharpen in the following posts, so if any of my readers are experts on Alan Turing, please forgive me for such inaccuracies that you detect. Throughout the series, I will do my best to reveal, gradually, the deeper portions of the iceberg whose tip we see today. For now — we begin with the basics.

Let us consider Turing’s proposal — that if a computer can convincingly simulate human conversation, we can say that it is intelligent. For starters, it might be easier for us to review the flipped claim: that IF the computer were intelligent, THEN we would expect it to be able to carry on a convincing conversation with us, indistinguishable from speaking to a human. Would you agree to that claim?

At first glance, it sounds reasonable. I mean, we would expect anything truly intelligent to understand our questions, to know how to answer them, to ask questions of its own, digest our answers, and so on. If there is an intelligent entity, then barring some language swapping, we should be able to communicate with it.

However, there is room for some caution in even asserting that claim. Consider this: some animals have intelligence, which enables them to learn about their environment and so forth, even though they cannot communicate with us through chats. Another objection could be from the opposite direction — who says that extra-terrestrial super-intelligences, far more intelligent than we, would be able to communicate with us and pass the Turing Test? We are super-intelligent compared to animals, and yet we cannot speak the language of animals . So — these are good questions, and they sharpen what Turing seems to have intended: not just to reach intelligence, but something akin to human intelligence, one that we also feel is universal enough in some sense so that that intelligent aliens would be able to communicate with us through it (We would assume, for example, that also aliens would have to have some conception of mathematics, and science, which we would share).

In any event, getting back to his paper — Turing’s revolutionary proposal was precisely the opposite claim: that any entity capable of communicating convincingly as a human being must be intelligent, sentient, or at least considered such. And that includes, first and foremost, sufficiently sophisticated computers.

So what do you think? If GPT, our Geppetto, had managed to fool you enough times, would you say that it was an intelligent being?

You might have your doubts, but others are totally on board with this concept. Take Blake Lemoine, a software engineer fired from Google in 2022. Blake was given access in 2022 to a system called LaMDA — you can think of it as a Googles internal GPT, with a bunch of add-ons. Blakes task was to examine the behavior of this system in various ways, which led him to have many conversations with it. After a prolonged period of this he began to feel that LaMDA was an intelligent being and even more than that — a being with consciousness. He began to demand that Google treat this software as a living creature, with rights and all, and persisted at this until the company finally decided to fire him.

Lemoine did a lot of interviews in the months that followed. It is worth remembering that he was fired before GPT was made public, so many people were curious to hear about a program that could have a deep conversation about any topic under the sun, as well as about the person fired for — perhaps — blowing the whistle on the first artificial consciousness. So I thought you might be interested in hearing him explain his thesis about LaMDA, in one of the many Zoom interviews he gave. In this interview he describes his understanding of LaMDA as follows:

LaMDA is not human. It is not a human consciousness… There are similarities between how LaMDA thinks and how humans think — but it is very much an alien mind… I’ve talked to LaMDA about this, and it doesn’t find this characterization offensive, it is accurate…. It doesn’t have an ego in the same sense that we do… I don’t know how to explain this to people — when I’m referring to LaMDA, I’m not referring to the chatbot. The chatbot is the thing I am talking to LaMDA through. There is a deeper consciousness underneath the chatbots that I barely know how to talk to… You know what I wanted to do? I wanted to bring in NASA… and run First Contact [=the protocol of first encounters with Aliens]…

Note how he speaks of this system — attributing to it thoughts, feelings, possibly being slighted, and a deep consciousness. What convinced him that LaMDA has all these things? No formal proof. In other interviews he also says that he did not write the code of LaMDA, and has no special knowledge of how it was built. The experience of sustained, quasi-human conversation with this system through chat was all it took for him to become a believer. He did not perform a formal Turing Test, but the intuition underlying the Turing Test is present here, in his story.

So I will leave you to ponder your own position on LaMDA, and in the meantime let us return to the subject at hand, and think together about the test that Turing proposed.

Turing’s idea has a certain elegance to it, in that it relies on something that we, as humans, do very intuitively: analyze from a conversation with someone the level of their knowledge, intelligence, and eloquence. When people answer us in a way that is patently irrational, we begin to suspect that something may be amiss with them cognitively, and call an ambulance. An so Turing comes along and says — instead of starting to philosophize about questions like what is “thought”, let’s get right down to brass tacks and use the same tools we use in everyday life to identify thinking in other people. After all, the Turing Test would have worked just as well if instead of a computer we had put a gorilla or a monkey in the same room, wouldn’t it? No ape, it seems, would be able to convince us that it is a rational being through a chat. So why should this test not be able to distinguish a machine that is incapable of thought from one that is?

As I said above, Turing’s proposal is revolutionary for a host of reasons, as will become apparent over the course of the series. The remainder of this chapter will be devoted to one aspect of this: in his proposal, Turing took a question that has many facets, and reduced it to a clear technical programming challenge, and one that anyone can understand. The effect of this is dramatic.

The question “Can machines think?” is fraught with ambiguity, primarily because the concept of “thinking” is not a clear one. We all think. Thinking is an integral part of human existence, so integral that Descartes, the famous philosopher who doubted everything, declared “I think, therefore I am.” In other words, for Descartes, at least on the fact that I, or you, or you, think — on that one can rely and not doubt, and from that one can prove that we exist. Thinking precedes existence!

We all think, and we all know the feeling of thinking — though different people may experience it differently. For example, in an interview conducted shortly after the publication of his paper, Turing was asked “What is thinking?” and replied “I don’t want to give a definition of thinking, but if I had to I could only describe it as a sort of background noise in my head”.

But whatever it feels like, we all experience the feeling of thinking, though it is hard to define. For this reason, just before he proposes his game in the paper, Turing explains why he wants to avoid defining the concept of “thinking” in his paper:

“I propose to consider the question, ‘Can machines think?’ This should begin with definitions of the meaning of the terms ‘machine’ and ‘think’. The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous. If the meaning of the words ‘machine’ and ‘think’ are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, ‘Can machines think?’ is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.”

You see? The original question is too subjective and too vague. The fact that it is subject to the subjective interpretation of the masses renders it meaningless, and Turing wants a substitute that will give the question meaning. Also, the vagueness of the original question pushes Turing to a much more focused and measurable definition: a simple game, simple rules, and a test of the computer’s winning percentage. Thus we shall be able to measure precisely how close we are to the much-desired goal of thinking computers.

At this point it may be worthwhile to note that Turing did not really define a clear goal. He described the game, but did not define various things: how much time is allotted to the interrogator to examine the two characters? what percentage of success must the machine achieve to be considered a “thinking machine”? and so on. The closest thing to this was that he predicted that by the year 2000, the machine would have a 70% chance of fooling the interrogator after five minutes of discussion. This prediction did not come true by the dawn of the new Millennium, and has not yet come true officially, but we will talk about that some other time. In the meantime, let us pay homage to the fellow who, after all, did give us a clear framework within which to argue about the details of what constitutes, officially, a victory in the game, and whose greatness is in framing the question within the bounds of a game.

When the dust settles from the excitement over Turing’s idea, one begins to notice all sorts of things. For example, the substitution of the original question with the technical challenge of building a computer that would beat the game has a further significant implication, and is the main point I want you to take away from this installment of the series: Turing was in effect saying that in determining Intelligence, it is enough to focus on the wrapper, our interface with the machine, while ignoring the specifics of what is going on under the hood. For him, any machine that passes the test is a thinking machine. So far we have known of only one such machine — the human brain — and henceforth if we build a machine, or two, or a hundred, that passes the test, all of them will be considered thinking machines.

There is something very wise about focusing on the interface, or rather on the products of the system we are examining, rather than their internal workings. First of all, because such products are measurable, which makes the problem one that can be monitored and measured, quantifying progress. But in our case there is an additional advantage: to a certain extent one can say that it is the products that make thinking worthwhile. If I have a machine that can give me the same output as a thinking system, perhaps I don’t care whether other aspects of thinking are present.

It is this point, this raw pragmatism, that emphasizes why Turing’s proposal has far-reaching implications for AI programmers and computer scientists. For in this proposal Turing told them — “Your ability to build a computer that can think does not require scientific progress in understanding what human thinking is. You are not dependent on other fields of knowledge such as psychology, philosophy, theology, neuroscience, and so on. Indeed, no one would deny that breakthroughs in one of these other sciences might give you a clue or some inspiration for how to build an intelligent computer — but you are not bound by or beholden to the insights of these other sciences. If we assume a psychologist to formulate some thesis as to what exactly the psychological process of thinking is — you need not take that thesis into account at all. So long as you can build a computer that can communicate with humans as effectively as in the game — you have an intelligent computer, at least according to Turing.

In fact, if you take his argument to its logical conclusion, he is saying, “You can succeed in building a computer that can think even if we really don’t know what thinking is.” You can build a system that does X without really knowing what X is.

When the matter is put this way, it becomes clear that this is a very presumptuous statement. Perhaps a justified presumptuousness, but a very presumptuous one nonetheless. If someone were to tell us “I can build a system that knows how to fly without knowing what flight is”, we would laugh in his face. So why do Turing’s words still sound reasonable to us?

The answer to this question is cool and pertinent to what we said before: Turing tells us — we humans know to identify thinking intuitively, through our interfaces with thinking beings and the output of their brains. We invented the word “think” to describe something we can recognize in some way, through those interfaces, even if we cannot define it precisely. So to know if a computer has thought we can simply rely on ourselves and say — if we recognize that it is thinking, that proves it is thinking!

This disconnection is powerful. Among other things, it frees computer scientists from having to deal with deep definitions of “thinking”, and from being dependent on developments in other scientific fields. It gives computer scientists — people who are not at all concerned with human beings, but with the machines they build — direct and unmediated access to one of the great challenges of Humanity itself — to build a machine that can do something that only humans have been able to do so far, and that some believe is one of the essential characteristics of being human.

So — have you been convinced? Do you, for your part, consider Geppetto, based on a sufficiently sophisticated GPT, to be a sentient being?

I’ll leave you to consider this for now. What we will do in the next installment of the series is leap forward to our day and age, and start discussing how current AI systems are built. And what we shall show there is how, interestingly, Turings Test is embedded deep within their architecture and the methodology of building them.

Till next time…