AI does not think like us
By David Weinberger, Writer-in-Residence, Google PAIR
AI Outside In is a continuing feature by PAIR’s writer-in-residence, David Weinberger, who offers his outsider perspective on key ideas in machine learning. His opinions are his own and do not necessarily reflect those of Google.
When I was serving twenty years to life at the notorious Devil’s Island prison in the 1870s, I learned a new game. I was in isolation for seventeen years before my daring escape — a story for another day — and could only communicate with the person in the cell next to me by tapping on the wall. Over the many years, Francois (for such I learned was my fellow inmate’s name) managed to convey a simple game to me. He called it “Nines.”
Through trial and error I learned that Nines consists of putting your mark into a sequence of nine blanks. Back and forth we would go, taking turns as tropical day turned to chilling night. I fairly quickly figured out that getting three of my initials in sequence sometimes won. But it took me months to figure out that there were in fact eight winning configurations:
It didn’t matter where Francois placed his F’s so long as my D’s were in any of those eight sets of positions.
Within another couple of years, unless my pervasive hunger cost me my focus, I could quite reliably either win or tie Francois, the friend I knew only through his taps on our shared wall.
Once I escaped, I went back to my haunts in turn-of-the-century Marseilles, where I introduced Nines at my favorite bistros. I found I could rely on making beer money over the course of a night because beginners rarely perfectly remembered the eight winning sequences.
I often wondered who would have designed a game with such an arbitrary set of winning patterns. I could see a certain rhythm in them, but still, there seemed to be no reason to have chosen those patterns rather than any others.
Until one night.
I had engaged an elderly newcomer to the cafe in a “friendly” game of Nines, at first for low stakes. Then, over the course of fifteen games, he wiped me out. “How, sir, have you taken to Nines so readily?” I inquired. “How do you even remember which patterns you are aiming for?”
He looked at me not unkindly. “I was a window washer for many years,” he said, “Until infirmity left me unable to perform those duties.” He glanced down at his hands and continued. “In many of the finer residences, I cleaned doors and windows with many glass panels. It is not a fascinating occupation, so I found myself playing a game against myself using the gridwork of panes, three by three. The aim was to complete three squares in a row before my counter-self did. Three across, three down, or three diagonally through the middle.”
“Fascinating,” I said, “But what does that have to do with Nines?”
“When working with a 3x3 grid of panes, if you number the panes one through nine, the winning patterns are represented by the winning sequences of Nines.”
“Drat!” I said as the light dawned. “I’ve been playing TicTacToe all along!”
Trying to teach a machine learning system to play TicTacToe is a silly waste of computing power since you can easily specify the logic by which a computer — or a person — can routinely win or at least tie a game: Always first check if you need to block your opponent from winning, try to occupy three corners…that sort of thing. But suppose you’re trying to learn about machine learning, so you decide to train a system to play TicTacToe by giving it no rules or hints. Instead you give it thousands of completed games to learn from. Just suppose.
The whole point of machine learning is that it’ll learn from data rather than having to be explicitly instructed in the rules or logic of the domain it’s making predictions about. So, your machine learning system will analyze the games you’ve input, looking for statistical relationships among the moves. Then its algorithms will build a complex web — a neural network — of those relationships so that it’s able to output a probabilistic suggestion about the best next move, just as other machine learning apps suggest the next book you might like to read, the best way to Santa Jose, or the likelihood of a tornado forming.
So, let’s feed it thousands of games. But what’s a game to a computer? In one interpretation, it’s a series of moves, each resulting in a new state of the board. For example, one complete game might consist of these boards, reading from left to right:
Now you want to translate those images into data that expresses the information in those images as succinctly as possible. For example, we might assume that for a computer, the best representation of the game depicted in Figure 2 might be:
where each three-line grid represents a board, and each character is the sort of thing you might type on a keyboard.
But there’s an issue. To the computer, each three-line grid in Figure 3 doesn’t look like nine symbols but like eleven: the nine plus the carriage returns (or, as we say in techier lingo, the “newline” characters) at the end of the first two lines of each 3x3. They may be invisible to us, but not to the computer. To a computer, the first board in Figure 2 in fact looks like this sequence:
where “↩︎” stands for that newline character.
Now, it might be that the machine learning system will learn to ignore those carriage returns. But why are we even trying to represent the board as a two dimensional graphic? Why not just give it a sequence of characters like the lines of Nines in Figure 1? Is there information in the two-dimensional 3x3 that the machine learning system wouldn’t get from a one-dimensional sequence like a move in Nines?
Perhaps. The fact that squares #1,#3, #7, and #9 on a TicTacToe board are corners matters to us humans because we can see that they each can be part of three different successfully filled-in rows, whereas #2, #6, #8 and #4 only have two ways to be part of a winning combo. But to the machine learning system, what counts is the statistical advantage those four squares bring, not that they’re corners.
That’s why given enough years on Devil’s Island playing Nines we would learn that slots number 1,3,7, and 9 have strategic value. At that point our play would become indistinguishable from that of someone playing TicTacToe on a 3x3 grid. The same for the machine learning system. The AI doesn’t need the concept of corners in order to learn the statistical patterns that make corners strategically important. Those patterns are present in Nines’ nine-character strings.
In fact, the AI wouldn’t care if we constructed those strings differently than Nines does, as shown in Figure 5. Nines happens to read the TicTacToe board left to right, from the top. If the Hebrew version read it right to left, or another version read from the bottom up, the winning 9-character strings would be different, but AI would learn how to play equally well.
In fact, the string of characters could be in any random order, and so long as each of the strings is consistent in that representation, the machine would presumably learn to play the game equally well.
As a result, even if the moves a machine learning system makes are indistinguishable from the moves a human would play, TicTacToe and Nines are different games by our way of thinking. Not only are the boards different, but a fundamental rule of TicTacToe is that to win, you have to get three in a row. But that’s not true for Nines. And if the boards are different and how you win is different, what’s left? The games are different.
In fact, machine learning isn’t even playing a game. Rules define a game, but the AI doesn’t know about any such thing. It’s looking for statistical relationships among sequences of numbers. If those same calculations worked for routing cars or sorting cucumbers, the machine learning wouldn’t know or care.
Machine learning does not think the way we do…if machine learning thought at all.
Towards the end of my life I revisited Devil’s Island as a free person. I turned aside all offers of hospitality by the warden and went immediately to Francois’ door. The warden let me in without awaiting a response from the cell’s occupant.
Francois was famished and insane from close to 150 years of solitary confinement. Yet the light in his eyes shone just as I had always imagined.
After inquiring about his health I asked “Did you know that when we were playing Nines we were actually playing TicTacToe?”
“I am puzzled,” he hoarsely whispered, “by your use of the word ‘actually.’ There seems to be an entire metaphysics hidden in that word choice.”
While I struggled to find a response, Francois’ strength gathered and he continued. “It is true that at times over the years I imagined what the geometries would look like if we displayed the nine slots in two dimensional space, including the 3x3 grid you imagined, but other arrangements as well.”
With this he scratched some figures into the dirt floor:
“I also envisioned some three dimensional and even four dimensional arrangements. Some of the patterns the Nines draws into these arbitrary arrangements are quite beautiful.”
Now that he was invigorated enough to unsteadily stand, I learned that he was a man of considerable height. “Indeed,” he said from above me, “I did not think we were ‘playing’ a game at all.”
“Surely you understood that the game play ended when one of us won,” said I.
“I am greatly distressed to learn that for those many years in which Nines was the nourishing wellspring of our deep friendship, you thought we were competing with one another.”
“But …”
“I thought we were engaged in an aesthetic pursuit, like two musicians playing back and forth until the melody was complete.”
“I…”
But Francois was too overwhelmed to let me talk. “Playing Nines with you became my life. It let me enter a meditative state in which the patterns were music composed with the dear friend I never met. But now I learn you were out to win, to beat me, to vanquish the only person you could communicate with. I am devastated.”
A silence rose between us, a wall I feared I would never be able to scale.
At last I spoke.
“Then I guess you won’t be needing this,” I said, showing him the TicTacToe app on the mobile phone I’d brought for him.
“Never!” he said.
And then more feebly, “Does it come with Netflix?”
