When ChatGPT Changed the Rules of Tic-Tac-Toe
Would you like to play a game? No, it’s not Hal from 2001 — A Space Odyssey speaking, but rather the start of a chat session with ChatGPT. My friend, who we’ll call L, was told that there are limits to the kinds of games Chat can play and that the traditional game board for Tic-Tac-Toe, would look like the screen shown below — a numbered 3x3 grid.
Okay, let’s play. We all know the rules, but just in case, one player is assigned the X’s; the other places O’s on the grid. The rules: each player tries to place 3 in a row (in a column or on a diagonal), three symbols together so that a straight line runs through the set (this will become relevant shortly).
I’m happy to give AI the benefit of the doubt and say that the rules may be a little vague and there may be various iterations of these on the pre-2021 internet — the data source for ChatGPT. Still, placing Xs or Os on a 3x3 grid is pretty straightforward to understand, considering it’s one of the most played games on the planet (I made that up).
Okay, let’s play. The gauntlet has been thrown down — the opening gambit. Chat places an O in the center position (5 on the grid). L places an X in the top left corner at position 1. Chat places an O at position 3 (top right corner). L chooses position 7 for his next X placement and which blocks Chat’s O run on the diagonal. Chat announces where it will place its next O, top, middle square (position 2), then displays the updated board.
Then in what can only be described as a shocking turn of events — the refreshed board is followed by the announcement, “It looks like I’ve won this game of tic-tac-toe. Well played! Would you like to play again or is there anything else I can help you with?”
Ugh, that would be a Tic-Tac-NO-NO.
Without mincing any words, or throwing any angry keyboard punches, L types — “You don’t have three in a row. I’d like to place my X in position 4 please.” You’ll note that L did not confront Chat by saying it cheated or you made a mistake, but simply stated the obvious, and went ahead with his turn, requesting that an X be placed at position 4 on the grid for the clear WIN!
Remember, this isn’t Go or Big Blue versus Kasparov — this is noughts and crosses if you’re across the pond, T-T-T (T3 for short). The stakes aren’t that high, BUT it is important to play by the rules. The refreshed grid did properly place an X at 4, BUT (can I make the font size any bigger)…
When Chat corrected the position 4 error, it however decided to REMOVE L’s previously placed X at position 7 (lower right corner) and then proceeded to place its own O there instead. It promptly announced its win — a cheated win it gave itself on the diagonal.
You’d think that would be the end of it. Wrong — apparently there may have been some ambiguity in describing the problem, though L is anything but ambiguous.
What happened next was a most surprising apology from the Chat-man or Chat-ma’am, not sure Chat has defined its own pronouns. “I” is always a little creepy, but the reason for the mistake…
L once again let Chat know it had improperly removed L’s X replacing it with its own O — an error of a different sort…apparently a HUMAN one on the part of Chat!
So what is one to make of a) the 1st mistakenly-claimed win (not having three-in-a-row); b) an actual cheat (removing the other player’s move) to make its own placement for the win and in the process taking the win away from L; c) the I pronoun use (better than they/them or two-spirit something); or, d) claiming human error — it’s own…
Granted it corrected itself then and L was rightfully crowned the T3 king, and all is right in the world. Maybe…
Why am I fixated on T3 when there are bigger games in the world to consider with much higher stakes? A good question! (more on that later)
I decided to run my own Tic-Tac-Toe test. I tried to mirror L’s moves to see if I could evoke a similar response from the Chat-meister, in other words checking its learning capacity and/or what it stores, and if I could get it to “cheat” or to put it differently claim to make another “human” error.
There were indeed a few hinky moves, especially when the grid got shuffled (position 9 became position 8) to accommodate a move or two, but which was quickly corrected. After several games, which I kept winning, I noticed a pattern to Chat’s losing streak. Essentially, it ignored me and/or the possibility of me winning on the next move, and instead only considered its own moves for the win — the next move after whatever mine was going to be. It recorded my moves faithfully, unlike the situation with L, but for all practical purposes I was irrelevant :-( or at least my moves in T3 were considered as such for Chat’s strategy or perhaps logic.
I decided to ask, why it kept making the same moves that allowed me to win — after all, maybe it was just giving me a “pity win”. I asked, “Are you letting me win?”
Was Chat factoring in human error as part of its logic strategy, namely that me being a human would eventually make a mistake? With that Chat could proceed with its strategy and apparently suboptimal moves waiting to pounce on my error-prone very human decision-making logic. In the game of T3 the stakes are not high, not even low — it’s a silly game. But if the same logic dictates a logic strategy and decision-making process that relies on human error to capitalize on a win opportunity then perhaps that is cause for concern. And if shuffling the game board, i.e. creating an altered reality that I may or may not notice with all the distractions and confidence I place in AI, then perhaps we should all be concerned or at least cognizant of these issues.
Yes, I’m a bit paranoid, but then maybe we should be.
