The Reality of Red Dead Redemption 2’s AI (Part 1)
How Red Dead Redemption 2 uses AI to create a believable world.
(This is the first in a two-part series. Part two will focus on the implications for users, as well as real-world applications.)
The first, unforgiving slivers of light creak through the old saloon’s cracked windows. Dawn brings to an end another rowdy night of drunken card-playing with old friends, foes and familiar strangers. Joe resistantly pulls himself away from the scene, dragging his half-drunk self to the construction site driving him to drink in the first place. After lazily pretending to clean the floor for the last half hour, it’s time for another well-earned drink. The daily construction routine begins; hammering nails into surfaces. But today he’s building the floor of what will become the fanciest emporium in town. The beating sun is already burning a hole in his back. High time to return to the saloon for another round or three.
With the faint simmer of roast beef in the back of his mouth, made by the bartender he secretly swoons for, he returns to hammering nails. Eventually, it’s almost dark. Time to clean up the sawdust from the day’s plank produce. When that’s done, he’s set free. Free to return to that charming old saloon to enjoy the company of all the young ladies and the whiskey he’s promised to remain faithful to. Honky-tonk piano music begins to blend with voices, before fading into another hazy blackout. Tired and drunk, Joe collapses onto the saloon’s largest table. Another day has gone. Another is ready to begin.
This isn’t the character plot out of a Sergio Leone Spaghetti western (although it could have been; if only). It’s a day in the life of one of the thousands of non-player characters (NPCs) populating Red Dead Redemption 2 (RDR2). If you’re wondering how I learned about Joe the construction worker’s daily routine, it’s thanks to YouTubers who document the virtual lives of NPCs in RDR2. While this form of prying might not seem like the funniest pastime ever, it reveals the depth of the stories and behaviours of the characters filling the RDR2 world. (The fact that someone would spend time tracking the behaviours of virtual people is a nice reminder of how wonderfully odd us humans are).
RDR2 isn’t just a commercially successful adventure game launched in October 2019 by Rockstar Games, that garnered near-perfect scores from most specialised game magazines. It’s a complex simulation where thousands of agents — both human and non-human — act independently and create a somewhat believable virtual world by following rules, acting on needs and making appropriate plans.
Developing massive sandbox games is Rockstar’s forte. GTA V, for example, is a vast Southern California-inspired, primarily urban world populated by an array of characters. However, their actions and behaviours are quite schematic and, for the most part, fixed. With RDR2, Rockstar Games brought real-life simulation in commercial games into a bold, new terrain. That’s because of their particular emphasis on developing a complex game AI.
What’s video game AI?
When game developers refer to AI, they usually mean something different from AI researchers. Game AI rarely leverages machine learning models that can be trained on large datasets (does a buzzword like deep learning ring any bells?). Rather, it implements intelligent control agents, which mostly focus on deciding which action to take based on the current game state. In the context of a game, an agent is often a character, but can also be a vehicle, an animal or a group of agents such as a Roman legion or a civilisation.
Agents perceive elements of their environment and act upon them. We can refer to this process as the sense-think-act cycle. Imagine our friend Joe from RDR2 was dead in the middle of a fire-fight raging inside the saloon. What would he do? First, he’d have to understand how many allies and enemies there are (sense). Given the intel he gathers, he should decide whether to attack or flee (think). Finally, he should act on his decisions. If he decides to attack, for example, he should start moving towards one of the enemies to start punching him (act).
The main goal of game AI is to create realistic agents. Differently from the AI system used to optimise the path to bring you the TV you’ve bought on Amazon, we don’t want game agents to always choose the best course of action. And we also don’t want them to cheat. In game AI, this refers to agents having superior access to in-game information or ‘complete knowledge’. In some cases, enemies could ask the game engine directly for the player’s location, or spawn in unfairly advantageous positions. This wouldn’t be credible when trying to recreate human-like behaviour. Humans are a lot things, which is why they’re far from perfect. A degree of imperfection in decision-making has to be embedded in the AI, so that it makes believable decisions, rather than ideal ones.
Another aspect to consider when implementing believable agents is the breadth of actions available to an NPC. One of my favourite games is Metal Gear Solid (MGS) on the Playstation 1 (I might be giving away my age here). MGS was an amazing stealth game with a compelling spy story. Often you would crawl through industrial compounds and hide in boxes, sniping around. If you made too much noise or walked into an enemy’s line of sight, you could be detected. Then the inevitable chase would start. This is stealth game 101, and it’s quite believable. The issue though, was that, after a certain amount of time outside their radar, the enemies would go back to their scripted patrol mode and completely forget you were there in the first place. This feature made for an enjoyable combat system, but wasn’t ideal for immersion and believability (try hopping onto the White House lawn in real life, then patiently wait around to go ‘off the radar’ to understand why). The enemy agent’s scripted two-state-based AI (patrol state vs combat state) was too simple, its mechanics easily exposed to the player.
Many modern games still use the rudimentary techniques employed in MGS to implement NPCs. However, virtual world simulations have improved dramatically since the days of the PS1, and since the game F.E.A.R. was one of the first to break free from static ‘scripted’ NPCs. Large sandbox games like GTA V and Skyrim have propelled the immersion envelope; RDR2 raised the bar of believability.
From 1- to 2.5-dimensional NPCs
The world of RDR2 is teeming with life. If you haven’t played it yet, the best way to visualise RDR2 is to think Westworld meets video games (and if you haven’t watched Westworld, go and do so right now). While wandering around in a Spaghetti-western-like open world, you can engage with thousands of NPCs, and watch an ever-evolving environment filled with incredibly vivid wildlife. All of these elements augment the immersive gaming experience. But, at least for me, NPCs are the peak of this simulation.
In video games, it often seems that characters are in the simulation just to assign the player a mission or, even worse, to contrast the horror vacui — the hopeless feeling of emptiness often found in open-world titles. Their behaviours are generally highly scripted and, as in the case of MGS, limited in scope. Borrowing the terminology from narrative theory, we can say traditional game characters are generally monodimensional. They don’t have a life; they don’t have goals, needs or beliefs.
NPCs in RDR2 are different. They behave more like people who operate according to certain storylines that give meaning to their actions. Characters don’t follow the same endless loop, wandering aimlessly in the virtual space. Rather, the actions they perform are determined by how they are connected with the world, via the work they carry out. By following his daily routine, we can infer that our friend Joe is a construction worker, probably unmarried and on the brink of depression. His only comfort is a bottle of whisky and the nightly exchanges with the girls at the saloon.
These characters have distinct personalities enriched by mood states and the capacity to ‘remember’ events. Rockstar Games used 1200 actors to create NPCs, and wrote 80-page scripts to ensure that each character would feel multidimensional; a quasi-real person. Much like earthlings, their dialogues and actions are a complex negotiation of what NPCs ‘feel’ in the moment and what they’ve ‘experienced’ in the past.
The type of response you’ll get when engaging with NPCs depends both on who they are and how they perceive you. If your character is decorated in mud, NPCs will be less inclined to help you. If you’ve hit a character in the past, they may remember that and therefore be hostile. NPCs also respond to subtleties in facial expression and posture. It’s the attention to these apparently minor aspects that helps bridge the uncanny valley in virtual simulations. Richness in character depth reaches its peak with the gang members whom the player gets to know best. The player can familiarise themselves with these characters while chatting around the campfire, preparing meals and gathering food. Unlike any other large game I’m aware of, in RDR2 the player can interact with every single NPC, expecting a behaviour that’s tailored to the context.
I love this wild-life
Although anthropomorphic characters are central to creating immersive worlds, there’s another aspect that deeply enhances the believability: wildlife. In RDR2, the ecosystem is filled with 200 animal species that behave in entirely unique ways, responding to the stimuli of the environment. The game implements a system that simulates food chains. Coyotes hunt prey, but are scared off by bigger animals like buffalos. Carcasses decay over time and are scavenged by vultures. Both the player and the NPCs can spontaneously interact with animals, reshaping the natural course of the food chain.
The player can interact with the environment and the surrounding materials. When the player is caught by a heavy rain shower, he and his horse will get muddy. This will actually slow the player down and impact the functionality of their guns.
A brave new (synthetic) world
RDR2 brings game simulation up a considerable notch. In RDR2, the player explores a world filled with details and interaction that, in many aspects, feels similar to that of Westworld. Despite incredible progress, limitations still persist that innovative AI methodologies can help overcome.
For instance, plotlines and NPCs’ dialogues in RDR2 are still scripted. Researchers have developed AI systems that can generate fairly believable plots and conversations. This AI technology could be used at runtime in sandbox games to determine emergent story lines and dialogues. In such games, the world would provide constraints and resources the AI system could consider to create a unique narrative. The narrative would be customised to each user, based on the interaction they may have taken in the world.
NPCs have a fixed daily routine and fixed goals. What makes humans efficient is not only our capacity to plan our actions to achieve certain goals, but also to update our goals based on events we live. People who’ve gone through a near-death experience will look at life in a completely different way, the moment they’re out of the woods. They may want to reconsider their life goals and make the most of their limited time; prioritising family, helping others, or doubling down on their career. While NPCs in RDR2 can react to their immediate environment, it doesn’t seem like they can change their goals or their beliefs. Implementing a dynamic AI system that could reset NPC’s beliefs and goals based on their experience in the simulation would create a more immersive experience for the player. Perhaps an intervention would inspire Joe to take up a simpler life, tending to the tomato garden and supporting a family of three. And then he might decide to rob a bank out of sheer boredom. The beauty is whatever choice he makes, it would be as random as the possibilities of everyday life.
