How generative artificial intelligence could radically change video games
When you read the words "artificial intelligence," you probably think of innovations like ChatGPT, but artificial intelligence (AI) has been used in video games since the 1950s.
From Pacman's signature ghosts to "The Sims'" autonomous decision-making, AI is essential to creating adaptable characters and storylines.
Generative AI, artificial intelligence that creates text, images and audio in response to a request, is set to revolutionize one of the characteristic components of video games: non-playable characters, also known as NPCs or NPCs. These characters usually have a fixed pattern of behavior, and their gestures and speech are often stilted and unnatural.
"When we think about those NPCs, they seem a little strange to us," says Alexis Rolland, director of development at La Forge China, the Chinese branch of the R&D Unit of video game publisher Ubisoft. "You realize that there is something strange about what you see or hear."
This is where generative AI comes in: earlier this year, La Forge launched Ghostwriter, a text-generating AI tool designed to help writers create a greater variety of original dialogue for NPCs, and in 2022, it tested a new technology that helps generate more realistic and natural gestures that match the tone and mood of your speech.
"It takes speech as input and generates body gestures as output, so we can imagine those NPCs expressing themselves with unscripted dialogue, having almost natural body animation, synthesized from speech," says Rolland.
Combining generative AI elements like dialogue and animation could create "an AI NPC that might have a little more natural, unpredictable behavior," Rolland says.
Jitao Zhou, a student at Rikkyo University in Tokyo, is doing just that to generate more realistic, smarter, and less predictable NPCs.
"This NPC, which uses deep learning, has no fixed pattern, so it can have a greater variety of movements," Zhou says, adding that smarter NPCs will make games more entertaining and challenging.
Some publishers are already employing generative AI on NPCs to make conversations more realistic. Chinese video game company NetEase uses ChatGPT to generate NPC dialogue in its newly released mobile game "Justice", while Replica Studios recently introduced "AI-powered intelligent NPCs" for game engine giant Unreal Engine, which allows game developers to use AI to read NPC dialogue, instead of hiring a human voice actor.
However, one of the risks of using AI-generated NPCs is that game designers could lose control of the game's narrative, says Julian Togelius, an associate professor at New York University, where he researches AI and games. "They may say things that break the game, are rude, or break immersion," he explains.
Creating "good" AI-generated NPCs that help the player is also much more difficult than creating enemies that fight you, Togelius adds. "We haven't had as many advances in artificial intelligence that controls the other characters in games, or that tries to model the player, or tries to generate the world... so we will see a lot of advances in these directions."
open worlds
Open-world games, such as "Grand Theft Auto," "Skyrim" and "Elden Ring," present gameplay with non-linear missions and stories. This offers another opportunity for generative AI to modify the gaming experience.
Togelius imagines a "huge open game" with endless opportunities, new cities, landscapes and people, each with its own story and interactive elements. Using player data collected from previous games, generative AI could create unique stories and tailored quests for players of the right level, “like a custom dungeon master,” he adds.
Some research is already being done in this field. Takehiko Hoshino, also a student at Rokkyo University, created an AI tool that he is teaching to generate its own mazes and dungeons from square to square, based on previous ones he has found.
It's still in the early stages of development, but Hoshino says the next step is to embellish the maze with elements such as "treasure chests, enemy characters, and other game-specific features such as traps and other tokens."
Near-infinite open worlds are already possible to some extent: "No Man's Sky" (2016) is a "virtually infinite game," says Togelius, which uses a technique called procedurally generated content to create fauna, flora, geology and conditions. custom atmospherics for your planets, of which there are 18 trillion unique variations.
For the average citizen, procedural content generation is very similar to generative AI, explains Togelius. But he uses algorithms that generate content according to predefined rules, based on the data entered by the game developer.
While developers retain control over the content generated in this way, generative AI has the potential to develop unplayable levels or unintentionally deviate from the game's narrative. "Games have functional limitations, such as levels being complete and NPCs not lying about the game world," Togelius adds.
But generative AI could impact the game world in other ways.
Players are already tailoring the game to their own preferences and needs with user-generated content (UGC), which is a key component of many games, including "Fortnite," "Minecraft" and "The Sims." Generative AI could make UGC production easier and more accessible for players, as well as raise the quality of content.
"Generative AI has the potential to enable a much broader and emerging set of personalized and reactive player experiences," a spokesperson for Maxis, the developer behind "The Sims," told CNN in an email.
"Player customization today is limited by the complexity of the tools and [user experience] we can expose players to, but some of the new models may make it easier for the game to interpret and respond to what the player player wants to do," says Maxis.
One more tool
Although gamers are excited about the gameplay potential, generative AI is likely to impact development before altering the user experience.
Maxis is developing generative AI tools, currently in various stages of maturity, that can help game designers by eliminating repetitive tasks and allowing developers to work on more interesting problems, according to a spokesperson.
At La Forge, generative AI tools like Ghostwriter or ZooBuilder, a 3D animator that animates quadruped animals based on videos, could help designers "speed up the most tedious part of their process, so they can really focus on the most creative parts." and interesting," says Rolland.
Creatives across all industries worry that generative AI will take away their jobs, but Rolland is quick to add that this new technology won't replace human game developers. Animators faced a similar existential threat with the advent of motion capture, which Rolland said didn't really affect jobs but instead became a tool for creating better graphics.
"We've never had as many animators as we do now, and we still need more. Motion capture became part of their workflow as just another tool," says Rolland. "I think with generative AI, it's essentially the same thing, or at least, we're approaching it with exactly the same mindset here at Ubisoft." However, there are still many "legal and ethical issues" to resolve in the use of generative AI, including artists' copyrights, he adds.
La Forge is eager to explore opportunities, such as the potential to increase iteration speed, and independent game designers are also going to benefit from it, he adds. "This increasingly accessible technology will allow many of the smaller studios to produce games and scale their productions, and perhaps achieve higher quality than they would have had without generative AI."
Ghostwriter generative AI and voice-to-gesture animation have "moved beyond the simple prototype phase," Rolland says, and La Forge is now studying how these technologies would work in the development process.
"Video games have a long journey ahead of them in the next ten or twenty years," says Togelius. "It requires us to change the way we think about designing games, but I think when that happens, the games will be much better."
