When artificial intelligence in video games becomes...artificially intelligent

Artificial intelligence in video games helps bring virtual worlds to life; it lurks beneath the surface, determining the way a player interacts with a game.

Artificial intelligence in video games helps bring virtual worlds to life; it lurks beneath the surface, determining the way a player interacts with a game. As the brains of a game, AI engages our brains.

Often, we think of AI in terms of obvious AI agents like NPCs or enemies that duck and roll away from gunfire. But speaking with numerous AI developers inside and outside of game development over recent weeks, the potential for AI goes far beyond making "smarter" enemies or more convincing NPCs. As game AI slowly converges with general AI concepts and techniques, AI can change the way games are played, and even the way they're developed.

Artificially barely-intelligent

At a GDC 2013 panelabout the next big leap in game AI, AiGameDev.comeditor-in-chief and nuclai Conferenceorganizer Alex J. Champandard quipped, "My prediction is a little bit controversial. I think the next giant leap of game AI is actually artificial intelligence."

The statement got a laugh out of the audience of game AI developers—but why? It’s because a good amount of game AI developers see their craft as outside of general AI. Some developers in the AI Game Programmers Guild even said that game AI has little to nothing to do with the advanced AI techniques that come out of academic and industrial organizations—that game AI has wholly different challenges from AI at large.

So what are the common challenges of game AI versus the challenges of industrial and academic AI?

For game developers, AI challenges are related to player experience. AI needs to operate in such a way as to convey an experience that provides for fun, drama or some form of enjoyment. In games, it often serves a tightly-controlled, authored experience, whether you’re talking about enemy AI that ducks behind a wall in Halo: Combat Evolved , or the squad AI in Half-Life .

"My prediction is a little bit controversial. I think the next giant leap of game AI is actually artificial intelligence." - Alex J. Champandard at GDC 2013

"Most game AI is designed to be exactly as intelligent as it needs to be in order to give the player a good experience," said former Sims 4 programmer Bruce Hill. "For many games, this means the AI should behave in predictable and barely intelligent ways, like the enemies in a Mario game or most first person shooters."

Dave Mark, who runs the AI Summit at Game Developers Conference and is founder of AI consultancy Intrinsic Algorithm, said much of what game AI developers work on is "more like 'artificial behavior'" than "artificial intelligence."

Advanced AI development happening outside of games isn't about creating AI that is "barely" intelligent, and it's not typically hinged on authoring an entertainment experience for an intended user. Rather, it attempts to create systems that closely mimic true biological intelligence. Here, there's much focus on machine learning, in which computers learn to perform tasks without being explicitly programmed to perform those tasks.

Challenges on this side of AI include creating AI that can learn and adapt to new environments; creating more robust systems for understanding speech and noise; pattern extraction; and developing AI that can evolve and take on non-trivial tasks. Self-driving cars, autonomous stock market traders, targeted advertising, autonomous military drones, and natural language recognition are all in the realm of general AI.

The trouble with convergence

The solutions to every one of advanced AI's challenges, and their applications, seem fantastic for video games. Playing with AI agents such as NPCs or training bots that can learn, understand, and perform interesting tasks sound like fun...theoretically.

But machine learning and its potential applications in games is a bit of a controversial topic among game AI developers.

"The video game industry takes almost no note of cutting edge AI research, which I find unfortunate," said Jacob Schrum, assistant professor of computer science at Southwestern University in Georgetown, TX. "Many advanced AI techniques can produce amazing results."

The reason games don't often used advanced AI techniques (as Schrum did acknowledge) is that implementing concepts such as machine learning in games isn't so simple, and often not desirable from game production and game design perspectives.

"It's not that we can't get good behavior out of machine learning techniques... it's that it may not be the behavior we want for the game experience we are looking for."

Game AI developers point out that advanced AI is unpredictable, and could not only destroy the player experience (imagine an AI bug in a new Call of Duty game in which enemies experience catastrophic unlearning and go completely haywire), but also that unpredictability could throw a wrench in more traditional game production processes.

"If you train up your AI agents [with machine learning], show your designer, and he or she says, 'yeah, that's cool... but in this one situation here, can you make it do this [action] a little more often?' you're kind of hosed," Mark said. "What do you do? Re-train your data set and hope for the best? In most architectures, we turn one or two 'knobs,' restart it, and say, 'how's that?'

"It's not that we can't get good behavior out of machine learning techniques... it's that it may not be the behavior we want for the game experience we are looking for."

That's a recurring theme: that game AI needs to fit the experience, and often, particularly in commercially-proven genres and franchises in video games, advanced game AI and the production issues that come with it eclipse any potential benefits of implementation. One game AI developer said he mostly uses AI techniques that were well-known in the 1970s.

Risto Miikkulainen, professor of computer science and neuroscience at University of Texas at Austin, said, "I don't think it's really 'game AI' vs. 'academic and industry AI.' Games just have different types of AI, and it's been like that for a long time. [Game AI] is what we like to call 'Good Old-Fashioned AI' - 'GOFAI.' You do A* pathfinding, you have your behavior lists and behavior trees, and you have lots of authoring go into it."

The resistance to convergence of these types of AI is rational: If the game you want to make doesn't need advanced AI, then don't waste the time developing that kind of AI. And if your existing, entrenched production pipeline, development process, or even studio culture isn't conducive to newfangled AI techniques, it's difficult to convince a team to rework the entire system, unless they clearly understand how the benefits outweigh the pain.

All of this is not to mention the common understanding that AI that's too "smart" can destroy the enjoyment of a game. "Everybody hates to lose to Deep Blue," said Dan Kline, former tech designer on The Sims 4 at GDC 2013's AI Summit. "Nobody's looking to get crushed by awesome AI all the time."

The promise of convergence

Even though more advanced AI techniques only seem to be trickling into game development, there have already been successful implementations of such AI in commercial games: there's Black & White (machine learning), F.E.A.R (context-sensitive behavior), Façade (natural language parsing) , Spore (data-driven lifeform simulations), and The Sims (Sims’ interaction with objects, each other), to name a few.

Game design that hinges on more advanced AI techniques is slowly but surely becoming more commonplace, said AiGameDev.com’s Champandard. He said his tongue-in-cheek prediction from two-and-a-half years ago—that artificial intelligence will be the next giant leap for game AI—is coming true, in a measured way.

"Developers are more willing to let go and embrace more complex systems," he said. "Since the rest of the tech industry is getting great results with machine learning, games are following behind quickly. [For example,] Killer Instinct using case-based reasoning, [ Forza 's car AI] Drivatar using neural networks, and Fable Legends using Monte-Carlo tree search."

There's also more recognition that advanced AI is useful in areas other than bot behavior. AI can, and has, been used to improve game production and design itself.

Champandard, who also has worked on game AI in studios including Killzone developer Guerrilla Games, said an increasing number of studios are using hierarchical task network(HTN) planners, an AI implementation used for tasks such as logistics planning and military coordination.

"Developers are more willing to let go and embrace more complex systems. Since the rest of the tech industry is getting great results with machine learning, games are following behind quickly."

For Mark with Intrinsic Algorithm, a medium-term AI implementation in games could come from big data. For example, games are already toying with the idea of interacting with speech and written language, but the next step would be for machines to understand more natural, unscripted speech. To get there, it'll take big data, AI to parse that data, and design to make it enjoyable for players. This tack presents its own challenges, not the least of which is where does one get that data.

Mark also described a scenario in which AI collects player data, then autonomously creates new content to fit the player's play style (e.g., an RPG that takes modular pieces of content and fits them together in data-optimized ways within a quest template). Such an approach would help keep an MMORPG fresh, and alleviate a developer's pressure to constantly create all-new content.

Paul Tozour's company Intelligence Engine Design Systems implemented advanced AIto take on the complex job of game design, using it to balance and optimize the strategy game City Conquest . In a 2012 interview, he said he had a revelation: "I had a huge change in perspective. I came to the realization that machine learning isn't a tool for game AI-- it's a tool for game design ."

Today, he believes that in the long term, machine learning will be the basis for a new set of tools that facilitate systems-based game design. "Someday, designers will have vastly more powerful tools at their disposal, including some tools to help evolve and grow and tune and optimize the design itself, not just the content," he said. "And that will have to involve a lot of machine learning."

What would you say…you do here?

There’s a sentiment among some game AI developers that their non-AI colleagues don’t quite "get" what AI programmers do, which in itself could be a barrier to the adoption of newer, interesting techniques and implementations.

Asking AI programmers to describe in one sentence exactly what they do was an interesting exercise that showed a variety of responses:

@AIGPGWe teach the game to make the decisions that provide the context for the player's own decisions.

— Paul Tozour (@PaulTozour) September 16, 2015

@AIGPG @gamasutraTake a system that knows everything and can do anything, and dumbs it down just enough to be fun to beat.

— Steve Martin (@Purple_Steve) September 16, 2015

@AIGPG @gamasutraan AI programmer tricks the player into thinking that elements of the game are making informed intelligent decisions.

— Leonard Ritter (@paniq) September 16, 2015

@krisgraftThey build interactive systems in code that generate performances or content.

— Alex J. Champandard (@alexjc) September 16, 2015

@AIGPGIt's not an easy question to answer, however I did put "I make the baddies" on my business cards! pic.twitter.com/gSpLnIY2yp

— Rich Welsh (@Welshy31) September 16, 2015

@AIGPG @IADaveMarkAI developers make tools that allow for more interesting worlds to be created

— Lance Hitchcock (@lancehit) September 16, 2015

Game AI developers have different ideas of what game AI is, what its applications are, and what the future holds. And amid the spirited debates about game AI, there is typically a sense among developers that the field is moving forward. For game AI developers, it’s just a question of which direction forward.

Convergence between AI in games and general AI is happening, even if for some developers it is practically imperceptible or even irrelevant. Champandard argued that instances of convergence are not as isolated as they used to be, and that in the past couple of years, convergence has become a bona fide trend.

"As much as some industry developers like to distance themselves from academia--or the rest of the tech industry--there are many parallels, and that's increasing," he said. "It's an amazing time."

Thanks to Intrinsic Algorithm's Dave Mark for the help on this article, and to the AI devs who kindly entertained my nightmare scenario of evil game AI taking over the world.