AI Agent for Games: What It Is and How It Works

An AI agent in a game is like an independent player in the computer-generated environment. The agent is a program that does not just run code for acts that have been pre-written, rather is aware of the state of the game and how things happen, what has been done, etc., and can use that information to make decisions about what to do and actions to take. The main difference between the AI agent and the traditional scripted model of AI agents is that an AI agent will adapt.

For example, if a player chooses to take a new approach, then the agent will adapt the behaviour of the AI agent (if the AI agent is an NPC) to reflect that. One of the coolest things about this is the AI agent does not repeat actions blindly; but rather uses its perception, along with previous experience, to make a choice that will lead to the action that is most effective.

As an example of this, consider a guard NPC. The guard NPC is not just following a predetermined path, but also takes into consideration all the sounds that it hears, the position of the player, and the location of all other NPC guard as it determines whether to sound an alarm or to watch the player quietly.

The AI agent is made up of three parts: 1) the perception (via sensors) of the environment in which the agent resides; 2) the decision process (decision module) that enables the agent to take action; and 3) the actuator that determines what action the agent will take in the game. In Unity, the majority of AI agents are written using C# scripts in conjunction with the ML-Agents Toolkit. In Unreal Engine, most AI agents are created using a combination of Blueprints and Python API calls for predictive capabilities.

AI agents accomplish a variety of functions (solve multiple problems) in the game at one time. To begin with, these systems manage NPCs and not just foes, but NPCs that appear realistic and can learn how to evade, conceal and support each other. This methodology requires the user to evaluate their strategies and adapt to those strategies; users cannot just memorize the means of the patrols.

The second factor regarding the use of AI is its ability to dynamically modify the level of challenge pursuant to the performance metrics of the user. For example, should you experience difficulty completing a level, the agent would decrease the difficulty. Conversely, should you complete a level readily, the agent may increase the challenge level. There are many games incorporating this form of adaptive gameplay, including Left 4 Dead and modern roguelike games.

Lastly, AI agents provide the capability for games to create content in real-time, i.e., new quests, dialogues and levels that are based upon the player's play style and preferences. In educational games, an AI agent will provide the player with tasks that are congruent with the player's skill level. Based on successes or failures, these tasks are modified accordingly.

AI Agent Implementations in Gaming: Primary Use Cases & Examples

Dynamic NPCs Changing Gameplay Factors

Standard NPCs are predictable; e.g., a guard walks a path and sees a user and runs to them. This leads to boredom in gameplay. Therefore, AI agents alter the rules of the game utilizing either behavior trees or reinforcement learning.

A relevant example is in the game, "Far Cry 5," where through the utilization of the AI system, the NPCs will analyze the play style of the user. If the user prefers stealth as the primary mode of play, the number of patrolling NPCs will be decreased in open areas engaging the user through ambushes from close quarters. Alternatively, if the user plays using a "run 'n gun" strategy, the difficulty of the NPCs will be adjusted based on defensive strategy and equipment are upgraded. Adaptive behavior is not predetermined but instead is done in real-time. Take for example the Nemesis mechanic in Middle Earth: Shadow of Mordor. Each captain will retain an awareness of every captain's actions (Did I run away? Did I hurt him?) so that when they encounter you again, they will act much more cleverly (setting traps, adapting their fighting styles) to counteract whatever combat methods you've been utilizing in their previous encounters.

If you happened to kill a captain, the next time you see him he'll be back with all sorts of new scars and abilities. This really gives the game some depth!

AI Agents in Interactive Storytelling

Non-linear plots with many dialogue choices (such as in Detroit: Become Human or The Witcher 3) are also difficult to manually account for all possibilities. This is where AI agents come in to create conversations and quests based on context (what you have done up until this point) and the current actions of the player as well as where the rest of the world has been up to this point.

AI Dungeon is a prime example. In AI Dungeon, you can type out any action you want to take, and the AI dungeon will provide you with a unique and creative description of what that would look like, thus creating the story as you go along. There's no single script—only your imagination and the AI's responsiveness.

In the position of making commercial products, Inworld AI is a no-code platform which allows you to define the personal characteristics of a non-player character (NPC) and for the agent to generate the lines of dialogue it produces for the player in real-time. In The Ascent (you'll remember), non-player characters will often recall recent events like gunfights, and say things about them to you over and over again.

Educational AI Agents in Games

When it comes to educational games, the agent is your personal tutor. If you get something wrong, it will provide you with something more simplified or offer you hints. And if you succeed, it raises the difficulty of the tasks you are given.

The most important aspect here is that of collecting data (i.e., how long did it take the player to respond, how many times did they get it wrong, and how many attempts did the player make). The agent is able to tailor an individual learning path based on what the user needs. For example, Duolingo uses spaced repetition and machine learning to help users learn new words more effectively. In DragonBox Elements, the agent can track which types of tasks a user is having trouble with and add game-like activities to assist them in solving those issues while continuing to motivate and engage them.

Artificial Intelligence Agent Technologies and Algorithms for Games

Two of the algorithms used by AI agents in games are Behavior Trees (BT) and Finite State Machines (FSM).

In a Finite State Machine (FSM), an agent switches between three states of behaviour: patrol, attack and flee, based on the environment. As the number of behaviours increases, it becomes increasingly difficult to manage and track all of the various state transitions.

Behavior Trees (BT) offer an alternative approach to managing this problem in that they use modularization to maintain a hierarchy of conditions and behaviours that continuously cycle, and select the optimal behaviour at the time of being evaluated. For example, a guard has three sequences of behaviour based on the following sets of conditions:

1. Root
   1. Patrol Sequence
      1. Condition: I do not see the player
      2. Action: Move along the route
   2. Pursuit Sequence
      1. Condition: I see the player
      2. Condition: I have more than 50% health
      3. Action: Run toward the player
   3. Flee Sequence
      1. Condition: I have less than 20% health
      2. Action: Run to cover

There are several Behaviour Designer tools available to Unity developers for creating Behaviour Trees, while Unreal Engine users have access to a Behaviour Tree builder that they can use to create trees without writing any code. Using these algorithms allows game developers to create lifelike and believable behaviour in NPCs. Reinforcement Learning (sometimes called "RL") is an intuitive method that uses an exploratory approach for the development of RL agents by using multiple attempts or tries to explore all possible actions within an environment and their resulting outcomes in relation to the reward or punishment received from the actions performed, which allows the agent to determine which actions are optimal in order to maximise long-term rewards. This method of using rewards and punishments is best seen in a standard maze situation where exploring different paths throughout the maze results in receiving a reward for each successful path taken and a punishment for taking a path that was incorrect.

One of the simplest forms of representing this is using Q-learning to create a basic action value table; however, as RL agents became more complex, Q-learning was eventually replaced or augmented by the use of deep learning model techniques utilising the development and implementation of neural networks via deep Q-network (DQN) technology. More recently, Reinforcement Learning algorithms such as PPO and SAC have been heavily utilised due to the rapid speed in which the agent learns compared to the Q-learning method of training or DQNs, as well as their ability to provide RL agents with a level of stability. As an example, Ubisoft incorporated the use of the PPO algorithm for the non-player characters (NPCs) in their game "Watch Dogs: Legion," in which the NPCs could support each other, take cover for protection, and perform tactical manoeuvres to create real-world challenges for players by giving the NPCs behavioural characteristics similar to those of real-world players.

Game Engines and Tools Utilised to Develop RL Agent-Based Games (Unity ML-Agents and Unreal Engine)

Unity ML-Agents Toolkit is the primary set of tools used to incorporate RL agents into Unity Games. The process for implementing a model trained using Unity ML-Agents requires creating the RL agent in Unity with an associated Python client to train the RL agents using Python, with the training data being imported into Unity using C# script. The following procedure is used to create and train RL agents with Unity ML-Agents:

1. Develop an environment or scenery in Unity where the RL agent will exist.
2. Define what the RL agent will be able to observe, define what actions the RL agent will be able to do, and define what rewards will be attributed to an RL agent as it relates to the RL agent's performance in each of the environmental variables.
3. Execute the appropriate command line function to start the training process while executing hundreds or thousands of training episodes for the RL agent.
4. Once the model has been saved (using the file location specified when setting up the agent in Unity), switch the RL agent's state or settings from training mode to inference mode (i.e. the agent will now be able to play the game).

Unreal Engine uses a similar Python application program interface (API) to perform the same function through the use of Unreal Engine Blueprints to export and convert the trained RL agents to the ONNX model format. Embark Studios utilized machine learning to improve balance in its game, The Finals, by continuously modifying its machine learning models using player statistics.

A Comparison of Some of the Most Popular AI Agents for Gaming: Pros and Cons

The Unity ML-Agents Toolkit is the official Unity-supported agent toolkit that allows for integration with PyTorch and has good documentation (including full examples). It can also be trained in the cloud at no cost. However, the user must have a working knowledge of both Python and Reinforcement Learning with the ability to train for as long as it takes to complete.

The Unreal Engine AI system is integrated with a Behaviour Tree editor in addition to supporting ML models using ONNX format. Both of these systems can be configured to perform well, but neither one provides a complete set of ML tools nor does either provide adequate training time.

Inworld AI allows for no-code dialogue generation with real-time response generation from created conversational content. In addition to providing quick prototypes, Inworld AI is a paid service with all processing requiring an active internet connection and providing minimal logic outside of conversational logic.

The Behaviour Designer product for Unity provides a visual and simple interface for creating agent behaviour and supports ready-made templates with no coding experience. However, Behaviour Designer does not support ML in any form and can only be used for agents that follow rules.

ASCN.AI NoCode + AI Agent is a product that provides complete no-code automation for the creation and operation of agents, including pre-built integrations and ability to launch quickly for purposes of analytics and decision-making, but is business-focused and will need special adaptations for use in games.

Statistics and Trends Regarding the Use of AI Agents in Gaming

As reported in the GDC State of the Game Industry Survey 2024 71% of respondents in the survey reported using or planning on using AI. Most commonly these AI technologies are being used for procedural content generation, to enhance non-playable character in-game behaviour and for test functions.

• 42% of survey respondents stated they use AI to create levels/quests/textures.
• 38% of respondents reported they use AI to enhance NPC behaviours with machine learning.
• 27% of gaming companies use automated testing via bots.
• 19% of gaming companies adapt to create personal experiences for each gamer by modifying difficulty based on ability level.

Due to the expected growth of the artificial intelligence (AI) market from $1.2 billion in 2024 to an estimated $7.3 billion by 2032 and an average annual growth rate of 25.4%, indie developers have been leveraging new technologies, such as the GPT-4 tool, to create dialogue for video games and thus accelerate development time considerably.

Different Ways to Purchase AI Agents for Games: Pricing and Terms

Commercial Solutions

• Inworld AI - Offers a free tier (for 5,000 requests per month); Indie level (20,000 requests for $20); Studio level (200,000 requests for $200); and Enterprise level (customized plans available).
• Behavior Designer - Available for a one-time purchase of $95 (Unity).
• ASCN.AI NoCode - Subscripton fees starting at $29 per month with individual plans available for studios wishing to integrate into their game(s).
• Unity ML-Agents - Cost-free with additional (cloud) training fees ranging from $50 to $500+ depending on the task to be accomplished.

Freelance/Outsourcing

Average Costs for Developing an AI Agent:

Creating AI Agents Services Via No-Code

For example: An Indie Studio who set up ASCN.AI (to collect and analyze gamer feedback using a Telegram Bot with automated sentiment classification) spent $29/month on services and had it configured within hours. This was the ideal solution for a developer looking for a way to implement working solutions quickly without having to hire (programmer) personnel.

Using an AI Agent in Games - An 8-Step Methodology

1. Define the Goals of the AI Agent in Games

What Issues Do You Want to Solve? An Example - Getting NPCs to be more predictable, Decreasing Complaints about Game Difficulty, Creating More Content. What Metrics do You want to Improve? Retention, Average Play Time, Contraction.

2. Decide What Kind of AI Agent You want to Use

A Rule-Based AI Agent (Behaviour Trees, or Finite State Machines) will Solve Basic Logic Issues and be quicker to design and develop; whereas, a Machine Learning AI Agent (Reinforcement Learning) will Solve Complex Problems that Require Continued Adaptive Behaviour, but the Design and Development Process is longer. A Hybrid Model is created when the Behaviour Tree is used as a High-Level Structure; and Machine Learning is used to Create Low-Level Decisions (for example: Creating A Quests Using Generative-AI).

3. Build the Rules and Behaviours

For Rule-Based Models, Create a Visual Tree. For Machine Learning Models, Write Scripts; Create Inputs, Actions, and Rewards; and then Train the Model. For Generative-AI, Choose a Platform to Use and Set-Up Your Prompts.

4. Test and Pеrfect AI Agents

Conduct A/B Testing, Measure Game Parameters, Adjust Rewards/LogicMappings as Necessary, and Collect User Feedback. It is important to Continuously Improve AI Agent Behaviour and User Interaction.

Common Questions - FAQ

What is an AI agent?

An AI agent is an Autonomous Component of a Game that interprets inputs from the Game World and processes those inputs using either Rules-based Logic or Machine Learning Algorithms, and has the capability of being Adaptive (I.E. Rather than just Executing Scripts).

What is the most common algorithm?

The four primary options for developing agents are: finite state machines (FSM), behavior trees (BT), reinforcement learning, with its various algorithms like Q-learning, DQN, PPO, SAC, and generative language models (GPT) for developing dialogue.

Can you do this without any programming knowledge?

Yes, there are no-code platforms that provide the ability to create dialogue and automate processes without writing any code. However, if you wish to create deep integrations with your game engine, you will likely need some level of technical knowledge to do some of the necessary setup.

How will agents influence performance?

While training an agent requires many resources, the amount of time it takes to infer (run the model in the game) usually occurs within an acceptable timeframe. When you use generative AI for dialogue generation there will be latency introduced (200-500ms), which will be acceptable for some types of conversations, but not in cases of high-speed action.

How ASCN.AI will enable you to earn revenue by using AI agents

ASCN.AI is a no-code platform that allows users to automate and create AI agents without needing a development team. It helps you adapt your operational processes to be able to create and use AI agents for all of your game/technical development tasks and to integrate them with all aspects of accounting and analytical reporting.

Scenario 1 - Automate Game Metric Analytics

Prior to using ASCN.AI, a small game studio required hours of time each week to collect data from Firebase and Unity Analytics. By using ASCN.AI, they now trigger a request for specific data every day at 9 am that is retrieved using the API. Once the data is received, an AI agent reviews the incoming data to identify any problems (for example with a specific game level). Once findings are established, the data is saved to Google Sheets, and a report is sent to the team using Telegram. This saves the development team at least 10 hours of manual data entry each week.

Scenario 2 - Processing Player Feedback/Generating Roadmaps

Prior to ASCN.AI, the development manager was responsible for manually reviewing player reviews which were received via Discord, Steam and Reddit. Using ASCN.AI, every time a new review is submitted, an automated process is triggered that automatically classifies the review into three categories (type, sentiment and priority). Any bugs identified are sent to QA, any suggestions are added to the Google Sheets development roadmap, and an automated message of appreciation is sent to the player. In addition, weekly summary reports are emailed to the team. This enables the manager to save at least 8 hours each week.

Scenario 3 - Utilizing Personalization Offer and Upsell for AI Generated Messages

One free-to-play game was trying to raise its conversion rate from 2% - 5%. When the player completed level 10 an automated triggering event was enabled in ASCN.AI's system, which caused ASCN.AI to analyze the players previous purchases and make personalized recommendations. These were provided to the player by means of an individual message, complete with a deadline. The game finally sent a push notification to the player, and subsequently conversion rates jumped to 4.5%, adding $15,000 in additional margin per month.