Doing things with others is always a more fulfilling experience, but sometimes your friends can’t be around to play a game, enter the NPC to keep you engaged in the game. Human and AI interactions are an important aspect of video games. Here are some elements of human and AI interactions that are commonly seen in video games:
Dialogue: Dialogue is a key element of human and AI interactions in video games. Players often engage in conversations with NPCs, and the quality of the dialogue can greatly affect the player’s engagement with the game.
Decision-making: Decision-making is another important element of human and AI interactions in video games. The decisions players make can affect the behavior of NPCs, and the decisions NPCs make can affect the outcome of the game.
Emotion: Emotion is a key aspect of human and AI interactions in video games. NPCs can be designed to display a wide range of emotions, from fear and anger to joy and humor, and the player’s interactions with these NPCs can affect their emotional state.
Personalization: Personalization is an important aspect of human and AI interactions in video games. NPCs can be designed to respond to the player’s actions and preferences, making the game feel more personalized and immersive.
Combat: Combat is another element of human and AI interactions in video games. AI-controlled enemies can be designed to respond to the player’s actions in realistic ways, making combat more engaging and challenging.
Learning: Learning is an important element of AI interactions in video games. AI-controlled characters can learn from the player’s actions and adjust their behavior, accordingly, creating a more immersive and dynamic game world.
Believability in AI and human interactions in gameplay is a crucial factor in creating an engaging gaming experience. When players interact with non-player characters (NPCs), they expect a certain level of believability in terms of the characters’ behaviors and reactions. The same applies to interactions with other players in multiplayer games.
AI is often used to create NPC behavior in games. In this context, believability refers to how closely the NPC’s actions and responses resemble those of a human player. For example, if an NPC in a first-person shooter game always runs in a straight line and never takes cover, players are likely to find it unrealistic and unengaging. On the other hand, if an NPC acts like a real human player by taking cover, shooting back, and even fleeing when overwhelmed, players are more likely to be drawn into the game world and enjoy the experience.
Similarly, human interactions in multiplayer games also require a level of believability. In this context, believability refers to how closely a player’s actions and reactions resemble those of a real person. For example, if a player always rushes into combat without any thought or strategy, other players may find it unrealistic and frustrating to play with them. Conversely, if a player communicates effectively, takes appropriate actions, and behaves in a way that is consistent with the game’s objectives and setting, other players are more likely to find them engaging to play with.
Believability in AI and human interactions in gameplay is critical to creating an immersive gaming experience. Players expect to interact with characters and other players that behave in a way that is consistent with the game’s context and objectives. As AI technology continues to advance, game developers will have even more tools to create more believable and engaging gameplay experiences.
Squads of NPCs can greatly enhance a game player’s experience by adding a sense of teamwork and cooperation to the gameplay. In many games, NPCs may serve as teammates, providing assistance, support, and even combat capabilities. One good example of an NPC squad is the squad of soldiers in the video game “Halo: Combat Evolved.” The game’s protagonist, Master Chief, is often accompanied by a team of AI-controlled soldiers known as the “Marines.” These Marines provide cover fire, take down enemies, and provide commentary on the game’s events. They also follow the player’s lead and respond to orders given by the player.
In addition to combat support, NPCs can also provide other forms of assistance to the player. For example, in the game “Mass Effect,” the player controls a squad of characters who are all controlled by AI. Each squad member has unique abilities, such as hacking or healing, which can be used to assist the player in completing missions.
NPCs can also interact with the player in more subtle ways, such as providing dialogue that enhances the game’s story and setting. In the game “The Last of Us,” the player is accompanied by a young girl named Ellie. Although she cannot provide combat support, Ellie provides emotional support and helps the player navigate the game’s story and world. Squads of NPCs can greatly enhance a game player’s experience by providing combat support, assistance with objectives, and even emotional support. Good examples of NPC squads include the Marines in “Halo: Combat Evolved,” the squad in “Mass Effect,” and Ellie in “The Last of Us.”
There are many examples of believable NPCs in video games. One example is Ellie from The Last of Us, a game developed by Naughty Dog and released in 2013.
Ellie is a companion NPC who accompanies the player character, Joel, throughout the game. She is a young teenage girl who has grown up in a post-apocalyptic world where a fungal infection has wiped out most of the human population. Despite the challenges she faces, Ellie is a well-developed character who is both tough and vulnerable. Her dialogue and actions are realistic and engaging, and she forms a strong bond with the player as they progress through the game. Ellie’s behavior in the game is controlled by a complex AI system that allows her to react to the player and the environment in realistic ways. For example, when the player character is injured, Ellie will search for medical supplies to help him. She also reacts realistically to enemies and can engage in combat to help the player.
Ellie’s believability is enhanced by her voice acting and facial animations, which were captured using motion capture technology. This allows her to display a wide range of emotions, from fear and sadness to humor and courage. Her well-developed character, realistic behavior, and engaging dialogue make her an essential part of The Last of Us and a fan favorite among gamers.
Dialogue and Chatbots in AI Agents
Along with Video or Images in AI Games and Simulations, Audio also plays an important part in the game world. Through interactions with AI Agents gamers can learn narratives of the story, they can learn where to go next, they can be bullied by oppositional AI Agents or NPCs to try to steer them toward game play, or away from their goals—winning the game. AI Assistants can leverage this Audio into hidden ways to win the game, depending on how generative an AI Assistant is.
Dialogue systems of non-player characters (NPCs) in video games have come a long way since the early days of gaming. While NPCs were once simply programmed to repeat a set of pre-recorded lines, modern dialogue systems have become much more sophisticated, allowing for more dynamic and immersive interactions between players and NPCs.
One of the most important aspects of dialogue systems in video games is their ability to create a sense of agency and immersion for players. In a game with a well-designed dialogue system, players should feel like they are truly engaging with a living, breathing world, where the NPCs have their own personalities, motivations, and goals. This can be achieved through a variety of techniques, including branching dialogue trees, dynamic AI-driven conversations, and even voice recognition technology.
One common approach to dialogue systems in video games is the use of branching dialogue trees. These are essentially sets of pre-written conversations that are triggered based on the player’s choices and actions. For example, in a game like Mass Effect, the player’s choices in conversation can lead to different outcomes, affecting the overall story and the player’s relationships with NPCs. This approach can be effective, but it can also be limiting, as it requires a lot of pre-written content and can make conversations feel somewhat scripted.
Another approach to dialogue systems is the use of dynamic AI-driven conversations. These are conversations that are generated on the fly, based on the player’s actions and the current state of the game world. This approach allows for a much greater degree of player agency, as the NPCs can respond to the player’s actions in more natural ways. For example, in the game Red Dead Redemption 2, NPCs will react differently to the player depending on the time of day, their location, and their reputation. This can make conversations feel much more organic and immersive.
It is relatively straightforward these days to create your own custom dialogue engine by using the HuggingFace (https://huggingface.co) model zoo, where there are many pretrained models that you can fine tune with a dataset based on texts of things a particular character has said in books or films, for example this one fine tuned to Harry Potter dialogue that could be inserted into a video game:
from transformers import AutoModelWithLMHead, AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(‘s3nh/DialoGPT-small-harry-potter-goblet-of-fire’)
model = AutoModelWithLMHead.from_pretrained(‘s3nh/DialoGPT-small-harry-potter-goblet-of-fire’)
for step in range(4):
new_user_input_ids = tokenizer.encode(input(“>> User:”) + tokenizer.eos_token, return_tensors=’pt’)
bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids
chat_history_ids = model.generate(
bot_input_ids, max_length=200,
pad_token_id=tokenizer.eos_token_id,
no_repeat_ngram_size=3,
do_sample=True,
top_k=100,
top_p=0.7,
temperature=0.8
)
print(“HarryBot: {}”.format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
# https://huggingface.co/s3nh/DialoGPT-small-harry-potter-goblet-of-fire
Voice recognition technology is another exciting development in the world of dialogue systems. While this technology is still in its early stages, it has the potential to revolutionize the way we interact with NPCs in video games. With voice recognition, players could speak to NPCs using natural language, rather than selecting pre-written dialogue options. This could allow for more dynamic and personalized conversations, as well as greater immersion in the game world.
Of course, creating effective dialogue systems for NPCs is no easy task. It requires a combination of good writing, strong AI programming, and a deep understanding of what players are looking for in terms of immersion and agency. Additionally, dialogue systems must be carefully balanced to avoid overwhelming players with too much choice, while still allowing them to feel like they have agency in the game world.
Dialogue Systems in AI are of varied designs. For instance in a combat simulation, you will have a Battle Chatter system- enemy AI will shout commands to one another and provide exposition of their intended behaviour. Not all AI Agents are the same, one factor of differentiation in NPCs is Knowledge of Player- regular archetypes need to find the player, while specialists always know where you are.Which is reminiscent in Drone Swarm programming that a lead AI Agent (drone) tells and coordinates the other drones in the swarm, a hierarchical C2 within drone swarms.
Facade, a groundbreaking interactive drama game released in 2005, revolutionized the video game industry with its innovative dialogue system. Developed by Michael Mateas and Andrew Stern, Facade pushed the boundaries of interactive storytelling by creating a game in which the player’s choices and actions directly influenced the game’s narrative. At the heart of Facade’s gameplay is its dialogue system. The game places players in the role of a guest at the home of a married couple, Trip and Grace. As the player interacts with the couple, they are presented with a wide range of dialogue options, ranging from polite small talk to personal attacks. The player’s choices influence the couple’s reactions, as well as the overall trajectory of the game’s story.
One of the key innovations of Facade’s dialogue system is its use of natural language processing. Unlike most games, which limit player choice to pre-written dialogue options, Facade allows players to type in their own responses to the characters. The game’s AI then analyzes the player’s input, attempting to understand their intent and crafting an appropriate response from the characters. This approach to dialogue makes Facade feel much more immersive and dynamic than most other games. Instead of simply selecting from a list of pre-written responses, players can respond to the characters in the same way they might in a real conversation. This allows for a much greater degree of player agency and immersion in the game world.
However, creating a dialogue system that can accurately parse natural language is no easy task. Mateas and Stern had to develop a sophisticated AI system that could analyze and understand the player’s input, and then generate appropriate responses from the game’s characters. This required a deep understanding of natural language processing and a significant investment of time and resources. Despite these challenges, Facade’s dialogue system was a major success. Players were impressed by the game’s ability to understand and respond to their input, and the game received widespread critical acclaim for its innovative approach to interactive storytelling. Facade paved the way for a new generation of dialogue systems in video games, inspiring developers to create games with more dynamic and immersive conversations.
To fully understand how Dialogue works in games, the following overview of Dialogue Systems is presented. A way for an NPC to show intelligence in a game is to engage in a dialogue with a player, this can of course also be used as strategy in the game. The vast majority of dialogue in games is scripted often leading to clunky implementations where NPCs repeat themselves unnecessarily or they will ignore the player entirely, especially when they have input that matches no handler for dialogue responses. Dialogue trees are a common technique to simulate dialogue in the game world. Branching dialogue trees allow a gamer to choose from pre-selected questions or options to speak to the NPC. Some dialogue trees are not trees but can be graphs that have cycles (Rose 2018).
Understanding inputs involves the use of Natural Language Processing libraries. Rose talks about the limitations of using NPL:
The use of simple natural language processing has the following major disadvantages:
NPC responses are usually not automated
o Most of the time, NPC responses are prewritten by a human author; it is very rare for them to be automated. Thus, lots of time and effort must go into writing them because “the only way to increase interactivity is to author extraordinary amounts of content by brute force”. NPC may ignore what player says
o If no rule exists on how to respond to the player’s input, an NPC will either ignore the player entirely or suddenly change the topic of
conversation. Either scenario breaks the illusion that the NPC is intelligent because it becomes obvious that it does not know how to respond.
With sentiment analysis, the system could recognize that
the player has a negative sentiment towards the sword’s sharpness. In response, it could perhaps recommend either a place to get it sharpened or a shop to purchase a new sword. Furthermore, players may be either friendly or hostile towards an NPC. Sentiment analysis would allow the system to recognize the player’s positive or negative sentiment about the NPC and have it respond in a sensible manner. For example, if a player insults the NPC, then the NPC could respond aggressively, while if the player compliments it, it could respond kindly. By adjusting the NPC’s reaction based on how the player is perceived to be feeling, the dialogue will seem more natural and the NPC will seem more autonomous. (Rose, 2018, 24-5)
With these limitations in mind recent game developers have relied on other means of developing Game dialogue systems. Such as Information Extraction, which extracts structured information from unstructured sources. Sentiment Analysis, analyzes the sentiment of the gamer and makes recommendations or misdirects based on its perceptions of the gamers dialogue. Question Answering, which posts information based on a natural language question.
Combined with information extraction and sentiment analysis, question answering could potentially offer very sophisticated replies. While none of these techniques have been perfected yet, their implementation in a dialogue engine could greatly add to the believability of an NPC compared to traditional methods. (Rose 2018, 25-6)
A key factor that facilitates this sort of Turing Test in games, is that the NPC has Episodic Memory, which is responsible for recalling episodic memories based on previous topics in conversation. Keywords are used which trigger the activation of certain memories. When the topic handler decides a match exists in memory the most recently activated memory is retrieved (Rose, 2018, 47). Episodic memory can be created ahead of time in a knowledge base or assigned dynamically during the game, each memory receives a weight based on ‘importance’. Two pools of memories are created based on these weights: short term and long term memory. With low weight short term memories being deleted like a forgetful or immemorable experience.
An effective Dialogue system can go a long way to making the NPCs in a game or simulation believable. It can also make the game interactions even richer. One developer, Alec John Larsen, has proposed a solution that allows authors to create different NPCs using a single chatbot, by using the open source project Artificial Linguistic Internet Computer Entity (A.L.I.C.E.).
To facilitate the dialogue behaviors are facilitated by A Behavior Language (ABL).
Also, there is a drama manager, to facilitate beats in dialogue, beats are selected using preconditions (causal dependence), weights, priorities and the current tension levels (Larsen, 16). A rule-based system for translating player actions and textual input into discourse acts. This can provide a great experience in Interactive
Drama in the game as Larsen explains:
…interactive drama, interactive storytelling and interactive experiences, to name a few. There are also many definitions of what interactive diction is. It is stated that interactive dramas are systems in which the audience experience the story as interactive participants. The users may be first person protagonist but this is not an explicit requirement of an interactive diction system. Some works of interactive diction make use of artificial intelligence (AI) techniques, to manage the way the story is generated, and some do not (e.g. hypertext diction). The focus of interactive storytelling is audience experience, this includes building worlds with character and story, where the virtual worlds are dramatically interesting and contain computer-controlled characters, with whom the user can interact. From the design focus when creating interactive experiences is agency, immersion (a player feels like an integral part of the story) and transformation (a player can experience different aspects of the story), where agency is the most important. Agency is defined in as the satisfying ability of the player to make meaningful decisions, which have tangible effects on the play experience. It is not that the players have an unlimited number of actions at their disposal but rather there are context-based actions that can be performed, which have a very real impact on the experience. The reason agency is the main focus when designing interactive diction is because it allows the player to have the most substantial influence on the experience and requires the system to dynamically generate the story based on the player’s unpredictable actions.
(Larsen, 14-5)
He goes on giving an account of how the dialogue is programmed:
Dialogue graphs are made of stimulus response elements (SREs), represented by AIML patterns (which map an input to an output). A GUI is used to create the dialogue graphs and the scenes out of SREs. Scenejo requires that the authors take the NPCs’ moods into account and as such cannot dynamically assign mood. Scenejo uses AIML extremely verbosely and requires a knowledge-base per character per seen, resulting in an inefficient writing process. SentiStrength is a textual affect sensing framework, which makes use of keyword spotting and statistical methods…to assess the sentiment of informal text. SentiStrength gives text a rating of positive (1 to 5) and negative (1 to 5) simultaneously, which differentiates it from other opinion mining systems which do not give both positive and negative ratings simultaneously and do not give strengths of the sentiment but rather an indication of whether it is positive or negative. SentiStength was created using MySpace user comments (which provided a large quantity of informal text) as training data sets and evaluation data set. (Larsen, 18)
An interesting example of AIML at work is this dialogue with ChatGPT, which is used to generate small parts of this book:
Dialogue is dependent on mood. Which is computationally complex, see next section. How does one rank the emotive qualities of the player? This is handled by the field of Textual Affect Sensing, commonly used libraries are ConceptNet and SentiStrength. ConceptNet uses the model of Paul Ekman, where every possible emotion can be given a weighted sum of six emotions: happiness, sadness, anger, fear, disgust and surprise. And of course each would also have other behaviors or markers in the game interface. The weights are then calculated into a decision making structure to produce desirable and hopefully believable behaviors. The mulitplicative weights are modeled using a Feelings Modelling Language (FML).
So one can see how dialogue systems can be crafted in games or serious games or simulations. Of course one could take this a step further and input a deep psychological profile of the gamer into the system to get even more specific responses to the player. Making the illusion that the game is real, enticing the player to get lost in the ‘flow’, is tantamount to an emotionally invested and adrenaline rushed gamer, to get them into the game, to lose sight of reality and be entertained. Creating believable characters with effective dialogue is just one aspect of the open world game that is popular in current game development. Open world games require a large diverse and unique set of background NPCs to create the illusion of a populated virtual world that mirrors the diversity of the real world. One can imagine the difficulties is trying to hard code all these unique NPCs, which means that many NPCs are repeated characters and patterns, which makes an open world game seem repetitive, predictable and not believable (Fallon, 2-3). The highest priority should be given to Personality in a Game. Researcher in making games believable extrapolates on the Personality problem in games:
Emotion: Just as in the traditional character-based arts, as well as acting, where emotion is regarded as a fundamental requirement to bring the characters to life, so too is it in making believable agents. In order for an agent to be believable, it must appear to have emotions and be capable of expressing these emotions in a manner that corresponds with its personality.
Self Motivation: In order to create believable agents, a requirement is that they should be self motivated. In other words agents should be able to carry out actions of their own accord and not just react to stimuli, as is often the case in the creation of autonomous agents. The power of this requirement is quite substantial and can lead to increased admiration and interest in the agent when it appears to complete tasks according to its own intentions.
Change: Essential to the believability of agents is how they grow and change. However, this growth and change cannot be arbitrary, rather it should correspond with the agents personality.
Social Relationships: Since humans lead a generally social lifestyle, an important requirement for believable agents is social relationships. There should be interaction amongst agents, influenced by the relationships that exist between them, and in turn those interactions may influence the relationships.
Consistency of Expression: A basic requirement for believability is the consistency of expression. An agent can have a multitude of behavioral and emotional expressions such as facial expression, movement, body posture, voice intonation etc. In order to be believable at all times, these expressions must work in unison to communicate the relevant message for the personality,
feelings, thinking, situation etc. of the character. If this consistency is
broken, the suspension of disbelief is likely to be lost.
The Illusion of Life: Some requirements necessary for believable agents are overlooked or taken for granted by the conventional media. For example, it is not necessary to specify that an actor should be able to speak while walking since any human actor reading the script can do this easily. This is simply not the case when creating believable agents where those properties must be built into the agent. A list of such properties, which alltogether form this final requirement for believability, includes: appearance of goals, concurrent pursuit of goals and parallel action, reactive and responsive, situated, resource bounded, existence in a social context, broadly capable and well integrated (capabilities and behaviours).
(Fallon 2013)
The desire to have believable AI Assistants has driven a lot of research into the area of believability, which is to say Turing Test passable characters, even with some NPCs seeming more human than humans we meet in real life. With these AI Agents or NPCs the internal model of the agent is built on a system known as the Beliefs-Desires-Intentions (BDI) architecture (Fallon 2013). Other models are also incorporated into the model of the agent, the Consumat model uses dominant psychological theories on human behavior, being put into a 2×2 matrix. One hand you have the level of need of satisfaction (LNS) and behavioral control (BC), while on the other hand based on certainty, type of needs and cultural perspective (Fallon 2013, 12). Another model which is a meta-model developed by Carley and Newell is the Fractionation Matrix (C&N Matrix) on social behavior to provide understanding of the complexity of an AI Agent. This model is based on an assortment of sociological theories that can be used to classify, categorize and silo human like behavior in agents, the goal of which is to create a “Model social agent” (MSA) which has strong, human-like social behavior (Fallon 2013, 13-4). A derivative from MSA is the Model Social Game Agent of Johansson and Verhagen which takes that there exists an emotional state and a social state that contribute to a behavior selection, which subsequently leads to an action from the agent ((Fallon 2013, 15). One last entry in this field is that of SIGVerse as explained by Fallon:
More comprehensive frameworks have been recently proposed with a focus on serious games. Inamura et al propose a simulator environment known as SIGVerse which includes a combination of dynamics, perception and communication simulations, with a background in robotics simulation and therefore with a strong model of embodiment and situatedness. Embodiment involves equipping a body with sensors and actuators which enables structural coupling with the agents surroundings. A video game agent can therefore be said to be embodied in that sense (or ‘virtually embodied’) since it procures exteroceptive and proprioceptive sensing data through its software sensors and actions can also be performed using its software actuators. Situatedness is concerned with interaction with the world. Take for example a video game character, if it interacts with the simulated world, the game environment is affected, which in turn affects the agent. The SIGVerse project does not, however, incorporate social interaction very well. (Fallon 2013, 18)
For more on dialogue systems in video games see, Tommy Thompson in his channel ‘AI and Games’, How Barks Make Videogame NPCs Look Smarter, https://www.youtube.com/watch?v=u9VkW18IMzc
Emotions in Video Games
As seen in the previous section dialogue is a key component to believable AI Agent NPCs. It was also drawn out that dialogue must have a calculation for emotion to give a proper context to the dialogue. So when we say emotion modeling what is it we are talking about:
‘Emotion modeling’ can mean the dynamic generation of emotion via black-box models that map specific stimuli onto associated emotions. It can mean generating facial expressions, gestures, or movements depicting specific emotions in synthetic agents or robots. It can mean modeling the effects of emotions on decision making and behavior selection. It can also mean including information about the user’s emotions in a user model in tutoring and decision-aiding systems, and in games. There is also a lack of clarity regarding what affective factors are modeled. The term ‘emotion’ itself is problematic …it has a specific meaning in the emotion research literature, referring to transient states, lasting for seconds or minutes, typically associated with well-defined triggering cues and characteristic patterns of expressions and behavior. (More so for the simpler, fundamental emotions than for complex emotions with strong cognitive components.) (Hudlicka, 1)
Yet, there is a question as to what is being modeled and the consistency of the models.
Recognizing the lack of consistent terminology and design guidelines in emotion modeling, this paper proposes an analytical framework to address this problem. The basic thesis is that emotion phenomena can usefully be understood (and modeled) in terms of two fundamental processes: emotion generation and emotion effects, and the associated computational tasks. These tasks involve, for both processes: defining a set of mappings (from triggers to emotions in emotion generation, and from emotions to their effects in the case of emotion effects), defining intensity and magnitude calculation functions to compute the emotion intensities during generation, and the magnitude of the effects, and functions that combine and integrate multiple emotions: both in the triggering stage (antecedents), and in the emotion effects stage (consequences). This analysis represents a step toward formalizing emotion modeling, and providing foundations for the development of more systematic design guidelines, and alternatives available for model development. Identifying the specific computational tasks necessary to implement emotions also helps address critical questions regarding the nature of emotions, and the specific benefits that emotions may provide in synthetic agents and robots. (Hudlicka 2008, 8)
Sergey Tarasenko, has sought out an emotional model based on Reflexive Control. In McCarron 2023, we showed how that Reflexive Control was combined with Game Theory, RC would be a good tool for steering game behavior of the player along the lines of an established narrative or goal in a game. One researcher, currently with Mizuho Securities, has done extensive work on Reflexive Game Theory and Emotions, Tarasenko (2016). Sergey Tarasenko’s work, which was originally devised by Soviet military programmer, Lefebvre, also deals with using robots interacting with humans in Reflexive Games He writes regarding the incorporation of emotion into games in a paper which he also cites Ekman, from above:
The semantic differential approach originally proposed by Osgood et al [1957]. considers three dimensions to characterize the person’s personality. These dimensions are Evaluation, Activity and Potency. This approach was further tested from the point of view of emotions. Russell and Mehrabian proved in their study that the entire spectra of emotions can be described by the 3-dimensional space spanned by Pleasure (Evaluation), Arousal (Activity) and Domination (Potency) axes. For every dimension, the lower and upper bounds (ends) are recognized as negative and positive poles [-1,0,1], respectively. Consequently, the negative pole can be described by negative adjectives, and positive one by positive adjectives. It was shown by Russel and Mehrabian that these three dimensions are not only necessary dimensions for an adequate description of emotions, but they are also sufficient to define all the various emotional states. In other words, the Emotional state can be considered as a function of Pleasure, Arousal and Dominance. On the basis of this study, Mehrabian proposed Pleasure-Arousal-Dominance (PAD) model of Emotional Scales. The emotional states defined as combinations of ends from various dimensions are presented. In this study, we discuss the matter of how the PAD model can be used in Reflexive Game Theory (RGT) to emotionally color the interactions between people and humans and robots.
Dialogue is dependent on mood. Which is computationally complex, see next section. How does one rank the emotive qualities of the player? This is handled by the field of Textual Affect Sensing, commonly used libraries are ConceptNet and SentiStrength. ConceptNet uses the model of Paul Ekman, where every possible emotion can be given a weighted sum of six emotions: happiness, sadness, anger, fear, disgust and surprise. And of course each would also have other behaviors or markers in the game interface. The weights are then calculated into a decision making structure to produce desirable and hopefully believable behaviors. The mulitplicative weights are modeled using a Feelings Modelling Language (FML).
So one can see how dialogue systems can be crafted in games or serious games or simulations. Of course one could take this a step further and input a deep psychological profile of the gamer into the system to get even more specific responses to the player. Making the illusion that the game is real, enticing the player to get lost in the ‘flow’, is tantamount to an emotionally invested and adrenaline rushed gamer, to get them into the game, to lose sight of reality and be entertained. Creating believable characters with effective dialogue is just one aspect of the open world game that is popular in current game development. Open world games require a large diverse and unique set of background NPCs to create the illusion of a populated virtual world that mirrors the diversity of the real world. One can imagine the difficulties is trying to hard code all these unique NPCs, which means that many NPCs are repeated characters and patterns, which makes an open world game seem repetitive, predictable and not believable (Fallon, 2-3). The highest priority should be given to Personality in a Game. Researcher in making games believable extrapolates on the Personality problem in games:
Emotion: Just as in the traditional character-based arts, as well as acting, where emotion is regarded as a fundamental requirement to bring the characters to life, so too is it in making believable agents. In order for an agent to be believable, it must appear to have emotions and be capable of expressing these emotions in a manner that corresponds with its personality.
Self Motivation: In order to create believable agents, a requirement is that they should be self motivated. In other words agents should be able to carry out actions of their own accord and not just react to stimuli, as is often the case in the creation of autonomous agents. The power of this requirement is quite substantial and can lead to increased admiration and interest in the agent when it appears to complete tasks according to its own intentions.
Change: Essential to the believability of agents is how they grow and change. However, this growth and change cannot be arbitrary, rather it should correspond with the agents personality.
Social Relationships: Since humans lead a generally social lifestyle, an important requirement for believable agents is social relationships. There should be interaction amongst agents, influenced by the relationships that exist between them, and in turn those interactions may influence the relationships.
Consistency of Expression: A basic requirement for believability is the consistency of expression. An agent can have a multitude of behavioral and emotional expressions such as facial expression, movement, body posture, voice intonation etc. In order to be believable at all times, these expressions must work in unison to communicate the relevant message for the personality,
feelings, thinking, situation etc. of the character. If this consistency is
broken, the suspension of disbelief is likely to be lost.
The Illusion of Life: Some requirements necessary for believable agents are overlooked or taken for granted by the conventional media. For example, it is not necessary to specify that an actor should be able to speak while walking since any human actor reading the script can do this easily. This is simply not the case when creating believable agents where those properties must be built into the agent. A list of such properties, which alltogether form this final requirement for believability, includes: appearance of goals, concurrent pursuit of goals and parallel action, reactive and responsive, situated, resource bounded, existence in a social context, broadly capable and well integrated (capabilities and behaviours).
(Fallon 2013)
The desire to have believable AI Assistants has driven a lot of research into the area of believability, which is to say Turing Test passable characters, even with some NPCs seeming more human than humans we meet in real life. With these AI Agents or NPCs the internal model of the agent is built on a system known as the Beliefs-Desires-Intentions (BDI) architecture (Fallon 2013). Other models are also incorporated into the model of the agent, the Consumat model uses dominant psychological theories on human behavior, being put into a 2×2 matrix. One hand you have the level of need of satisfaction (LNS) and behavioral control (BC), while on the other hand based on certainty, type of needs and cultural perspective (Fallon 2013, 12). Another model which is a meta-model developed by Carley and Newell is the Fractionation Matrix (C&N Matrix) on social behavior to provide understanding of the complexity of an AI Agent. This model is based on an assortment of sociological theories that can be used to classify, categorize and silo human like behavior in agents, the goal of which is to create a “Model social agent” (MSA) which has strong, human-like social behavior (Fallon 2013, 13-4). A derivative from MSA is the Model Social Game Agent of Johansson and Verhagen which takes that there exists an emotional state and a social state that contribute to a behavior selection, which subsequently leads to an action from the agent ((Fallon 2013, 15). One last entry in this field is that of SIGVerse as explained by Fallon:
More comprehensive frameworks have been recently proposed with a focus on serious games. Inamura et al propose a simulator environment known as SIGVerse which includes a combination of dynamics, perception and communication simulations, with a background in robotics simulation and therefore with a strong model of embodiment and situatedness. Embodiment involves equipping a body with sensors and actuators which enables structural coupling with the agents surroundings. A video game agent can therefore be said to be embodied in that sense (or ‘virtually embodied’) since it procures exteroceptive and proprioceptive sensing data through its software sensors and actions can also be performed using its software actuators. Situatedness is concerned with interaction with the world. Take for example a video game character, if it interacts with the simulated world, the game environment is affected, which in turn affects the agent. The SIGVerse project does not, however, incorporate social interaction very well. (Fallon 2013, 18)
For more on dialogue systems in video games see, Tommy Thompson in his channel ‘AI and Games’, How Barks Make Videogame NPCs Look Smarter, https://www.youtube.com/watch?v=u9VkW18IMzc
Emotions in Video Games
As seen in the previous section dialogue is a key component to believable AI Agent NPCs. It was also drawn out that dialogue must have a calculation for emotion to give a proper context to the dialogue. So when we say emotion modeling what is it we are talking about:
‘Emotion modeling’ can mean the dynamic generation of emotion via black-box models that map specific stimuli onto associated emotions. It can mean generating facial expressions, gestures, or movements depicting specific emotions in synthetic agents or robots. It can mean modeling the effects of emotions on decision making and behavior selection. It can also mean including information about the user’s emotions in a user model in tutoring and decision-aiding systems, and in games. There is also a lack of clarity regarding what affective factors are modeled. The term ‘emotion’ itself is problematic …it has a specific meaning in the emotion research literature, referring to transient states, lasting for seconds or minutes, typically associated with well-defined triggering cues and characteristic patterns of expressions and behavior. (More so for the simpler, fundamental emotions than for complex emotions with strong cognitive components.) (Hudlicka, 1)
Yet, there is a question as to what is being modeled and the consistency of the models.
Recognizing the lack of consistent terminology and design guidelines in emotion modeling, this paper proposes an analytical framework to address this problem. The basic thesis is that emotion phenomena can usefully be understood (and modeled) in terms of two fundamental processes: emotion generation and emotion effects, and the associated computational tasks. These tasks involve, for both processes: defining a set of mappings (from triggers to emotions in emotion generation, and from emotions to their effects in the case of emotion effects), defining intensity and magnitude calculation functions to compute the emotion intensities during generation, and the magnitude of the effects, and functions that combine and integrate multiple emotions: both in the triggering stage (antecedents), and in the emotion effects stage (consequences). This analysis represents a step toward formalizing emotion modeling, and providing foundations for the development of more systematic design guidelines, and alternatives available for model development. Identifying the specific computational tasks necessary to implement emotions also helps address critical questions regarding the nature of emotions, and the specific benefits that emotions may provide in synthetic agents and robots. (Hudlicka 2008, 8)
Sergey Tarasenko, has sought out an emotional model based on Reflexive Control. In McCarron 2023, we showed how that Reflexive Control was combined with Game Theory, RC would be a good tool for steering game behavior of the player along the lines of an established narrative or goal in a game. One researcher, currently with Mizuho Securities, has done extensive work on Reflexive Game Theory and Emotions, Tarasenko (2016). Sergey Tarasenko’s work, which was originally devised by Soviet military programmer, Lefebvre, also deals with using robots interacting with humans in Reflexive Games He writes regarding the incorporation of emotion into games in a paper which he also cites Ekman, from above:
The semantic differential approach originally proposed by Osgood et al [1957]. considers three dimensions to characterize the person’s personality. These dimensions are Evaluation, Activity and Potency. This approach was further tested from the point of view of emotions. Russell and Mehrabian proved in their study that the entire spectra of emotions can be described by the 3-dimensional space spanned by Pleasure (Evaluation), Arousal (Activity) and Domination (Potency) axes. For every dimension, the lower and upper bounds (ends) are recognized as negative and positive poles [-1,0,1], respectively. Consequently, the negative pole can be described by negative adjectives, and positive one by positive adjectives. It was shown by Russel and Mehrabian that these three dimensions are not only necessary dimensions for an adequate description of emotions, but they are also sufficient to define all the various emotional states. In other words, the Emotional state can be considered as a function of Pleasure, Arousal and Dominance. On the basis of this study, Mehrabian proposed Pleasure-Arousal-Dominance (PAD) model of Emotional Scales. The emotional states defined as combinations of ends from various dimensions are presented. In this study, we discuss the matter of how the PAD model can be used in Reflexive Game Theory (RGT) to emotionally color the interactions between people and humans and robots.
Rather than having a 2X2 Matrix as the above example illustrates we have a 3×3 matrix [not unlike the Enneagram and it’s 3 divisions and 3 values per division, again see Krylov space from Ch4 McCarron 2023]. Tarasenko sees this as a way of influencing people, it might also be used to create great NPCs. By applying his math to a computer game it could be possible to create more believable characters. Of course, all of this is very similar to the ideals presented by Norseen in ‘Thought Injection’ (McCarron 2023). Tarasenko points out how different variances in the matrix can lead to different emotional states:
Summarizing the facts about the RGT [Reflexive Game Theory] and PAD model, we highlight that RGT has been proven to predict human choices in the groups of people and allows to control human behavior by means of particular influences on the target individuals. Next we note that PAD model provides a description of how the emotional states of humans can be modelled, meaning that a certain emotional state of a particular person can be changed to the desired one. Furthermore, it is straight-forward to see that the coding of the PAD emotional states and alternatives of Boolean algebra are identical.
Therefore, it is possible to change the emotional states of the subjects in the groups by making influences as elements of the Boolean algebra. In such a case, vector {1,0,0}, for example, plays a role of influence towards emotional state Docile. Besides, we have distinguished three basis emotional states Docile ({1,0,0}), Anxious ({0,1,0}) and Disdainful ({0,0,1}). The interactions (as defined by disjunction and conjunction operations) of these basic emotional states can result in derivative emotional states such as Dependent, Relaxed, etc. Before, considering the example of PAD application in RGT, we note that reflexive function Φ defines state, which subject is going to switch to. This process goes unconsciously. We have discussed above the reasons the conjunction and disjunction represent alliance and conflict relationships, respectively (Tarasenko, 2016) [emphasis, bold, added]
In a video game this could also be incorporated into dynamic AI vs Human, AI vs AI, interactions. In his paper he gives the example of a Director of Enterprise and how to measure his assistants, this is also of utility in Quantitative Finance Behaviorism. One can see how in squad based combat simulations this could come in handy in single player mode, as the gamer is in a squad with only AI NPCs. The interactions could become very interesting rather than scripted. Indeed, he does see AIs taking on human emotions and believes this will make computation faster. Tarasenko writes:
Furthermore, we have not only illustrated how to apply RGT to the control of subject’s emotions, but uncovered the entire cascade of human reflexion as a sequence of subconscious reflexion, which allows to trace emotional reflection of each reflexive image. This provides us with unique ability to unfold the sophisticated structure of reflexive decision making process, involving emotions. Up to date, there has been no approach, capable of doing such a thing, reported. The emotional research based on PAD model is transparent and clear. The models of robots exhibiting human like emotional behavior using only PAD has been successfully illustrated in recent book by Nishida et al. The ability to influence the factors (variables) is the major justification for such approaches and for the application of the RGT. Yet, it is just a “mechanical” part of the highly sophisticated matter of human emotions.
The present study introduces the brand new approach to modeling of human emotional behavior. We call this new breed of RGT application to be emotional Reflexive Games (eRG). At present RGT fused with PAD model is the unique approach allowing to explore the entire diversity of human emotional reflexion and model reflexive interaction taming emotions. This fusion is automatically formalized in algorithms and can be easily applied for further developing emotional robots. Since the proposed mechanism has no heavy negative impact on human psychological state, robots should be enabled to deal with such approach in order to provide human subjects with stress free psychological friendly environment for decision making. (Tarasenko, 2016)
Another aspect of emotions in games is that of the beat of the game. The beat or pacing of the game is a key element of emotional engagement as studied by Thompson. In Left 4 Dead (2009) the use of an AI Director System was introduced (Thompson, 2014), the director system builds pace in the game, it does not just constantly through a steady stream of NPCs at the player rather it uses 3 phases: build-up (anticipation/expectations), peak, relax, with the amplification of each phase variable. The Director system uses player stress levels based on player performance, worse performance = greater stress. The stress level rate of increase tells the system how skilled the player is. The Directors present something novel: an opportunity to create unique experiences every time you play the game.
Another emotional manipulating trick in games is that of creating a personal relationship with the player by NPCs, as shown with the Nemesis System in Shadow of Mordor (2014):
“Middle-earth: Shadow of Mordor has a lot of ambition in what it’s trying to do. The action-adventure title stands out for a couple new features that help its open world feel alive. One of those things is the unique Nemesis system, where the game populates a hierarchical system of hostile Uruks who all have unique names, features, and traits. Better yet, these AI-controlled enemies remember specific things about you and the world, such as whether or not you beat them in combat, wound them, run away, or do something else entirely. What’s fantastic here is that not only does the system make each player’s experience unique, it gives the title’s setting a personality that other open-world games struggle to create.
These efforts to humanize your enemies, and then use those enemies to more or less troll the player in a way we enjoy, is brilliant. A few simple systems work together to create something that feels larger than life and complex, and it helps to make Shadow of Mordor one of the most interesting games of the season.”
https://www.polygon.com/2014/10/13/6970533/shadow-of-mordors-nemesis-system-is-simpler-than-you-think-and-should
Leave a Reply