Emotions and Context in Decision-making for a Socially Intelligent Agent

Abstract

Artificial intelligence (AI) research has largely focused on rational thinking, decision making, goal achievement, and reward maximization. Emotions have been considered nonessential, or even detrimental, when designing and implementing AI systems. With the advancement in affective computing research and the increasing adoption of AI agents as part of human society, there is a growing need to have a deeper connection between human and machine. Although significant efforts have been made in affective computing towards recognizing human emotions and generating human-like emotions, there has been less progress towards using emotions to guide decision and understanding human social context. This thesis focuses on emotions and context in decision-making, towards building socially intelligent agents, that are adaptive and emotionally aligned with humans. I first conducted a systematic review of the literature on implemented systems for decision making that used emotions. I synthesized extracted data into four conceptual model types, viz. Matching, Appraisal and Coping, Decision-theoretic and Parametric and provided a process view of each type. Then, I implemented one such model as a brain-inspired neural model. The aim was to model the role of affect guiding decisionmaking, resulting in interactions that are similar to human interactions, while inhibiting some behaviors based on the social context. The model was implemented using Nengo, a python library for building and simulating large-scale neural models, using spiking neurons. I then investigated how to supply such a model with context, known to be a very important factor in emotional-based decision making. I proposed a computer vision spatio-temporal transformer model and its variations for joint learning and prediction, and evaluated on an existing Video Group Affect dataset. Improvements to social event prediction were shown by utilizing affective information. Finally, I considered a real-world care-giving scenario which demonstrates the potential of our model for establishing an emotional relationship and interaction between older adults, care partners, people living with dementia, and three exemplar robots. The insights gained in this thesis may encourage AI and affective computing research to develop agents that can simulate human affective and decision-making mechanisms, and in the process understand humans better.