Design and Study of Emotions in Virtual Humans for Assistive Technologies

Abstract

This thesis presents the design and study of emotionally aligned prompts given by virtual humans for persons with cognitive disabilities such as Alzheimer’s disease and related dementias (ADRD). Our goal is to understand how emotions in virtual humans are interpreted by people. Persons with ADRD often need assistance from a care partner to complete activities of daily living such as washing hands, making food, or getting dressed. Artificially intelligent systems have been developed that can assist in such situations by giving automated prompts or cues. Our long term aim is to enhance such systems by delivering automated prompts that are emotionally aligned with individuals in order to help with prompt adherence and with long-term adoption. As a step in this direction, we designed and conducted user study with three different virtual humans, who expressively communicate prompts for a simple handwashing task. The user study was conducted in two phases. The phase I study had all age group people as participants and involved a female virtual human character with facial expressions and body gestures. The phase II study had elderly people as participants and involved both male and female virtual human characters with a focus on their facial expressions. Prompts were evaluated with respect to three basic and important dimensions of emotional experience: evaluation, potency, and activity. The results of the phase I study showed that, people generally agree on the evaluation dimension, whereas in phase II, we had more consensus on evaluation and potency dimensions and were close to consensus on activity. This thesis gives an overview of the hand washing system, and then details the design of the virtual human character and prompts and the results and analysis of the user study for both phases.