An exploratory study into the use of an emotionally aware cognitive assistant

Abstract

This paper presents the results of a survey of human emotional responses to a set of audio-visual prompts or cues designed for assistive technology applications. Our goal in this paper is exploratory: we wish to understand how audio-visual prompts are understood by people on an emotional spectrum as a first step towards the more challenging task for designing emotionally aligned prompts for persons with cognitive disabilities such as Alzheimer's disease and related dementias (ADRD). Persons with ADRD often need assistance from a care partner to complete activities of daily living such as washing hands, making food, or getting dressed. Artificially intelligent systems have been developed that can assist in such situations by giving automated prompts or cues. Our long term aim is to enhance such systems by delivering automated prompts that are emotionally aligned with individuals in order to help with prompt adherence and with long-term adoption. As a step in this direction, this paper presents a set of prompt videos of a virtual human 'Rachel', wherein she expressively communicates prompts at each step of a simple handwashing task, with various human-like emotions and behaviours. A user study was conducted for 30 such prompt videos with respect to three basic and important dimensions of emotional experience: evaluation, potency, and activity. The results show that, while people generally agree on the evaluation (valence: good/bad) of a prompt, consensus about power and activity is not as socially homogeneous. This paper gives an overview of the hand washing system, and then details the creation process of the virtual human and the results and analysis of the user study.