An Assistive Handwashing System with Emotional Intelligence

Abstract

Whether emotional intelligence (or, affective reasoning) is included can influence the effectiveness of a cognitive assistive system. This thesis presents a novel emotionally intelligent hand-washing assistant that aims to help older adults with Alzheimer's disease complete hand-washing tasks more independently. The thesis reviews previous works in the development of cognitive assistants and in the study of emotional intelligence, and then designs a hand-washing system prototype that combines the two research streams. The difficulties in designing such a system, including probabilistic and decision-theoretic reasoning of the user's functional and emotional states, computer-vision based activity monitoring and affective recognition, and embodied prompting are discussed. Designing the hand-washing system as an integration of independent components, the thesis also discusses coordination between the components. The thesis implements the system in the end, and shows by preliminary tests in laboratory settings that the system implemented (1) runs in real-time from the perspective of the user group, (2) is able to provide a level of functional assistance, (3) produces system prompts that have encoded to some extent the emotional state of its user. The preliminary tests also indicated that a user with emotions with high potency levels (and high activity levels) is more likely to receive system prompts with low potency levels (and high activity levels).

This thesis is one of the exploratory works in the area of integrating emotional intelligence with cognitive intelligent assistive systems. It provides a solution to fitting emotional intelligence in a functional system, as well as points out directions for future improvements. The framework designed in this thesis is portable and extensible, and can be generalized to be used in other applications.