An Investigation of Overt Visual Attention and Gaze Behaviour in Social Human-Robot Interaction and Human-Computer Interaction Contexts

Abstract

In human-human and human-robot interaction gaze has a consequential role as a type of non-verbal communication behaviour, affecting the social interaction depending on gaze behaviour's characteristics. As such, gaze behaviour has been a topic of major research throughout the past number of years since a better understanding of gaze behaviour could lead to design of robot behaviour for social interactions.

In the context of the human-human interaction (HHI) and human-robot interaction (HRI) studies, gaze behaviour has been seldom investigated while taking into consideration all social interaction elements including interaction partners' personalities and social roles in addition to the social context. There are a number of studies which investigate conversational roles and personality matching in relation to gaze behaviour in the context of HRI in separate studies. However, works which investigate gaze behaviour in tandem with these social interaction elements are needed since such a study will contextualize gaze behaviour in relation to variations in these social elements (e.g. gaze behaviour characteristics based on introverted and extroverted personalities) while taking into consideration the compounded effects of these social elements in combination.

What this thesis accomplishes is incorporation of all these social elements in tandem with gaze, all under the umbrella of one body of research. Utilization of this integrative approach was inspired by recent HRI literature, encouraging the investigation of verbal and non-verbal social interaction elements together with social interaction elements. This thesis investigates gaze behaviour in the context of HRI while taking into account social role and designed personalities in robotic platforms. As the social context, this thesis explores dyadic human-robot interactions involving objects of discourse from a gaze-centric point of view while considering the robot's gaze-centric perspective and the participant's gaze-centric perspective. Four major studies are conducted in the context of this thesis to fulfill this exploration.

Tools for recording overt visual behaviour are vital in conducting human-computer interaction (HCI) research. However, specific tools enabling the recording of these metrics in online settings, facilitating video viewing were not available, therefore Study 1 created the FocalVid platform. This platform collects cursor location attentional data for the participants in online settings such as Amazon Mechanical Turk. The cursor metrics gathered through this platform were then compared to eye tracking data and our rendition of another relevant platform (BubbleView). It was determined that human gaze and cursor movements are distinct but have similarities in relation to velocities and dwell timing. This platform allowed for large-scale data collection for HCI and HRI studies, which is not possible in the context of in-person studies.

Personality and social role are major elements of social interactions; however, perception of designed introverted/extroverted personalities for the humanoid iCub robot were not previously examined and additionally these two elements have not been explored simultaneously in the previous literature involving the iCub robot. In the second study, I explore the participants' perception of a robot in interactions between a robot and a human actor utilizing recorded online scenarios. In this study, the robot takes on different social roles while embodying different personalities. The robot is either a teacher, a student or a collaborator while either introverted or extroverted. To conduct this study, the Amazon Mechanical Turk platform and HRI video recordings were used. I discovered the presence of perceiver effects in participants’ assessment of the robot’s Ten-Item Personality Inventory (TIPI) dimensions perception vs. self TIPI dimensions, where participants' self-assessment of their personality correlated to their assessment of robot’s personality. TIPI questionnaire is a measure used to assess personality dimensions. It was also determined that the designed robot personality was perceived accordingly by the participants. These findings indicated that even though participants’ self-assessment of their personality dimensions affects their perception of the robot, they could still perceive the robot’s designed personality as intended.

Observation and analysis of people’s overt visual attention dynamics in HRI could allow for better understanding of these interactions however, such overt attention while considering social interaction elements have not been previously explored in detail. The third study investigated participants' overt visual attention in the context of dyadic social settings using the FocalVid platform. In this study, I was also interested in the efficacy of the use of the FocalVid platform to collect attention metrics relating to such social settings. This study, taking advantage of the HRI scenarios designed in Study 2 and using the FocalVid platform, recorded the cursor attentional data for participants while the robot was enabled with different social roles and personalities. It was determined that the robot’s social role and personality significantly affected the participants’ overt visual attention. It was also determined that the presence of the FocalVid platform did not adversely affect the perception of the robot.

Gaze studies in Human Robot Interaction should investigate both the human partner’s gaze behaviour’s effect on the social interaction, in addition to the robot’s gaze behaviour’s effect on the social interaction. A limited number of studies have explored the effects of gaze-architecture-enabled robots' behaviour on social interaction. In the fourth study, after the design of gaze-based interaction architectures based on Social Gaze Space taxonomy in dyadic interactions involving objects of discourse, the effects of using these gaze interaction architectures for robot gaze control were evaluated utilizing eye tracking data and Human Robot Interaction questionnaires. Through this study, it was determined that the SGS-IA architecture led to higher visual engagement by the participants towards the robot’s face and eye region compared to the TutorSpotter architecture, which was used for comparison purposes. One of the main contributions of this thesis is the design and evaluation of these gaze-based interaction architectures for anthropomorphic humanoid robots involved in human-robot interactions.

All four of these studies were geared towards gathering a better understanding of gaze behaviour in HRI and HCI. Studies 1 and 2 had a preparatory role to this end. Study 1 allowed us to design the FocalVid platform and to investigate the attention metrics gathered through this platform against gaze metrics in this Human Computer Interaction platform. Study 2 allowed us to design the Human Robot Interaction scenarios needed for Studies 3 and 4. Study 3 investigated gaze behaviour of the human interaction partner involved in Human Robot Interaction using the FocalVid platform, and in Study 4 we designed and evaluated a gaze interaction architecture for the iCub robot through an in-person Human Robot Interaction study. These studies allowed for better understanding of the role of gaze behaviour in social HRI settings. These studies also enabled us to design gaze-specific interaction architectures for the iCub robot.