A Learning Social Referencing Disambiguation Framework for Domestic Service Robots

Abstract

The successful integration of domestic service robots into home environments can bring significant services and convenience to the general population and possibly mitigate important societal issues, such as care provision for older adults. However, home environments are complex, dynamic, and object-rich. It is, therefore, very probable that service robots will encounter ambiguity while interacting with household items. Moreover, service robots need to have the capability to obtain knowledge and continuously learn from the environment to ensure high adaptability and persistent functional adequacy.

This thesis presents a learning object disambiguation framework for domestic service robots that is inspired by the cognitive mechanism of social referencing and the human visual mental imagery perceptual experience. The framework allows the service robot to resolve various ambiguities in the object selection task and learn objects through bidirectional human robot interactions. The framework's technical details are explained in depth. We first describe the base framework, which consists of five functional components: the user command interface, fuzzy ambiguity determination, fuzzy human attention assessment, social referencing disambiguation, and short-term long-term memory object learning. We then explain our extended framework with the expansion of robot mental imagination and user objection absence detection to further enhance the robot's ability to handle novel objects and improve interaction robustness.

To showcase and validate our robot framework, we developed a system validation study with human participants. The framework is implemented on our mobile robot manipulator, Fetch, for our testing scenarios. Our experiment is designed to measure the success of the framework objectively and to understand the human perception of the robot with the framework. The study design, implementation, results, and discussions are illustrated.

Finally, we discuss the current limitations of the framework and summarize many valuable lessons learned in this research project. We conclude the thesis with an exploration of many promising and exciting future directions for the proposed framework. We believe this framework forms an important conceptual foundation for future service robots to become "lifelong learners" with human guidance.