System Development and Evaluation of a Social Robot as a Public Speaking Rehearsal Coach

Abstract

Skills are developed through practice and rehearsal. Mastery and leadership often arise from dedicated effort. Among the skills that demand ongoing attention and practice, public speaking holds a crucial place. It is a skill by which individuals are assessed in various facets of life, be it in academia or as they progress in their professional careers. Nonetheless, when it comes to advancing in this skill, it is notably more time and energy-efficient to receive feedback from an individual capable of providing a reliable analysis of our performance. Ideally, engaging a human public speaking coach would be the optimal choice. However, accessing a human coach may not always be feasible due to a variety of reasons.

In this master's thesis, I delved into the journey of exploring the potential of using a social robot as a coach for public speaking rehearsals, with the aim of enhancing individuals' presentation skills.

My MASc project begins by conducting a comparison between two interactive technologies: a voice assistant agent and a social humanoid robot, both considered potential artificial systems for enhancing presentation skills. This comparison explores the embodiment of these technologies and underscores the significance of incorporating social non-verbal behaviours in an agent. To carry out this comparison, we organized an online study through Amazon Mechanical Turk, involving a diverse sample of participants across different age groups. Participants were asked to share their perspectives on various aspects of the agents depicted as coaches for public speaking rehearsals. They were presented with a scenario involving a video showing a student's brief presentation rehearsal in front of both agents, followed by feedback provided by these agents. Our analysis encompassed both quantitative and qualitative methods. While participants generally displayed a preference for using socially interactive agents as coaches for public speaking rehearsals, the social robot outperformed the voice assistant agent on several metrics. Notably, it received higher rankings in terms of human nature attributes, perceived likeability, and perceived warmth.

Building upon the findings of the first study, and considering the more favourable evaluation of the social robot, we selected the social robot as the coaching agent for public speaking rehearsals involving university students in a single in-person presentation rehearsal session. The system development involved analyzing participants' speech quality and (simulated) audience orientation and providing verbal feedback based on the calculated performance analysis.

In the second user study, we sought to gauge participants' interactions with the robot, specifically their level of acceptance and their willingness to use the system in the future. We also administered questionnaires to assess participants' perceptions of the robot's behaviour during the session and the accuracy of its feedback. The results of this study highlighted strong agreement among participants on the use of the social robot as a public speaking rehearsal coach in the university, and their expressed intention to utilize it in the future. Furthermore, participants' perceptions of the robot's competence and usefulness were closely linked to the accuracy of its feedback and the perceived quality of its listening behaviour. Finally, we asked a human public speaking coach to evaluate the robot coach, and presented the results. Future work can improve on this work and provide an accessible public speaking coach that university students can benefit from.