Nonprehensile Manipulation via Multisensory Learning from Demonstration

Abstract

Dexterous manipulation problem concerns control of a robot hand to manipulate an object in a desired manner. While classical dexterous manipulation strategies are based on stable grasping (or force closure), many human-like manipulation tasks do not maintain grasp stability, and often utilize the intrinsic dynamics of the object rather than the closed form of kinematic relation between the object and the robotic fingers. Such manipulation strategies are referred as nonprehensile or dynamic dexterous manipulation in the literature. Nonprehensile manipulation typically involves fast and agile movements such as throwing and flipping. Due to the complexity of such motions (which may involve impulsive dynamics) and uncertainties associated with them, it has been challenging to realize nonprehensile manipulation tasks in a reliable way. In this paper, we propose a new control strategy to realize practical nonprehensile manipulation tasks using a robot hand. The main idea of our control strategy are two-folds. Firstly, we make explicit use of multiple modalities of sensory data for the design of control law. Specifically, force data is employed for feedforward control while the position data is used for feedback (i.e. reactive) control. Secondly, control signals (both feedback and feedforward) are obtained by the multisensory learning from demonstration (LfD) experiments which are designed and performed for specific nonprehensile manipulation tasks in concern. We utilize various LfD frameworks such as Gaussian mixture model and Gaussian mixture regression (GMM/GMR) and hidden Markov model and GMR (HMM/GMR) to reproduce generalized motion profiles from the human expert's demonstrations. The proposed control strategy has been verified by experimental results on dynamic spinning task using a sensory-rich two-finger robotic hand. The control performance (i.e. the speed and accuracy of the spinning task) has also been compared with that of the classical dexterous manipulation based on finger gating.