Experimental Evaluation of Affordance Detection Applied to 6-DoF Pose Estimation for Intelligent Robotic Grasping of Household Objects

Abstract

Recent computer vision research has demonstrated that deep convolutional neural networks can be trained on real images to add context to object parts for simultaneous object detection and affordance segmentation. However, generating such a dataset with expensive hand annotations for pixel-wise labels presents a challenge for training deep convolutional neural networks. In this thesis, a method to automate dataset generation of real and synthetic images with ground truth annotations for affordance detection and object part 6-DoF pose is presented. A variant of Mask R-CNN is implemented and trained to perform affordance detection and integrated within DenseFusion, a two-stage framework for 6-DoF pose estimation. The primary contribution of this work is to experimentally evaluate 6-DoF pose estimation with object segmentation and affordance detection, which was done on the YCB-Video benchmark dataset and the ARL AffPose dataset. It was demonstrated that 6-DoF pose estimation with object segmentation slightly outperforms pose estimation with affordance detection, as the latter operates on a subset of RGB-D data. However, the advantage of pose estimation with affordance detection is realized when the trained model is deployed on a robotic platform to grasp complex objects, such that an 11% improvement in terms of grasp success rate was experimentally demonstrated for a power drill.