Vision-Based Adaptive Impedance Control for Soft Material Manipulation

Abstract

The manipulation of soft materials presents a longstanding challenge in robotics due to their nonlinear, variable, and often unpredictable physical properties. Traditional control strategies either require complex physical modeling or extensive data collection, limiting their applicability in dynamic, real-world settings. This thesis proposes a model-free, vision-based adaptive impedance control framework that enables collaborative robots to regulate contact forces during manipulation using only real-time visual feedback. By dynamically adjusting stiffness based on vision measurements of object deformation, the framework allows for compliant and adaptive interaction with a wide range of soft materials without prior knowledge of their physical characteristics.

The effectiveness of the framework is demonstrated through two representative studies. The first, VAIRO (Vision-based Adaptive Impedance-control Robotic framework), addresses the challenge of rolling croissant dough in a craft bakery setting. This study involved using a standalone stereo vision camera to monitor the rolling process of puff pastry into croissants with a robotic manipulator. Employing RGB-D data and real-time image processing, VAIRO detects and segments croissant layer thickness and inter-layer gaps, which are used to adapt the stiffness of a Cartesian impedance controller. The result of the study showed VAIRO creating croissants closer to artisinal quality compared to a constant stiffness approach, where VAIRO produced croissants with up to 6 times less variance in heights and mean error of heights less than 0.51mm.

The second study, VAISI (Vision-based Adaptive Impedance-control for Surgical Incisions), applies similar methodology to perform depth-controlled incisions in soft biological tissues using point-clouds captured by a stereo vision-enabled scalpel end-tool on the robotic manipulator. Using geometric measurements on the point cloud, the scalpel's depth and skin tissue deformation is tracked in real-time to adapt both the incision depth and force through visual servoing and stiffness adaptation of the Cartesian impedance controller. VAISI demonstrated the capability to create incisions with sub-millimeter accuracy and a maximum variance in incision depths of 1.23mm.

The vision-based adaptive impedance control framework presented in this thesis offers a foundation for future research in flexible automation, medical robotics, and other domains requiring interaction with soft materials. This contributes towards the goals of Industry 4.0—namely, adaptable and robust robotic systems.