Active Perception for Robotic Grasping under Occlusion: Semantic-Aware and VLM-Distilled Approaches

dc.contributor.authorKweon, Tae Hyeon
dc.date.accessioned2026-08-26T16:06:52Z
dc.date.issued2026-08-26
dc.date.submitted2026-08-25
dc.description.abstractRobotic grasping in cluttered environments is challenging when target objects are heavily occluded, as a single viewpoint is often insufficient for reliable grasp planning. This thesis addresses this problem through two approaches to next-best-view (NBV) planning for active grasping under occlusion. The first approach is a semantic-aware NBV framework that maintains a unified semantic–geometric volumetric representation, integrating geometric and semantic information gain to guide viewpoint selection toward regions most likely to reveal graspable target surfaces. Semantic detections are incrementally accumulated across views through a confidence-weighted update strategy, which adds robustness to false positives and enables autonomous target localization without any prior knowledge of object location. The second approach is a learned NBV policy distilled from Vision-Language Model (VLM) viewpoint preferences. By ranking candidate views offline based on target visibility, occlusion, and grasp feasibility, the VLM provides training labels for a lightweight view scoring network that efficiently selects NBVs at inference without further VLM queries or ray casting. Experiments in simulation and real-world settings validate both approaches, demonstrating that semantic volumetric reasoning and learned VLM preferences are effective and complementary strategies for active perception in robotic grasping.
dc.identifier.urihttps://hdl.handle.net/10012/24058
dc.language.isoen
dc.pendingfalse
dc.publisherUniversity of Waterlooen
dc.titleActive Perception for Robotic Grasping under Occlusion: Semantic-Aware and VLM-Distilled Approaches
dc.typeMaster Thesis
uws-etd.degreeMaster of Applied Science
uws-etd.degree.departmentMechanical and Mechatronics Engineering
uws-etd.degree.disciplineMechanical Engineering
uws-etd.degree.grantorUniversity of Waterlooen
uws-etd.embargo.terms0
uws.contributor.advisorSoo, Jeon
uws.contributor.affiliation1Faculty of Engineering
uws.peerReviewStatusUnrevieweden
uws.published.cityWaterlooen
uws.published.countryCanadaen
uws.published.provinceOntarioen
uws.scholarLevelGraduateen
uws.typeOfResourceTexten

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Kweon_TaeHyeon.pdf
Size:
60.72 MB
Format:
Adobe Portable Document Format

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
6.4 KB
Format:
Item-specific license agreed upon to submission
Description: