Advancing Semi-Supervised Domain Adaptive Semantic Segmentation Through Effective Source Integration Strategies

Abstract

Semantic segmentation is a highly valuable visual recognition task with applications across fields such as medical imaging, remote sensing, and manufacturing. However, training segmentation models is challenging because it requires large-scale, densely labeled data specific to the target. Semi-supervised learning (SSL) addresses this challenge by leveraging unlabeled data alongside limited labeled data, reducing the reliance on fully labeled datasets. Semi-supervised domain adaptation (SSDA) further mitigates this issue by incorporating labeled data from a source domain alongside minimally labeled target data.

While existing SSDA methods often underperform compared to fully supervised approaches, recent SSL methods that utilize foundation models achieve near fully supervised performance. Given the strength of current SSL methods using foundation models, this thesis investigates effective strategies for integrating source-domain data from a different distribution into existing pipelines to improve segmentation performance. First, we explore a simple source transfer mechanism that merges target and source data into a single unified labeled set for SSL pipelines. Our analysis demonstrates the accuracy benefits of this setup while also highlighting some downsides, particularly in terms of training efficiency.

We also examine the use of ensembling SSL and SSDA models to enhance target-domain performance. This ensemble combines a model trained solely on target data with a source-transferred SSDA model. We find that ensembling can improve performance in certain cases but is less effective in others, and training efficiency remains suboptimal due to the need to train two models.

Given the training inefficiencies of simple source transfer and ensembling, we propose a dual-curriculum source integration strategy to address and improve these limitations. This approach consists of two complementary learning strategies: curriculum retrieval, which progressively samples source examples from easy to hard, and curriculum pasting, which increases the diversity of target-labeled data.

Across our experiments, we compare against and outperform state-of-the-art SSL and SSDA methods on a variety of benchmarks, including synthetic-to-real and real-to-real scenarios. Our findings highlight the benefits of effective source data integration into modern SSL pipelines for boosting segmentation performance, opening a new avenue for label-efficient semantic segmentation.