Deep Image Prior for Disentangling Mixed Pixels

Abstract

A mixed pixel in remotely sensed images measures the reflectance and emission from multiple target types (e.g., tree, grass, and building) from a certain area. Mixed pixels exist commonly in spaceborne hyper-/multi-spectral images due to sensor limitations, causing the signature ambiguity problem and impeding high-resolution remote sensing mapping. Disentangling mixed pixels into the underlying constituent components is a challenging ill-posed inverse problem, which requires efficient modeling of spatial prior information and other application-dependent prior knowledge concerning the mixed pixel generation process.

The recent deep image prior (DIP) approach and other application-dependent prior information are integrated into a Bayesian framework in the research, which allows comprehensive usage of different prior knowledge. The research improves mixed pixel disentangling using the Bayesian DIP in three key applications: spectral unmixing (SU), subpixel mapping (SPM), and soil moisture product downscaling (SMD). The main contributions are summarized as follows.

First, to improve the decomposition of mixed pixels into pure material spectra (i.e., endmembers) and their constituting fractions (i.e., abundances) in SU, a designed deep fully convolutional neural network (DCNN) and a new spectral mixture model (SMM) with heterogeneous noise are integrated into a Bayesian framework that is efficiently solved by a new iterative optimization algorithm.

Second, to improve the decomposition of mixed pixels into class labels of subpixels in SPM, a dedicated DCNN architecture and a new discrete SMM are integrated into the Bayesian framework to allow the use of both spatial prior and the forward model.

Third, to improve the decomposition of mixed pixels into soil moisture concentrations of subpixels in SMD, a new DIP architecture and a forward degradation model are integrated into the Bayesian framework that is solved by the stochastic gradient descent approach.

These new Bayesian approaches improve the state-of-the-art in their respective applications (i.e., SU, SPM, and SMD), which can be potentially utilized for solving other ill-posed inverse problems where simultaneously modeling of the spatial prior and other prior knowledge is needed.