On Initializations for NMF

Abstract

In Ward and Kolda (2023) and Xu et al. (2024), the authors proposed a sketching based initialization for unconstrained two block matrix factorization (MF) with provable guarantee. The purpose of this thesis is to examine the limits of extending these two proof frameworks in the context of Non-negative Matrix Factorization (NMF), as well as its numerical implications.

The first chapter deals with fundamentals of NMF: its formulation, the identifiability issue and guarantees, a survey of iterative methods to solve it, and finally some complexity results. In the second chapter, I will quickly go over spectral initialization techniques for matrix optimization problems, and introduce sketching, a family of randomization techniques for numerical linear algebra, with a particular focus on its uses in iterative methods. The core focus of this chapter lies at the intersection of these two topics: a sketching-based initialization for the unconstrained two-block MF problem, using either non-alternating or alternating gradient descent (GD). We will go over the proof frameworks. In the third chapter, I will present my attempts to generalize the proof frameworks to NMF and the conclusions based on the partial progress.

The fourth chapter deals with numerics. We will compare the performance of different base methods in different data regimes using all of the initializations mentioned in this thesis.