A quadratic programming approach to find faces in robust nonnegative matrix factorization

Abstract

Nonnegative matrix factorization (NMF) is a popular dimensionality reduction technique because it is easily interpretable and can discern useful features. For a given matrix M (dimension n x m) whose entries are nonnegative and an integer r smaller than both n and m, NMF is the problem of finding nonnegative matrices A (dimension n x r) and W (dimension r x m) such that M = AW. The matrix M could be noisy, in which case one seeks a robust algorithm that solves M ≈ AW. The nonnegativity constraint in NMF has wide applications in data science problems like document clustering, facial feature extraction, hyperspectral unmixing etc.

Geometrically, the rows of M can be viewed as a set of points in m-dimensional space. If we think of the rows of W as the vertices of an (unknown) W-simplex, then the data points lie in this W-simplex. Therefore, NMF asks us to deduce the vertices of the simplex given the data points.

NMF is a computationally hard problem though certain assumptions like separability lead to polynomial time algorithms. This assumes that all the vertices of the unknown simplex are already present as data points. In practice, this is not true in many settings. Ge and Zou (2015) assumed subset separability which uses higher dimensional structures and gave a polynomial time algorithm to find the NMF robustly. In this thesis, we effectively replace one of their key algorithms that finds faces. We show a quadratic programming based approach which is efficient and can be employed in practice. Under bounded noise, our algorithm finds the faces of the simplex which contain enough data points, thus helping in finding the NMF.