Some Results on the Convergence of Anderson Acceleration

Abstract

Anderson acceleration (AA), also known as Anderson mixing, is an extrapolation technique used to accelerate the convergence of fixed-point iterations $\bm{x}_{k+1}=\bm{q}(\bm{x}_k)$, $k=0,1,\dots$, with $\bm{q}:\R^n\to\R^n$, $\bm{x}_k\in\R^n$. AA was first introduced by D.G. Anderson in the context of solving integral equations but has since been adapted to fixed-point iteration problems in general. Despite relatively little being known about its convergence properties, AA has seen considerable usage in several areas such as electronic structure computations and machine learning. This thesis presents a broad overview of the current convergence literature for AA and introduces a variety of new results concerning properties of AA, such as its asymptotic convergence rate, the possibility of stagnation, and an analysis of its coefficients. Additionally, some variations of the AA iteration are proposed with an accompanying analysis and comparison to the classical AA algorithm.