On primal-dual interior-point algorithms for convex optimisation

Abstract

This thesis studies the theory and implementation of interior-point methods for convex optimisation. A number of important problems from mathematics and engineering can be cast naturally as convex optimisation problems, and a great many others have useful convex relaxations. Interior-point methods are among the successful algorithms for solving convex optimisation problems. One class of interior-point methods, called primal-dual interior-point methods, have been particularly successful at solving optimisation problems defined over symmetric cones, which are self-dual cones whose linear automorphisms act transitively on their interiors.

The main theoretical contribution is the design and analysis of a primal-dual interior-point method for general convex optimisation that is ``primal-dual symmetric''---if arithmetic is done exactly, the sequence of iterates generated is invariant under interchange of primal and dual problems.

The proof of this algorithm's correctness and asymptotic worst-case iteration complexity hinges on a new analysis of a certain rank-four update formula akin to the Hessian estimate updates performed by quasi-Newton methods.

This thesis also gives simple, explicit constructions of primal-dual scalings---linear maps from the dual space to the primal space that map the dual iterate to the primal iterate and the barrier gradient at the primal iterate to the barrier gradient at the dual iterate---by averaging the primal or dual Hessian over a line segment. These scalings are called the primal and dual integral scalings in this thesis. The primal and dual integral scalings can inherit certain kinds of good behaviour from the barrier whose Hessian is averaged. For instance, if the primal barrier Hessian at every point maps the primal cone into the dual cone, then the primal integral scaling also maps the primal cone into the dual cone.

This gives the idea that primal-dual interior-point methods based on the primal integral scaling might be effective on problems in which the primal barrier is somehow well-behaved, but the dual barrier is not.

One such class of problems is \emph{hyperbolicity cone optimisation}---minimising a linear function over the intersection of an affine space with a so-called hyperbolicity cone. Hyperbolicity cones arise from hyperbolic polynomials, which can be seen as a generalisation of the determinant polynomial on symmetric matrices. Hyperbolic polynomials themselves have been of considerable recent interest in mathematics, their theory playing a role in the resolution of the Kadison-Singer problem.

In the setting of hyperbolicity cone optimisation, the primal barrier's Hessian satisfies ``the long-step Hessian estimation property'' with which the primal barrier may be easily estimated everywhere in the interior of the cone in terms of the primal barrier anywhere else in the interior of the cone, and the primal barrier Hessian at every point in the interior of the cone maps the primal cone into the dual cone. In general, however, the dual barrier satisfies neither of these properties.

This thesis also describes an adaptation of the Mizuno-Todd-Ye method for linear optimisation to hyperbolicity cone optimisation and its implementation. This implementation is meant as a window into the algorithm's convergence behaviour on hyperbolicity cone optimisation problems rather than as a useful software package for solving hyperbolicity cone optimisation problems that might arise in practice.

In the final chapter of this thesis is a description of an implementation of an interior-point method for linear optimisation. This implementation can efficiently use primal-dual scalings based on rank-four updates to an old scaling matrix and was meant as a platform to evaluate that technique. This implementation is modestly slower than CPLEX's barrier optimiser on problems with no free or double-bounded variables. A computational comparison between the ``standard'' interior-point algorithm for solving LPs with one instance of the rank-four update technique is given.

The rank-four update formula mentioned above has an interesting specialisation to linear optimisation that is also described in this thesis. A serious effort was made to improve the running time of an interior-point method for linear optimisation using this technique, but it ultimately failed.

This thesis revisits work from the early 1990s by Rothberg and Gupta on cache-efficient data structures for Cholesky factorisation. This thesis proposes a variant of their data structure, showing that, in this variant, the time needed to perform triangular solves can be reduced substantially from the time needed by either the usual supernodal or simplicial data structures. The linear optimisation problem solver described in this thesis is also used to study the impact of these different data structures on the overall time required to solve a linear optimisation problem.