A data-centric view of LQR algorithms in continuous time

Abstract

Many control theorists are interested in how measurements of input and state trajectories can determine the properties of a control system, in lieu of the differential or difference equation models that usually play this role. In situations of practical interest, such models may be wholly or partially unknown, while input and state data are readily available. In the discrete- and continuous-time linear quadratic regulator (LQR) settings, this area of research has produced a proliferation of algorithms that use input and state data to solve the LQR problem, the system identification problem, or both. In different algorithms, these data and the requirements imposed on them take different forms that are not directly comparable, making it difficult to assess the relative efficiencies with which different algorithms makes use of data.

In [30], the authors show that the LQR and system identification problems are essentially equivalent in the discrete-time setting. In this thesis, we extend this result to the continuous-time setting, showing that, assuming input and state data are collected on intervals, every algorithm that solves the LQR problem requires at least as much data as system identification. From this, we show that the map from the data to the optimal gain defined by these algorithms is continuous, establishing a connection between interval data and sampled data algorithms. The possibility of using sampled data in place of interval data leads to a weaker convergence criterion on sampled data approximations, and a natural connection with numerical integration. We do some numerical experiments that show the critical importance of choosing when to make input and state measurements, and emphasize the possibility of doing so without knowledge of the system or its optimal gain.