Optimal trajectory calculation using neural networks

Abstract

Optimal control methods for linear systems have reached a substantial level of maturity, both in terms of conceptual understanding and scalable computational implementation. For non-linear systems, an open-loop feedback control may be calculated using Pontryagin's Maximum Principle. Alternatively, the Hamilton-Jacobi-Bellman (HJB) equation may be used to calculate the optimal control in a state-feedback form. However, it is an established fact that this equation becomes progressively harder to solve as the number of state variables increases. In this thesis, we discuss a Neural Network (NN)-based method [1] to approximate the solution to the HJB equation arising from high-dimensional ODE systems. We leverage the equivalency between the HJB equation and Pontryagin's Principle to generate the training and test datasets and define a physics-based loss function. The NN is then trained using a supervised optimization approach. We also examine an existing toolkit [2] to approximate the optimal control based on a power series expansion of the system around an equilibrium point in an infinite time horizon setting. We examine the possibility of incorporating this toolkit in the NN training procedure at different stages. The proposed methods are applied to three problems: optimal control of a 6 degree-of-freedom rigid body and the stabilization of ODE systems arising from the discretization of a Burgers'-like non-linear PDE and the damped wave equation.

References: [1] Tenavi Nakamura-Zimmerer, Qi Gong, and Wei Kang. Adaptive deep learning for high-dimensional Hamilton-Jacobi-Bellman equations. SIAM Journal of Scientific Computing, 43(2):A1221–A1247, 2021. [2] Arthur J. Krener. Nonlinear systems toolbox v.1.0, 1997. MATLAB based toolbox available by request from ajkrener@ucdavis.edu