Bayesian Integral-based Methods for Differential Equation Models

Abstract

Ordinary differential equations (ODEs) are widely considered for modeling the dynamics of complex systems across various scientific areas. When inferring the parameters of ODEs from noisy observations, most existing parameter estimation methods that bypass numerical integration tend to rely on basis functions or Gaussian processes to approximate the ODE solution and its derivatives. Due to the sensitivity of the ODE solution to its derivatives, these methods can be hindered by estimation error, especially when only sparse time-course observations are available. Furthermore, in high-dimensional sparse ODE systems, existing methods for structure identification are predominantly frequentist, and uncertainty quantification for trajectory estimation remains a significant challenge.

In this thesis, we present a Bayesian collocation framework that operates on the integrated form of the ODEs and also avoids the expensive use of numerical solvers. In contrast to frequentist methods, the Bayesian framework provides better quantification of uncertainty. Specifically, we propose three methods to recover ODE systems with both known and unknown functional forms, as well as for model validation in systems with known functions. First, for ODEs with known functional forms, our methodology can handle general nonlinear systems to infer parameters from noisy observations. We then extend the approach to ODEs with unknown functional forms to identify system structure. Under the additive ODE model assumption, we develop a unified framework that combines the likelihood, integrated ODE constraints, and a group-wise sparse penalty, enabling simultaneous system identification and trajectory estimation. Finally, for cases involving several competing functional forms that exhibit similar dynamics, we incorporate model inadequacy as a function into the measurement model, allowing simultaneous model validation and parameter estimation. We demonstrate the favourable performance of our proposed methods, compared to existing ones, in terms of estimation accuracy and model validation. We also illustrate the methods with real data analyses.