Show simple item record

dc.contributor.authorDockhorn, Tim 20:01:16 (GMT) 20:01:16 (GMT)
dc.description.abstractNeural ordinary differential equations (NODEs) (Chen et al., 2018) are ordinary differential equations (ODEs) with their dynamics modeled by neural networks. Continuous normalizing flows (CNFs) (Chen et al., 2018; Grathwohl et al., 2018), a class of reversible generative models which builds on NODEs and uses an instantaneous counterpart of the change of variables formula (CVF), have recently proven to achieve state-of-the-art results on density estimation and variational inference tasks. In this thesis, we review key concepts that are important to understand NODEs and CNFs, ranging from numerical ODE solvers to generative models. We derive an explicit formulation of the adjoint sensitivity method for both NODEs and CNFs using a constrained optimization framework. Furthermore, we review several classes of NODEs and prove that a particular class of hypernetwork NODEs is a universal function approximator in the discretized state. Our numerical results suggest that the ODEs arising in CNFs do not need to be solved to high precision for training and we show that training of CNFs can be made more efficient by using a tolerance scheduler that exponentially reduces the ODE solver tolerances. Moreover, we quantify the discrepancy of the CVF and the discretized instantaneous CVF for two ODE solvers. Our hope in writing this thesis is to give a comprehensive and self-contained introduction to generative modeling (with neural ordinary differential equations) and to stimulate both theoretical as well as computational future work on NODEs and CNFs.en
dc.publisherUniversity of Waterlooen
dc.subjectgenerative modelingen
dc.subjectneural ordinary differential equationsen
dc.subjectdensity estimationen
dc.subjectdeep learningen
dc.titleGenerative Modeling with Neural Ordinary Differential Equationsen
dc.typeMaster Thesisen
dc.pendingfalse Mathematicsen Mathematicsen of Waterlooen
uws-etd.degreeMaster of Mathematicsen
uws.contributor.advisorRhebergen, Sander
uws.contributor.advisorDe Sterck, Hans
uws.contributor.affiliation1Faculty of Mathematicsen

Files in this item


This item appears in the following Collection(s)

Show simple item record


University of Waterloo Library
200 University Avenue West
Waterloo, Ontario, Canada N2L 3G1
519 888 4883

All items in UWSpace are protected by copyright, with all rights reserved.

DSpace software

Service outages