FJMP: Factorized Joint Multi-Agent Motion Prediction

Abstract

Multi-agent motion prediction is an important problem in an autonomous driving pipeline, and it involves forecasting the future behaviour of multiple agents in complex driving environments. Autonomous vehicles (AVs) should produce accurate predictions of future agent behaviour in order to make safe and informed plans in safety-critical scenarios. Importantly, AVs should generate scene-consistent future predictions that predict the joint future behaviour of multiple agents, as this enables reasoning about potential future multi-agent interactions, which are critical for downstream planning.

In this thesis, we address the problem of generating a set of scene-level, or joint, future trajectory predictions in multi-agent driving scenarios. To this end, we propose FJMP, a Factorized Joint Motion Prediction framework for multi-agent interactive driving scenarios. FJMP models the future scene interaction dynamics as a sparse directed interaction graph, where nodes represent agents and edges denote explicit interactions between agents. We then prune the graph into a directed acyclic graph (DAG) and decompose the joint prediction task into a sequence of marginal and conditional predictions according to the partial ordering of the DAG, where joint future trajectories are decoded using a directed acyclic graph neural network (DAGNN). We conduct experiments on two autonomous driving datasets and demonstrate that FJMP produces more accurate and scene-consistent joint trajectory predictions than existing approaches. Importantly, we show that FJMP produces superior joint forecasts compared to non-factorized approaches on the most interactive and kinematically interesting agents, which highlights the benefit of our proposed factorization.