Socially and Spatially Aware Motion Prediction of Dynamic Objects for Autonomous Driving

Abstract

The primary goal of this thesis project is to develop a robust object motion prediction framework enabling safe decision making for autonomous vehicles in various driving scenarios. Given the comparatively higher importance and complexity of urban driving settings such as stop-sign controlled intersections or non-signalized/sign controlled roads are of primary interest; the approach, however, is not limited to these settings and is applicable to other driving settings. Specifically, motion prediction for all moving objects surrounding the autonomous vehicle such as pedestrians, cyclists, cars, and trucks is considered.

In this thesis, estimation of the position and velocity of the objects surrounding the autonomous vehicle is performed using observed positions of the object in interest during a finite time window in the past, subsequent to which a socially and spatially informed model predicts the positions of these objects for a finite time window in the future through the use of the obtained position and velocity estimate as well as an artificial potential field (PF) modelling social interactions between surrounding objects and the scene.

The necessary inputs for prediction are the class, position, and velocity of object of interest which can be obtained through 3D object detection approaches. However, often times, intermittent noise and/or loss in detections is observed pointing to the need for a robust estimation scheme. Traditional one-step lookback-based filtering and estimation approaches do not perform well due to a lack of sufficient prior information and simplistic model assumptions. On the other hand, most data-driven approaches do not offer any explicit embeddings of physical motion models or constraints leading to lack of generalizability in unseen scenarios.

To this end, a constrained moving horizon state estimation (MHE) approach to estimate an object's states with respect to a global stationary frame including position, velocity, and acceleration that are robust to intermittently noisy or absent sensor measurements is proposed. Utilizing a computationally light-weight fusion of a Convolutional Neural Network (CNN)-based 2D object detection algorithm and projected LIDAR depth measurements, the approach produces the required measurements relative to the vehicle frame and combines them with the rotation and translation information obtained via a global positioning and inertial measurement system. The performance of the proposed approach is experimentally verified on an in-house dataset featuring urban crossings, with and without autonomous vehicle motion.

Taking the position and velocity estimates as inputs, three key observations in microscopic agent-agent behaviour are incorporated for motion prediction namely – inclination to maintain direction of heading for pedestrians and follow lane centers for vehicles when free of surrounding agents, tendency to maintain heading and speed unless a collision is anticipated, and most importantly social interaction demonstrating collision avoidance. Traditionally, a fixed model or a model chosen from a fixed set of models is used for modelling future behaviour. These models are applicable to a variety of scenarios, however, they have an inherent bias and may lead to inaccurate predictions. On the other hand, purely data driven approaches suffer from a lack of holistic set of rules governing predictions and hence do not generalize well to a variety of scenarios.

To address these issues, a novel potential field-based model predictive control (MPC) algorithm, MPC-PF, is proposed incorporating social interaction in a single cost function. Simulation results on a variety of scenarios including pedestrians and vehicles approaching directly head-on or otherwise show accurate predictions for a long future horizon. Furthermore, detailed qualitative and quantitative evaluation on a large public motion prediction dataset demonstrates state-of-the-art performance achieved by the proposed approach. Lastly, the potential field-based notion is integrated in a hybrid data driven Deep Deterministic Policy Gradient (DDPG) reinforcement learning (RL) agent, termed RL-PF, with a reward function governed by the potential field and is a valuable direction for further research and experimental validation.