Towards Domain Invariant Real-time Point Cloud Perception

Abstract

In recent years, autonomous driving has witnessed substantial advancements, owing in part to the rapid advancement of 3D sensor technologies, particularly Light Detection and Ranging (LiDAR) sensors.Despite these advancements, the challenge of open-world autonomous driving remains a pressing issue that requires resolution. This encompasses the need for dependable and resilient 3D perception across a multitude of environments. Particularly, point cloud is irregular, unordered, and continuous with large data size, which presents unique challenges for its real-time processing. Another major challenge came from the perception system’s inability to adapt to different environments, known as the domain adaptation problem. This is due in part to a lack of diverse and representative datasets, and in part to existing models’ insufficient generalization ability. To tackle this challenge, this dissertation conducts a thorough investigation into the domain adaptation challenges associated with real-time point cloud perception. This dissertation addresses these challenges associated with the deployment and train- ing of point cloud perception systems in a self-contained manner.To ensure sufficient real- time capability during deployment, a novel approach called task-attentive 3D perception is proposed. It incorporates HD-map, vehicle states, and emergency breaking distance to dynamically remove task un-related point cloud for driving safety and computation effi- ciency. Furthermore, this thesis looks at the problem of driving in varied domains from both the data and the model standpoint. First, a novel method for creating the target real- world domains in the simulator using real-world prior is presented. Using this method, a noval large-scale multi-task dataset called Domains-3D is developed, which comprises both real-world and synthetic domains. Finally, using this dataset, a novel Plug-and-Play (PnP) domain adaptation algorithm for 3D point clouds that minimizes domain shifts is presented. This algorithm improves a model’s performance for both cross-scene domain adaptation and intra-domain 3D object detection.