Augmenting Vehicle Localization with Visual Context

Abstract

Vehicle self-localization, the ability of a vehicle to determine its own location, is vital for many aspects of Intelligent Transportation Systems (ITS) and telematics where it is often a building block in a more complex system. Navigation systems are perhaps the most obvious example, requiring knowledge of the vehicle's location on a map to calculate a route to a desired destination. Other pervasive examples are the monitoring of vehicle fleets for tracking shipments or dispatching emergency vehicles, and in public transit systems to inform riders of time-of-arrival thereby assisting trip planning. These system often depend on Global Positioning System (GPS) technology to provide vehicle localization information; however, GPS is challenged in urban environments where satellite visibility and multipath conditions are common. Vehicle localization is made more robust to these issues through augmentation of GPS-based localization with complementary sensors, thereby improving the performance and reliability of systems that depend on localization information.

This thesis investigates the augmentation of vehicle localization systems with visual context. Positioning the vehicle with respect to objects in its surrounding environment in addition to using GPS constraints the possible vehicle locations, to provide improved localization accuracy compared to a system relying solely on GPS. A modular system architecture based on Bayesian filtering is proposed in this thesis that enables existing localization systems to be augmented by visual context while maintaining their existing capabilities.

It is shown in this thesis that localization errors caused by GPS signal multipath can be reduced by positioning the vehicle with respect to visually-detected intersection road markings. This error reduction is achieved when the identities of the detected road marking and the road being driven are known a priori. It is further shown how to generalize the approach to the situation when the identities of these parameters are unknown. In this situation, it is found that the addition of visual context to the vehicle localization system reduces the ambiguity of identifying the road being driven by the vehicle. The fact that knowledge of the road being driven is required by many applications of vehicle localization makes this a significant finding.

A related problem is also explored in this thesis: that of using vehicle position information to augment machine vision. An approach is proposed whereby a machine vision system and a vehicle localization system can share their information with one another for mutual benefit. It is shown that, using this approach, the most uncertain of these systems benefits the most by this sharing of information.

Augmenting vehicle localization with visual context is neither farfetched nor impractical given the technology available in today's vehicles. It is not uncommon for a vehicle today to come equipped with a GPS-based navigation system, and cameras for lane departure detection and parking assistance. The research in this thesis brings the capability for these existing systems to work together.