Road Condition Sensing Using Deep Learning and Wireless Signals

Abstract

Similar to human car drivers, future driverless cars need to sense the condition of road surfaces so that they can adjust their speed and distance from other cars. This awareness necessitates the need for a sensing mechanism that enables cars to sense the surface type (gravel versus asphalt) and condition (dry versus wet) of a road. Unfortunately, existing road sensing approaches have major limitations. Vision-based approaches do not work in bad weather conditions and darkness. Mechanical-based approaches are either expensive or do not have enough resolution and robustness.

In this thesis, we introduce VIVA, which uses mmWave to enable robust and practical road sensing. Our key insight is that mmWave radar devices enable high resolution ranging, which can be used to scan the roughness of a road surface. Moreover, mmWave radar devices use high-frequency signals, which are significantly reflected by water, and hence can be used to sense the moisture level of a road. However, due to the high sensitivity of mmWave radar devices, other factors such as car vibration also impact their signals, resulting in noisy measurements. To extract information about road surfaces from noisy signals, we have developed a cross-modal supervised model that uses mmWave measurements to sense road surfaces. Our prototype of VIVA costs less than $300 and achieves more than 98% accuracy in classifying road types (gravel versus asphalt) and 99% accuracy in classifying road conditions (wet versus dry), even in bad weather and darkness.