Learning-based Image Scale Estimation for Quantitative Visual Inspection of Civil Structures

Abstract

The number of assets of civil infrastructure (e.g., bridges or roads) have been increasing to meet the demands of growing populations around the world. However, they degrade over time due to environmental factors and must be maintained and monitored to ensure the safety of its users. The increasing number of infrastructure assets which deteriorate over time is fast outpacing the rate at which they are inspected and rehabilitated. Currently, the main mode of structure condition assessment is visual inspection, where human inspectors manually identify, classify, track, and measure, as needed, deterioration over time to make assessments of a structure’s overall condition. However, the current process is highly time consuming, expensive, and subject to the inspector’s judgement and expertise, which could lead to inconsistent assessments of a given structure when surveyed by several different inspectors over a period of time. As a result, there is a clear need for the current inspection process to be improved in terms of efficiency and consistency.

Developments in computer vision algorithms, vision sensors, sensing platforms, and high-performance computing have shown promise in improving the current inspection processes to enable consistent and rapid structural assessments. Recent work often involves rapid collection and/or analysis of imagery captured from personnel or mobile data collection platforms (e.g., smart phones, unmanned aerial or ground vehicles) to detect and classify visual features (e.g., structural components or deterioration). These works often involve the use of advanced image processing or computer vision algorithms such as convolutional neural networks to detect and/or classify regions of interest. However, a major shortfall of vision-based inspection is the inability to deduce physical measurements (e.g., mm or cm) from the collected images. The lack of an image scale (e.g., pixel/mm) on 2D images does not permit quantitative inspection.

To address this challenge, a learning-based scale estimation technique is proposed. The underlying assumption is that the surface texture of structures, captured in images, contains enough information to estimate scale for each corresponding image (e.g., pixel/mm). This permits the training of a regression model to establish the relationship between surface textures in images and their scales. A convolutional neural network was trained to extract scale-related features from textures captured in images. The trained model is used to estimate scales for all images captured from surfaces of a structure with similar textures in subsequent inspections. The capability of the proposed technique was demonstrated using data collected from surface textures of three different structures. An average scale estimation error, from images of each structure, is less than 15%, which is acceptable in typical visual inspection settings. The source code and data are available from a data repository (GitHub).