Enhanced detection of point correspondences in single-shot structured light

Abstract

The crucial role of point correspondences in the process of stereo vision and camera projector calibration is to determine the relationship between the camera view(s) and the projector view(s). Consequently, acquiring accurate and robust point correspondences can result in a very accurate 3D point cloud of a scene. Designing a method that can detect pixel correspondences quickly and accurately and be robust to factors such as object motions and color is an important subject of study. The information that lies in the point correspondences determines the geometry of the scene in which depth plays a very important role, if not the most important. However, point correspondences will include some outliers. Outlier removal is another important aspect of obtaining correspondences that can have substantial impact on the reconstructed point cloud of an object. During the Single-Shot Structured Light (SSSL) calibration process, a pattern consisting of tags with differently shaped symbols inside and separated by grids are projected onto the object. The intersections of these grid lines are considered to be potential pixel correspondences between a camera image and the projector pattern.

The purpose of this thesis is to study the robustness and accuracy of pixel correspondences and to enhance their quality. In this thesis we propose a detection method that uses the model of the pattern, specifically the grid lines, which are the largest and brightest feature of the pattern. The input image is partitioned into smaller patches and then the optimization process is executed on each patch. Eventually, the grid lines will be detected and fitted to the grid, and the intersections of those lines are taken as potential corresponding pixels between the views. In order to remove incorrect pixel correspondences, or in other words, outliers, Connected Component Analysis is used to find the closest detected point to the top left corner of each tag. The points remaining after this step are the correct pixel correspondences.

Experimental results show the improvement of using a locally adaptive thresholding method against the baseline in detecting tags. The proposed thresholding method showed a maintained accuracy compared to the baseline method while automatically tune all the parameters whereas in the baseline method some parameters need fine tuning. Introduced model-based grid intersection detection yields an approximately 50 times improvement in speed. Inaccuracy in point correspondences are compared with state-of-the-art method based on the generated final reconstructed point clouds using both methods against the CAD model as ground truth. Results show an average of 3 pixels higher error in distance, between the reconstructed point clouds and the CAD model, in the proposed method compared to the baseline.