Projector-Camera System Calibration and Non-planar Scene Estimation

Abstract

A projection mapping display system creates impressive 3D displays with light by mapping a 2D image from a calibrated projector onto a display surface. Projection mapping systems require that geometric information must be known about the projector, its spatial relationship to the display surface, and the surface itself. These relationships are constructed through observation of the projector and the display environment by a camera. The calibration process can be burdensome on the user, and different strategies will rely on prior information about the devices or upon enforcing display environment constraints. High capital costs are associated with generating a prior knowledge of cameras. Display environment constraints limit the range of possible display environments, in some cases requiring a 2D display surface, preventing non-planar 3D display environments. A selfcalibration projector-camera(s) process that does not rely on known or fixed cameras, nor calibration targets, is highly desirable to increase both the ease of use and the range of possible environments for existing projection mapping systems. This thesis develops a method for producing a geometric calibration estimate and 3D display surface estimate for non-planar projection mapping display environments. This approach assumes no prior information on the moving camera or fixed projector. Pixel correspondences relate observations across the camera and projector views, and are used to construct geometric relationships to produce a weak calibration estimate. Many applications of projection mapping technology involve artistic renderings that must be precisely mapped from 2D image projection to a 3D non-planar surface. The drafting of these artistic renderings often necessitates the existence of some prior virtual scene understanding. Limited scene understanding provides the basis for constructing virtual calibration targets to perform a geometric recovery of the weak calibration estimate recovery through bundle adjustment. Experimental results show that the geometric calibration estimate observed no error in the estimated projector intrinsic parameters, and less than 2 degrees of average angular error in the estimated projector and camera poses when considering 2500 pixel correspondences with σ = 1 px additive Gaussian noise. The performance accuracy decreases with increasing noise in the pixel coordinates.