ANALYTICAL FORWARD-PROJECTION AND SYSTEM CALIBRATION FOR IMAGING THROUGH REFRACTIVE WINDSHIELDS IN AUTOMOTIVE APPLICATIONS

Abstract. We study in this paper models to mitigate effect of a windshield-related refraction on imaging systems. The literature shows that the distortions introduced by this curved surface are non-linear in their effect and reduce the performance of image-based analyses and depth estiamtion algorithms. We show that using geometric optics and local approximation of the windshield’s surface to a spherical one, a direct analytical refractive forward projection (RFP) form of a 3-space point onto the image plane can be derived. However, an exact form requires solving a 22-degree polynomial which may become numerically unstable. To stabilize the solution, we demonstrate how the introduction of valid local assumptions on the interface allows reducing the polynomial degree down to 8 and 4. Utilizing these forms we then show that the RFP can be used to jointly estimate the camera pose and the windshield’s surface parameters through minimization of the reprojection error. The proposed models are tested on simulated data and validated on real-world observations. Results show stability and a sub-millimeter level of reconstruction accurecy, alluding to the validity and quality of our representations.