Impact of Rain on 3D Reconstruction with Multi-View Stereo, Neural Radiance Fields and Gaussian Splatting

Abstract. Image-based 3D reconstruction uncovers many applications in documenting the geometry of the environment. Nonetheless, the assumption that images are captured in clear air rarely holds in real-world settings where adverse weather conditions are inevitable. We are particularly interested in rain as dynamic occlusion which degrades image quality and can hinder complete and accurate 3D scene reconstruction of the underlying features. In this contribution we analyze the geometry behind rain reconstructed by traditional Multi-View Stereo (MVS) and radiance field methods, namely: Neural Radiance Fields (NeRFs), 3D Gaussian Splatting (3DGS) and 2D Gaussian Splatting (2DGS). To assess the impact of rain to the 3D reconstruction we consider occlusion masks with different mask coverage. The results demonstrate that although MVS shows lowest accuracy errors, the completeness declines with rain. NeRFs manifest robustness in the reconstruction with high completeness, while 2DGS achieves second best accuracy results outperforming NeRFs and 3DGS. We demonstrate that radiance field methods can compete against MVS, indicating robustness in the geometric reconstruction under rainy conditions, allowing applicability to large-scale scenes, city modeling, digital twins and urban planning which is important for a multidisciplinary approach in problem-solving environmental challenges.