Joint Stone Segmentation and Feature Driven Deformation Analysis at Water Dams

Abstract. Structural health monitoring of water dams is crucial to ensure their long-term safety and operational reliability. Traditional geodetic techniques, although precise, are limited to sparse observation points and cannot capture spatially heterogeneous deformations. Laser scanning enables comprehensive, area-wide acquisition, overcoming this limitation. Subsequent deformation analysis often relies on comparisons along the local surface normal, which are limited in detecting in-plane movements. To address this, this study presents an approach that combines image-based stone segmentation with point-cloud-based deformation analysis to estimate both in-plane and out-of-plane displacements across masonry dam surfaces. Individual stones are detected in unmanned aerial vehicle (UAV) imagery using a deep learning segmentation model (Mask R-CNN) and subsequently projected into corresponding point clouds acquired by terrestrial laser scanning (TLS) and UAV laser scanning. By establishing consistent stone correspondences across multi-epoch point clouds via centroid-based matching and local iterative closest point (ICP) alignment, the proposed method enables deformation analysis on a stone-by-stone level. Simulated deformations were applied to TLS- and UAV-based point clouds of a dam to evaluate the method. Results demonstrate that the approach achieves sub-centimeter accuracy for the TLS and low-centimeter accuracy for the UAV point cloud, as measured by the RMSE between the estimated and true deformation. Our approach outperforms conventional model-to-model comparison methods, such as Multiscale Model to Model Cloud Comparison (M3C2), for in-plane displacements. The integration of image segmentation and geometric analysis provides a powerful framework for full-field deformation monitoring of masonry structures, supporting the detection of instabilities and improving dam safety.