Autonomous UAV 3D Reconstruction using Prediction-Based Next Best View

Abstract. High-quality 3D reconstruction of infrastructure using UAVs is essential for inspection, monitoring, and digital twin applications. Traditional flight planning methods rely on predefined paths and often struggle with complex geometries, leading to incomplete models and inefficiencies. This paper evaluates a state-of-the-art autonomous Next Best View (NBV) of MACARONS model (Mapping And Coverage Anticipation with RGB Online Self-Supervision), which enables online, self-supervised 3D reconstruction of large-scale scenes using only a monocular RGB sensor. The MACARONS NBV model autonomously adjusts UAV trajectories in real time based on predictions of unseen scene structure to improve reconstruction accuracy and surface detail recovery. Despite its advantages, a key limitation is its lack of consideration for camera coverage percentage from a photogrammetric perspective, which makes it challenging to consistently obtain an informative point cloud. The simulation results demonstrate that the autonomous NBV strategy significantly enhances both reconstruction quality and operational efficiency. To evaluate its effectiveness, we applied the MACARONS NBV model to two open-access 3D bridge models. The generated camera trajectories were imported into Blender, where we rendered high-resolution images using realistic camera intrinsics to overcome the limitations of the low-resolution depth predictions. From these images, we reconstructed point clouds and compared them to those produced by a traditional flight planning approach, as well as to the ground truth models. The comparison highlights the added value of autonomous view planning for accurate and efficient UAV-based 3D reconstruction. The two experiments showed a high coverage percentage of 88 % compared to the ground truth and 90% compared to traditional flight planning based on a 37.5% efficiency raise. This work highlights the potential and current limitations of prediction-based NBV in UAV photogrammetry and motivates further research into integrating coverage-aware planning.