End-to-End Pothole Detection and Semantic 3D City Model Enrichment Using LoRA-Adapted Foundation Models and Open Street-Level Imagery

Abstract. Regular road inspection is essential for maintaining pavement service life and ensuring traffic safety. Conventional approaches require specialized survey vehicles and sensor payloads, while damage assessment relies predominantly on supervised computer vision models trained on large annotated datasets. Recent advances in foundation models, notably the Segment Anything Model 3 (SAM 3), have demonstrated strong zero-shot and few-shot segmentation capabilities guided by natural language prompts, offering a promising alternative for domain-specific defect detection. However, most existing studies address detection or segmentation in isolation without integrating results into standardized spatial data infrastructures suitable for infrastructure management. This paper presents a Location-Based Support Decision System (LBSDS) that bridges this gap: from openly available Mapillary street-level imagery acquisition, through pothole detection using a parameter-efficient Low-Rank Adaptation-adapted SAM 3 model, to monoplotting-based geolocation, and semantic enrichment of CityGML 3.0 road network models via direct 3DCityDB insertion.Comparative benchmarking against a YOLO-based instance segmentation model and fully fine-tuned SAM 3 shows that Low-Rank Adaptation detects 2.5 times more potholes than the supervised baseline at competitive precision, while training only 4.96% of model parameters; the supervised detector retains an advantage in pixel-level mask quality on the defects it does detect. The resulting pipeline produces schema-validated features within the CityGML Transportation Module, enabling interoperable road condition representation within semantic 3D city models. We demonstrate the practical feasibility of our pipeline through a real-world case study in Hamburg, Germany.