Object Detection for the Enrichment of Semantic 3D City Models with Roofing Materials

Abstract. Semantically rich 3D city models play a vital role in a variety of applications, such as urban planning. Enhancing these models with currently unavailable attributes, such as roof material types, can unlock new opportunities to tackle pressing challenges, including climate change mitigation and sustainable urban development. In this work, we present an end-to-end pipeline for the automatic detection of roof materials to semantically enrich 3D city models. To support this, a comprehensive training dataset was prepared by automatically annotating roof materials across Germany using OpenStreetMap (OSM) attributes and high-resolution orthophotos. Our object detection pipeline classifies five distinct roof material types using the YOLOv11-L architecture. Our detection results enabled the automatic augmentation of CityGML-based 3D models, filling in missing roof material information. This enrichment supports advanced applications, such as assessing roof suitability for green infrastructure or simulating urban heat island mitigation strategies. We validated the feasibility of our approach with real-world data and applied the method to a district in the city of Bremen, Germany. The paper also includes a detailed discussion of the learning process quality, the integration, and the visualization of the enriched 3D city model. The used code is available at: https://github.com/hcu-cml/citydb-roofmats-ai.