Domain-Adaptive Object Detection for Enriching Semantic 3D City Models with Building Storeys from Street-View Images

Abstract. Semantically rich 3D city models play a vital role in a variety of applications, such as urban planning. Enhancing these models with currently unavailable attributes, such as building storey numbers, can unlock new opportunities to address pressing challenges, including sustainable urban development. In this work, we present an end-to-end pipeline for the automatic estimation of the number of storeys to semantically enrich 3D city models. We employ volunteered geographic information street-view imagery from Mapillary, using a COCO-pretrained object detection model to identify windows in fac¸ade images as key visual indicators for inferring building storey counts. Our detection pipeline, based on the YOLOv3 architecture, estimates storey numbers using an ensemble of clustering methods including Gaussian Mixtures and DBSCAN and enables the automatic augmentation of CityGMLbased 3D city models by filling in missing attributes. This enrichment supports advanced applications, such as assessing buildingscale energy demand, evaluating vertical urban growth patterns or population density estimations. We validated the feasibility of our approach with unfiltered Mapillary and applied it to a district in the city of Heidelberg, Germany. The paper also includes a detailed discussion of learning process quality, integration workflows, and visualization of the enriched 3D city model. The developed code is available at: https://github.com/hcu-cml/citydb-buildingstoreys-ai.