Using Passive Multi-Modal Sensor Data for Thermal Simulation of Urban Surfaces

Abstract. This paper showcases an integrated workflow hinged on passive airborne multi-modal sensor data for the simulation of the thermal behavior of built-up areas with a focus on urban heat islands. The geometry of the underlying parametrized model, or digital twin, is derived from high-resolution nadir and oblique RGB, near-infrared and thermal infrared imagery. The captured bitmaps get photogrammetrically processed into comprehensive surface models, terrain, dense 3D point clouds and true-ortho mosaics. Building geometries are reconstructed from the projected point sets with procedures presupposing outlining, analysis of roof and fac¸ade details, triangulation, and texturing mapping. For thermal simulation, the composition of the ground is determined using supervised machine learning based on a modified multi-modal DeepLab v3+ architecture. Vegetation is retrieved as individual trees and larger tree regions to be added to the meshed terrain. Building materials are assigned from the available visual, infrared and surface planarity information as well as publicly available references. With actual weather data, surface temperatures can be calculated for any period of time by evaluating conductive, convective, radiative and emissive energy fluxes for triangular layers congruent to the faces of the modeled scene. Results on a sample dataset of the Moabit district in Berlin, Germany, showed the ability of the simulator to output surface temperatures of relatively large datasets efficiently. Compared to the thermal infrared images, several insufficiencies in terms of data and model caused occasional deviations between measured and simulated temperatures. For some of these shortcomings, improvement suggestions within future work are presented.