Spectral and Spatial Attention Fusion for Building Segmentation in Remote Sensing Imagery

Abstract. The building segmentation in very high resolution remote sensing imagery presents challenges due to the need to delineate features accurately in a wide range of urban landscapes. Many existing building segmentation methods struggle in discerning complex structures and providing fine grained generalisation over different geographic regions. Additionally, these methods often require to extensive preprocessing and struggle to combine multispectral data. Addressing the different challenges, we introduce the Multi-Band Spectral-Spatial Fusion Attention Network (MBSSFA-Net), a novel method for semantic segmentation. MBSSFA-Net implements a dual encoder designed to exploit the complementary spectral information provided by the Near-Infrared and the RGB bands, to improve the feature representation and the segmentation accuracy. The approach incorporates multiscale spectral and spatial attention fusion blocks in the encoder to fuse the extracted features to enhance boundary delineation, and a spectral and spatial attention fusion blocks in the decoder to merge the spatial and abstract features. The proposed framework can extract buildings in different environment since it can process multispectral data. The experiments have been performed on GaoFen-7 and WHU-Satellite II datasets. The experiments prove that our method outperforms current state of the art Deep Learning segmentation techniques, demonstrating its potential for building segmentation in complex urban environments.