A Transformer-Based Framework for Spatiotemporal Unmixing of Land–Water Mixtures in Multispectral Satellite Data

Abstract. Spectral unmixing is essential for analyzing mixed pixels in remote sensing, though it has traditionally focused on hyperspectral data. Multispectral Sentinel-2 imagery, despite its wide availability and relevance for environmental monitoring, has seen limited application in this domain and is affected by spectral variability caused by environmental conditions, atmospheric residuals, and temporal changes, which are often neglected in existing methods. We propose the time-dependent Deep Transformer MultiSpectral Unmixing Model (tDTMSUM), a multimodal deep generative framework designed to extract pure water spectra from mixed Sentinel-2 observations, particularly in narrow rivers where water pixels are frequently mixed with adjacent land. The model integrates Sentinel-2 reflectance with auxiliary variables contributing to spectral variability, including the geographical position of water bodies, to capture the spatial dynamic transition of water properties. For example, in the study area in this work, the model successfully detected the change of the water body from standing water in the southern reservoir to sediment-laden flowing water in the northern river. tDTMSUM combines a Variational Autoencoder with a channel-wise Transformer and is trained on augmented dataset derived by synthetic mixtures from real Sentinel-2 data to perform supervised endmember extraction and abundance estimation, focusing on the water endmember. Evaluation using Sentinel-2 imagery and close range spectrometer measurements demonstrates that tDTMSUM outperforms state-of-the-art methods in efficiency, robustness, and accuracy, providing a practical tool for real-world water monitoring even in the absence of extensive ground truth.