Accurately predicting crop yields at a large scale is crucial for safeguarding global food security and optimizing the management of agricultural resources. Although diverse data sources, such as satellite imagery, climatic variables, and soil characteristics, have become increasingly available, traditional machine learning models still struggle to capture the complex nonlinear interactions and underlying spectral–temporal dependencies inherent in these datasets. This study proposes a novel Spectral–Temporal Attention-Based WaveNet (STAW-Net) architecture that integrates dilated causal convolutions with combined spectral and temporal attention mechanisms. The proposed model effectively learns long-range dependencies and adaptively extracts and fuses informative features from heterogeneous data sources, thereby enhancing the accuracy and robustness of crop yield predictions. The model was trained using a comprehensive dataset comprising radar imagery from Sentinel-1, multispectral optical data from Sentinel-2, daily meteorological variables from Daymet, and soil characteristics from SoilGrids. The dataset spans major U.S. corn-producing regions between 2016 and 2022; data from 2016–2020 were used for training, while 2021–2022 were reserved for independent testing to evaluate model generalization. Results that the proposed STAW-Net outperforms other models, including Random Forest, DeepRF, the standard WaveNet, Temporal-Attention WaveNet, and Spectral-Attention WaveNet models. The STAW-Net achieved R² values of 0.81 and 0.75, MAE of 12.80 and 15.18, and RMSE of 16.44 and 20.55 for 2021 and 2022, respectively. These findings highlight the model’s strong generalization capability and its effectiveness in capturing complex spectral–temporal patterns for reliable crop yield prediction across diverse environmental and climatic conditions.