Speckle noise is an intrinsic property of Synthetic Aperture Radar (SAR) imagery, which affects the quality of image. Single-temporal despeckling methods usually pay attention to the utilization of spatial information, but sometimes due to lack of sufficient information, the despeckling image is too smooth or losses some information about edge details. However, multi-temporal SAR images can provide extra information for despeckling resulting in better performance. Therefore, in this paper, we proposed a novel multi-temporal SAR despeckling method based a convolutional neural network (MSAR-CNN) embedded temporal and spatial attention (TSA) module to deeply mine the spatial and temporal correlation of multitemporal SAR images. The whole network, which is end-to-end trained with simulate realistic SAR data, consists of several residual blocks. In addition, the simulated and real-data experiments demonstrate that the proposed MSAR-CNN outperforms most of the mainstream methods in both the quantitative evaluation indexes and visual effects.