DMPCONV: DECOUPLING MULTI-BRANCH POINTWISE CONVOLUTIONS FOR LIGHT-WEIGHT REMOTE SENSING SCENE CLASSIFICATION

Abstract. The use of multi-branch architectures in off-the-shelf light-weight residual series neural networks can significantly improve their performance in remote sensing scene classification tasks. However, such architectures come at the expense of an increased number of parameters and calculations. In this paper, we propose the Decoupling Multi-branch Pointwise Convolutions (DMPConv), which works without a corresponding increase in parameters and calculations during inferencing, and at the same time, can maintain the same performance improvement ability as the multi-branch architectures. DMPConv can be decoupled into two states, the training-time DMPConv and the inferencing-time DMPConv. The training-time DMPConv enhances the expressivity of the network by using weighted multi-branch 1×1 convolutions. After training, we use structural reconstruction to convert the training-time DMPConv to the inferencing-time DMPConv, which has the same form as vanilla 1×1 convolution, so as to realize the inferencing-free. Extensive experiments were conducted on multiple remote sensing scene classification benchmarks, including Aerial Image data set and NWPU-RESISC45 data set to demonstrate the superiority of DMPConv.