Improving Deep Learning based Point Cloud Classification using Markov Random Fields with Quadratic Pseudo-Boolean Optimization

Abstract. 3D point clouds are a relevant source of information for multiple applications, including digital twins, building modeling, disaster and risk management, forestry, autonomous driving, and many others. Assigning points to the semantic classes is one of the essential data interpretation steps to effectively use them for further analysis. Deep learning models for semantic segmentation, such as RandLA-Net, are state-of-the-art methods for this task. Although the overall accuracy of classification is usually satisfactory,there are still several shortcomings not allowing assigning correct labels across all the classes. For instance, the receptive field of these networks is often too small to correctly classify point clouds in all cases. These networks suffer also from class imbalance, typical in real-world data sets, and tend to oversmooth small classes. Post-processing approaches help to overcome these problems and achieve better classification accuracy. In this work, we investigate the feasibility of improving the deep-learning outputs by introducing prior knowledge. To do this, the output probabilities of point classes obtained using RandLA-Net are post-processed with a workflow based on Markov Random Fields, in which the unary potentials are adjusted to preserve smaller classes while the pairwise potentials take into account. a hand-tailored inter-class reliability matrix. To validate our method, we apply it to the Hessigheim benchmark. Our MRF-based approach further optimizes these prediction results, effectively and efficiently improving the overall accuracy by approximately 1 to 2 percentage points.