RANSAC-Based Planar Point Cloud Segmentation Enhanced by Normal Vector and Maximum Principal Curvature Clustering

: Planar feature segmentation is an essential task for 3D point cloud processing, finding many applications in various fields such as robotics and computer vision. The Random Sample Consensus (RANSAC) is one of the most common algorithms for the segmentation, but its performance, as given by the original form, is usually limited due to the use of a single threshold and interruption of similar planar features presented close to each other. To address these issues, we present a novel point cloud processing workflow which aims at developing an initial segmentation stage before the basic RANSAC is performed. Initially, normal vectors and maximum principal curvatures for each point of a given point cloud are analyzed and integrated. Subsequently, a subset of normal vectors and curvature is utilized to cluster planes with similar geometry based on the region growing algorithm, serving as a coarse but fast segmentation process. The segmentation is therefore refined with the RANSAC algorithm which can be now performed with higher accuracy and speed due to the reduced interference. After the RANSAC process is applied, resultant planar point clouds are built from the sparse ones via a point aggregation process based on geometric constraints. Four datasets (three real and one simulated) were used to verify the method. Compared to the classic segmentation method, our method achieves higher accuracy, with an RMSE from fitting equal to 0.0521 m, along with a higher recall of 93.31% and a higher F1-score of 95.38%.


Introduction
Light Detection and Ranging (LiDAR, renowned for its ability to promptly generate detailed three-dimensional (3D) spatial data.With the advancement of hight-precision LiDAR sensors, an increasing array of fields are incorporating this technology, such as autonomous vehicles (Li andIbanez-Guzman, 2020 , Zhao et al., 2020), cultural heritage management (Chan et al., 2011, Rodrí guez-Gonzá lvez et al., 2017, Chan et al., 2021, Teppati Losè et al., 2022), mobile robotics (Weiss andBiber, 2011 , Yang et al., 2022), and smart cities (Garnett andAdams, 2018 , Ortega et al., 2021).Three-dimensional point clouds, as a data type for recording LiDAR measurements, require extensive post-processing in which the plane segmentation is one of most significant processing operations.
Plane segmentation algorithms can be broadly categorized into two types: parameter-based methods and clustering-based methods (Wu et al., 2019).For parameter-based methods, the RANSAC algorithm (Fischler and Bolles, 1981) is one of the most frequently-used due to its robustness.It operates by iteratively selecting random points to construct initial planes, then extracting the best-fitting plane by comparing the number of points falling on each plane.Kurban et al. (2015) utilized RANSAC algorithm to segment point cloud obtained from Kinect sensor.Qian and Ye (2014) combine the Normalized Cross-Correlation technique with the RANSAC algorithm to swiftly and accurately identify planar surfaces in complex 3D scenes.Li and Shan (2022) utilizes the RANSAC algorithm to identify and extract multiple primitive shapes or structures that collectively represent a building.While this approach can reduce the influence of outliers, it often requires considerable time to identify the correct plane and can sometimes segment numerous inaccurate planes in complex scenes, such as those involving stairways point cloud (Awwad et al., 2010).Nonetheless, the clustering-based methods group points that are close in both normal vectors and spatial positions.For example, Xu et al. (2019) employed an improved density clustering algorithm to group points together, facilitating the identification and fitting of planes in the point cloud data.Sağlam et al. (2020) segmented the point cloud data into patches using K-means clustering and then refining these nonplanar patches into plane segments.These methods typically cluster points based on either spatial proximity or normal vectors at a time, often requiring a secondary segmentation step.Region growing algorithm (Gorte, 2002, Tóvá ri and Pfeifer, 2005, Nurunnabi et al., 2012) is another clustering-based plane segmentation method that can simultaneously consider both spatial and normal vector information.However, it currently faces challenges related to slow computational speed and the selection of an appropriate seed point strategy (Wu et al., 2019).
In order to accelerate the speed of the point-based region growing and to enhance the precision of the original RANSAC algorithm for plane segmentation in complex scenes, we present a comprehensive workflow that it begins with an initial segmentation operation on downsampled point clouds, employing a seed point selection strategy based on finding the minimum value of the maximum principal curvature.Following this initial segmentation, the algorithm uses the obtained information to perform the secondary segmentation on the original point cloud data with RANSAC algorithm and distance and normal vector constrained clustering.

Method
Firstly, local regions' normal vectors and maximal principal curvature are estimated by constructing a K-d tree structure (Friedman et al., 1977) to search for the adjacent points of each point.The normal vectors are estimated by calculating the three principal components of the local point cloud and taking the direction of the third principal component as its estimate.Subsequently, a downsampling operation is employed to accelerate the coarse segmentation of the subsequent region growing algorithm.Since a portion of the planar points has already been clustered, the RANSAC algorithm can more rapidly identify the correct plane through random point selection and obtain the corresponding plane parameters.Finally, based on the obtained plane parameters, the distance from the original point cloud before downsampling to each plane is calculated.The final segmentation of the planar point cloud is obtained through constraints based on distance and normal vectors.The described workflow is shown in Figure 1.
Figure 1.Workflow of our method.

Normal Vector and Maximum Principal Curvature
The local adjacent points obtained by K-d tree searching algorithm is fitted to the surface to estimate the normal vector and maximum principal curvature.Principal Component Analysis (PCA) is employed to calculate the three principal components for each local region of the point cloud (Jolliffe, 2002).Based on the planar characteristics, the eigenvector corresponding to the smallest eigenvalue represents the direction with the least information, which is perpendicular to the local plane and can be used as an estimation for the normal vector (Hoppe et al., 1992).
The normal curvature of a point is the degree of curvature of the intersection curve between the surface and a plane, which passes through that point and is perpendicular to the tangent plane.The planes that meet the above conditions are countless, and the principal curvatures are the two curvatures among them that represent the maximum and minimum degrees of curvature (Figure 2).The maximum principal curvature (Gray, 1997) is incorporated into our method.
After performing a least squares quadratic surface fitting (Eq. 1) to the adjacent point cloud, Gaussian curvature and mean curvature (Gray, 1997) can be obtained using the Eq.(2-3), and subsequently solving the Eq.(4-5) yields the maximum principal curvature.where G, M are Gaussian curvature and mean curvature respectively, ki denotes principal curvature.

Region Growing based on Geometric Parameters
The region growing algorithm can be used to aggregate points with similar normal vectors and proximity in distance.This algorithm fundamentally begins from a set of seed points and progressively incorporates adjacent points into the region of these seed points, guided by specific criteria.The process continues until certain predetermined stopping conditions are met.The key to the region growing algorithm's effectiveness lies in the strategies employed for selecting seed points and defining criteria for the inclusion of adjacent points.
For planar point clouds, the degree of curvature is approximately zero, and the normal vectors are similar throughout the same plane.Consequently, curvature can be adopted as a criterion for selecting seed points, and the angle between normal vectors (Ө) can serve as a condition for growth.
In this paper, the point with the smallest maximum principal curvature is selected as the seed point and the angle between the normal vectors serves as the growth condition.The complete workflow can be seen in Figure 3.

Segmentation Result Evaluation
The accuracy of the final segmented planar will be assessed using the root mean square error (RMSE) of the parameters fitted through least squares method (Eberly, 2000), and the average precision, recall, F1-score of the segmented point clouds for multiple planes.This evaluation will be conducted on simulated stairway data.The equation as follow: where N is the number of planes, Pij and Pij′ respectively represent the fitted and true values of the j-th parameter of the ith plane, TPi represents the number of points correctly classified as belonging to plane i, FNi represents the number of points incorrectly classified as not belonging to plane i, FPi represents the number of points incorrectly classified as belonging to class i.

Experiments
We applied our method to point clouds from four different scenes, and compared it with classic RANSAC plane segmentation and region growing methods in terms of visual effects and computational speed.quantitative accuracy.Additionally, the segmentation results of each method were quantitatively evaluated using a simulated stairway point cloud data.
The experimental data for this study consists of point cloud data from four different scenes: (1) A room point cloud at a university, acquired by a Trimble SX 10 scanner, consisting of 538099 points (Figure 4a).( 2) A simulated straight stairway point cloud, composed by stitching together seven rectangular cuboid point clouds with 0.01 m Gaussian noise, consisting of 167556 points (Figure 4b).(3) A spiral stairway point cloud at a university, obtained by a Trimble SX 10 scanner, consisting of 526706 points (Figure 4c).( 4) An electric tower body point cloud, collected by a Faro Focus 3D scanner, consisting of 569667 points (Figure 4d).

Preliminary Plane Segmentation
In the scenes with planes, the distribution of normal vectors presents a clustered state (Figure 5).However, due to the influence of point cloud noise and errors in the normal vector estimation method, especially significant at the edges of planes, the planar points clustered by region growing often has omissions (Figure 6).

Final Plane Segmentation
During the RANSAC algorithm process, iterative searches are conducted to find planar points, and the preliminarily segmented planar points can enable it to fit rapidly.Finally, based on the information of the fitted planes, coupled with constraints on the point-to-plane distance and normal vectors, complete planar points are segmented from the original dense point cloud.

Comparison
The classic RANSAC method and region growing method were used to segment the point clouds of these four scenes, and the results are shown in Figure 8.Additionally, the operating speeds of different methods were compared, and the results are presented in Table 1.It is evident that the RANSAC method tends to segment numerous inaccurate planes, such as the window frame in Scene 1 and the inclined planes in Scene 2 and Scene 3.For the region-growing algorithm, the segmentation result for Scene 2 is quite good, but there were some errors in Scene 1 and 3.For instance, in Scene 1, the mutually perpendicular left plane and bottom plane were segmented as a single plane, while in Scene 3, multiple steps were segmented as one plane, which are mainly due to the inaccurate estimation of normal vectors.In the case of the most complex Scene 4, although all three methods exhibit some confusions, our method demonstrates a slightly lower error rate in terms of visual effectiveness.In comparison to these two methods, our approach exhibits superior segmentation results visually.In terms of speed, the region growing algorithm is the slowest because it requires iterative searching of adjacent points for growth.Since our method also employs the region growing algorithm, its operating speed is not as fast as the RANSAC method when dealing with a large number of point clouds.Additionally, an increase in the number of point clouds will lead to longer times for calculating curvature and normal vectors.However, the use of a K-d tree can accelerate this process to a certain extent.Due to the fact that the speed of RANSAC also depends on the number of planes in the scenes, our algorithm achieves a reverse exceedance in speed when processing scenes with multiple planes such as Scene 2, 3 and 4, where it is faster by 7.18 s, 136.19 s and 190.77 s, respectively.Compared to the region growing algorithm, the speed of our method is significantly faster, being respectively 624.77 s, 411.96 s, 9140.43 s and 30094.18squicker in Scene 1, 2, 3 and 4.
Due to the generation of numerous inaccurate segmented planes by RANSAC in the simulated stair point cloud, quantitative accuracy evaluation was conducted exclusively for our method and the region growing algorithm.The results of accuracy evaluation are shown in Table 2. Our method demonstrated higher accuracy, achieving an average RMSE of 0.0521 m, which is superior to the region growing algorithm at 0.116 m.Although our method is approximately 1% lower in precision compared to region growing, our recall and F1-score are higher by 3.86% and 1.68%, respectively.This indicates that our method exhibits fewer cases of both error and missed segmentations, demonstrating its robustness.At the end of this paper, we replaced the RANSAC algorithm with the least squares method to fit the plane on the preliminarily segmented point cloud, in order to validate the necessity and superiority of the RANSAC method in our method.Due to the sensitivity of the least squares method to noise, there exists a problem of parameter bias in the fitting process, leading to errors in plane segmentation, as illustrated in the first stair shown in Figure 9b.

Conclusion
This paper proposes an algorithm that clusters points with similar normal vectors and spatial proximity in a downsampled point cloud to enhance the subsequent RANSAC-based plane segmentation process.The algorithm is both fast and accurate, addressing the efficiency issues of region-growing algorithms and the problem of inaccurate segmented plane generation in classic RANSAC algorithms.Furthermore, the algorithm offers a strategy for selecting seed points for the region-growing

Figure 2 .
Figure 2. Principal curvature viewed from two different angles.

Figure 3 .
Figure 3. Workflow of region growing for geometric parameters.
Figure 4. (a) Scene 1: a room at the university (real point cloud).(b) Scene 2: a straight stairway (simulated point cloud).(c) Scene 3: a spiral stairway at a university (real point cloud).(d) Scene 4: a electric tower body (real point cloud).

Table 1 .
Operating Speed of Different Methods.

Table 2 .
RMSE of plane parameters for different methods.
algorithm based on the minimal value of the maximum principal curvature.Through comparative experiments applying three different methods to four different scenes, our approach demonstrates the best visual effects and superior computational efficiency.Additionally, it exhibits the highest accuracy in terms of RMSE, recall, and F1-score, achieving 0.0521m, 93.31%, and 95.38%, respectively.The results indicate the superiority of our workflow in terms of execution speed and accuracy.Our study integrates normal vector clustering constraints into plane segmentation algorithms, leveraging the strengths of both region-growing and RANSAC techniques.This fusion aims to enhance the robustness of plane segmentation methods, providing a directional reference for research in this field.improveddensityclusteringalgorithm for laser radar.InfraredPhysics & Technology, 96, 133-140.Yang T., Li Y., Zhao C., Yao D., Chen G., Sun L., Krajnik T.,  Yan Z., 2022.3DToFLiDAR in mobile robotics: A review.arXiv preprint arXiv:2202.11025.Zhao X., Sun P., Xu Z., Min H., Yu H., 2020.Fusion of 3D LIDAR and camera data for object detection in autonomous vehicle applications.IEEE Sensors Journal, 20,[4901][4902][4903][4904][4905][4906][4907][4908][4909][4910][4911][4912][4913]