EIGENVALUE AND GRAPH-BASED OBJECT EXTRACTION FROM MOBILE LASER SCANNING POINT CLOUDS

The mapping of road environments is an important task, providing important input data for a broad range of scientific disciplines. Pole-like objects, their visibility and their influence onto local light and traffic noise conditions are of particular interest for traffic safety, public health and ecological issues. Detailed knowledge can support the improvement of traffic management, noise reducing infrastructure or the planning of photovoltaic panels. Mobile Mapping Systems coupled with computer aided mapping work-flows allow an effective data acquisition and provision. We present a classification work flow focussing on pole-like objects. It uses rotation and scale invariant point and object features for classification, avoiding planar segmentation and height slicing steps. Single objects are separated by connected component and Dijkstra-path analysis. Trees and artificial objects are separated using a graph based approach considering the branching levels of the given geometries. For the focussed semantic groups, classification accuracies higher than 0.9 are achieved. This includes both the quality of object aggregation and separation, where the combination of Dijkstra-path aggregation and graph-based classification shows good results. For planar objects the classification accuracies are lowered, recommending the usage of planar segmentation for classification and subdivision issues as presented by other authors. The presented work-flow provides sufficient input data for further 3D reconstructions and tree modelling.


INTRODUCTION 1.1 Background
The automatic classification of urban road environments and the detection of single objects such as traffic signs and trees is an important task for a variety of topics.Point cloud classification provides a fundamental input to the reconstructions of 3D scenes, which are used in road safety, public health, ecological and micro climatic applications (e.g.Zhou and Vosselman 2012, Pu et. al 2011, Brunner 1998, Endelaw et al. 2009).For road safety issues the mapping of traffic signs, curbstones and trees is of importance.For the management and assessment of urban vegetation, i.e. for above ground biomass and leaf area measures, the extraction and classification of single trees is required.Trees, and their complex transparency properties, are of particular interest for the assessment of light and traffic noise conditions and visibility studies in urban planning.In order to provide appropriate input data for such applications, surveys with mobile mapping systems mounted on cars or other vehicles are combined with automated computer-aided extraction work-flows allowing efficient data processing and mapping of larger areas e.g.along road corridors.

Related work
For the automatic detection of objects from mobile laser scanning (MLS) data a variety of methods is available.For example Zhou and Vosselman (2012) are detecting curbstones by analysing local height jumps.
Trees can be detected by the echo ratio measures (Rutzinger et al. 2011) and pole-like structures can be identified by height percentile techniques (Zhong et al. 2013, Pu et al 2011).Many work-flows apply a planar segmentation to the point cloud which is an indirection for the detection of non-planar objects.As poles and trees show rather linear or cylindrical patterns, an alternative approach might be better suited.Gross and Thoennessen (2006) and Jutzi and Gross (2009) provide discriminating features to select points belonging to linear, planar and other structures in point clouds.Object reconstruction approaches relying on graph-analysis such as tree skeletonization (Dai et al. 2009, Dai et al. 2010, Livny et al. 2010, Bremer et al. 2013) or building modeling (Oude-Elberink and Vosselman 2009) allow the separation of complex semantic groups.Especially for trees, the separation of interlinked tree crowns and nearby objects is an important task.While Zhou and Vosselman (2012) and Zhong et al. (2013) use Voronoi-regions and related approaches for the separation of tree crowns, Livny et al. ( 2010) investigate the suitability of a Dijkstra cost algorithm (Dijkstra 1959) to associate crown points to the tree trunk.

Overview
Our approach focuses on rotation and scale invariant point and object features for point cloud classification (Gross and Thoennessen 2006)  (4) Pole objects including trees are separated using a Dijkstra region growing approach.Artificial pole objects such as lamps, traffic signs and traffic lights are further separated from trees using branching levels derived from skeletonization (see Bremer et al. 2013) and graph-analysis.

Multi scale feature computation
For each point, local point neighbourhood features are computed.All barycentric coordinates of points lying in a spherical radius around a search point are encoded into a covariance matrix A T A (Eqn. 1).The encoding is done for both a 0.1 m radius (r01) and a 0.5 m radius (r05).The radii were chosen after visual analyses considering the optimized extraction of smaller poles (e.g.thin branches (r01)) and larger posts (r05).
From the covariance matrices, three specific eigenvalues, three eigenvectors, and the vertical angles of the longest and the smallest eigenvectors are computed for each radius (r01, r05) respectively.
According to the usually high point density of the original MLS data, a search radius of 0.5 m is increasing the computation time significantly.Thus we use a hierarchical approach, analysing the neighbourhood of a 0.1 m block-thinned point cloud for each point of the original point cloud.This is reducing the time required for computing and leads to a generalization of the input data, increasing the coarse scale effect of the 0.5 m radius search. (1)

Primitive classification
The specific relationship of the normalized largest eigenvalue (eL), intermediate eigenvalue (eI) and the smallest eigenvalue (eS) is a characteristic feature for the shape of individual point sets.
In order to classify the raw points into the primitive classes 'linear', 'planar', and 'volumetric', we define three specific eigenvalue patterns for each class (values between 0 and 1): linear: eL = 0.75, eI = 0.16, eS = 0.04 planar: eL = 0.45, eI = 0.45, eS = 0.01 volumetric: eL = 0.45, eI = 0.45, eS = 0.30 The eigenvalues of the patterns are averaged values derived from small training areas of the selected classes.
In the 3-dimensional feature spaces spanned by the eigenvalues r01 and the eigenvalues r05, each data point is assigned to the primitive pattern to which the shortest Mahalanobis Distance (D) is observed.The Mahalanobis Distance (Eqn.2) is the statistical distance measure (D) where, in addition to the euclidean distance, the covariance matrix (S) of the dataset is considered.Here, S is computed as shown in equation 1 using the feature space vectors eL, eI, eS instead of the x,y,z coordinates.

( ) ( ) ( )
This leads to the primitive classifications (linear01, planar01, volume01, linear05, planar05 and volume05).The linear and planar classes are further split into vertical, horizontal and other orientations (vert_planar01, hor_planar01, vert_linear01, etc.).For linear structures the orientation of the longest eigenvector is used.A horizontal structure shows a vertical angle of the longest eigenvector < 10°, a vertical structure > 80°.For planar structures the orientation of the smallest eigenvector (corresponding to the normal vector) is used.A horizontal structure shows a vertical angle of the smallest eigenvector > 80°, a vertical structure < 10°.For volumetric structures, the orientation is not defined.

Object classification
In the next step, the multi scale primitive classification is used to aggregate and separate semantic groups.Based on the primitive classification, a region growing in object space is performed in order to separate smaller and larger homogeneous point clumps.For the separation of wall-associated elements (e.g.columns and window frames) and ground-associated poles (e.g.traffic signs and tree trunks) we additionally apply distance thresholds.
A ground inventory feature needs to show a distance to a ground cluster smaller than 1 m and a minimum distance to a wall cluster of 3 m.A wall feature needs to show a distance to a wall cluster smaller than 1 m

Graph-based classification for pole supported objects
As a lot of applications need reconstructed 3D models as data input, automatic mobile mapping work-flows often include linear and polygonal geometry reconstruction.Thus, we integrate a skeleton graph reconstruction step into our classification work-flow.In order to extract trees and tree trunks from the data set, we analyse the point neighbourhood in the upper half-space of a pole feature.Object clumps are grouped using a Dijkstra region growing procedure.
Using the pole as a seed-object for a skeletonization, a graph is build for each pole component and its associated neighbourhood.

DATA SET
The test site is located in Rheine, Germany.The MLS data was aquired by TopScan GmbH in March 2013 using the Optech Lynx Mobile Mapper system with two rotating laser scanners mounted in a 45°angle with the driving direction of the vehicle (Optech 2013).For this study an exemplary road segment, 300 m in length, is processed.It was scanned in one driving direction.The point cloud merged from both sensors contains 5,838,701 pts.and shows an averaged point distance of 2 cm.Simultaneously with the scan, panoramic images were shot using a Ladybug panoramic camera system (Point Grey, 2013).
The threshold values given in section 2 were obtained from a small training area in the centre of the data set.These values were validated against the whole MLS data set.Therefore, the whole point cloud, was manually classified into the target classes (reference) comparable to the automatic classes.

Qualitative results
Using a 0.1 m radius (Fig. 1), small tree branches and poles of traffic signs are classified as linear structures.Volumetric patterns are restricted to sharp edges on building walls and curbstones.However, linear and volumetric patterns are also found on planar features such as roof and ground surfaces.A 0.5 m radius search (Fig. 2) allows the detection of larger poles.Roofs are detected as planar features.Curved free-form features such as cars and other moving objects show volumetric patterns.Volumetric patterns are also found in tree crowns.The Dijkstra region growing for artificial poles and trees shows correct results, even in dense and closely connected neighbourhoods.The threshold-based classification leads to some less reliable classifications (see Fig. 3, 4 and 5 as reference).As can be seen in Fig. 6, the maximum hierarchical branching level of an artificial pole-like object (lamp or traffic sign) is between 0 and 4, while trees show much higher branching levels.This leads to a visually sufficient separation of both classes (Fig. 3 and 4).

Accuracy measures
The accuracy measures (table 7 and 8) show the highest values for the base classes 'ground', 'wall' and 'pole', coming from the the raw classification.Besides that, the pole supported classes separated by the graph-based classification also show high accuracy values.For these classes (G,W,AP,T) only small point counts are missing compared to the target point counts given by the reference data (user accuracies > 0.98).The producer accuracies are also high for these classes.The producer accuracy of the AP class is slightly lowered due to the wrong classification of columns of buildings and gutters as single poles.
The associated classes WI, GI and R also show lowered accuracies, because of the lowered robustness of the distance criteria compared to the the Dijkstra classification approach.Table 8.Error matrix of the classification accuracies 5. 5. 5.

DISCUSSION
The eigenvalue based classifications show advantages and disadvantages due to the scale of the analysed neighbourhoods (0.1 and 0.5 m radius search).An Advantage of the 0.1 m search classification is the appropriate description of small geometrical patterns and discontinuities.On the other hand, a disadvantage is the strong sensitivity to scanning patterns, data gaps and differences in point densities.This partly results in linear and volumetric patterns on planar features such as roof and ground surfaces.Additionally, the detection of larger pole objects such as thicker tree trunks is not possible (see section 4).In contrast to this, the 0.5 m radius eigenvalue classification offers a straightforward detection of larger poles.Additionally, surface elements suffering from lower point densities in the scan coverage such as roofs are clearly detected as planar features.
Reference Data The raw classification of cars allows the separation of these objects as undefined points.A disadvantage of the 0.5 m radius approach is a blurring effect at edges and discontinuities in the point cloud.This also leads to volumetric patterns in tree crowns, well known from lower resolution data such as airborne LiDAR (Fig. 2).
In order to make use of both advantages, we combined both raw classifications by applying reasonable constraints as described in section 2. This results in a reliable classification of walls, ground and poles.
The connectivity analysis for ground inventory, wall inventory and roof shows less reliable results as for artificial poles and trees.In the latter case, the shortest path competition of the Dijkstra algorithm results in distinct clusters.The thresholdbased classification leads to some less reliable classifications in areas where two classes are spatially connected (see Fig. 3 and  4).
As shown in Figure 6 the maximum branching level of a pole supported object is a suitable object feature for the separation of artificial objects and trees. of the hierarchical levels on artificial objects do only occur if traffic lights and signs are inhomogeneously covered by data points.
With respect to the applied work-flow typical systematic errors occur.One main problem is the discrimination between wall inventory such as pipes, gutters and columns in front of building entrances, and poles of the road inventory.The application of a connected component analysis results in systematic errors in case of traffic signs that are directly adjacent to building walls (see Fig. 9).An other source of error is associated with low point densities and data gaps due to occluding effects of other objects.In many cases occlusions are caused by moving objects.One example is the car on the left traffic lane in Fig. 5, which is moving in the same direction as the mobile mapping device, causing a large data gap in the point cloud (Fig. 4).
In areas where the point density is too low and clumps of points with similar raw classification become too small, planar target objects are falsely classified as UD.This is visible for ground segments (Fig. 10) and roof segments.In particular roof surfaces are under-represented in the MLS data and thus are difficult to classify.
In general, the presented work-flow shows a reduced performance regarding the distinct subdivision of planar surfaces (see Fig. 9 and 10.).Here, a classification based on planar segmentation is expected to show better results.Additionally, the thresholding approach used for the clump sizes and adjacencies lowers the accuracy of the results.Here, threshold-less decisions, such as the presented Dijkstraapproach, are desirable.However, focussing on the detection of poles and other nonplanar objects as well as the separation of pole supported objects, the presented approach is robust and straight forward.6. 6. 6.

CONCLUSION AND OUTLOOK
Object classification of mobile laser scanning point clouds is providing an essential input for many applications.We presented an automated classification approach separating the eight classes ground, ground inventory, walls, wall inventory, roofs, artificial poles, trees and undefined objects.The workflow is independent from planar segmentation and height slicing steps, focusing on pole-like objects.
For this semantic group, we can provide a robust and straightforward approach for its detection, separation and classification.This can provide adequate input data for future light, noise and visibility modelling and leaf area estimations.
For leaf-off trees, as presented in this paper, the leafage could be modelled making the consideration of complex crown densities possible (Cote et al. 2009, Rutzinger 2011).
For the other semantic groups, thresholding has a more important role, reducing the comparability of the results.Planar object groups for example can be handled less accurate.In order to improve the results, more intelligent and threshold-less solutions are necessary.
For future work on planar objects the presented approach might be useful for the detection of planar candidate classes, followed by planar segmentation for robust and refined subdivisions.

ISPRS
Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey The following criteria are used for classifications: Poles: vert_linear05 OR (vert_linear01 AND NOT planar05) AND clump size > 100 Ground: (hor_planar01 OR hor_planar05) AND clump size > 30000) Walls: (vert_planar01 AND NOT vert_line05) AND clump size > 3000Applying the 0.1 m radius leads to small blurring effects.Smaller branches and thin poles are detected as linear structures.Larger objects appear as vertical planes and not as poles.Showing a more intense blurring, the 0.5 m radius classification is applied in order to also classify larger trunks as linear structures.Combining both classifications exploits the advantages of both scales and leads to a reliable extraction of pole-like structures.Due to inhomogeneous point densities and scan patterns in MLS point clouds, single scan lines on the road and other flat surfaces may be classified as linear or volumetric structures using a 0.1 m radius.Therefore, both 0.1 m and 0.5 m radius are used for ground classification.Additionally, ground segments are usually large and include a lot of points.Since building walls are mostly vertical, the use of the vert_planar01 feature leads to a sufficient separation of building walls.In order to avoid noise in the object classification results, minimum cluster sizes are introduced.Connected ground clusters need to show a minimum point count of 30.000 pts.wall clusters need to show a minimum point count of 3.000 pts.and pole clusters of 100 pts.

Figure 1 .
Figure 1.Raw classification into volumetric (red), linear (dark blue) and planar (light blue) patterns using a 10 cm radius search.

Figure 2 .
Figure 2. Raw classification into volumetric (red), linear (dark blue) and planar (light blue) patterns using a 50 cm radius search.

Figure 4 .
Figure 4. Final classification.Panoramic rendering of the classified point cloud.

Figure 5 .
Figure 5. Panoramic image of the scene.

Figure 6 .
Figure 6.Skeleton reconstructions of pole supported objects.The branching levels are colour coded from low (blue) to high level (red).The estimated thickness of the point cloud around each skeleton segment is modelled as a connected pipe model.

Figure 9 .
Figure 9. Example of an undetected pole of a traffic sign, near to a building wall (falsely classified as wall inventory).

Figure 10 .
Figure 10.Example of a low point density ground segment, falsely classified as undefined (UD = magenta).
Fig. 11  shows the separation of a traffic sign and a tree standing next to each other.The clumps of both objects are separated by Dijkstra region growing.This leads to an improved separation compared to e.g.Voronoi-approaches and related methods presented by e.g.Rutzinger et al. (2011) orZhong et al. (2013).

Figure 11 .
Figure 11.Example of a correctly separated traffic sign and a tree.
Further enhancements could focus on the incorporation of corrected or normalized intensity values improving point cloud classification and differentiating even more classes.In order to ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey strengthen the evaluation of point cloud classification approaches, further research on appropriate error assessment strategies are required.This would allow a decision on the suitability of point cloud classification results for specific applications.
avoiding planar segmentation and height slicing steps.ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, VolumeII-5/W2, 2013  ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, TurkeyFor this work-flow, we are focusing on the detection and subdivision of artificial pole based objects (AP) such as lamps and traffic signs, and natural pole supported objects such as trees (T).As additional classes, we separate ground (G), ground inventory (GI) such as curbstones and lower objects, walls (W), wall inventory (WI) such as window-frames, doors and building columns, roofs (R), and undefined objects (UD) such as moving cars or people.Points associated with mirror effects of windows, are treated as undefined objects.