DETECTING AND ANALYZING CORROSION SPOTS ON THE HULL OF LARGE MARINE VESSELS USING COLORED 3 D LIDAR POINT CLOUDS

This work presents a new method that automatically detects and analyzes surface defects such as corrosion spots of different shapes and sizes, on large ship hulls. In the proposed method several scans from different positions and viewing angles around the ship are registered together to form a complete 3D point cloud. The R, G, B values associated with each scan, obtained with the help of an integrated camera are converted into HSV space to separate out the illumination invariant color component from the intensity. Using this color component, different surface defects such as corrosion spots of different shapes and sizes are automatically detected, within a selected zone, using two different methods depending upon the level of corrosion/defects. The first method relies on a histogram based distribution whereas the second on adaptive thresholds. The detected corrosion spots are then analyzed and quantified to help better plan and estimate the cost of repair and maintenance. Results are evaluated on real data using different standard evaluation metrics to demonstrate the efficacy as well as the technical strength of the proposed method.


INTRODUCTION
Nowadays, there are close to 6 million large ships in service (Review of Maritime Transport, 2014), and all of them need to inspect, clean and paint their hulls regularly (every 4-5 years) (Ortiz et al., 2007).Traditional manual surveying of ship hulls is costly, time consuming and of limited accuracy (Ortiz et al., 2007) (Biskup et al., 2007).On the other hand, 3D scanners have emerged as a powerful technology solution for many industries in recent times (Biskup et al., 2007), and painting/repairing ships is not an exception.
Ship hull inspection for detecting surface defects such as corrosion for re-painting and repairing purpose poses a number of challenges, ranging from time consumption due to large dimensions of the ship, limited accuracy to poor illumination and limited visibility, etc.Moreover, these defects could be of any shape and size and can be located at any part of the hull.Detection of these defects is currently done manually by experts who inspect the hull and mark the areas to be treated/repaired.This is a subjective task which makes it very dependent on the experience of person doing it and is also affected by his cumulative fatigue (Navarro et al., 2010).
Alternatively, laser scanners can operate in total darkness, in relatively severe weather conditions and provide fast 3D scans of the hull at high resolution.These high resolution scans can then be used to analyze the hull's surface to detect these defects.
The main aim of this work is to develop a method that uses commercially available 3D laser scanners to scan complete ship hulls and then use these scans to automatically detect and analyze defects such as corrosion spots on the surface of the ship hull.This will not only help in reducing the inspection time by many folds but also increases the reliability as well as the accuracy of the detection and estimation of these defected regions, as compared to manual inspection.As a result, this could lead to better optimization of different repair and maintenance processes saving millions of dollars every year in the shipping industry.

RELATED WORKS
Over the years, the task of detecting surface defects has mostly been considered as a texture analysis problem as presented in (Fernández-Isla et al., 2013).In (Xie, 2008), texture analysis techniques for detecting defects are classified into four categories: structural approaches, statistical approaches, filter based approaches, and model based approaches.Whereas, in (Kumar, 2008), they are classified into three: statistical, spectral, and model based.Ngan et al. (Ngan et al., 2011) divide defect detection methods largely into nonmotif-based and motif-based approaches.The motif-based approach (Ngan et al., 2010) uses the symmetrical properties of motifs to calculate the energy of moving subtraction and its variance among different motifs for detection.Many defect detection methods rely on clustering techniques which are mainly based on texture feature extraction and texture classifications.These features are collated using methods such as Fourier transform (Tsai and Huang, 2003), co-occurrence matrix (Han and Shi, 2007), Gabor transform (Kumar and Pang, 2002) and the wavelet transform (Ngan et al., 2005).
The wavelet transform is an attractive option when attempting defect detection in textured surfaces (Truchetet and Laligant, 2008).In the literature review we find two main categories of defect detection methods based on wavelet transform.The first category includes direct thresholding methods that rely on the fact that wavelet decomposition can attenuate texture background (Tsai and Huang, 2003) (Han and Shi, 2007).This allows the use of existing defect detecting techniques for non-textured images, such as (Sezgin and Sankur, 2004).Textural features extracted from wavelet-decomposed images are another category which is widely used for defect detection (Wong et al., 2009) (Lin, 2007).Features extracted from the texture patterns are used as feature vectors to feed a classifier (Euclidean distance, Neural Networks, Bayer, or Support Vector Machines).This has certain limitations when handling large image data obtained for different inspection tasks.
The authors of (Zheng et al., 2002) presents a method based on images of external metallic surfaces using an intelligent approach based on morphology and genetic algorithm to detect structural defects on bumpy metallic surfaces.The approach employs genetic algorithms to automatically learn morphological parameters such as structure elements and segmentation threshold, etc.
In (Armesto et al., 2011), a light sweeping method for detecting defects on car surfaces is presented.A series of images are obtained which are then merged together.After blurring and matching, defects appear as dark pixels after subtracting from the original image.However, the method has practical limitations for large volumous objects/surfaces.
A sensor system based on thresholding technique is introduced in (Navarro et al., 2010), that is especially suited for image segmentation under variable and non-uniform lighting conditions.A global reference value is calculated using denominated Histogram Range for Background Determination.This value is subsequently used to calculate the local threshold of each area of the image, making it possible to determine whether or not a pixel belongs to a defect.The drawback of this method is that the camera's optical axis should always be placed perpendicular to the plane of the surface to be inspected.
Laser scanning as a new technology has been introduced in the marine industry for the last few years (Biskup et al., 2007).3D scanning technology has begun to emerge in shipyards, but has not yet been exploited for the inspection process and specially detection and analysis of defects on the hull.For these operations, in recent years, a tendency of improvement appeared using the techniques of vision (Navarro et al., 2010).However, these solutions do not provide acceptable results in the case of inspection of very large surfaces, in varying and non-uniform lightening conditions in the open air (Navarro et al., 2010) (Zheng et al., 2002).Such drawbacks and others can be overcome in addition to high accuracy and scanning rates by using 3D scanning technology.Biskup et al. (Biskup et al., 2007) used a terrestrial laser scanner FARO LS 880 to model the hull and the deck of a ship.Analysis of data from the scanner was performed with the aid of commercially available software Geometric Studio 8.At the end, they get 3D model of two differentiated parts (deck and hull) of the ship.Based on this model, certain series of analysis could be made as detection of construction defects, possible asymmetries, along with a variety of different measures.However, surface defects are still not detected.In this work we present a new method of detecting and then analyzing surface defects like corrosion on the ship's hull exploiting data from a 3D scanner.According to the best of our knowledge no prior work exists that exploits the 3D LiDAR point clouds to detect and then analyze surface defects on ship hulls.
In the proposed method several scans from different positions and viewing angles around the ship are registered together to form a complete 3D point cloud (Section 3.).The R, G, B values associated with each scan, obtained with the help of an integrated camera are converted into HSV space to separate out the color component.Different surface defects such as corrosion spots of different shapes and sizes are automatically detected, within a selected zone, using two different methods depending upon the level of corrosion/defects (Section 4.).
The first method relies on a histogram distribution whereas the second on adaptive threshold based method.The detected corrosion spots are then analyzed and quantified to help better plan and estimate the cost of repair and maintenance (Section 5.).The results are evaluated on real data using different standard evaluation metrics to demonstrate the utility as well as the efficacy of the proposed method (Section 6.).Conclusion is presented in Section 7..

DIGITIZATION OF SHIP HULL
The digitization of a complete ship's hull, using a stationary ground based LiDAR scanning system, requires scanning from multiple positions at appropriate distances.For high resolution scans, as necessary for our application, the distance is kept nominal with slower scanning rates and large scan overlaps.These multiple scans after transformation in a global frame of reference are registered together and filtered to obtain a 3D point cloud of the complete ship hull as shown in Figure 1.The 3D scanning system, as shown in Figure 1

Registration of Multiple Scans
In order to obtain multiple scans the ground based LiDAR scanner is placed at different positions all around the ship to ensure full coverage at suitable resolution.The scans are taken such to ensure some overlapping to facilitate the registration process as shown in Figure 1(b).In order to further aid the process, additional targets are also placed all around the ship.The scans are registered, one by one, using a standard ICP algorithm (Besl and McKay, 1992).In order to satisfy the equations, it is ensured that at least 3 common targets are visible in each successive scan.

DETECTION OF CORROSION SPOTS
Once the 3D point clouds of the complete ship's hull is obtained after the registration step, corrosion spots are detected.As the 3D point cloud also contains parts of the ship and surrounding other than the hull, we allow the user to manually select a zone on the hull to be analyzed for corrosion spots, using a simple GUI as shown in Figure 2. The detection of corrosion spots are then conducted within this selected zone automatically.Although it is possible to analyze the complete ship's hull all at once, this manual selection of smaller zones of the hull (a common industrial practice) allows the user/expert to quickly and efficiently identify the zone of interest (with larger chance of corrosion spots) for further processing and prevents wasting resources on less interesting parts.As the corrosion spots are usually more apparent as visual defects (change in color, intensity, etc.) and less of physical deformation (bending, breaking, etc.), the color information plays an important role in the detection process.This is also supported by the fact that usually the ship's hull itself is mono color with very little variation and so corrosion spots are easily visible.
As the R, G, B (Red, Green and Blue) color values are prone to lightning variation, they are converted into HSV (Hue, Saturation and Value) color space, for each 3D point.This conversion separates the color component from the intensity component.Also, the intuitiveness of the HSV color space is very useful because we can quantize each axis independently.Wan and Kuo (Wan and Kuo, 1996) reported that a color quantization scheme based on HSV color space performed much better than one based on RGB color space.The component, invariant to the lightening conditions, is then analyzed.It is referred to in this paper as the color component as it provides more stable color information.Based on the description presented in (Hughes et al., 2013), the following equations were used for the conversion.2).Once the upper and lower bounds are determined, the 3D points belonging to the corroded region are segmented using (3), as shown in Figure 4. (1) Here Pi is the i th 3D point in the selected zone while P c i value of its color component.H c max is the value corresponding to the maximum peak in the distribution, while H c bin i is the bin size of the i th peak for the color component c. m + and m − are the number of peaks considered in the cutoff region along the +ve and −ve directions as shown in Figure 3.
The 3D points belonging to the corroded region are then clustered for further analyses as explained in Section 5..

Threshold based Detection
If the selected zone is heavily corroded as shown in Figure 2(b) then the histogram based method is not useful as the predominant background (i.e.non corroded area) is not readily available and we cannot base our model on the dominant corroded zone due to a lot of variation in the color components.Thus, we employ an adaptive threshold based method in which a smaller sample volume is used to calculate the upper and lower bounds B c U and B c L respectively.This smaller sample volume of any non-corroded region can be selected from within the selected zone or from any other part of the hull as the ship hull is mainly the same (material and paint color, etc.) all around.The effect of illumination variation on the color values of 3D points on different parts of the hull due to overlaying shadows, reflections, etc., is already catered for by separating the intensity and the color component by converting into HSV color space and using only the color component for the analysis.Once B c U and B c L are calculated using ( 4)&( 5), the 3D points belonging to the selected zone are segmented using (3). (5) Here P c s is the mean value of the color component c of all the 3D points in the sample zone, while σ c s 2 is the variance respectively.
The 3D points belonging to the corroded region are segmented out as shown in Figure 4 and further analyzed as explained in the next section.

ANALYSES AND ESTIMATION OF THE CORRODED REGION
In order to analyze the corroded region, the 3D points belonging to these regions are first extracted out (as shown in Figure 4) and then clustered together using a k-means clustering algorithm as presented in (Zhou and Liu, 2008).The initial k clusters are selected randomly and the algorithm minimizes the dissimilarity measure between all 3D points and their respective associated cluster centroids.It uses the objective function defined as: where P j i − C j 2 is a chosen distance measure (Squared Euclidean distance) between a 3D point P j i and its associated cluster centre C j .This distance measures the dissimilarity of the 3D points to their respective associated cluster centers.Based on the clustered n 3D points, new centroids are estimated as the barycenters of the clusters and the process is repeated until the centroids seize to move.

Estimation of Corroded Region
Once the 3D points are clustered, the total volume and surface area of the corroded regions are determined.This helps in estimating the amount of work and material (such as paint) required for renovation/overhauling of the ship's hull.
As the clusters formed of different corroded regions are of varying shapes and sizes, it is difficult to calculate the exact surface area and volume.Hence, we use a finite element method by dividing these regions into finite number of very small 3D voxels (Aijazi et al., 2013).Although the maximum voxel size is limited to 5 cm 3 , the actual voxel size varies depending on the maximum and minimum values of the constituting 3D points along each of the three axes as shown in Figure 5.This way, a certain corroded region may be divided into tens or hundreds of smaller voxels depending upon the density and size while the adaptive voxel sizes ensure that the profile of the region is preserved.The volume is then simply calculated as the volume of the bounding cuboid (refitted/adapted voxel).The total volume is then the sum of all individual voxels as shown in (11).
In order to calculate the surface area, an arbitrary plane is determined by calculating the best-fit plane through the 3D points contained in the refitted/adapted voxel using planar regression of data as presented in (Fernández, 2005).A best-fit plane is defined with the equation: where x, y, and z are the respective mean values of X, Y , and Z coordinates of all points.To find the equation of the best-fit plane for a given set of points, Press et al. (Press et al., 2007) present the following equations that are solved for B and C: The result of the regression is a plane that passes through a point with coordinates (x, y, z) and is returned in the form of a vector normal to the best-fit plane.The equations in (Press et al., 2007) are corrected to deal with traces/residue, by replacing ( 7) with the following definition: and modifying ( 8) and ( 9) accordingly.
These 3D points contained in the refitted/ adapted voxel are then projected on this plane as shown in Figure 5 and area is calculated as the area of the bounding rectangle.The total surface area is again the sum of individual areas as presented in ( 11) and ( 12).
Here N is the total number of voxels, X, Y , Z and X P , Y P are the 3D and 2D-projected coordinates of the points respectively.As the voxel sizes are very small, the volume and surface area estimations of the corroded region, V olcorr and Areacorr respectively, are quite accurate.

EXPERIMENTS, RESULTS AND EVALUATION
The proposed method was evaluated on real data.700 × 10 6 3D points were obtained after scanning a ship (120 m long and 20 m wide) using Leica's P20 laser scanning station as shown in Different zones on the ship's hull were selected and the corroded regions were detected and then analyzed using our method.Some qualitative results are presented in Figure 6.The figure shows that the method is able to successfully segment out most of the corroded regions.As the method is non parametric (does not rely on particular shapes and sizes) it is able to segment corroded regions of different shapes and sizes.For quantitative results, some of these zones were manually segmented to obtain ground truth.The detection results are presented in Table 1 using different standard evaluation metrics as described in (Vihinen, 2012).
The analysis was conducted with 3D points.
Although, all these metrics are commonly used to evaluate such algorithms, MCC (Matthews Correlation Coefficient) is regarded as most balanced measure as it is insensitive to different class sizes (like in our application sometimes the number of points belonging to the corroded regions are significantly less or more than that of the non-corroded regions).
The MCC, like the other measures, is calculated based on the count of the true positives (i.e.correct detection of 3D points belonging to corroded region), false positives, true negatives, and false negatives.A coefficient of +1 represents a perfect prediction, 0 no better than random prediction and −1 indicates total disagreement.The detailed results including overall accuracy ACC and MCC greater than 88% and +0.6 respectively, clearly demonstrate the efficacy of the proposed method.It is also observed that the PPV is generally found higher than NPV which suggests that the method is more conservative and is more likely to detect corroded zones once highly certain otherwise it does not.Hence, false positives are usually less than false negatives.
The estimation of V olcorr and Areacorr by the proposed method was also evaluated.Estimated values for five different zones were compared with the corresponding ground truth.The ground truth was obtained by manual labeling of corroded zones followed by Volume and Surface Area calculation based on the equal distribution of 3D points using a 3D CAD software.This supposition of equal distribution is justified by the fact that the sizes of the selected zones were relatively small and the position of the scanner was fixed.This ensures that there is minimal to no variation in the point density in the selected zone.The results are presented in Table 2.The results show a relatively small error which demonstrates the efficacy of the method.This error difference between the estimated and the ground truth value also includes the error due to some False Positives FP (zones wrongly detected as corroded zones) as well as False Negatives FN (corroded zones not detected).
We also evaluated the effect of registration errors in the estimation of V olcorr and Areacorr.For this analysis, multiple scans of a particular zone were registered with small but different registration errors.The results, presented in Table 3, show that registration errors play an important part in the detection and analysis of corroded regions as this effect the estimation of the dimension and magnitude of the corroded regions.This may in turn result in higher or lower estimation of repair costs (in terms of material required such as paint, man hours and other processes, etc.).

CONCLUSION
In this work a new method for automatic detection and analysis of surface defects such as corrosion spots of different shapes and sizes, on large ship hulls is presented.In the proposed method multiple scans from different positions around the ship are registered together to form a complete 3D point cloud.The R, G, B values associated with each scan, are converted into HSV space to separate out the illumination invariant color component from the reflectance intensity.Using this color component, different surface defects such as corrosion spots of different shapes and sizes are automatically detected, within a selected zone, using two different methods depending upon the level of corrosion/defects.These detected corrosion spots are then analyzed and accurately quantified.The different aspects of the method are thoroughly evaluated on real data using different standard evaluation metrics and the results clearly demonstrates the efficacy as well as the applicability of the proposed method.
This method not only helps to increase the reliability but also the accuracy of the detection and estimation of these defective regions, as compared to manual inspection that is currently the standard practice.As a result, this could definitely lead to better estimation of cost and optimization of different repair and maintenance processes in the shipping industry.
Figure 1.(a) shows the complete 3D point cloud of the ship and the docking area before the filtering phase.Each 3D point is coupled with the corresponding R, G & B value.(b) shows the 10 different scanning positions (and different viewing angles), of P20 laser scanner, used to scan the complete ship.

Figure 2 .
Figure 2. The two images show the zones selected by the user.(a) shows a zone with minor corrosion whereas (b) shows a zone with dominant corroded area.
M AX, and V = M AX Here M AX = max(R, G, B), M IN = min(R, G, B), δ = M AX − M IN and H, S, V are the corresponding point of R, G, B, in the HSV space.Also, to be noted that the normalized values of R, G, B are used, i.e.R, G, B ∈ 0 . . . 1 , and so as a result H ∈ 0 . . .360 • and S, V ∈ 0 . . . 1 .In case of R = G = B = 0, H is undefined, hence it is assumed to be −1.After the conversion, the color component is then used in our analysis.Two different techniques are proposed to detect corrosion spots depending on the type of zone selected.They are explained as follows.4.1 Histogram based Detection If the selected zone has a majority of non-corroded area as shown in Figure 2(a) then a histogram distribution based method is used.Similar to 2D image segmentation, the larger non-corroded surface is considered as background with corrosion spots as foreground.A histogram is obtained for each channel of the color component in the HSV space.Based on these histograms upper and lower bounds for the color component (c = {H, S}) of the background region (i.e.non corroded region) B c U and B c L respectively are automatically calculated.Based on the distribution, the B c U and B c L are calculated by analyzing the dominant peaks as shown in Figure 3. Centered on the highest peak, only the peaks with more than 50% of this maximum height are considered for the determination of the cutoff region (1)&( Figure 3. (a) and (b) show the histogram distribution for H and S respectively for 3D points of a selected zone.Based on the dominant peaks, B c U and B c L are automatically selected.
Figure 4. (a) shows the colored 3D point cloud of a small part of a selected zone.(b) shows the corroded region segmented out after detection while in (c) we find the 3D point cloud with the corroded region extracted out.

Figure 5 .
Figure 5. Estimation of V olcorr and Areacorr for the 3D points of the corroded part contained in each adapted Voxel.
Figure 1.Multiple scans were obtained from ten different locations, all around the ship, and then registered together as explained in Section 3..
Figure 6.(a), (c), (e) & (g) show the colored 3D point clouds of different zones while (b), (d), (f) & (h) present the 3D point clouds with the corroded region segmented out along with the corresponding V olcorr and Areacorr respectively.

Table 1 .
Detection results using different standard evaluation metrics.

Table 2 .
Evaluation of volume and surface area estimation.

Table 3 .
Effect of registration errors in the estimation of V olcorr and Areacorr.