DEVELOPMENT OF A STEREO-LASER-PROFILE-SYSTEM FOR THE OPTICAL INSPECTION OF WELDING SEAMS

This research project focuses on the development of an optical 3D measuring system that enables high accurate surface measurements of welding seams in order to detect impurities. The systems concept is based on a stereo camera system in conjunction with a projecting line laser. A second camera system is used as tracking or positioning component to obtain the position of the measuring systems in object space. The complete stereo laser-profile system will be used as a hand-held system. The development, optimization and testing of the system components (stereo camera system with projecting laser line and tracking component) for surface measurements as well as calibration and accuracy evaluations are the main objectives within this research project. Testing procedures and probes are constructed and evaluated to verify the results. The development considers conditions for a future adaption to underwater use.


INTRODUCTION
The increasing construction of off-shore building sites like platforms, ports and wind power plants as well as installations in harbours and industry require further developments in terms of underwater welding and inspection techniques.The 3D acquisition of welding seams is needed for documentations and quality control when instantiating repairs and recurring inspections.
Up to now no optical assisted testing systems are known for the automatic detection of the surface topography and geometry of a welding seam in underwater use.However, different investigations and developments of 3D measurement techniques in underwater conditions are given by Höhle (1971), Kotowski (1987) or Korduan et al. (2003).Harvey et al. (2003), Korduan and Lämmel (2004) and Shortis et al. (2000) report on investigations of 3D-stereo-measuring systems for underwater conditions but focussing on lower accuracy requirements.
In order to provide high measuring sensitivity and reliability new developments are necessary to create a suitable assisted and automated testing technique with respect to the measuring task and its corresponding conditions.Due to the targeted underwater application high demands in terms of stability, accuracy and multi-media model exist for the measuring system.Extended mathematical models have to be developed for optical imaging and 3D reconstruction for a system that allows for multi-media optical interfaces.Kotowski (1987); Maas (1995) and Mulsow (2010) discuss multi-media approaches and their application to high-precision measurement.
The desired measuring concept consists of a system for 3D measurement of welding seams based on a multi-camera set-up with projected laser lines (Figure 1).One part of the system is composed by the measuring unit of the laser profiles while a second component is used to position the device, i.e. to provide the absolute orientation of single profiles within a common coordinate system.The positioning device can, in principle, be utilised by mechanical means as given by Vilaca et al. (2006) and Klimentjew et al. (2010) or, as done in this case, be provided by an additional camera system.In principle, the welding seam geometry can be measured by a laser line sensor consisting of one camera and a laser plane.The method is widely used and well known (Fraunhofer, 2008). Mc Ivor (1999) discusses calibration possibilities for laser line sensors.One benefit of such systems is given by the fact that they can be configured individually.As a drawback system calibration under water is not trivial.Since the measurement principle does not allow for redundant information, errors of laser plane calibration directly affect the measurement result without control.In addition, occlusions can occur for the camera.
Main objective of the project is the investigation of different concepts for camera and laser arrangements according to given specifications.As a result, a stereo camera system with laser line projection has been defined.The advantages of such a setup are given by higher redundancy, better resolution and accuracy, and the use of standardized photogrammetric calibration methods.The current research is based on extended experiences in camera technology and calibration (Luhmann, 2010), and in modelling and calibration of 3D measuring devices as they will be applied for this work (Hastedt et al., 2005;Luhmann et al., 2008).

SYSTEM CONCEPT
In order to reconstruct the surface geometry a stereo camera system with a projecting line laser (stereo laser-profile system) will be developed.In order to define the orientation of each 3D laser profile in a global object space an optical tracking or positioning component is added to the systems concept.This enables the definition of the surface in three-dimensional object space.Firstly, the concept will be investigated and a demonstrating system (demonstrator) will be developed.The demonstrator including the tracking or positioning component will be evaluated under normal conditions.Secondly, in further project stages the stereo camera system will be modified and evaluated for underwater use.
The systems accuracy mainly depends on the resolution of the cameras (defining the amount of measurable points in object space) and the accuracy of the positioning component.In addition, factors like the configuration of the stereo camera and line laser, the measuring strategy of the profile lines and the quality of the systems calibration have to be considered.
A high measurement resolution in the order of 1/10 mm shall be achieved by the introduced stereo laser-profile system.In the ideal case welding pores and cracks can then be detected.The wider objective is to provide a documentation of the surface topography of the welding seam and to enable a subsequent visual testing of the 3D surface.

Stereo laser-profile system
In order to measure the surface geometry a stereo camera system is combined with a projecting line laser.The line laser is used as structural light and therefore, in contrast to its function within a laser triangulation system, the projected laser plane does not have to be calibrated.Within the stereo images homologous points representing the projected laser line are automatically detected.The resulting 3D profile based on the homologous points is calculated by spatial intersections.The six degrees of freedom (6DOF) that define the three-dimensional orientation of each 3D profile in object space are determined by the tracking component.
High accuracy requirements with respect to the surface topology of a welding seam induce alike requirements to the stereo camera system.A high resolution of the cameras is needed as well as a minimum measuring frequency of 20 Hz to allow for an adequate scanning speed.Addressing the future underwater use the working distance of the measuring head has to be minimised.This helps to minimise the thickness of water and therefore unpredictable circumstances, e.g.caused by turbulences.The shorter the working distance the higher the achievable resolution in object space.Additionally a convergent arrangement of the cameras (30°) is applied to minimise shades on the laser line due to buckles and holes on the surface.
The measuring task requires a high qualitative line laser optics.
An even, homogenous brightness on the length of the laser line and a Gaussian distribution of the lasers brightness in its crosssection is necessary for the laser projection.Favorable for underwater purposes is a high laser power in order to compensate for a loss of brightness due to the medium water and containing particles.
As reported by (Bass et al., 2009, Hildebrandt et al., 2008) the wavelength of the line laser should be in the green spectral band.Hence, it is a good compromise between high light energy and absorption effects of the water caused by suspended loads and dissolved solids.

Positioning component
The positioning component can either be realized using a single or a multi-camera system.In any case the position is estimated by means of a fixed object reference.Different alternatives for positioning or tracking are conceivable.
As a first alternative an optical tracking solution is applied.A probe is mounted on top of the stereo laser-profile system which is observed by an additional tracking system.The 3D profiles acquired by the stereo laser-profile system are then oriented in object space (Figure 2).As a second alternative a positioning solution is used.The stereo laser-profile system and the positioning component are physically combined to a measuring head.The orientation is calculated through an object reference (Figure 3).

MEASURING CONCEPT AND SYSTEM CALIBRATION
The calibration of the stereo laser-profile system and the tracking and positioning component follows a combined calibration procedure.In a first step all interior camera orientations are calibrated beforehand by using a photogrammetric standard calibration procedure.The relative orientation of the system is calculated by using a calibrated reference and the corresponding interior orientations.In the following step the combined calibration of the stereo laserprofile system and positioning component is estimated.This calibration procedure differs with respect to the applied tracking-alternative.

Tracking alternative no. 1
For this approach the stereo laser-profile system respectively its probe on top of it is tracked by a second camera system (tracking component, Figure 2).The fixed probes and targeted points need to be estimated within the local coordinate system of the stereo laser-profile system.The calibration of the probe is done within a bundle adjustment using a Nikon D2X camera with a 24mm lens.This bundle covers the stereo laser-profile system, its probe and the reference body represented by target points.Synchronously the stereo laser-profile system acquires the reference body, too.The probe coordinates are estimated within a bundle adjustment using AICON 3D Studio.Subsequently they are transformed to the local coordinate system of the stereo laser-profile system based on the observations of the reference body.The probe coordinates result with a precision of RMS XYZ = 0.004 mm (1σ).
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey The final measurement of the welding seam is done within the coordinate system of the tracking component.

Tracking alternative no. 2
For the second approach the stereo laser-profile system and the positioning component are physically assembled to a combined system building the measuring head.Then the positioning component acquires a fixed object reference (Figure 3).The 6DOF orientation in object space is provided by space resection, thus the measured 3D profile is also given in object space.In this case the positioning component can either be formed by a multi-camera system or a single-camera system.Using a single camera a suitable configuration of the object reference has to be chosen in order to gain a straightforward result (Luhmann, 2009).In order to calibrate the measuring head respectively to estimate the orientation of all optical components of the measuring head, both systems synchronously observe a calibrated object reference.The relative orientation is estimated within a bundle adjustment of different perspective images by using fixed precalibrated interior orientations.If the systems components (positioning and measuring cameras) do not provide a corresponding field-of-view, a suitable volumetric calibration body could be used.

LASER PROFILE ESTIMATION
The 3D point estimation of the laser profile is done by extracting corresponding points in the normal case images of the measuring system.The normal case images are calculated with calibrated interior and exterior orientations of the system.Different possibilities for finding homologous points are offered by the large variety of image processing algorithms.In the following two methods will be introduced that are used for these investigations.
The resulting 2D profile based on the homologous points in image space is calculated by spatial intersection in order to give the 3D profile in the coordinate system of the stereo laserprofile system.In order to provide the profile geometry within a common coordinate system a transformation is used with respect to the chosen tracking alternative.

Sub-pixel estimation of local extrema
For standard laser triangulation sensors usually the middle of the projected laser line is requested.This part of the line represents the laser plane of the triangulation sensor in the image.For the image measurements the brightest pixels (Figure 4a) or the edge pixels (Figure 4b) estimated by edge detectors can be used as starting values.After calculating the grey value gradients a curve (parabola, Gaussian function, etc.) or a centroid is determined through the starting value (R 0 ) and its neighbor pixels (R -1 , s +1 ).In order to get the sub-pixel position the distance d between R 0 and the maximum of the interpolated function is estimated (Figure 5). Figure 4b shows the detected edge pixels that can be used to calculate the centre of the projected laser line.

Least-squares matching
As an alternative to sub-pixel localisation of local profile extrema a least-squares matching (LSM) approach is used (Bethmann and Luhmann, 2010).A best fit between two image patches is calculated along the corresponding epipolar line (Figure 6).Since LSM is processed for normal case images the parameters of affine transformation for y (b 0 , b 1 , b 2 ) are not introduced as unknowns for the adjustment.Otherwise matching of linear structures in y direction would not be reliable.Thus the simplified functional model is given:

EXPERIMENTAL TESTS
The first tests are evaluated by using existing stereo-camera systems of AXIOS 3D which are designed for optical tracking and positioning in three-dimensional object space (Axios 3D, 2013).The camera systems are pre-calibrated and use an infrared band-pass filter of 810 nm.Therefore infrared line lasers are used that work within the same wavelength range.
The cameras of the stereo system operate with VGA resolution, have lenses with an 8 mm focal length and are arranged with different base lengths.

Quality of the laser line image
Depending on the line laser lens and the chosen camera settings the quality of the laser line image varies significantly (Figure 7) and therefore its measurement results.Figure 7a shows a grainy structure, the so called laser speckle or speckle granulation.A reduced speckle noise and more homogeneous structure within the line laser image is shown in Figure 7b.The speckle noise is a natural feature of the laser light.If the laser light (coherent light) hits an uneven surface and the magnitude of the surfaces' roughness is within the range of the laser's wavelength interferences occur.These interferences lead to an extinction or gain in the light wave.The speckle noise then results from these interferences.Observing a laser line from different perspectives, as it is given by using a stereo-camera system, the speckle noise is presented with different characteristics (Meschede, 2008).

Verification of the laser line measurement
In order to verify the measured profiles and surfaces a reference artifact consisting of two parallel height levels is used (Figure 8a).This allows for a verification of the laser line image on an even surface by comparing the results with the reference data.
In the following some practical results are presented.Within the tests the distance (step height) between the two levels is measured.The tests additionally allow for a verification of the profile measurement quality.In order to eliminate the influence of the laser image a narrow wrapper is stucked onto the reference body in order to simulate an optimized laser line (see Figure 8b).Using this practical arrangement the algorithms can be tested, too.The nominal step height is specified to 19.05 mm ± 0.02 mm.The expected accuracy in object space is estimated to 0.044 mm (2σ) by assuming an image measurement precision of 1/20 pixel of the used camera system.
One measurement solution can be obtained by applying the approach of local extrema.The Canny algorithm is used to identify the edge pixels within the stereo images (see Figure 4b).At both sides of the line a sub-pixel interpolation is implemented by using the two neighbours in image row.
Building the mean of all sub-pixel values the centre of the wrapper is estimated.Subsequently the homologous image points are used for a spatial intersection.Then the step height (S 1, S 2 ) is determined through three adjusted lines defined by the 3D point cloud (Figure 9).The analysis by using the local extrema approach results to 19.001 mm.
Figure 9. Determination of the step height Alternatively a least-squares matching approach is applied to the test images for which different patch sizes are used (Figure 10).
(a) (b) (c) Figure 10.Section of a test image with different patch sizes for LSM (a) 14 x 2 pixels (b) 14 x 8 pixels und (c) 14 x 14 pixels The determination of the step height is equivalent to the local extrema approach.

Verification of tracking
In order to evaluate the accuracy of a measured single point with respect to the two introduced tracking alternatives different camera configurations are tested in practise.For this purpose a fixed circular target is observed by moving the measuring head.
Tracking the measuring head simultaneously the resulting 3D ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey coordinates in object space can be compared or be compared to their reference value In the following the spans of measurements results of point P and their standard deviations (precision) for tracking approach 1 are reported.In contrast, for tracking approach 2 differences between measured and nominal values are given since P is known in that case.Here minimum and maximum deviation are calculated and the resulting standard deviation as well.Remaing systematic deviations can be observed in the order of 0.4 to 0.8 mm which have still to be analysed.
Tracking-alternative 1(a).For the first practical scenario an AXIOS CamBar B2M is used as tracking and measurement component each.The CamBar B2M system has a base of 200 mm with a minimum operating distance of 800 mm.The measuring camera is tracked by the tracking component by observing its probe on top (see Figure 11).Table 2 summarizes the results.The results show relatively high spans with respect to the 3D comparison of up to 3 mm.Inhomogeneous standard deviations for the three coordinate components can be expected as they are dependent on the probe geometry and mutual orientation of the camera components.
Tracking-alternative 1(b).The first practical scenario (1a) is repeated by using a CamBar B1 as measuring component.The CamBar B1 forms a base of 60 mm with a minimum operation distance of 200 mm.Combined with a CamBar B2 as tracking component this meets the characteristics of the system developed within this project.
12 measurements X Y Z span in comparison 0.38 mm 0.58 mm 0.54 mm standard deviation 0.13 mm 0.15 mm 0.17 mm The results of test 1b (see Table 3) point to an improvement that can be identified to the more compact system structure, the resulting higher accuracy and a better height-to-base ratio.
Tracking alternative 2(a).For the second practical scenario the tracking alternative no. 2 is applied.This test uses the same components as test 1a (Figure 12).Compared to the results of tracking alternative no. 1 explicitly better results could be achieved.One reason can be found within the object reference.In proportion to the probe the object reference dimension is much larger and therefore the measurements are based on a higher amount of reference points.
A second reason is given by the advantageous error propagation of the tracking component due to the physically docked system.

Tracking alternative 2(b).
The second practical scenario (2a) is repeated by using a CamBar B1 or either the self-developed system consisting of two Basler ace cameras (BasProject) as measuring component ( Even though the chosen configuration of this test (2b) is adverse the results show an accuracy level comparable to test 2a.The relative orientation is estimated with a five times worse precision.By optimizing the configuration and the object reference a higher achievable accuracy level is expected for the relative orientation as well as for the point measurement quality for further tests.
Due to a more compact system concept and easier handling with respect to its further underwater use, tracking alternative no.2 is applied to the demonstrator.The presented results give information about the achievable accuracy level of a single point measurement.With respect to future work using a laser line, a decrease in accuracy can be expected because of a lower ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey imaging quality of the laser projection and the resulting lower measurement quality.

OUTLOOK
The demonstrator which is currently under development consists of two synchronized cameras with high resolution (2048 x 2048 pixels) and a suitable line laser (higher laser power, wavelength 532 nm).After intensive testing and optimization the system will be further developed and modified for underwater conditions.
As a key research issue the image analysis algorithms for the detection of corresponding laser line points have to be tested and optimized in terms of accuracy and robustness.The tracking approach 2 based on two physically combined camera systems will further be investigated using an adapted reference body.

APPENDIX
This project is funded by the German Federal Ministry of Economics and Technology in cooperation with the German Welding Society project funding number IGF 17.333N / DVS V4.005.

Figure 1 .
Figure 1.Projected laser line on a welding seam (hollow seam)

Figure 3 .
Figure 3. Schematic illustration of tracking alternative no. 2 Example image with projected laser line: (a) middle of laser line (b) edge pixels estimated with Canny detector

Figure 5 .
Figure 5. Principle of sub-pixel interpolation with three points After determination of sub-pixel coordinates corresponding points in the stereo images have to be identified.Since the images have been rectified to normal case images corresponding epipolar lines are horizontal and homologous points are detected in parallel image lines under consideration of a possible y shift of the principal point.

Figure 6 .
Figure 6.Principle of correspondence search for LSM Laser line image with (a) obvious speckle noise and (b) reduced speckle noise Figure 8.(a) Reference body with two parallel arranged levels and (b) section of an image illustrating a fixed wrapper

Figure
Figure 11.Tracking-alternative no.1: principle of single point measurement 34 measurements X Y Z span in comparison 2.22 mm 2.83 mm 2.91 mm standard deviation 0.50 mm 0.81 mm 0.62 mmTable 2. Accuracy of a single point measurement by applying tracking-alternative no. 1 (measuring and tracking component: CamBar B2M)

Figure 12 .
Figure 12.Tracking alternative no.2: principle of single point measurement With this scenario the measuring and tracking components are physically attached to each other.The measuring camera observes the fixed circular target whereas an object reference (1 m x 0.8 m x 0.5 m) is measured by the tracking system.The coordinate of point P is known, therefore a 3D comparison between reference and measured value can be determined.Table4summarizes the results.

Table 1 .
Table1shows the comparison of the different results.All solutions result within the estimated accuracy of 0.044 mm (2σ).Comparison of the results of step height determination

Table 4 .
Table 4 summarizes the results.Accuracy of a single point measurement by applying tracking alternative no. 2 (measuring and tracking component: CamBar B2M)