HYPERSPECTRAL REFLECTANCE SIGNATURES AND POINT CLOUDS FOR PRECISION AGRICULTURE BY LIGHT WEIGHT UAV IMAGING SYSTEM

The objective of this investigation was to study the use of a new type of a low-weight unmanned aerial vehicle (UAV) imaging system in the precision agriculture. The system consists of a novel Fabry-Perot interferometer based hyperspectral camera and a high-resolution small-format consumer camera. The sensors provide stereoscopic imagery in a 2D frame-format and they both weigh less than 500 g. A processing chain was developed for the production of high density point clouds and hyperspectral reflectance image mosaics (reflectance signatures), which are used as inputs in the agricultural application. We demonstrate the use of this new technology in the biomass estimation process, which is based on support vector regression machine. It was concluded that the central factors influencing on the accuracy of the estimation process were the quality of the image data, the quality of the image processing and digital surface model generation, and the performance of the regressor. In the wider perspective, our investigation showed that very low-weight, low-cost, hyperspectral, stereoscopic and spectrodirectional 3D UAV-remote sensing is now possible. This cutting edge technology is powerful and cost efficient in time-critical, repetitive and locally operated remote sensing applications. * Corresponding author.


INTRODUCTION
The unmanned aerial vehicle (UAV) based remote sensing with low-weight imaging systems offers low-cost and flexible tools for the agricultural applications.Based on the precise measurements of energy emission and reflection from the vegetation, a wide range of variables that affect the crops can be monitored, such as soil moisture, surface temperature, photosynthetic activity, and weed or pest infestations.This information is of increasing importance to ensure the cost-efficiency of the agricultural production, for the harvest forecasts and from the wider perspectives of the climate change mitigation and adaptation, and the environmental sustainability.
An information system for the crop production is shown in Figure 1 (Saari et al., 2011).The UAV imaging service providers collect and process the images of the crop fields of farmers.The images and the interpretation results are stored on servers of a farmer service provider company.The results are utilized by the farmers in order to enhance the production, such as precision farming tasks and production planning, and also by the food industry for instance for the forecasting and traceability purposes.Our investigation emphasizes the information flow 3 in Figure 1 by providing information of the state of the crop, which are required in precision farming tasks.In Finland, the optimization of fertilizers and pesticides are expected to be the first operational applications of the technology; the time window for the UAV data collection is two weeks and the maximum allowable processing time is one week.
A new method for the UAV based agricultural applications is under development.The method utilizes a novel Fabry-Perot interferometer (FPI) based light weight hyperspectral sensor developed by the VTT Technical Research Center of Finland (Unmanned Aerial System Innovations, UASI) (Saari et al., 2011).The sensor is operated using a UAV having the maximum payload of 1 kg.A high-resolution camera is operated from the same platform for the purposes of georeferencing and digital surface model (DSM) generation.The dense image matching methods and quantitative processing of image radiometry are recent important innovations in the airborne photogrammetry and have also potential with UAV imaging (Scholten and Wewel, 2000;Honkavaara et al., 2012aHonkavaara et al., , 2012b;;Leberl et al., 2010;Hirschmüller, 2011;Rosnell et al., 2011Rosnell et al., , 2012)).Accurate point clouds and DSMs together with hyperspectral reflectance signatures will be input for a support vector regression (SVR) based estimation method, which will provide biomass estimate maps for agricultural applications.
The advantages of the frame images are the strong constraints provided by the rigid rectangular image geometry and multiple overlapping images (Leberl et al., 2010).This is important in particular in the UAV applications, which typically utilize images collected in dynamic, vibrating and turbulent conditions.Typically, hyperspectral sensors are line scanners requiring accurate GNSS/IMU observations for georeferencing; this technology cannot be operated currently reliably from lowweight UAV platforms (Nagai et al., 2009).

6.
Knowledge on the validity of the EU support conditions 6.

Support for product development INFORMATION SERVICES/ FARMING PROGRAMS
Figure 1.The UASI concept for the application of hyperspectral and high resolution false color imaging in crop production.The information flows are marked with numbers 1-6 (Saari et al., 2011).
The use of UAVs in precision farming tasks have been successfully demonstrated (Lelong et al., 2008;Berni et al., 2009;Hunt et al., 2010), but the existing methods do not utilize rigorously the full potential of the photogrammetric data.For example, the utilization of the point clouds is still limited and the methods for the rigorous radiometric utilization of the multiple overlaps of rectangular UAV images are missing.Furthermore, the novel hyperspectral imaging technology provides new possibilities for the processes.
The objective of this investigation is to study the use of the UASI hyperspectral imager in the precision agriculture.We will go through the complete data collection, processing and analysis process for the precision farming.Our focus is in the development and assessment of methods for producing point clouds and radiometrically corrected images.We describe the proposed process and an empirical study in Section 2. We give the results in Section 3 and discuss our findings Section 4.

UAV imaging system
The Microdrones MD4-1000 quadrocopter UAV was used as the platform, allowing for a maximum payload of 1000 g.In this study, the major analysis with the UASI was carried out using five channels (central band width, full width of half maximum FWHM): G: 568.6 nm, 23.3 nm; R: 658.5 nm, 22.3 nm; Red edge (RE): 739.1 nm, 29.7 nm, NIR-1: 802.9 nm, 38.4 nm, NIR-2: 857.1 nm, 35.6 nm.These channels were considered as the most interesting for the biomass estimation process.More details of the sensors and the set-up are given by Saari et al., (2011) and Honkavaara et al. (2012b).

Flight campaigns
An empirical campaign was carried out at the MTT Agrifood Research Finland agricultural test site (N 60° 25' 21'', E 24° 22' 28'') on 6th July 2011.The area consisted of wheat and barley test plots where the seed and fertilizer amounts were varied to cause wide variability in vegetation; applied amounts can be seen in Figure 2. Ground truth for the campaign included a reference DSM collected by a tractor during the sowing, height measurements of the vegetation during the image campaign and physically measured dry biomass values of vegetation samples.
The reference targets for the imaging system included four reflectance reference tarps (P05, P20, P30 and P50 with nominal reflectance of 0.05, 0.2, 0.3 and 0.5, respectively), reference spectrums of six vegetation samples, a Siemens star and 11 ground control points (GCPs) (Figure 3).In-situ reflectance reference measurements were carried out using the ASD Field Spec Pro FR spectroradiometer and the measurements were normalized to a calibrated white 30 cm by 30 cm Spectralon reference standard from Labsphere.
Images were collected from the flight altitude of 140 m above the ground level, providing a ground sampling distance (GSD) of 3 cm for the GF1 images and 13 cm for the UASI images.Sensors were not operated simultaneously because of the lack of a suitable camera mount.In this study, a single image strip from both sensors was used; the forward overlaps were 81% and 91% for UASI and GF1, respectively.The length of the area was about 500 m and the width of the image strip was about 80 m with UASI and 120 m with GF1.Solar zenith and azimuth angles were 45º and 132º with the UASI and 38º and 166º with the GF1, respectively.The weather conditions during the campaign were fine with almost a cloud free sky and moderate wind; more details of the campaign are given by Saari et al. (2011) and Honkavaara et al. (2012b).

Data processing
Data were processed in the photogrammetric environment consisting of the Bae Systems SOCET SET photogrammetric workstation and in-house implemented components (see details in Rosnell et al., 2011Rosnell et al., , 2012;;Honkavaara et al., 2012b).

Orientation determination:
The self-calibrating bundle block adjustment method was applied to determine the orientations (geometric sensor calibration was not available).
Because the GNSS/IMU information provided by the UAVsystem was of the poor quality, some manual interaction was required in the determination of the approximate orientations of the images; the subsequent tie point measurement was fully automatic.Number of GCPs was 11 for the GF1 and 30 for the UASI (about 20 additional GCPs were measured from the GF1 images).The existing process flow was functional with the GF1.The challenge with the UASI was that the individual channels of the spectral data cube are collected in a short period of time (in this set up approximately in 1 s) and due to the platform movements, the images of different channels are not accurately overlapping, as is demonstrated in Figure 4.In this study, each of the five selected channels were oriented separately.

Point cloud generation:
Point clouds were extracted from GF1 images by novel dense image matching techniques using the Next Generation Automated Terrain Extraction software (NGATE).The point cloud was created with 10 cm point interval using the default NGATE strategy (ngate.strategywith a correlation window size of 5 × 5 pixels).

Radiometric processing:
The radiometry of passive imaging is influenced to a great deal by the sensor quality and stability and the atmospheric and illumination conditions during the image collection campaign.A further complication is the anisotropy of the object reflectance (Bidirectional Reflectance Distribution Function; BRDF) (Schott, 2007).In the previous study, a radiometric block adjustment method was developed to enable extraction of bidirectional reflectance factors (BRFs) from the images (Honkavaara et al., 2012b).In this study, the method was extended to carry out the BRDF correction.We used the BRDF model of Walthall (Walthall et al., 1985): where ρ is the reflectance, θ v is the viewer zenith angle, φ is the relative azimuth angle of the sun and the viewer, and a, b and c are the unknown model parameters.A least squares method was used to estimate the unknown parametrs.A grid of radiometric tie points was created in the object area and the object coordinates were projected to the images by taking the heights from the DSM and using the orientation information (details are given in Honkavaara et al., 2012b).The transformation from the DNs to the reflectance was determined by using the empirical line method.The average reflectance value in an object window of size 5 m x 5 m was used as the observation in the BRDF model estimation process and the view/illumination geometry was calculated in each image for the window center.It was considered that the use of 5 m x 5 m window size provides appropriate correction for the BRDF effects, but in the future the influence of window size will be investigated further.
Multiplicative BRDF correction was applied.
Finally, a hyperspectral orthophoto mosaic was calculated with a 0.20 m GSD utilizing the image orientations, DSM and radiometric correction parameters.

Biomass estimation using SVR
A SVR based machine learning approach (Drucker et al., 1997) is under development for the biomass estimation process (Figure 5).After the data collection and pre-processing (steps 1 and 2), features are extracted from the images and DSM (step 3), the training data are collected (step 4) and the SVR based estimates are calculated.The first experiments were carried out using simple features extracted from the five-channel multispectral images.The features (1-7) were: 1-5: average DN in G, R, RE, NIR-1 and NIR-2 channels, respectively, 6: NDVI ((NIR-2 -R)/(NIR-2 + R)), 7: simple ratio (NIR-2/R); in all cases the average DN in 1 m x 1 m window was used.The physical measurements of the dry biomass were used for the training of the SVR classifier.
5. The SVR based process for the biomass estimation.

Quality of images
The central image quality indicators influencing the application are the signal-to-noise ratio (SNR) of the images and the radiometric accuracy and stability of the sensor.Corners of the UASI images were black due to the lens-falloff effects and there appeared some periodic noise (Figure 4).The noise level was the highest in NIR-2 channel.The signal level of UASI images was relatively low because of the relatively high F-number (around 7.0) and the low quantum efficiency of the CMOS image sensor above 850 nm.The selected exposure time used for all wavelength bands was too short for NIR-2 channel.
When the signal level is low the detector related noise sources are dominating (Sandau, 2010).

Orientation processing:
The processing of the GF1 images was relatively efficient as was expected based on the previous experiences (Rosnell et al., 2011(Rosnell et al., , 2012)).The processing of the UASI images was more time consuming and the further complication was that the adjustment appeared to be quite unstable with self-calibration parameters; only the first order radial distortion parameter was used to model image distortions; in the future campaigns the sensor calibration should be carried out in a separate process.The root-meansquare values (RMS) of the residuals of the GCPs is typically a realistic estimate of the orientation accuracy; in the case of the UASI, the variation in RMS values of 30 GCPs was 0.21-0.45m in X, 0.22-0.33m in Y and 0.10-0.22m in Z in different channels.Planimetric and vertical accuracy of the georeferencing was estimated to be better than 0.2 m for the GF1 and 0.5 m for the UASI.

Point cloud generation:
The NGATE was capable of extracting point clouds from high quality GF1 images with one pixel point density and with a height RMSE of 10 cm as was expected based on the previous experiences (Rosnell et al., 2011(Rosnell et al., , 2012)).Examples of point clouds are shown in Figure 7.

Relative and absolute radiometric block adjustment:
There appeared radiometric differences in the subsequent UASI-images; the relative block adjustment eliminated these disturbances (Figure 8a) (Honkavaara et al., 2012b).

BRDF-correction:
The BRDF-effects were clearly visible in the left side of the image strip in the original image data (Figure 8b) (higher brightness in the back scattering area).
The BRDF-model parameters (Equation 1) could be estimated statistically significantly (values of parameters were significantly greater than their estimated standard deviations) and the parameters of different channels indicated some correlation with each other, which is the expected and desired behaviour (Figure 9a).The reflectance differences between the image center and the extreme backscattering angles were less than 0.05 on green and red channels, while on infrared they were 0.13-0.2;with 5° view zenith angle the differences were 0.01-0.05depending on the channel (Figure 9b).Plots of the reflectance observations and the corresponding modelled BRFs in the area of one image indicated similar behaviour (Figure 10).The correction eliminated efficiently the BRDF-effects (Figure 8b).1).b) Reflectance range (maximumminimum modelled reflectance) for the view zenith angels of 0-25º (maximum range), 0-10 º, 0-5 º.
Figure 10.Measured (left) and modelled (right) BRF in the area of one NIR-2 channel image; x-axis is the view zenith angle in degrees, y-axis is the relative azimuth and z-axis is the BRF.

Quality of the reflectance products and hyperspectral observations
The accuracy of reflectance images was assessed by using the field reflectance measurements as the reference.Differences were calculated for the tarpaulins, the vegetation and the bright tarpaulins (nominal reflectance 0.2-0.5).The following reflectance products were evaluated: 1) empirical line based method (el), 2) 1 + BRDF correction (el, brdf), 3) radiometric block adjustment with absolute and relative (imagewise) parameters (ba, abs, rel) (Honkavaara et al., 2012b), 4) 3 + BRDF correction (ba, abs, rel, brdf).Results are shown in Figure 11.In the case of tarpaulins, the best results were obtained without the BRDF correction, which was the expected behaviour, because the BRDF correction was based on the vegetated surfaces.With the vegetation, the BRDF correction improved the accuracy in many channels.The relative correction did not improve the accuracy with the reference targets.The reflectance accuracy was for the bright tarps on the level of 5% of the reflectance value; for the vegetation the values were clearly higher due to the larger variation of the object.This assessment shows that the methods are functional, but that the reliability and accuracy should be further improved.

SVR-based biomass estimation
The first SVR-based biomass estimation was carried out using the image mosaic without any radiometric corrections.The R 2 values of the estimation process were 0.56 which can be considered as a mediocre result (Figure 12).The important factors influencing the deviation include the quality of images, the object, the reference values, and the performance of the regressor.In this study, the two first causes are the most likely.First of all, the features used are sensitive to the relative radiometric differences of the images and the BRDF effects; these effects were not corrected in the test data.The possible issues with the object include the variability caused by the soil type, and furthermore, the features in the test data are quite small in size, so also the georeferencing errors can cause problems.The biomass estimate map (Figure 12) shows that in some cases high biomass estimates were obtained in the areas with strong backscattering even though the measured biomass reference values were low.In the further studies we expect that the results would improve, when using rigorous radiometric processing methods, by combining the spectral features with the 3D geometric features and by improving the regressor.A biomass estimate map computed using the image mosaic without any radiometric corrections; the amount of biomass is low in red and high in green areas.The grey dots show the locations of drybiomass measurements; the brighter the dot the higher the amount of dry biomass.

DISCUSSION AND CONCLUSIONS
We investigated the use of a new type of a UAV imaging system in agricultural applications.The system consists of a novel Fabry-Perot interferometer based hyperspectral camera (Saari et al., 2011) and a high-resolution small format camera Panasonic Lumix GF1.Both sensors provide stereoscopic imagery in a 2D frame-format and weigh less than 500 g each.A processing line was developed for the production of high density point clouds and hyperspectral reflectance signature mosaics.We demonstrated the use of this new technology in the agricultural biomass estimation process by using support vector regression (SVR) estimation method.
The 2D format images collected in an image block structure with large forward overlaps provide strong geometric and radiometric constraints, which is a key aspect for reliable as discussed also by Leberl et al. (2010).The results showed that the block adjustment approach was functional with the UASI system.
The data sets used in this investigation were collected under a sunny weather, providing a very high radiometric image quality with the GF1 while the UASI images had poorer SNR.Point cloud generation from the high quality image data collected by the GF1 was successful in the agricultural fields, which is consistent with the expectations (Hirschmüller, 2011;Rosnell et al., 2011Rosnell et al., , 2012;;Honkavaara et al., 2012a).The radiometric block adjustment and BRDF correction provided promising results and will be further optimized and also improved to allow reliable data processing in different conditions.The SVR-based biomass estimation is sensitive to the radiometric and geometric quality of the image block.In geometric terms, the accuracy of estimated canopy height is critical; this is dependent on the quality of the point cloud, which is discussed in details by Rosnell et al., (2012) and Honkavaara et al., (2012a).In radiometric terms, the image mosaic has to be relatively corrected, which requires balancing of individual images and BRDF-correction; in some applications also absolute correction to reflectance signatures is required.More testing in practical conditions is necessary in order to provide detailed requirements for the data quality.The presented method is expected to provide improvements to the existing approaches in terms of cost, accuracy, efficiency and information content (Lelong et al., 2008;Berni et al., 2009;Hunt et al., 2010).
The signal level was low especially in the NIR-2 channel.The hyperspectral imager prototype to be built for the year 2012 will have larger signal levels due to the lower F-number ( 2.8 instead of 7.0).It is expected that this improvement will help all the tasks in the UASI processing flow where image matching is applied.This is also very important in order to be able to operate the sensor in poor illumination conditions and to cope with shadows (Rosnell et al., 2011;Honkavaara et al., 2012a).
In the first experiments, only five channels of the hyperspectral data cube were evaluated.It is expected that the developed approach can be extended to all the other channels, and this is now under development.In the future studies we will emphasize the processing and utilization of the entire hyperspectral data cube to obtain more detailed information of the object.As soon as the functional methods are available, it will be necessary to optimize the processing times in order to meet the requirements for the data availability (Section 1).This article presents a new approach for the measurement of the geometric and radiometric properties of environment.The method is expected to be functional in wide variety of remote sensing applications, and we are investigating these possibilities.

Figure 4 .
Figure 4. Examples of UASI images (mirror images).From top left: image with green, red and NIR-2 channels and the corresponding individual images.

Figure 7 .
Figure 7. Example point clouds: points colour based on the height (left, top right) and a spectral reflectance with RE, G and R channels (bottom right).

Figure 11 .
Figure 11.Reflectance errors (RMSE % of the reflectance value) in tarps, vegetation and bright tarps for different radiometric correction methods (Section 3.4)