IMPACT OF UAV AND SENTINEL-2A IMAGERY FUSION ON VEGETATION INDICES PERFORMANCE

: Image fusion techniques can improve the quality of remote sensing images by combining high spatial resolution images with low spectral resolution images. This enhancement of the images may impact the performance of various vegetation indices (VI’s). This study investigates the impact of image fusion on the quality of vegetation indices by fusing UAV (Unmanned Aerial Vehicle) bands with Sentinel-2A images using Principal Component Analysis (PCA) and Brovery Transform (BT) fusion techniques. The fused images were used to calculate the Normalized Difference Vegetation Index, Normalized Difference Red Edge, Green Red Vegetation Index, and Normalized Difference Water Index. To assess the performance of the fused images, several image quality assessment metrics were used, including Root Mean Square Error (RMSE), Entropy, etc...The results showed that image fusion techniques can improve the quality of images which is important to assess crop health. The PCA image fusion technique showed higher quality than the BT technique. The PCA fused images had lower RMSE, ERGAS, and Entropy Difference and higher UIQI, CC, and SSIM values than the original images. Moreover, the fused images produced higher VIs values than the Sentinel-2A images. Finally, scatter plots were created to compare the correlation between the VIs calculated from the original and fused images. The results showed a strong correlation between the VIs calculated from the Sentinel-2A and fused images, indicating that the fused images can accurately estimate vegetation health parameters. Overall, this study demonstrates the potential of image fusion techniques to improve the quality of VI’s for monitoring vegetation health.


INTRODUCTION
Monitoring vegetation health and biophysical parameters is crucial for understanding ecological processes and managing natural resources, as vegetation plays a vital role in maintaining the balance of ecosystems.Remote sensing technology has greatly improved our ability to monitor vegetation through highresolution and spectral images of the Earth's surface (Almalki et al., 2022).Vegetation indices (VIs) derived from remote sensing images are widely used for characterizing parameters like leaf area index, vegetation cover, and photosynthetic activity.However, in order to ensure accurate monitoring of vegetation health parameters, the VI's need to be assessed accurately, which often requires high-resolution images (Guo et al., 2021).Obtaining such high-resolution images can be expensive and not economically feasible for such applications (Sishodia, Ray, and Singh, 2020).
UAV images typically have high spatial and low spectral information, while satellite images have high spectral and low spatial information.Fusing UAV and satellite images can result in high spatial and spectral information, which can be done more effectively for monitoring of vegetation health parameters (Li et al., 2022).
Image fusion refers to the process of combining high spatial panchromatic image with low spatial high spectral resolution images to create a high quality image (Prakash Ghimire et. al., 2020).Image fusion have been shown to enhance the spatial and spectral resolution of remote sensing images, which may improve the accuracy of VI's.Various fusion techniques have been proposed by researchers, PCA and BT have shown to improve the quality of remote sensing images (Prakash Ghimire et. al, 2020;Klonus and Ehlers, 2009).
Image fusion can lead to improvements in the quality of the image, but the fusion process may introduce some distortions or artifacts in the image, which depend on the fusion method used.Therefore, it is important to assess the quality of the fused images visually and statistically (Somvanshi and Kumari, 2020).Visual quality assessment of the fused images can be done by comparing the original and fused images using human perception (Han et al., 2013).Statistical method for assessing the quality of fused images by using various image quality assessment (IQA) metrics such as Root Mean Square Error (RMSE), ERGAS (Relative Global Dimensional Synthesis Error), SSIM (Structural Similarity Index Mapper), UIQI (Universal Image Quality Index), CC (Correlation Coefficient), and Entropy (H) (Klonus and Ehlers, 2009;Somvanshi and Kumari, 2020).RMSE, ERGAS, and SSIM quantify the differences between the original and fused images, while UIQI, CC, and H measure the amount of information present in the fused images (Somvanshi and Kumari, 2020).Various methods such as correlation analysis, coefficient of determination (R2), RMSE, scatterplots and hypothesis testing are available to assess the quality of VI's (Kamenova and Dimitrov, 2021;Ryu, Na, and Cho, 2020;Kong et al., 2021;Qin et al., 2021;Guo et al., 2021).A combination of these methods can be used to assess the performance of fused vegetation indices (FVI).This can help identify errors or biases in the data and improve the accuracy of vegetation monitoring and analysis (Kong et al., 2021).
The main objective of the study is to assess the quality of fusional images of the different band combinations of UAV and Sentinel-2A images and to assess the quality of VI's obtained from fused images for monitoring the crop health parameters.

STUDY AREA
The study area for the proposed work is located in Rentachinthala village, Guntur, India (Figure 1) laying between 79°33'35" E, 16°34'17"N and 79°33'42" E, 16°34'25"N, with an average altitude of 130m above sea level.The predominant soil type in the area is black cotton soil, and commonly cultivated crops are paddy, cotton, and chilli.

Data Collection
Sentinel-2A Level 1C imagery of the study area is collected for August 1, 2021, from the USGS GloVis (https://glovis.usgs.gov/).To capture crop information on the same day, aerial imagery was acquired using a DJI Phantom 4 V2.0 quadcopter equipped with a 20MP RGB camera.The UAV was flown at an altitude of 100m while maintaining a 70% overlap between the images.Flight planning is performed using the Pix4D Capture mobile application.The sensor characteristics of the UAV and Sentinel-2A imagery are presented in the

Data Pre-Processing
Sentinel-2A Level 1C products are orthorectified, meaning they have undergone geometric and radiometric correction to remove distortions caused by terrain and sensor effects.However, they still contain atmospheric effects such as scattering and absorption (Bui et al., 2022).To correct for these atmospheric effects, Sen2Cor algorithm is used to generate bottom-of-atmosphere reflectance data by enhancing the quality of data and the corrected image of the sentinel-2A is showed in Figure 1(a).
The aerial images captured by the UAV were initially calibrated and aligned based on the camera position and orientation of each image.A dense point cloud was then constructed from the aerial images, and this point cloud was used to generate a DSM (Digital Surface Model) and it is used to generate an orthomosaic image, using reflectance bands in the red, green, and blue wavelengths with a GSD (Ground Sample Distance) of 0.03meters.The study area is then extracted from the reflectance maps of the UAV and Sentinel-2A layerstacked image, which were used for performing the image fusion and VI calculations.

Image Fusion
In this study, two image fusion techniques, namely PCA and BT, were utilized to combine the red, green, and blue bands of the UAV and Sentinel-2A multispectral imagery.

PCA:
The PCA technique is a linear transformation technique that works by transforming the original bands into a new set of bands, known as principal components, have maximum variance and minimum correlation (Ghassemian, 2000).PCA is applied on UAV bands (R, G and B) and Sentinel-2A imagery to extract the spectral information from the images.The steps involved (Eq. 1 -5) in the PCA image fusion are: A. Construction of dataset matrix: a. UAV dataset matrix: Xuav (m x n) b.Sentinel-2A dataset matrix: Xs2a (m x n) where, m is the number of pixels and n is the number of spectral bands B. Normalizing the dataset matrices: Subtract the mean of each spectral band/feature from the corresponding dataset matrix.C. Calculating the covariance matrix of the combined dataset: D. Performing eigen decomposition: where, V is the eigen vectors and it contains the principal components as its columns, λ is the eigen values.

E. Selecting principal components:
The number of principal components to retain, k (k < n), based on the explained variance and Select the first k eigenvectors corresponding to the largest eigenvalues: F. Fusing the principal components from both datasets: G. Reconstructing the fused image: The first three principal components were selected from the UAV imagery, while the first three bands of Sentinel-2A imagery were used.The PCA components were then merged together to form a fused image.

Brovery Transform:
The Brovery Transform (BT) is a nonlinear transformation technique that works by enhancing the low-resolution image with the high-resolution image by adjusting the spectral values of the low-resolution image (Ghassemian, 2000).It is applied on UAV data (R, G, B bands) and Sentinel-2A imagery, which involved dividing each band of the multispectral imagery by the sum of all the bands, and subsequently multiplying it by the corresponding UAV band (Eq.6).The resulting panchromatic image was merged with the multispectral imagery to produce a fused image.
where, Snorm is the normalised pixel value of multispectral band of Sentinel-2A imagery and UAVnorm is the normalised pixel value of UAV imagery.Rescale the pixel values of each fused band between the original minimum and maximum values of the corresponding multispectral band from Sentinel-2A.

Image Quality Assessment
IQA of the fused image are assessed by using the quality parameters such as RMSE, ERGAS, SSIM, UIQI, CC and H and the equations of these parameters (Eq.7 -12) are listed below: where, M, N = row and column size of the image i, j = pixel index r = scale ratio of the image n = number of bands µ  = mean of the k th band between original image R, F = digital number of reference and fused image R ̅ , F ̅ = local means of the R and F σ R , σ F = local standard deviations of the R and F σ RF = sample correlation of R and F after removing their means C1, C2, C3 = small positive constants to make denominator as nonzero P(k) = probability of the occurrence of the image G = total number of grey levels

Vegetation Indices
VI's are useful for various remote sensing applications such as vegetation health monitoring, soil properties, etc.These VI's can be measured by using the spectral information from imagery.There are many VI's that have been developed for different applications, each with its own strengths and weaknesses.Some of the most commonly used VI's for measuring the vegetation properties include: a.The Normalized Difference Vegetation Index (NDVI) is a commonly used vegetation index that is calculated using the Near-Infrared (NIR) and Red bands of remote sensing data.The value of the NDVI ranges between -1 and +1 and the higher values indicate dense vegetation (Kong et al., 2021;Sagan et al. 2019).The formula for NDVI is: b. Normalized Difference Water Index (NDWI) is similar to NDVI, but it utilizes the Near-Infrared (NIR) and Short-Wave Infrared (SWIR) bands of imagery and it is useful for identifying changes in the water content of vegetation (Qin et al., 2021) and is computed using equation -14: c. Normalized Difference Red Edge (NDRE) is calculated using the Red Edge 1 (705nm) and NIR bands of Sentinel-2A data and it is sensitive to changes in vegetation structure and chlorophyll content (Kamenova and Dimitrov, 2021).The formula for NDRE is: d. Green-Red Vegetation Index (GRVI) is useful for detecting changes in vegetation biomass and it is calculated by using the green and red bands of remote sensing data (Yeom et al., 2019).The formula for GRVI is These vegetation indices are can be helpful for calculation of Sentinel VI's (SVI) and FVI to analyse the impact of fusion on FVI.

Image Fusion
Image fusion was performed between the UAV bands (R, G, and B separately) and Sentinel-2A multispectral image using the PCA and BT fusion methods.The fusional methods are generated the multispectral images with a GSD of 0.03m/pixel and are displayed in the true color composite (TCC) shown below:  2.
Table 2 shows the results of the image quality assessment based on various metrics for the PCA and BT fusion techniques applied to the Red, Green, and Blue bands of UAV imagery fused with Sentinel-2A imagery.The green band of UAV fused with Sentinel-2A (Green_Fusion) using PCA technique had the lowest RMSE value (75.635), suggesting that the fused image similar to the original image.However, the green and blue bands of UAV fused with Sentinel-2A (Green_Fusion and Blue_Fusion) using BT fusion had the highest RMSE values (241.958),indicating significant differences from the original images.
Green_Fusion using the PCA technique having the lowest ERGAS value (31.242%) which indicates the lowest error compared to the other combinations.In contrast, the Green_Fusion (99.927%) and Blue_Fusion (99.943%) using BT had the highest ERGAS values, implying the highest relative error.
The Green_Fusion (0.252) and Blue_Fusion (0.249) methods using BT fusion technique showed the lowest SSIM values, which indicates that they had the least structural similarity and the Green_Fusion using PCA technique had the highest SSIM value (0.869) compared with the original images.
The PCA fusion technique producing the highest UIQI values (0.915) for all bands of UAV fused with Sentinel-2A.In contrast, the Green_Fusion and Blue_Fusion methods using BT fusion technique showed the lowest UIQI values (0.583 and 0.418 respectively), indicating that they were least similar to the original images.
The fused images from both PCA and BT fusion techniques has high correlation coefficient compared with the original images and the values ranging from 0.975 to 0.999 and it represents the fused and original images are highly correlated.
The red band of the UAV fused with Sentinel-2A (Red_Fusion) using the BT method showed the lowest Entropy Difference (H) value which is 0.050, it suggests that fused images has less difference in the amount of information or randomness compared to the original image.The Green_Fusion using BT had the highest H value (0.511).
Overall, the fusion techniques employed has a significant impact on the quality of the fused images.PCA technique generally outperformed BT, with the green and blue bands of UAV fused with Sentinel-2A using BT fusion having the lowest similarity and highest relative error with the original images.The Red_Fusion using BT fusion had the lowest difference in entropy with the original image.However, further analysis and testing may be required to determine the optimal fusion technique for assessing the performance of the fusional vegetation indices by comparing the Sentinel-2A VI's.

Quality Assessment of Vegetation Indices
To evaluate the quality of VI's, several indices such as NDVI, NDWI, NDRE, and GRVI were computed for both the satellite and fused images.Figure 4 illustrate the SVI's and FVI images, along with their corresponding minimum and maximum values.
From Figure 4, it can be observed that the FVI values of the vegetation was increased compared to the Sentinel-2A VI (SVI) and soil between the vegetation is also discriminated in the VI of PCA fusion technique and Red band of UAV fused with the Sentinel-2A (BT Red) imagery.But the FVI values of the Blue and Green band of UAV fused with the Sentinel-2A (BT Green and BT Blue) are nearly similar to the SVI's and it is not apparently discriminating the soil between vegetation gaps compared to the PCA fusion which is shown in Figure 5. From the figure 6, it can be observed that the maximum and minimum values of the VI's are quite similar for the Sentinel-2A and fused images.However, there are some differences that can be observed between the PCA and BT fusion techniques.
For NDVI, both fusion techniques exhibit maximum values similar to the Sentinel-2A image, but their minimum values are slightly higher.Notably, the BT Blue fused image displays a higher maximum value for NDVI when compared to the original and PCA fused images.Similarly, for NDWI, the maximum values are slightly higher for the fused images, while the minimum values are slightly lower.
In terms of NDRE, all three band UAV fused images have quite similar maximum values, but the minimum values are slightly lower for the fused images.As for GRVI, the BT Green fused image has a slightly higher maximum value compared with the Sentinel-2A and PCA fused images.The minimum values of GRVI are slightly lower for both fused images compared to the Sentinel-2A image.
From the maximum and minimum values of the SVI and FVI, it can be concluded that the fusion techniques used in this study do not significantly impact the FVI's.However, there are some minor differences that could potentially affect the interpretation of the data.Scatter plots and regression analysis between the FVI and SVI are effectively represented the variation of indices value from the randomly collected samples which is shown in Figure 7. Linear regression models were applied to compare the FVI's and Sentinel-2A VI's in order to evaluate the performance of VI's.A scatter plot was created by collecting 1000 random samples from each image, and a linear equation, coefficient of determination (R 2 ), and RMSE were calculated to evaluate the performance of the regression models.

Statistical Analysis:
The scatter plots between the VI's estimated from UAV bands (R, G, and B) fused with the satellite imagery using two different techniques (PCA and BT) and the SVI were drawn.The scatter plots show the relationship between the FVI and the SVI, and a linear equation was fitted to the data points in each scatter plot.The linear equation provides information about the slope and intercept of the line that best fits the data, and the R 2 value and RMSE were calculated to evaluate the goodness of fit of the linear equation.
From the scatter plots of NDVI indices, the linear equations for PCA_Red (Red band of UAV fused with the Sentinel-2A imagery using PCA fusion technique) and BT_Red images had a positive slope, indicating that the FVI values increase compared with the SVI values.However, the R 2 values for both equations were quite low, suggesting that only a small proportion of the variability in the FVI can be explained by the SVI.The RMSE values for both equations were also relatively high, meaning that the FVI deviates from the SVI by an average of 0.054 and 0.0546, respectively.

Figure 7. Scatter plots between FVI and Sentinel-2A VI
The linear equation had a negative slope for the PCA_Green image, which means that the FVI values decrease with the SVI values.The R 2 value was slightly higher than for PCA_Red and BT_Red, but still quite low at 0.0099.The RMSE value was also relatively high at 0.0523.
On the other hand, the linear equations for PCA_Blue and BT_Blue had a positive slope with moderate to high R 2 values of 0.0531 and 0.8151, respectively.The RMSE values for both equations were also relatively low at 0.0431 and 0.0286, respectively.
Regarding NDWI, PCA_Red, PCA_Green, and BT_Red had a positive slope in the linear equations, indicating that an increase in SVI values is associated with an increase in FVI values.However, the R 2 values for these equations were quite low, indicating that the observed VI explains only a small proportion of the variability in the estimated VI.The RMSE values were also relatively high, indicating significant deviation from the observed SVI.
For BT_Green and BT_Blue, the linear equations had a positive slope with relatively high R 2 values of 0.8668 and 0.8065, respectively.The RMSE values were also relatively low, which indicates that the smaller deviation between FVI values from the SVI values.
For NDRE, the linear equations for PCA_Red, PCA_Blue, and BT_Red had a positive slope, but with low to moderate R 2 values and relatively high RMSE values.The linear equation for BT_Green had a positive slope with a relatively high R 2 value of 0.7982 and a low RMSE value of 0.0196.
In terms of GRVI, the linear equations for all fusion techniques had a positive slope, indicating an increase in FVI values compared with SVI values.However, the R 2 values for PCA_Green and PCA_Blue were quite low at 0.0058 and 0.0717, respectively, while the RMSE values for all equations were relatively low.
Overall, the results from the scatter plots, R 2 and RMSE suggests that, green band of UAV fused with the Sentinel-2A imagery using the BT fusion method to calculate VI's in the green band is generally more effective in capturing the observed vegetation patterns compared to PCA fusion method.However, the RMSE values of the fused indices compared with the SVI are very low (<0.05) in the PCA and BT fusion indices which may be considerable due to the discrimination of soil and vegetation in the fused images with in a pixel of Sentinel-2A imagery.

CONCLUSION
In conclusion, this study investigated the impact of UAV-Sentinel-2A fusion on vegetation indices performance.Two fusion techniques, PCA and BT, were applied to the Red, Green, and Blue bands of UAV imagery fused with Sentinel-2A imagery.The results of image quality assessment metrics showed that PCA generally outperformed BT, with the Green and Blue bands of UAV fused with Sentinel-2A using BT fusion having the lowest similarity and highest relative error with the original images.The fused images of both techniques showed improved visual quality compared to the original imagery, but some artifacts were presented in the individual bands of the fused imagery.The fused images developed the multispectral images with a spatial resolution of 0.03m/pixel.To evaluate the quality of the FVI's, several indices such as NDVI, NDWI, NDRE, and GRVI were computed for both the satellite and fused images.
From the maximum and minimum values of the SVI and FVI, it can be concluded that the fusion techniques used in this study do not significantly impact the FVI's minimum and maximum values, but the features in the FVI images are discriminated by the indices values.Overall, this study provides valuable insight into the use of fusion techniques for assessing the performance of the vegetation indices, which can be useful for various applications, including agriculture, forestry, and land-use mapping.

Figure 1 .
Figure 1.Location of the study area a) Sentinel-2A RGB image, b) UAV ortho image

Figure 2 .
Figure 2. Methodology flowchart of quality assessment of fusional images

Figure 3 .
Figure 3. Fusional images of UAV and Sentinel-2A imagery in TCC (a, b, and c are the fusional images of Red, Green and Blue bands of UAV fused with the Sentine-2A image using the PCA fusion technique respectively and d, e, and f are the fusional images of Red, Green and Blue bands of UAV fused with the Sentine-2A image using the BT fusion technique respectively)

Table 1 .
Sensor characteristics of UAV and Sentinel-2A imagery

Table 2 .
Statistical measures of fusional image quality assessment