MULTISPECTRAL IMAGE RESTORATION USING A VECTOR-VALUED REACTION-DIFFUSION BASED MIXED NOISE REMOVAL TECHNIQUE

: A novel multispectral image filtering technique is proposed in this article. Since the multispectral images are often corrupted by mixed Poisson-Gaussian noise during the sensing and acquisition process, a nonlinear anisotropic diffusion-based restoration approach that deals efficiently with this type of noise mixture is considered here. A second-order vector-valued reaction-diffusion model that leads to a system of well-posed single-valued anisotropic diffusion equations coupled by correlation terms is introduced for this purpose. A finite difference method-based fast-converging approximation algorithm that solves numerically this nonlinear diffusion-based system is then proposed. This iterative numerical approximation scheme is successfully used for removing both the additive Gaussian and quantum noises while preserving the essential features of the multi-valued image. The effectiveness of the described mixed denoising technique is illustrated by the results of the restoration experiments and method comparisons that are also presented here. The proposed restoration approach enhances considerably the spectral image quality, making it well-prepared for the further MSI analysis and computer vision processes, such as the geospatial and remote sensing image analysis.


INTRODUCTION
The multi-spectral imaging (MSI) is based on capturing the image data within specific wavelength ranges across the electromagnetic spectrum.These wavelengths can be separated by filters or detected using some devices that are sensitive to particular wavelengths, such as light from frequencies beyond the visible light range.Thus, the multispectral camera captures the infrared and ultra-violet lights that are invisible to humans.(Hagen and Kudenov, 2013;Reddy and Pawar, 2020).A MSI system employs a wide array of sensors with specific frequency bands of light illumination.Since its sensors may detect energy beyond what is humanly visible, the multispectral imagery enable us to observe various things that are not so apparent.Thus, a multispectral image represents a collection of several monochrome layers of the same scene, which are acquired at multiple wavelength bands.These spectral images provide more authentic representations of the real-world scenes than the RGB color images and the 2D gray-level images, thus improving the effectiveness of various image analysis and computer vision tasks (Reddy and Pawar, 2020).
Since they allow the scientists to view and examine many objects and events that would be normally hidden, the multispectral images have found applications in numerous domains, such as the remote sensing, medical imaging, astronomical imaging, fluorescence microscopy, environmental management, geographical tracking, military target tracking, weather forecasting, thermal signature detection, maritime surveillance or document and artwork investigation.Earth exploring represents also a major application field, since almost the entire earth is photographed and monitored by a lot of sensors mounted on aircraft, satellites and drones (Brown and Harder, 2016).The multispectral imagery generated by these imaging devices captures valuable information about our world (Schowengerdt, 2007).
Hyperspectral images represent a special category of spectral images.While the multispectral images are characterized by a low number of frequency bands, which is usually between 3 and 15, the hyperspectral images are acquired in more than 100 contiguous spectral bands (Chang, 2003;Lennon et al., 2002).
Unfortunately, the most multispectral and hyperspectral images are corrupted during the acquisition process.Since, the image capturing devices involve some physical measurements, the acquired multispectral images get affected by various sources of noise and multiple noises may come across them (Reddy and Pawar, 2020).The elimination of those noises constitutes a necessary and important pre-processing step that facilitates the further spectral image analysis processes, such as multispectral image classification and pattern recognition (Shen et al., 2001).
The multispectral image denoising and restoration still represents a very challenging task.Filtering each frequency band separately does not represent an efficient denoising solution and would produce loss of information, since the channels of the multispectral image are often highly correlated (Peng et al., 2014).So, any effective noise removal approach has to deal properly with this spectral band correlation and also must preserve the essential image details and overcome the unintended effects during the restoration process.
Although the additive white Gaussian noise (AWGN) is mostly considered by the restoration approaches (Aiazzi et. al., 2002), there are many other types of noise that contaminate the MSI and deteriorate their quality, such as Poisson noise (Mansouri et al., 2016), stripe noise (Chang, 2015), spectral impulse noise and various noise mixtures (Sun et al., 2022).In fact, the hyperand multi-spectral image capturing devices generate very often a type of mixed noise that follows the Poisson-Gaussian distribution (Haight, 1967).Since the noise in the multispectral data has a signal-dependent character, the Poisson-Gaussian mixed noise model is the most accurate one to describe it.While its AWGN component comes from natural sources, like the spontaneous thermal generation of electrons, its Poisson noise, which is also called quantum or shot noise, is generated by the mechanism of quantized photons and uniform exposure.
Various mixed noise removal techniques for hyperspectral and multispectral images have been developed in the last years.They include methods using spatial-spectral structure similarity (Yang and Zhao, 2013), the tensor-based techniques (Peng et al., 2014;Dong et al., 2018) or the 2D approaches extended for volumetric data, like 3D NLM (non-local means) filter (Qian et al., 2012), 3D Median filter or 3D Gaussian filter combined to variance-stabilizing transform (VST) (Mäkitalo and Foi, 2012).
Here we consider a partial differential equation (PDE)based mixed noise reduction technique for MSI.The PDE models have been used successfully in the image processing and analysis fields in the last 35 years, since they succeed in regularizing the data while preserving essential image features like edges and corners.Many variational and nonlinear PDEbased filtering models for Gaussian (Weickert, 1998;Barbu, 2019;Barbu, 2013), Poisson (Sawatzky et al., 2009;Barbu, 2020) and mixed noise removal (Thanh and Dvoenko, 2016;Pham et al., 2020) from the 2D and RGB images have been proposed.The PDE-based Poisson-Gaussian noise removal approach introduced here is based on a novel non-variational well-posed second-order nonlinear anisotropic diffusion-based model for vector-valued images (Tschumperle and Deriche, 2002;Tschumperle and Deriche, 2005;Lennon et al., 2002), described in the next section.It consists of a system of several coupled reaction-diffusion equations that evolve simultaneously and represents the major contribution of this research work.
A stable and fast-converging iterative numerical approximation algorithm that is consistent to the proposed nonlinear PDEbased model and has been developed by applying the finite difference method is also presented in the second section.This numerical solving scheme has been applied successfully to many multispectral image datasets affected by noise mixture.It has provided effective mixed noise removal results while preserving the essential details, as illustrated in the third section.This method can be applied properly to environmental remote sensing image analysis domain.The main conclusions and future research plans are discussed in the fourth section.

Second-order vector-valued reaction-diffusion model
A multispectral image with M frequency bands can be represented as a vector-valued function :

M uR 
, where the image domain 2 R  (Tschumperle and Deriche, 2005).Therefore, we have: where each single-valued function channel acquired at a particular wavelength band.The noise mixture representing here a combination of white additive Gaussian and quantum noises occurs because the multispectral image acquisition sensors have 2 noise sources: a signal-dependent one coming from the way light intensity is measured and a signal-independent source which is thermal and electronic noise.The Poisson-Gaussian mixed noise model has the form:  (Haight, 1967) and N  is AWGN characterized by 0 mean and variance σ.
A nonlinear anisotropic diffusion-based model extended to vector-valued images is proposed here for the removal of this multiple noise combination.Since we consider an anisotropic diffusion process described by a sequence of evolution equations, a time parameter has to be introduced in the spectral image function definition.The proposed secondorder vector-valued diffusion model has the form: a 2D Gaussian kernel, and 0 u is the observed multispectral image corrupted by mixed noise.We propose the following diffusivity (conductance) function, that is positive, monotonic decreasing and converges to 0 (Weickert, 1998), for the model (3): where where The components provided by these functions, the conductance term , assure the coupling of the single-diffusion based processes obtained from (3), in order to deal with the inter-channel correlation.
Since u  represents the generalized Jacobian matrix of u , we get: the vector-valued PDE model (3) becomes equivalent to the following system of single-valued diffusion equations: These second-order parabolic PDEs represent nonlinear reaction-diffusion equations that evolve simultaneously but not independently, since they share the two coupling terms: The M anisotropic diffusion processes are thus correlated by these coupling components that model properly the interchannel relations, preserving the meaningful boundaries of all the spectral layers.
Each nonlinear diffusion-based equation of the system (7) is non-variational, since it cannot be derived from the minimization of any energy cost functional (Fox, 1987).It removes successfully both the additive Gaussian and the Poisson noise from the corresponding image channel k u , due to the   term, where the controlling parameters  and  depend on the amounts of the AWGN and quantum noise respectively, and preserves efficiently the edges and other essential details, while overcoming the unintended effects, like blurring and staircasing (Buades et al., 2006).It is also well-posed, admitting a unique weak solution.
The solution of the vector-valued diffusion-based model ( 3) is then determined by solving numerically the equation system (7).A numerical approximation algorithm is constructed for this system by applying the finite difference method and is described in the next subsection.

Numerical Approximation Algorithm
The presented nonlinear PDE-based filtering model is then discretized by using finite differences.An approximation scheme that solves numerically the diffusion-based system (7) is created applying the finite difference method (Johnson, 2008) The left term of ( 8) is approximated numerically applying the central differences (Johnson, 2008), as follows: Then, in the right term, , Np is the set of pixels representing the 4neighborhood of the pixel p given as a pair of coordinates (i, j), One may consider the parameter values h = 1 and 1 t  .Therefore, by using these discretization results, one obtains the following explicit numerical approximation scheme for the m th equation of the system (7): . The iterative approximation algorithm ( 10) is stable and consistent to the m th equation of the nonlinear diffusion system (7) and converges fast to its numerical solution, for Therefore, because of the shared terms, the state of the evolving image m u determined at iteration n + 1 depends not only on its initial observation and its state at the previous iteration, but also on the evolving images of the other channels determined at iteration n.Thus, the numerical solution of the vector-valued diffusion-based model ( 3) is computed by applying (10) iteratively for each n from 0 to N and for each m from 1 to M.
The proposed finite difference-based numerical approximation scheme is successfully applied in the mixed noise reduction experiments discussed in the next section.

EXPERIMENTS AND METHOD COMPARISON
The described vector-valued diffusion-based multispectral image restoration framework has been tested on numerous datasets and some effective results have been obtained.The numerical simulations have been performed on an Intel (R) Core (TM) i7-6700HQ CPU 2.60 GHz processor on 64 bits, which operates Windows 10.The MATLAB programming platform has been used for the software implementation of the proposed filtering technique.
Several multispectral image databases have been used in our mixed noise removal experiments.So, we have used 135 8channel multispectral images from the UGR Spectral Saliency Database that contains urbanization scenes (Martínez-Domingo et al., 2021) and 55 [31 x width x height]sized spectral images collected from 2 multispectral image databases containing full spectral resolution reflectance data from 400 nm to 700 nm at 10 nm steps (31 wavelength bands), built using SpectraCube (Finlayson et al., 2004) and Cooled CCD Camera (Yasuma et al., 2010) respectively, which have been corrupted by various amounts of Poisson and Gaussian noise.
Since nonlinear diffusion improves images qualitatively by removing noise while preserving and even enhancing details, the proposed denoising approach filters successfully both the signal-independent AWGN and the signal-dependent shot noise from the spectral images and overcomes the undesired effects like blurring and staircasing.Also, it deals properly with interchannel correlation problem, preserving successfully essential features, such as edges and corners, among different layers.This MSI restoration technique is characterized by a rather low execution time, due to its fast-converging numerical solver.However, the number of iterations required by this algorithm and its running time depend on the multispectral image dimension and the levels of quantum and white additive Gaussian noises.The performance of our nonlinear PDE-based multispectral image smoothing approach has been assessed by using performance metrics like the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index Measure (SSIM) (Thung and Raveendran, 2009).The proposed mixed denoising framework achieves high values of these quality measures.
Method comparison have been also performed.The anisotropic diffusion-based technique introduced here outperforms many other Poisson-Gaussian noise MSI filtering approaches, such as the 2D TV-based mixed noise removal models and some 3D image filters like 3D Median, 3D Gaussian combined to VST and 3D NLM filter, achieving higher average PSNR scores than them, as illustrated by the method comparison results displayed in Table 1.

CONCLUSIONS
A nonlinear vector-valued diffusion-based mixed noise removal framework for MSI was described in this research paper.The main contribution of this work is the novel vector-valued second-order anisotropic diffusion-based filtering model proposed here.It is equivalent to a system of parabolic nonvariational well-posed single-value reaction-diffusion equations corresponding to the MSI layers.These nonlinear PDEs evolving simultaneously deal properly with both signalindependent and signal-dependent noise components from the related spectral channels, removing successfully both the AWGN and the quantum noise, while avoiding the unintended effects.Also they take into account the channel correlation issue, being coupled by two shared correlation terms, based on gradient and Laplacian respectively, that model efficiently the inter-channels relations and capture the multi-edges, providing an essential detail-preserving multispectral image filtering.
The finite difference method-based fast-converging numerical approximation algorithm that solves numerically the proposed model is another contribution of this paper.It was applied successfully in the described Poisson-Gaussian noise reduction experiments.The proposed noise mixture removal technique outperforms clearly the 2D restoration approaches that filter each MSI channel separately and also some volumetric 3D denoising schemes.
The proposed spectral image restoration framework can be further used as an effective pre-processing step by many MSI analysis and computer vision applications, since the successful mixed noise removal and the enhancement of essential details provided by it may facilitate some important tasks, such as the multispectral image classification, the multispectral pattern recognition, the multispectral unmixing or the multispectral target detection and tracking.Thus, our MSI filtering results might aid the extraction of valuable information from some important types of multispectral data, such as the geospatial, remote sensing, astronomical and microscopy images.Some important information related to air pollution, air and water quality, land degradation or land use change and dynamics, natural disasters, urban development tracking and vegetation could be retrieved properly from the remote sensing images and sequences for environment management.
Although this mixed noise filtering method was proposed for multispectral images, it could be extended to the hyperspectral images.Since the hyperspectral images are made up of much higher number of channels characterized by a continuous spectrum and implicit correlations between adjacent bands, modelling some proper cross-channel correlation terms for the diffusion equations could become a more difficult task.Such an improved extension of this anisotropic diffusion-based mixed denoising framework will represent the focus of our future research work in the spectral imaging domain.Also, we will consider some multispectral image pattern recognition solutions that extend our past pattern recognition approaches (Barbu, 2006).

u
is the observation impaired by this noise,   u  represents the photon-limited image u corrupted by a Poisson noise with distribution   other positive function used within this PDE-based model has the next form:

Figure 1 .
Figure 1.Multispectral image denoising exampleAlso, the implementation is sensitive to parameters which are necessarily tuned to sharpen a narrow range of edge slopes.The optimal model's parameters are determined empirically by using a trial-and-error approach.A mixed noise reduction for MSI example, involving an urban scene, is described in Figure1.The [512×612×8] multispectral image whose channels are displayed in a) is contaminated by a noise mixture composed of a high amount of AWGN, characterized by the mean 0   and the variance 2 0.12   , and an amount of Poisson noise.The deteriorated spectral image depicted in b) is then filtered applying the proposed technique and the denoising result achieved after N = 12 iterations is displayed in c).One can see that all the 8 channels of the restored MSI have good visual quality. .

Table 1 .
The average PSNR scores achieved by several filters