INVESTIGATION OF LATENT TRACES USING INFRARED REFLECTANCE HYPERSPECTRAL IMAGING

The detection of traces is a main task of forensics. Hyperspectral imaging is a potential method from which we expect to capture more fluorescence effects than with common forensic light sources. This paper shows that the use of hyperspectral imaging is suited for the analysis of latent traces and extends the classical concept to the conservation of the crime scene for retrospective laboratory analysis. We examine specimen of blood, semen and saliva traces in several dilution steps, prepared on cardboard substrate. As our key result we successfully make latent traces visible up to dilution factor of 1:8000. We can attribute most of the detectability to interference of electromagnetic light with the water content of the traces in the shortwave infrared region of the spectrum. In a classification task we use several dimensionality reduction methods (PCA and LDA) in combination with a Maximum Likelihood classifier, assuming normally distributed data. Further, we use Random Forest as a competitive approach. The classifiers retrieve the exact positions of labelled trace preparation up to highest dilution and determine posterior probabilities. By modelling the classification task with a Markov Random Field we are able to integrate prior information about the spatial relation of neighboured pixel labels.


INTRODUCTION
Detecting latent traces is a key field of forensics.Light illumination and screening by goggles form the public image of crime scene investigation.The need for instant examination and clearance of the crime clarifies the importance of efficient and comprehensive techniques.Chemical contrast enhancement techniques, such as leucocrystal violet treatment and luminol searches, are two of the main methods used to analyse crime scenes.A common contact free method for the task is the use of forensic light sources (FLS), which combines illumination and detection of light by established combinations of forensic lamps and camera filters or goggles, specific for each expected latent trace.We believe that the use of hyperspectral imaging (HSI) (1) allows for analysis of several traces at once, (2) extends the classical concept to the conservation of the crime scene for retrospective laboratory analysis, and (3) leads to a tremendous reduction of time effort.In this paper, we expose biological traces to light of a wide range as well as record its reflectance in many different light colors in infrared (IR) light spectrum (940 to 2543 nm).The goal is to investigate, to what extent, methods from image processing, pattern recognition, and hyperspectral remote sensing are applicable in this domain and can be integrated into the investigation of latent traces.This paper gives an overview of the current state of research in forensic spectroscopic applications and presents an approach to analyse forensic traces addressing criminal investigators and researchers.

Task
Guided by the applications in forensics, we investigate two tasks: First, the visualization of latent traces by spectroscopic examination, and second, the automatic detection of latent traces by pixelwise classification.The former aims at the enhancement of contrast of latent traces with respect to the background.For that, we select suitable features or extract new features from hyperspectral spectra, which are characteristic for each specimen.The latter aims at determining the positions of traces, i.e. a classification procedure on the task of identification of specimen with unknown trace positioning.We investigate different supervised learning approaches, in order to distinguish trace and background or traces among themselves.All of them result in a posteriori probabilities, which allow to smooth the results by a post-processing step using a Markov Random Field (MRF) to incorporate the dependency of neighbouring pixels.The work of Edelman (2014) serves as the main reference for detection of latent traces using hyperspectral imaging.We adopt the band and ratio method for contrast enhancement and additionally provide the development of spectral indices for FLS similar to Lee and Khoo (2010) and Edelman et al. (2012).For FLS and spectroscopic properties of forensic traces, we refer to Stoilovic (1991) dealing with the absorption behaviour of semen and blood and its light sources for detection.Our contribution is the proposal of important spectral regions and indices (i.e.combinations of light colors) for the use of forensic light sources and hyperspectral imaging each associated to the responsible biological components, so that classification through common pattern recognition techniques can be applied.Furthermore, we provide catalogue wavelengths for direct application in forensic light sources.We evaluate the suitability of hyperspectral imaging for crime scene application and provide an outlook for further extensions.

TRACES AND DATABASE
A set of specimen is provided, which comprises the traces blood, semen and saliva.Three cardboard substrates, one for each trace, contain three stains in six dilutions each, thus arranged in three columns and six rows depicting the dilution rates undiluted, 1:2, 1:16, 1:125, 1:1000 and 1:8000.The dilution liquid is water.
The visual detectability of the traces is strongly varying, cf. Figure 1.For example, blood is clearly visible in undiluted condition as well as up to dilution rate of 1:125, semen up to 1:16.Saliva is even barely visible in pure condition.To allow for supervised classification, and evaluation of results, we realize ground truth through a coordinate grid, indicated by markers on the border, and depositing traces at the grid points.
For the data acquisition we use a SPECIM shortwave infrared (SWIR) hyperspectral line scanning camera, in the spectral range of 940 to 2543 nm, i.e.SWIR light and parts of NIR light.The spectral range is captured within 256 wavelength bands.The capturing setup, cf. Figure 2, therefore requires a one-dimensional movable positioning system (drive unit).For image acquisition, the camera and illumination system are moved with constant velocity over the specimen.In advance of the recording, regarding an optimal sensitivity of the camera sensor, we adjust aperture, exposure time and height over the specimen beforehand.
The number of pixels, acquired for an image, depends on the resolution of the line sensor in one direction and on the scan distance in the other, while the former additionally restricts the height of the cameras above the specimen.The SWIR-camera samples 314 pixels arranged in the line sensor.The illumination is provided by six lamps emitting a polychromatic white light of visible and ultraviolet (UV)-region, as it is comparable to sunlight, which emits the whole spectrum.Using the camera system, we perform image acquisition in emission mode under laboratory conditions, i.e. we illuminate with a fixed excitation light, while recording the emission spectrum over the hyperspectral range.Finally, our database are hyperspectral images showing each specimen separately.For a multiclass application we can, however, handle pixels of different images jointly, which makes image normalization necessary.
We denote Īλ the intensity captured at a certain wave length λ.If it is clear from context we skip the index to denote any intensity value I.As a data pre-processing step we aim at reflectance intensity values I being normalized between minimal and maximal intensity.In order to achieve this, we put a white reference into the image, next to the specimen during the image scan.The material, made of barium sulfate, is highly reflective over the whole spectrum providing a 100% reflectance standard Īref.After the image acquisition the aperture is closed and the dark response Īdark is measured.Therefrom, we derive normalized reflectance intensity values by (1)

VISUALIZATION OF TRACES BY SPECTROSCOPIC EXAMINATION
This section focuses on our first task, the visualization of latent traces.We describe features derived from captured intensities, which we will use to analyse spectroscopic properties of pure and latent traces concerning absorbance and reflectance characteristics, in order to make latent traces visible.Using these features, we will investigate, in our experiments, which light colors (i.e.detection wavelengths) cause an enhancement of contrast including comparison to FLS.

Spectral Indices
Spectral indices, such as the normalized differenced vegetation index (NDVI), used in remote sensing, are a common device for deriving features from hyperspectral data.In contrast to single wavelength bands, substraction and rationing of images is capable of suppressing background interference and variations in illumination (Wagner, 2008;Bao et al., 2009).Ratio images describe a pixelwise difference of intensity values of two wavelengths λi and λj normalized by their sum A non-normalizing ratio of the form I λ i /I λ j and other calculations, such as bracketing an absorption peak by two wavelengths are also reported to eliminate background influence and enhance contrast (Wagner, 2008).

Fisher's Ratio
Fisher's ratio is a measure for the discriminative power of classes.
It is based on the assumption that maximal separation is obtained when classes have a large inter-class variability while having a low intra-class variability.The fraction of both is defined as Fisher's ratio (also F-ratio).Given the vectors fg x and bg x, containing all 1D samples of the respective classes fg and bg, standing for foreground and background, we evaluate their means fg/bg µ and standard deviations fg/bg σ, respectively, to obtain Fisher's ratio by which can be generalized for the multi-class case (Casella, 2008).Please note, that the representation by mean and standard deviation assumes Gaussian distributed data.The Fisher's ratio is the same as Fisher's criterion, which gets applied in Linear Discriminant Analysis.
For the task of contrast enhancement, we use Fisher's ratio in two ways, which are band and ratio method.Using the band method, Fisher's ratio is calculated for each wavelength with spectral data gathered at the selected regions of interest.Given labelled image data, using class labels y ∈ {fg, bg}, and pixel positions (r, c), we collect intensities I λ i of band λi for all trace specific pixels )|y=fg and all other within bg x λ i = [I λ i ] (r,c)|y=bg .For each wavelength and each class, we obtain means fg µ λ i and bg µ λ i and standard deviations fg σ λ i and bg σ λ i from fg x λ i and bg x λ i and we obtain Fisher's ratio for each wavelength by The Ratio image method uses spectral indices of two wavelengths λij = [λi, λj], as given in (2), as features fg x λ ij = I (λ i ,λ j ) (r,c)|y=fg and bg x λ ij = I (λ i ,λ j ) (r,c)|y=bg , respectively.For each pair of wavelengths and each class we obtain means fg µ λ ij and bg µ λ ij and standard deviations fg σ λ ij and bg σ λ ij from fg x λ ij and bg x λ ij and estimate Fisher's ratio of ratio images by The Ratio image method requires two iterations over the spectral dimension.Due to the commutative property if the ratio images, we obtain (n 2 − n)/2 combinations of wavelengths.
In order to identify single bands λi and pairs of wavelength λij, which maximises the contrast between specimen and background, thus best class separability, we maximise the according Fisher ratios λmax = argmax Identified wavelengths, together with the respective maximal Fisher-ratios, allow exposition of spectroscopically important wavelength bands and indices and comparison of detectability between different cameras and traces.

CLASSIFICATION
This section aims at our second task, the automatic detection of latent traces.As we are interested in identifying the positions of the traces on the fabrics, we are dealing with the task of separating foreground and background and the question for each pixel of the image whether it is trace or not, i.e. a classification procedure on the task of identification of traces.We shortly review the methods for supervised classification we use and describe the different procedures we choose for our experiments.
For evaluation we apply different classifiers partially based on methods of dimensionality reductions.As classifiers we consider maximum likelihood (ML) and Random Forest (RF).Principal Component Analysis (PCA) provides dimensionality reduction prior to Linear Discriminant Analysis (LDA), which itself reduces to one dimension, in the two-class case.All methods are pixel-wise, solely dealing with the spectral feature space.Finally, we smooth the classification results by a MRF, incorporating the spatial arrangement of the pixels in a grid-structure and the evidence for each pixel given by the probabilistic output the classifier.
Again, we denote feature vectors by x, which may differ for various contexts, and class labels by y ∈ {background, blood, semen, saliva}.We collect feature vectors of all pixels within data matrix X and their according labels within vector y.

Dimensionality Reduction
Dimensionality reduction methods are essential for a classifier to extract important features of the data, on which the decision function is based.PCA and LDA present an unsupervised and a supervised method for dimensionality reduction.Due to a number of samples for training, which is lower than the number of dimensions, we use PCA to reduce the dimensionality of the feature space.In the supervised approach, we intend to project data to an one-dimensional subspace, in which thresholding serves as classifier.Thus, PCA is optionally performed prior to LDA.In the subspace estimated from LDA we use ML to obtain an optimal threshold decision.We fit Gaussian distributions to the data of each class, resulting in the Gaussians' intersections as class boundaries.

Random Forest
Random Forest is a classifier combining an ensemble of t = 1 . . .T decision trees using majority voting of the most popular decision over all randomly subsampled trees (Breiman, 2001).We obtain a posteriori probabilities for classified samples by learning discrete distributions in the leaf nodes of each tree.Normalized categorical histograms, i.e. frequencies of classes in the leafs, build up the discrete probability distribution Pt(y|x), which contribute to an average over the leaf nodes of all trees T , which the sample x reaches (Chawla and Cieslak, 2006).

Markov Random Field
To combine prior knowledge about the spatial relations between neighboured pixels with our evidence about each pixels class, given by the probabilistic output of the classifier, we model our classification task as a MRF, which is given by (9) The first, unary term is defined as the negative logarithm of the posteriors P (yi|xi) obtained by the classifier.Since we assume the final labelling of the pixels to be smooth within the image, we introduce this prior knowledge by means of a Potts model in the second, binary term.The term describes the interaction potential over a 2D lattice penalizing every dissimilar pair of labels and therefore heterogeneous regions utilizing the Kronecker function δ.The set of spatial neighbours is denoted by Ni.The variable w is the weight between both terms and the normalization constant is given by Z.We use max-product to solve for the best labelling ỹ = argmax y P (y|X ), while weight w is empirically determined, see (Bishop, 2006, Chapter 8.4.5).

EXPERIMENTAL RESULTS
In the experiments, we intend to show the results for the two tasks, defined in Section 1.At first, we visualize stains of the hyperspectral images at wavelengths and wavelength ratios, which distinguish trace and background best, determined by Fisher's ratio, including an overview of the applied wavelengths.Secondly, we learn different classifiers from training data and apply them to test data in order to visualize it as semantic segmentation.Finally, using the a posterior probabilities given by each classifier, we smooth these results using the MRF model.
Training and test data were collected by picking pixels within a circle of fixed radius, at the grid positions, specified by markers at the border of the specimen, and samples for background from regions in between.We take two third as training data leaving one third as test data.Please note, that capturing of foreground pixel was done automatically, thus not as accurate as pixel-wise labelling.To avoid false data for training, we choose the radius of the circles for region of interest small, thus we expect real ground truth to be larger as we labelled.

Visualization of Traces by Spectroscopic Examination
Figure 3 contains visualizations of the regions around the stains.We display the images at band I λ i and ratio I (λ i ,λ j ) , which were identified as most discriminative by Fisher's ratio F λ i and F λ ij , respectively.We perform distinction with the respective dilution of each stain against background and indicate the resulting maximal F-ratio (i.e. the measure for the detectability) underneath the images.Band and ratio image show the same stain for each dilution.We observe that band and ratio images provide enhanced differences between trace and background.Thus, we make traces visible when RGB-view does not provide recognition.However, we realize that ratio images provide a better feature definition than single wavelength bands.We can clearly see that the value of Fisher's ratio is representative for the detectability in the grayscale images.
Table 1 lists up all wavelengths and Fisher's ratios allocated for dilutions, while Table 2 contains a list of spectral peaks, in range of SWIR, of several trace components as reported in the literature.Bold values in Table 1 indicate correspondences to trace-specific absorption peaks and italic notation signifies accordance to water peaks.The dominance of water interference is clearly given.Nearly each band and ratio contains at least one wavelength at an established water peak.Consequently, we can attribute most of the detectability of traces to the amount of water in the traces.Even blood features more accordances with water peaks than with hemoglobin influence.

Binary Classification
We show segmentation images as binary (black-white) images for binary classification in Figure 4. We realize that the SWIRimages provide good classification and reconstruction of the stains.All traces are successfully classified up to highest dilution.Visually, the best classifier is ML based on PCA and LDA.The Random Forest, however, exhibits connected misclassified regions instead of noise.The postprocessing step successfully achieves to reduce noisy Table 1: Wavelengths with maximal Fisher's ratios calculated for different methods (band, ratio) and different dilution rates of blood in SWIR-data, including accordances to established absorption peaks as given in Table 2. Bold values indicate correspondences to trace-specific absorption peaks.Italic notation signifies accordance to water peaks.classification.Nonetheless, the approach fails on the connected misclassified regions of Random Forest.In case of ML based on PCA and LDA, we can entirely retrieve the ground truth labelling mask.All images show that the identification of traces is accomplished clearly beyond visual detectability.

Multiclass Classification
Next, we investigate the multiclass performances of the classifiers.We show results for the multiclass approach in Figures 5 and 6.Next to the segmentation images we display the RGB-image (true-color).Although each specimen is captured in an own image, we can handle them jointly, due to performed image normalization.We observe that the RF outperforms the ML approach, cf. Figure 5. Blood and saliva are classified correctly up to the highest dilution, semen can only be retrieved up to dilution 1:2.It is noticeable that a fringe around a stain of blood is classified as semen.The ML classifier assumes nearly all traces as blood and exhibits higher noise in background regions.For both classifiers, background is mostly confused with saliva.Traces tend to attain blood labels if classified falsely.
A qualitative evaluation, in terms of confusion matrices, provides the same result, cf.Table 3.Using Random Forest, Table 3 left, 93% of background, 66% of blood, 35% of semen and 75% of saliva pixels are classified correctly.The overall accuracy amounts to 76.2%.Semen is assigned to other classes in large quantities.The accuracies of the ML, Table 3 right, approach completely decline.Overall accuracy adds up to 65.5%.Classification of semen is particularly vague because 48% and 29% are misassigned to blood and saliva leaving 21% correct allocations.Nonetheless, confusion with background is partially lower than for the RF classifier.Figure 3: Best band and ratio images I λ i and ratio I (λ i ,λ j ) , identified by maximal Fisher's ratios F λ i and F λ i , respectively, given by numbers under each image.According bands λi and wavelengths pairs λi,j are given in Table 1.For the mutliclass approach, MRF postprocessing does not yield as accurate results as the binary case, cf. Figure 6.Despite reduction of noise, we now obtain stains entirely occupied by a false class.Thus, we predominantly classify the water content of the traces instead of the individual trace itself.The traces seem to intersect due to dilution.We present a quantitative evaluation in terms of confusion matrices in Table 4.By postprocessing we can increase the classwise accuracies, mostly for blood.Confusion of some traces is reduced to zero as a consequence of reduced noisy misclassification.We can see that predominantly the strongest class receives more classified samples.Thus, in case of PCA, LDA, and ML the wrong classes get eliminated.A spectroscopic examination shows that detection of latent traces is predominantly water based.However, not the same regions of interference with water are relevant for the single traces.The SWIR-region accounts for excellent visibility of traces in band and ratio images up to the highest dilution (1:8000).We provide various options of normalized differenced indices (ratio images) and band images.Besides visual detectability, a classification approach has shown to what extent traces on fabrics can be labelled as such.We have successfully retrieved the positions in a classification task in SWIR-data up to maximal dilution (1:8000).Various classifiers have presented different forces and weaknesses towards the data.Random Forest and LDA in connection with a PCA provide the best classifiers.The evaluation from segmentation images favours LDA with PCA due to less connected regions and few widespread misclassified pixels.This is a sign that Gaussian distribution are appropriate for the data.For the multiclass case Random Forest yields best results.In an additional modelling of the classification task by a MRF we achieve smoother segmentation images by reduction of noisy classification, which exactly recovers the labelled regions.

Blood
Despite our encouraging results, there is further space for improvements.In order to achieve better results and an application to arbitrary images and specimen an increased control over background influences (i.e.inducing trace-specific interactions) is recommended.Supervision by targeted initiation of only tracespecific interference by appropriate illumination and variation over single bandpass light colors (i.e.excitation measurement mode) has to be considered.A possible extensions is the assimilation of excitation-emission maps (EEM) providing a modelling of the relation between excitation and emission as well as scatter corrections.This work has revealed the IR-region principal for biological traces.For the SWIR-camera higher resolutions are essential.Referring to Edelman et al. (2015) area scanning cameras with tunable filters are preferable and achieve a proper spatial resolution as well as easier repositioning at the crime scene.Next to that, they allow simpler live view (in situ) applications.Single band images and ratio images can be directly visualized on an external screen.Line-scanning cameras require on-the-fly image normalization for this task.As a conclusion, the investigation of the applicability of hyperspectral imaging for the detection of latent traces has revealed to have reasonable potential.

Figure 1 :Figure 2 :
Figure 1: Different traces on cardboard in the RGB-display of the hyperspectral images.From left to right pure blood, diluted blood (1:16), pure semen, pure saliva.
Results for binary classification.Traces on each specimen are provided, such that the fraction of dilution increases from top to bottom.Columns: a) RGB, b) ground truth, c) RF, d) PCA/LDA and ML, e) RF followed by MRF, f) PCA/LDA and ML followed by MRF.

Figure 5 :
Figure 5: Results of multiclass classification.Columns for each specimen: Left -RGB.Middle -RF.Right -PCA and LDA followed by ML.Meaning of colours given in the colour bar.(Best viewed in colour.) Results of multiclass classification postprocessed by MRF.Columns for each specimen: Left -RGB.Middle -RF.Right -PCA and LDA followed by ML.Colours as given in Figure 5. (Best viewed in colour.)6. CONCLUSIONS Latent traces have been successfully made visible exploiting hyperspectral data.The detectability can be significantly improved towards single wavelength images by the calculations of bands.

Table 2 :
List of spectral peaks, in range of SWIR, of several trace components as reported in the literature.The component blood comprises hemoglobin, albumin, and globulin.

Table 3 :
Confusion matrix with classwise accuracies for multiclass classification.

Table 4 :
Confusion matrix with classwise accuracies for multiclass classification postprocessed by MRF.