Open Access
Add to BibSonomyAbstract
Properties of the short wave infrared (SWIR) imaging spectrograph and the front lens along with the misalignment of optical elements contribute to positionally variant displacements and blur that can significantly degrade the overall quality of the acquired images. In this work, we devise a complete routine for simultaneous displacement correction and resolution enhancement of SWIR spectral images along the two spatial and the spectral direction. The proposed restoration routine requires images of widely available and inexpensive calibration targets from which the response function of the imaging spectrometer is extracted. Extensive validation reveals that the displacement error observed in the restored images is reduced to the manufacturing accuracy of the calibration targets. Furthermore, the restored images exhibit up to a two-fold improvement in the spectral and spatial resolution.
© 2016 Optical Society of America
1. Introduction
Short wave infrared (SWIR) pushbroom imaging spectrometers are gaining broad recognition in the scientific community. Their use is being investigated in various applications such as in the food industry for fruit bruise detection [1], in agriculture and forestry for tree species mapping [2], spruce seeds screening [3], fire severity assessment [4], detection of corn kernel contamination [5], differentiation between similar plants [6,7], for soil [8] and mineral [9,10] mapping, in biometrics [11], in chemistry for chemical composition identification [12] and in numerous biomedical applications [13].
SWIR pushbroom imaging spectrometers typically comprise an SWIR camera, an imaging spectrograph and a front lens. An illumination source and a motorized linear unit complement the image acquisition in laboratory settings. The overall quality of the acquired images can be significantly degraded by positionally variant displacements and blur arising from the properties of the imaging spectrograph, the front lens and the misalignment of optical elements. In order to perform reliable measurements, different calibration techniques are available and routinely performed in computer vision systems prior to image acquisition [14]. Nevertheless, in many of the laboratory imaging spectroscopy applications merely a flat-field correction is applied to eliminate the effect of illumination non-uniformity and sensor sensitivity, neglecting the effects of system optics on the acquired images [1,3,5,9]. In some applications the authors use off-the-shelf systems with internal spectral calibration, whereas the calibration in the across-track and along-track directions is left untreated [6,7,12].
In this contribution we propose a deconvolution-based restoration method for images acquired by SWIR pushbroom imaging spectrometer, which significantly reduces the effect of system response function in terms of displacements and blur observed in the two spatial and the spectral direction. The positionally variant response function is derived from the images of custom machined spatial calibration targets and gas-discharge calibration lamps. The performed restoration is accompanied by a comprehensive quantitative validation demonstrating the efficiency and accuracy of the proposed procedure.
2. Related work
The importance of pushbroom imaging spectrometer calibration in spectral and both spatial directions is recognized in the field of remote sensing where several studies have been published in recent years with thorough laboratory characterizations. Davis et al. [15] characterized Offner spectrograph-based Ocean Portable Hyperspectral Imager for Low-Light Spectroscopy (Ocean PHILLS). Spatial resolution in across-track direction was assessed from an image of a Ronchi ruling target by fitting a Gaussian to the edges in the image. Laser sources were used to analyze the spectral response function (SRF) at three spectral positions and four gas-discharge lamps were used to perform spectral characterization at each across-track position.
Ammannito et al. [16] reported on the characterization of Rosetta/VIRTIS-M Offner grating based Visual Infrared Thermal Imaging Spectrometer. A calibrated monochromator output was used for spectral characterization. Characterization was performed at three selected regions in the visible and three selected regions in the infrared spectral range. Measurements at different across-track positions were not conducted, therefore invariance of the SRF in the across-track direction was probably assumed. To validate the performed characterization, a spectrum of gas mixture was acquired with a Fourier transform spectrometer and subsequently convolved with the modeled SRF. The convolved spectrum and the spectrum acquired by the Rosetta imaging spectrometer were compared, obtaining a correlation coefficient of 0.94. Full width at half maximum (FWHM) characterization in both spatial directions was performed using a collimator illuminated by a Hg gas-discharge lamp through a slit located at the collimator focal plane. Subpixel measurements of the spatial response functions were conducted at three across-track positions and at several spectral positions defined by the emission spectrum of the Hg lamp. To assess the displacements between different spectral positions, a 5 × 5 grid of 1 mm pinholes, illuminated by tungsten microlamps, was placed at the focal plane of the collimator. The authors suggested that the obtained displacements could be removed from the acquired images by using a proper image restoration procedure.
Oppelt and Mauser [17] reported on the characterization and calibration of the AVIS-2 prism-grating-prism (PGP) spectrograph based Airborne Visible/Infrared imaging Spectrometer. The spectral resolution was characterized by measuring a strong oxygen absorption peak at around 760 nm and comparing the response to the results of MODTRAN 4.2 radiation simulations. The spatial resolution in the across-track and along-track directions was derived analytically and not based on experimental measurements. Characterization of the spatial displacements was not performed.
Zhang et al. [18] presented characterization of a PGP spectrograph-based field imaging spectrometer. A monochromator was used to determine the FWHM and central wavelengths of the spectral bands. The full spectral range of the instrument was scanned in 1 nm steps using a beam of 2 nm bandwidth. Spatial characterization of the instrument was not performed.
Lucke et al. [19] performed characterization of the Offner grating Hyperspectral Imager for the Costal Ocean (HICO). Lasers and spectral calibration lamps were used to inspect the spectral displacements and to determine the central wavelengths of the spectral bands. A Gaussian fit was used to find the exact position of the features in the spectra and the corresponding FWHM across the full spectral range of the instrument. The authors also performed a keystone characterization arising from the spectrally dependent spatial displacements. For this purpose, a point source was positioned at the focal plane of a collimator and imaged at eight across-track positions.
Mouroulis et al. [20] described a characterization of the Dyson Portable Remote Imaging Spectrometer (PRISM). A parabolic collimator with calibration objects positioned at the focal plane was used for spectral and spatial characterizations. Spectral FWHM was measured at five different across-track positions. For this purpose, an optical fiber placed at the collimator focal plane was fed by a wavelength-scanning monochromator. Central wavelengths of the spectral bands were determined using five emission lines of a He-Ne laser and a Hg lamp. Across-track and along-track characterizations were performed by scanning a slit placed at the focal plane of the collimator using subpixel sampling steps.
The most recent characterization of an imaging spectrometer was reported by Lenhard et al. [21]. Spectral characterization of NEO HySpex VNIR-1600 and NEO HySPEX SWIR-320m-e imaging spectrometers was performed using a collimated beam from a wavelength scanning monochromator with subpixel SRF sampling interval. Measurements were conducted at several across-track positions over the entire spectral range of both spectrometers. Spatial characterization in both directions was performed using an illuminated slit positioned at the focal plane of the collimator. Subpixel response function measurements were performed in both spatial directions at various across-track positions.
Since the lighting conditions in the field of remote sensing are not controlled and the acquisition of the white reference is usually not possible, the summarized studies also performed radiometric calibration of the imaging spectrometers. In contrast, laboratory setups employ controlled illumination sources, where a relative radiometric calibration of the system by a flat-field correction is sufficient, given the response of the imaging sensor is linear. Furthermore, the summarized characterization methods require the working distance to be set to infinity, which dictates a different characterization approach using a collimator. Finally, the main focus of the summarized studies is the characterization of resolution in terms of FWHM and assessment of displacements for subsequent restoration of acquired images by resampling. Nevertheless, it is overlooked, that image deconvolution could simultaneously reduce the effect of displacements and blur that are introduced by the imaging spectrometers.
3. Materials and methods
3.1. Calibration image model
In our previous work on calibration of pushbroom imaging spectrometers for VNIR range [22] we have established a methodological framework for modeling the response function of the imaging spectrometer in the across-track and spectral direction. In this section we provide a brief overview of the method and extend the framework to the along-track direction. The acquired spectral image o is modeled as
where u, w and z are the across-track, spectral and along-track directions, respectively, g is an undistorted image, * stands for 3D convolution, h is the response function of the imaging spectrometer, b is a polynomial baseline model to account for temperature variations of the camera sensor and the light source and ε denotes the difference between the modeled and the acquired image. The response function at each point (i, j) of the imaging plane is modeled by a positionally variant basis function f: where λk are the parameters of the basis function f. Since the response function is variant with respect to the position (i, j) in the imaging plane, the basis function parameters λk are modeled by either a bivariate cubic spline or a 2D polynomial. Table 1 summarizes the properties of the adopted basis functions over the imaging plane for the imaging spectrometer, used in this study. The degrees of polynomials and the number of spline nodes were determined by choosing the values, for which the restored validation images displayed the smallest displacement errors. In order to find the values of h and b from Eq. (1), an optimization procedure that minimizes the root mean squared error between the modeled and the acquired images of the calibration targets was employed: where 〈···〉 denotes the mean value. To accelerate the estimation of response function and improve the convergence, the optimization was performed iteratively by subsampling the calibration images along the directions orthogonal to the characterization direction. The optimization started with the calibration image subsampled to the level smax corresponding to a 1D intensity profile. After each completed iteration, the resolution of the calibration image and the number of free parameters was increased and a new iteration of the algorithm was started. The algorithm was terminated if the subsampling level reached s = 0 or if a relative change in the difference between the modeled and the acquired calibration image ε fell below 0.01. A detailed explanation of the optimization procedure can be found in [22] and an overview of the described iterative procedure is highlighted in Fig. 1.
Table 1. Adopted basis function and baseline models along the spectral and the two spatial directions. Abbreviations: G stands for Gaussian, x1 × x2 s.n. stands for the number of nodes in the bivariate spline model and x1 + x2 + ··· + xn p.d. stands for the degree of an n-dimensional polynomial defined as the sum of its powers.
3.2. Deconvolution
Deconvolution allows a simultaneous reduction of displacements and blur from the acquired image by using the retrieved response function h. Such deconvolution-based image restoration can be accomplished by an iterative algorithm [23,24]
ĥu,w is the flipped response function, Ω is the image domain and õk is the restored image after k-th iteration of the algorithm. The termination criterion of the algorithm is based on the standard deviation of the residual where The algorithm terminates if the relative difference between the standard deviations σ of two consecutive residuals falls under a predefined threshold, which was set to ξ = 0.01.3.3. Experimental setup
The pushbroom imaging spectrometer used in this study comprised an SWIR camera (Xenics, Xeva-2.5-320) with 320×256 pixels, an SWIR spectrograph (Specim, ImSpector N25E) with 30 μm slit, a lens (Specim, S31-f/2.0), a motorized linear unit (isel Germany AG, LES5, L290 mm) and a collimated axial cross-polarized illumination based on two ultra-broadband wire grid polarizers (Edmund Optics, 68–751), a beam splitter (Edmund Optics, 48–505), an aspheric lens (Edmund Optics, 46–661), an N-BK7 ground glass diffuser (Thorlabs, DG20-220) and a power controlled 250 W halogen lamp (Osram, 64659 ELC-10) (Fig. 2). Spatial characterization in the across-track direction was performed using a custom laser-machined 150 μm thick stainless-steel Ronchi ruling target with eleven edges contained in the field of view. An edge target was used for spatial characterization in the along-track direction. Both targets were manufactured with an overall accuracy of ±8 μm. The spectral characterization was based on all observable spectral lines of pencil style gas-discharge calibration lamps (Newport, Oriel Instruments, models 6030, 6032 and 6035) emitted in the target spectral range and a single spectral line at 2058.7 nm emitted by a He spectrum tube (Electro-Technic Products). The spatial displacement corrections were validated by a Glass Distortion Target (Edmund Optics, 57–985) and the spatial resolution enhancement was assessed on independent image sets of Ronchi ruling and edge targets. The spectral displacement corrections were validated by a Standard Reference Material® (SRM) 2035 (National Institute of Standards and Technology), gas-discharge calibration lamps (Newport, Oriel Instruments, models 6031 and 6033) and He spectrum tube (using three tightly packed spectral lines at 1082.909 nm, 1083.025 nm and 1083.034 nm). The spectral images of the gas-discharge lamps were also used to assess the spectral resolution enhancement. The acquired images were flat-field corrected using a dark image and an image of a diffuse reflectance standard (Labsphere, SRT-99-050). The dead pixels were identified by statistical analysis of eleven sets of dark corrected white reference images acquired by gradually increasing the exposure time. The imaging system was disassembled and a close to uniform illumination of the camera sensor was employed. Considering the gain, noise and nonlinearity, 0.4% of the worse performing pixels were designated as dead and bilinear interpolation was used to estimate the intensities from the neighboring pixels. Image acquisition was controlled by a custom Python-based control system with graphical user interface.
Since the acquired signal was low at the very high end of the spectral range, the images were cropped to contain an area of 320 × 235 × Z pixels which corresponds to approximately 54 mm in the across-track direction and the spectral range from 1000 nm to 2468 nm. The number of pixels Z in the along-track direction was driven by the extent of the imaged object.
4. Results
4.1. Response function characterization
The results of the proposed characterization procedure are the response functions h along all the three directions in each pixel of the imaging plane. These response functions are used in the subsequent deconvolution-based image restoration. Figure 3(a) shows the obtained response function at the 252th across-track pixel and at the 23rd and 168th spectral positions in all the three directions. Figures 3(b) and 3(c) show the corresponding fits between the modeled and the acquired intensity profiles at the selected positions. The high agreement between the two intensity profiles indicates proper selection of the employed response function models. The selection is later further confirmed by the restored images where an inappropriate response function model would result in a poor displacement correction and artifacts in the restored images.
Fig. 3 (a) Response functions in the across-track direction (top row), in the spectral direction (middle row) and in the along-track direction (bottom row) observed at the 252th across-track position and at the 23rd and 168th spectral position. The corresponding acquired intensity profiles and the model fits at the (b) 23rd and (c) 168th spectral positions.
4.2. Displacement correction
Spatial displacement correction was assessed on a restored image of the Glass Distortion Target (Fig. 4(c)). A center of mass was used to determine the dot centers in the acquired images independently for each spectral position. In an undistorted image of the target the dot centers would lie on a regular 3D grid. Therefore, a regular 3D grid was fitted to all the extracted dot centers across the entire spectral range. Figure 4(a) shows the average distance between the dot centers and the vertices of the fitted regular 3D grid for different across-track and spectral regions of the acquired and the restored image. The acquired images contain spatial displacement errors that generally increase with the distance from the image center. These errors are due to the keystone effect, which is inherent to the pushbroom imaging spectrometers. In contrast, the restored images exhibit a mean displacement error that is below the joint manufacturing accuracy of the two spatial calibration targets across almost the entire spectral range, with a few minor exceptions at the higher end of the spectral range. This could be attributed to the lower signal-to-noise ratio (SNR) observed in the corresponding regions of the acquired images that can slightly degrade the quality of characterization results as well as the extraction of dot centers in the validation step. The results of the 3D grid fit to the extracted dot centers also provided the image pixel size in the across-track and along-track directions with the across-track pixel size of 169 μm and the along-track pixel size of 99.1 μm, which was in accordance with the step size of the motorized linear unit.
Fig. 4 (a) Error bars for the positions of the dot centers in the acquired and the restored glass distortion target images at four spectral and five across-track regions. The dotted horizontal line represents the limit imposed by the manufacturing accuracy of the calibration targets. (b) RMSE distribution of the dot center positions in the spectral direction for the acquired and the restored glass distortion target images. (c) An image of the glass distortion target used for validation.
The restored image of the SRM 2035 and the restored images of the He, Xe and Kr gas-discharge lamps with three He spectral lines near 1083 nm, Xe at 1473.2 nm and Kr at 2190.9 nm, were used to assess the spectral displacement correction. The bands of the SRM 2035 spectrum were determined by the center-of-gravity method (COG) [25]. A SRM 2035 certificate contains certified values for spectral bandwidths of 1 and 3 nm and states, that in case the spectral bandwidth of the spectrometer does not match the certified values, the most representative spectral bandwidth should be used. However, the bandwidth of the tested spectrometer is approx. 6.2 nm, which is more than twice the nearest available certified bandwidth of 3 nm. To have a more realistic estimate for the uncertainties of the extracted spectral band locations, the uncertainties for 5 nm bandwidth of the SRM 2036, which is a SRM 2035-style target for diffuse reflectance [26], were used. Since the SRM 2035 and SRM 2036 spectra are comparable and the method for extraction of band locations is the same, it is reasonable to assume that the band location uncertainties should also be comparable. Figure 5 displays the spectral displacements as a function of the across-track and spectral position in the imaging plane, which are minimized in the restored image of SRM 2035.
Fig. 5 (a)–(e) Five SRM 2035 band locations at different across-track positions in the acquired and the restored image with the certified uncertainties. (f) SRM 2035 spectrum with the designated bands that were used for validation.
Table 2 reports mean displacement errors and corresponding standard deviations for SRM 2035 and the spectral emission lines of the three gas-discharge lamps. The standard deviations of displacement errors for the SRM 2035 and the spectral emission lines of the three gas-discharge lamps in the acquired images are similar and mostly due to the rotational misalignment between the spectrograph and the camera. In contrast, the standard deviation of displacement errors in the restored images of gas-discharge lamps is up to six times lower. This could be attributed to the different methodologies used to extract the positions of certified SRM 2035 spectral bands and emission lines of the gas-discharge lamps. The positions of SRM 2035 spectral bands were determined by the COG method using 0.1 band fraction. This approach can be highly inaccurate, if the spectral sampling interval is too low, which was the reason why the certified band at 1151.5 nm was excluded from the validation. Furthermore, one is limited to the certified sampling intervals only. Conversely, validating the spectral displacement errors by the emission lines of gas-discharge lamps does not depend on the sampling interval of the instrument and the entire line response can be used to determine the location in a more reliable and accurate way. The location of spectral lines in this study was determined using a Gaussian fit at each across-track pixel. Nevertheless, the SRM 2035 can prove advantageous over the gas-discharge lamps if the spectral sampling interval of the instrument is larger than the spectral bandwidth or the SNR of the acquired spectral lines is poor, which can be partially observed for the Kr 2191 nm spectral line with the standard deviation of the displacement error assessment being higher than for the remaining two spectral lines.
Table 2. Average values and standard deviations of the displacements and FWHM in each direction before and after the restoration.
4.3. Resolution enhancement
An evaluation of the resolution enhancement was performed independently along the spatial and spectral direction. To assess the resolution in the acquired and the restored images, a Gaussian fit was used in spectral direction and a scaled error function was used in both spatial directions. In the across-track and along-track directions the images of the displaced calibration targets were used for the evaluation. The resolution enhancement in terms of FWHM is reported at ten selected across-track locations over the entire spectral range (Table 2). Figures 6(a) and 6(b) show the average resolution enhancement of the restored images in the across-track and along-track directions depending on the spectral position. The restoration process enhances the spatial resolution over the entire spectral range, having the most profound effect at the lower spectral end, where the resolution in the acquired images is the lowest. It should be noted that the SNR in the acquired images decreases towards the upper and lower ends of the spectral range, which reflects in noisier FWHM assessments. The resolution enhancement in the spectral direction was assessed by using three He spectral lines near 1083 nm, Xe line at 1473.2 nm and Kr line at 2190.9 nm. Figures 6(c) and 6(d) show the FWHM of He and Kr spectral lines over the entire across-track range obtained for the acquired and restored images, and Table 2 lists the mean and standard deviations of the FWHM for all the spectral lines.
Fig. 6 FWHM in the acquired and the restored image in the (a) across-track and (b) along-track direction as a function of the spectral position. FWHM of the (c) He 1083 nm and (d) Kr 2191 nm spectral lines in the acquired and the restored image as a function of the across-track position.
4.4. Coregistration error
We have considered the metric from [27] and performed the evaluation of spectral and spatial coregistration error taking into account the presumptions of the characterization procedure. The coregistration error of restored images was assessed on the set of restored characterization images. The reason for choosing the characterization set instead of the validation set lies in the fact that the images with the strongest spectral features were already used in the characterization step, leaving images with too few features to compute a spectral coregistration error map. Since the characterization results were validated separately, we believe that the results obtained in this way should provide a reasonable estimate of the coregistration error. The response function in the restored images was assessed by performing characterization with a Gaussian model, which may not be the optimal choice but should suffice for the coregistration error assessment.
Figure 7 shows spatial and spectral coregistration error maps in the acquired and restored images. The spatial coregistration error map was obtained by averaging the differences between the spatial response function at a given spectral position and the spatial response functions at all the remaining spectral positions. Similarly, the spectral coregistration map was obtained by averaging the differences between the spectral response function at a given across-track position and the spectral response functions at all the remaining across-track positions.
Spectral coregistration error in the acquired image is mainly due to the displacement errors and amounts to an average value of 0.22, which drops to 0.03 in the restored image. Spatial coregistration error, however, does not decrease after the restoration with its average value of 0.21 in the acquired and 0.22 in the restored image. This can be attributed to the highly variant nature of the across-track and along-track response functions, particularly near the edges of the image. Figure 8 shows the across-track and the along-track spatial coregistration error maps. Due to large across-track displacement errors in the acquired image the average across-track spatial coregistration error after restoration is reduced from 0.17 to 0.07. On the other hand, since the displacement error in the along-track direction is due to the nature of image acquisition negligible, the main contribution to the along-track spatial coregistration error is the response function variability. The along-track spatial coregistration error increases from 0.12 to 0.20 after the restoration. This can be attributed to the resolution enhancement in the restored images, since blurred images tend to have a smaller coregistration error [27].
Fig. 8 (a) Across-track and (b) along-track spatial coregistration error maps of the acquired and restored images.
4.5. Examples of restored images
It is difficult to observe the displacement correction in the restored images of real samples lacking exactly defined features, therefore, we first show the visual improvement in the images of validation targets. Figure 9(a) shows selected intensity profiles for a spectral image near a group of five Kr spectral lines at five across-track positions, a selected edge of the displaced Ronchi ruling target at five different spectral positions (Fig. 9(b)), and profiles of the glass distortion target at one selected across-track position and at five different spectral positions (Fig. 9(c)). After restoration the displaced profiles of the Kr lines and Ronchi ruling target become perfectly aligned. In the along-track direction there are no evident displacements of profiles at different spectral positions in the acquired and the restored image. A similar image of the along-track profiles for the illustration of displacement correction at different across-track positions is not feasible, since it would require an image of an edge being perfectly aligned with the imaging system. The images of all the three validation targets also reveal a noticeable resolution enhancement.
Fig. 9 Acquired and restored profiles of the (a) Kr lines at 1678.5, 1689.0, 1689.7, 1693.6 and 1709.9 nm at five different across-track positions, (b) edge in the image of the validation target for the across-track direction at five different spectral positions and (c) the glass distortion target at the 151th across-track pixel and at five different spectral positions.
To demonstrate the proposed restoration method on realistic images, an image of coffee beans and pepper (Fig. 10) and an image of electronic circuit (Fig. 11) were acquired. The resolution of restored images is evidently improved which is best observed in the regions with highly dynamic content.
Fig. 10 The acquired (upper row) and the restored (bottom row) 2D slices of a 3D coffee beans and pepper image segment containing pixels [100–250, 1–235, 400–550] where the position in brackets represent the across-track, spectral and along-track directions, respectively. The displayed slices are located at the 175th across-track, 55th spectral and 475th along-track pixel.
Fig. 11 The acquired (upper row) and the restored (bottom row) 2D slices of a 3D electronic circuit image segment containing pixels [90–240, 1–235, 210–360]. The displayed slices are located at the 165th across-track, 115th spectral, and 285th along-track pixel.
5. Discussion
Deconvolution is generally an ill-posed problem [28] and many algorithms have been developed to recover the best estimate of the undistorted image [29]. The straightforward approach would be the direct inversion in Fourier space, dividing the acquired image by the system response function. This method, however, is not acceptable in practice due to the high noise content in the recovered image [30,31]. Statistical estimation methods which take into account the nature of noise in the acquired images are used for deconvolution in various fields [32–35]. A byproduct of deconvolution algorithms are often image artifacts (ringing) and several regularization schemes can be applied to reduce their effects [36]. These schemes often require additional parameter tuning which can be scene specific. In our studies, we have opted for Richardson-Lucy deconvolution algorithm due to a simple integration of positionally variant system response function and generally favorable performance [37]. As a regularization a stopping rule was adopted which terminates the algorithm, when the relative difference between two consecutive standard deviations of the difference between the measurement and the backprojected estimate falls under a certain level, as suggested in [29]. This stopping rule sets a compromise between the desired resolution enhancement and noise amplification and a fixed value of 0.01 has been used for all image restorations in our experiments. Iterations above a certain point tend to fit the noise in the acquired images without further improving the estimates [35].
On the other hand, the coregistration error in the acquired images is a result of the displacement term of the positionally variant response function, which is captured with our characterization procedure. Consequently, the coregistration error can be also reduced by resampling the acquired images. Such restoration does not influence the noise content in the image and is fully sufficient if there is no need to improve the spatial or spectral resolution. A complete estimation of coregistration errors according to [27] would require 3D point spread function measurements without the separability assumption, which we plan to investigate in the future. Nevertheless, the spectral and spatial coregistration error proposed in [27] can be assessed if its third term, i.e. the spectral-spatial interdependence error, is dropped.
The proposed characterization method requires a few minutes to extract the positionally variant response function model and can be efficiently used in various measurement settings. However, such a model-based characterization might in some cases conceal discontinuities or rapid changes of the response function. Spectral characterization methods in the field of remote sensing could theoretically resolve this issue by scanning the entire spectral range in subpixel steps using a monochromator. However, such a complete characterization would be a matter of months [38]. Therefore, practical characterizations are limited to several across-track and spectral positions and a model is used to extend the results over the entire imaging plane. Similar strategy is also applied for the across-track and the along-track characterizations.
6. Conclusions
In this contribution we have developed and validated a restoration routine for simultaneous displacement correction and resolution enhancement of spectral images acquired with a short wave infrared (SWIR) pushbroom imaging spectrometer. After initial selection of the response function model, the proposed method is fully automated and does not require any user intervention. The positionally variant response function of the imaging spectrometer is retrieved from the calibration target images by an iterative optimization procedure. Subsequent restoration employing the retrieved response function is performed by a 3D deconvolution of the acquired image. An extensive validation of the restored images reveals that the displacement errors are reduced to the limits imposed by the employed calibration and validation targets. Furthermore, a more than 2-fold spatial resolution and approximately 1.5-fold spectral resolution enhancement is attained.
Funding
Slovenian Research Agency (J2-7118, J2-7211, J7-6781, J2-5473, L2-5472, L2-4072).
References and links
1. P. Baranowski, W. Mazurek, and J. Pastuszka-Woźniak, “Supervised classification of bruised apples with respect to the time after bruising on the basis of hyperspectral imaging data,” Postharvest Biol. Technol. 86, 249–258 (2013). [CrossRef]
2. M. Dalponte, H. O. Orka, T. Gobakken, D. Gianelle, and E. Næsset, “Tree species classification in boreal forests with hyperspectral data,” IEEE T. Geosci. Remote 51, 2632–2645 (2013). [CrossRef]
3. J. Dumont, T. Hirvonen, V. Heikkinen, M. Mistretta, L. Granlund, K. Himanen, L. Fauch, I. Porali, J. Hiltunen, S. Keski-Saari, M. Nygren, E. Oksanen, M. Hauta-Kasari, and M. Keinänen, “Thermal and hyperspectral imaging for Norway spruce (Picea abies) seeds screening,” Comput. Electron. Agr. 116, 118–124 (2015). [CrossRef]
4. S. Veraverbeke, S. Hook, and G. Hulley, “An alternative spectral index for rapid fire severity assessments,” Remote Sens. Environ. 123, 72–80 (2012). [CrossRef]
5. L. M. Kandpal, S. Lee, M. S. Kim, H. Bae, and B.-K. Cho, “Short wave infrared (SWIR) hyperspectral imaging technique for examination of aflatoxin B 1 (AFB 1) on corn kernels,” Food Control 51, 171–176 (2015). [CrossRef]
6. N. Mncwangi, I. Vermaak, and A. M. Viljoen, “Mid-infrared spectroscopy and short wave infrared hyperspectral imaging–A novel approach in the qualitative assessment of Harpagophytum procumbens and H. zeyheri (Devil’s Claw),” Phytochem. Lett. 7, 143–149 (2014). [CrossRef]
7. I. Vermaak, A. Viljoen, and S. W. Lindström, “Hyperspectral imaging in the quality control of herbal medicines–the case of neurotoxic Japanese star anise,” J. Pharmaceut. Biomed. 75, 207–213 (2013). [CrossRef]
8. C. Gomez, R. Oltra-Carrió, S. Bacha, P. Lagacherie, and X. Briottet, “Evaluating the sensitivity of clay content prediction to atmospheric effects and degradation of image spatial resolution using hyperspectral VNIR/SWIR imagery,” Remote Sens. Environ. 164, 1–15 (2015). [CrossRef]
9. R. J. Murphy, A. Chlingaryan, and A. Melkumyan, “Gaussian processes for estimating wavelength position of the ferric iron crystal field feature at 900 nm from hyperspectral imagery acquired in the short-wave infrared (1002–1355 nm),” IEEE T. Geosci. Remote 53, 1907–1920 (2015). [CrossRef]
10. G. Notesco, V. Kopačková, P. Rojík, G. Schwartz, I. Livne, and E. B. Dor, “Mineral classification of land surface using multispectral LWIR and hyperspectral SWIR remote-sensing data. a case study over the Sokolov lignite open-pit mines, the Czech Republic,” Remote Sens. 6, 7005–7025 (2014). [CrossRef]
11. M. A. Ferrer, A. Morales, and A. Díaz, “An approach to SWIR hyperspectral hand biometrics,” Inform. Sciences 268, 3–19 (2014). [CrossRef]
12. N. Zaini, F. van der Meer, and H. van der Werff, “Determination of carbonate rock chemistry using laboratory-based hyperspectral imagery,” Remote Sens. 6, 4149–4172 (2014). [CrossRef]
13. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19, 010901 (2014). [CrossRef]
14. Z. Zhang, “A flexible new technique for camera calibration,” IEEE T. Pattern Anal. 22, 1330–1334 (2000). [CrossRef]
15. C. O. Davis, J. Bowles, R. A. Leathers, D. Korwan, T. V. Downes, W. A. Snyder, W. J. Rhea, W. Chen, J. Fisher, W. P. Bissett, and R. A. Reisse, “Ocean PHILLS hyperspectral imager: design, characterization, and calibration,” Opt. Express 10, 210–221 (2002). [CrossRef]
16. E. Ammannito, G. Filacchione, A. Coradini, F. Capaccioni, G. Piccioni, M. De Sanctis, M. Dami, and A. Barbis, “On-ground characterization of Rosetta/VIRTIS-M. I. Spectral and geometrical calibrations,” Rev. Sci. Instrum. 77, 093109 (2006). [CrossRef]
17. N. Oppelt and W. Mauser, “The airborne visible/infrared imaging spectrometer AVIS: Design, characterization and calibration,” Sensors 7, 1934–1953 (2007). [CrossRef]
18. L. Zhang, C. Huang, T. Wu, F. Zhang, and Q. Tong, “Laboratory calibration of a field imaging spectrometer system,” Sensors 11, 2408–2425 (2011). [CrossRef]
19. R. L. Lucke, M. Corson, N. R. McGlothlin, S. D. Butcher, D. L. Wood, D. R. Korwan, R. R. Li, W. A. Snyder, C. O. Davis, and D. T. Chen, “Hyperspectral Imager for the Coastal Ocean: instrument description and first images,” Appl. Optics 50, 1501–1516 (2011). [CrossRef]
20. P. Mouroulis, B. Van Gorp, R. O. Green, H. Dierssen, D. W. Wilson, M. Eastwood, J. Boardman, B.-C. Gao, D. Cohen, B. Franklin, F. Loya, S. Lundeen, A. Mazer, I. McCubbin, D. Randall, B. Richardson, J. I. Rodriguez, C. Sarture, E. Urquiza, R. Vargas, V. White, and K. Yee, “Portable remote imaging spectrometer coastal ocean sensor: Design, characteristics, and first flight results,” Appl. Optics 53, 1363–1380 (2014). [CrossRef]
21. K. Lenhard, A. Baumgartner, and T. Schwarzmaier, “Independent laboratory characterization of NEO HySpex imaging spectrometers VNIR-1600 and SWIR-320m-e,” IEEE T. Geosci. Remote 53, 1828–1841 (2015). [CrossRef]
22. J. Jemec, F. Pernuš, B. Likar, and M. Bürmen, “Push-broom hyperspectral image calibration and enhancement by 2D deconvolution with a variant response function estimate,” Opt. Express 22, 27655–27668 (2014). [CrossRef]
23. L. B. Lucy, “An iterative technique for the rectification of observed distributions,” Astron. J. 79, 745 (1974). [CrossRef]
24. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62, 55–59 (1972). [CrossRef]
25. S. J. Choquette and A. Urbas, The National Institute of Standards and Technology (email personal communication, 2015).
26. S. J. Choquette, D. L. Duewer, L. M. Hanssen, and E. A. Early, “Standard reference material 2036 near-infrared reflection wavelength standard,” Appl. Spectrosc. 59, 496–504 (2005). [CrossRef]
27. T. Skauli, “An upper-bound metric for characterizing spectral and spatial coregistration errors in spectral imaging,” Opt. Express 20, 918–933 (2012). [CrossRef]
28. T. J. Cornwell, “Image restoration,” in Diffraction-Limited Imaging with Very Large Telescopes, D. M. Alloin and J.-M. Mariotti, eds. (Springer, 1989). [CrossRef]
29. J. Starck, E. Pantin, and F. Murtagh, “Deconvolution in astronomy: A review,” Publ. Astron. Soc. Pac. 114, 1051 (2002). [CrossRef]
30. M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging (CRC Press, 1998). [CrossRef]
31. E. Thiébaut, “Introduction to Image Reconstruction and Inverse Problems,” in Optics in Astrophysics, R. Foy and F. C. Foy, eds. (Springer, 2005).
32. X. Liu, Z. Yuan, Z. Guo, and D. Xu, “A localized Richardson–Lucy algorithm for fiber orientation estimation in high angular resolution diffusion imaging,” Med. Phys. 42, 2524–2539 (2015). [CrossRef]
33. N. Chakrova, B. Rieger, and S. Stallinga, “Deconvolution methods for structured illumination microscopy,” J. Opt. Soc. Am. A 33, B12–B20 (2016). [CrossRef]
34. J. Núñez, A. Núñez, F. J. Montojo, and M. Condominas, “Improving space debris detection in GEO ring using image deconvolution,” Adv. Space Res. 56, 218–228 (2015). [CrossRef]
35. S. Eichstädt, F. Schmähling, G. Wübbeler, K. Anhalt, L. Bünger, U. Krüger, and C. Elster, “Comparison of the Richardson–Lucy method and a classical approach for spectrometer bandpass correction,” Metrologia 50, 107 (2013). [CrossRef]
36. Y.-W. Tai, P. Tan, and M. S. Brown, “Richardson-Lucy deblurring for scenes under a projective motion path,” IEEE T. Pattern Anal. 33, 1603–1618 (2011). [CrossRef]
37. J. Katrašnik, F. Pernuš, and B. Likar, “Deconvolution in acousto-optical tunable filter spectrometry,” Appl. Spectrosc. 64, 1265–1273 (2010). [CrossRef]
38. P. Gege, J. Fries, P. Haschberger, P. Schötz, H. Schwarzer, P. Strobl, B. Suhr, G. Ulbrich, and W. J. Vreeling, “Calibration facility for airborne imaging spectrometers,” ISPRS J. Photogramm. 64, 387–397 (2009). [CrossRef]
