We reported a high resolution snapshot imaging spectrometer (HR-SIS) and a fusion algorithm based on the properties of the HR-SIS. The system consists of an imaging branch and a spectral branch. The imaging branch captures a high spatial resolution panchromatic image with 680 × 680 pixels, while the spectral branch acquires a low spatial resolution spectral image with spectral resolution of 250 cm−1. By using a fusion algorithm base on grouping principal component analysis, the spectral image is highly improved in spatial resolution. Experimental results demonstrated that the performance of the proposed algorithm is competitive with other state-of-the-art algorithms. The computing time for a single frame is less than 1 min with an Intel Core i5-4200H CPU, which can be further reduced by utilizing a graphics processing unit (GPU).
© 2016 Optical Society of America
An imaging spectrometer is capable of capturing a three-dimensional (3D) datacube (x, y, λ) of an object scene. In recent decades, imaging spectrometers have been wildly applied in remote sensing and biomedical imaging [1–4]. In some specific applications, such as cellular dynamics research and tissue classification in vivo, imaging spectrometers are required to capture real-time scenes [5,6]. However, most traditional imaging spectrometers suffer from low temporal resolution caused by long scanning time which causes serious blur for dynamic scenes.
To overcome such drawbacks, snapshot imaging spectrometers, which can capture the 3D datacube in a single snapshot, were developed [7–14]. Gao et al. developed a snapshot Image Mapping Spectrometer (IMS) based on the dispersion of image zones by an image mapper, a lens array and a prism array . A snapshot hyperspectral imaging Fourier transform (SHIFT) spectrometer was derived from a Multiple-image Fourier Transform Spectrometer (MFTS) [8,9]. The computed tomographic imaging spectrometer (CTIS) utilized a two-dimensional (2D) disperser to project the object scene onto a detector and the 3D datacube was recovered by tomographic reconstruction techniques . The coded aperture snapshot spectral imaging (CASSI) uses a binary-coded mask to build a transmission pattern on the detector and estimated the 3D datacube during post-processing . Image replicating imaging spectrometer (IRIS) is compact and available in real-time applications because its direct acquisition of a 3D datacube is based on a Lyot Filter .
In general, some of these techniques acquire all voxels of the 3D datacube simultaneously by dividing it into multiple 2D elements. However, limited by the number of detector pixels, this mechanism causes a serious trade-off problem between spatial resolution and spectral resolution. Hegyi et al developed a hyperspectral imaging system based on a liquid crystal polarization interferometer . In this system a series of small images are acquired to reconstruct a 3D datacube. Trade-off among the spectral resolution, imaging speed, and spatial resolution can be implemented in software. However, the system only makes the trade-off selectable and it does not migrate it. The trade-off problem in CASSI was reduced by using compressive sensing theory and Wang et al. proposed a dual-camera CASSI , which utilized a grayscale camera to improve the reconstruction quality, though overall computational burden remains high.
Information fusion technique was first proposed in military survey field to produce an informative image from multiple sources . By applying the information fusion technique, the measurement systems can achieve high resolution both in spatial and spectral domain by merging a multispectral (MS) image and a high spatial resolution image. Therefore, the trade-off problem is reduced, since the high resolution in spatial and spectral domain is not required simultaneously. Recently, the information fusion technique was applied to spectral imaging. Ma et al. achieved content-adaptive high resolution hyperspectral video system based on a trilateral interpolation filtering approach [18,19]. A hybrid-resolution spectral imaging (HRSI) was proposed to combine a high resolution RGB image and a low resolution spectral image by utilizing piecewise Wiener estimation . For scenes with high sparsity in spectral domain, these techniques can achieve high spatial-spectral resolution. However, for the low sparsity situation, these techniques are unavailable because of their low sampling rate in the low spatial resolution hyperspectral image. Moreover, these techniques are very sensitive to metamerism, which is a phenomenon that different spectra appear as the same color to RGB cameras and human eyes.
Here, we report a high resolution snapshot imaging spectrometer (HR-SIS) and a matching fusion algorithm. In the proposed HR-SIS, a polarization beam splitter is used to separate the system into a spectral branch and an imaging branch. The spectral branch, which consists a spectral imager based on the SHIFT spectrometer, is compact and robust. A low spatial resolution MS image (which is also called 3D datacube. To make it explicit, we insisted on using “MS image” in the following sections.) can be acquired in the spectral branch while the imaging branch captures a high spatial resolution panchromatic (PAN) image. These images are merged by using of the fusion algorithm based on grouping principal component analysis (GPCA) with a high fusion accuracy. Meanwhile, the HR-SIS is insensitive to metamerism because there is no RGB camera in the system. However, it is worth noting that the fusion algorithm assumes that the object scene can be clustered into several groups per the correlation between each spectral band. In some extreme cases, such as a target with high frequency spatial variation and very different spectral lines for different spatial areas, the fusion algorithm will introduce significant errors.
2. Principle of HR-SIS
The layout of the proposed HR-SIS is shown in Fig. 1. It contains the imaging branch and the spectral branch based on a SHIFT spectrometer. The target is first imaged by the objective lens (OL) onto the field stop (FS). The polarizing beam splitter (PBS) behind the FS splits the object beam into two orthogonally polarized components. The reflected component is imaged by the imaging lens (IL) onto the first focal plane array (FPA1). The polarization orientation of the transmitted component is rotated by the first half-wave plate (HWP1). Then, the transmitted component enters into the spectral branch after collimation by the collimating lens (CL). In the spectral branch, the lenslet array (LA) forms the multiple images of the object scene. The Nomarski prisms (NP1 & NP2), the second half-wave plate (HWP2) and the analyzer (A) compose the birefringent polarization interferometer (BPI). Compared to the SHIFT spectrometer, the HR-SIS contains several extra components, including a PBS, an IL, a half-wave plate (HWP1) and a focal plane array (FPA1). The FPA1 can be much smaller than the FPA2.
In HR-SIS, the PBS acts as orthogonal polarizers for the reflected and transmitted beam. The corresponding polarization directions are 90° and 0° with respect to the x-axis. Their Jones matrices are denoted as and , respectively. Assuming that the Jones vector of incident light is , the Jones vector of the transmitted beam and the reflected beam are calculated by,Fig. 1(b), Eq. (1) and (2) can be transformed to,8], specific calculation of Δ is discussed in detail. The detected intensity of each FPA can be derived as,
According to Eq. (5), an MS image can be reconstructed by the reconstruction method described in . Meanwhile, a PAN image is obtained from the FPA1. There is only 25% energy lost in the HR-SIS since only one polarizer is installed in the transmitted branch of the PBS. On the other hand, there are two polarizers which cause a 75% energy lost in the SHIFT spectrometer. Therefore, theoretical optical throughput of the HR-SIS is triple that of the SHIFT spectrometer. However, the signal-to-noise rate (SNR) of the system is a more complicated issue , which is outside the scope of this article.
3. Fusion algorithm based on GPCA
The spatial resolution of the unfused MS image is low due to the trade-off problem, while the PAN image processes high spatial resolution but no spectral information. To solve the dilemma, the fusion technique is applied to achieve high spatial-spectral resolution by merging the MS image and PAN image.
Information fusion technique can synthetize an informative image from images of multiple sources. It is widely used in the remote sensing field . The image with a high spatial-spectral resolution can be generated by combining a high spectral resolution MS image and a high spatial resolution PAN image. This process, also called pansharpening, can reduce the required storage capacity on satellites and data transmission rates. Since 1980s, several effective image fusion techniques have been developed [22–24]. Component substitution (CS) and multiresolution analysis (MRA) are the two most popular classes in these techniques. CS techniques, which rely on the substitution of a component of the MS image by the PAN image, are fast and simple. However, serious spectral distortion is introduced when the MS image and the PAN image don’t share the same spectral bandwidth. MRA techniques can mitigate the spectral distortion and the computational burden depends on the method used to decompose the PAN image. In the snapshot spectral imaging field, the fusion process is required to be fast and accurate. CS techniques are suitable except for the spectral distortion issue. To settle the problem in CS techniques, several essential requirements are necessary ,
- (i) High correlation between the replacement component of the MS and the PAN image, where the replacement component is the component replaced by the PAN image.
- (ii) High percentage of variance accounted by the replacement component.
- (iii) Equal contribution from each MS band in the substitution component.
In the remote sensing field, the spectral responses of the MS image are not always perfectly overlapped with the bandwidth of the PAN image  which causes the loss of Requirement (i). In contrast, two visible light sensors, whose spectral responses cover the same range, are used in spectral branch and imaging branch of the HS-SIS. The arrangement ensures that the system can meet the Requirement (i). On the other hand, Requirement (ii) and (iii) can be met with high correlation among spectral bands of the MS image. The correlation is determined by the spectral complexity of the MS image. Higher spectral complexity lead to lower correlation. Fortunately, sparsity is widely present in nature. The spectral bands of the MS image always can be divided into several groups in which the correlation among group members is high.
Based on the property of the HR-SIS, we proposed a novel fusion algorithm based on grouping principle component analysis (GPCA) to fuse the MS and PAN image. The framework of the proposed fusion algorithm based on GPCA is shown in Fig. 2. The MS image is firstly clustered into several groups according to the factor loading matrix. Then a modified principle component analysis (PCA) fusion process is carried out in each group.
Firstly, a factor analysis based on PCA is operated to the MS image,
During the factor analysis, A is rotated based on the varimax criterion . This procedure, also called as varimax rotation, pushes the elements of A towards ± 1 and 0. According to the rotated factor loading matrix A, a clustering of X, , can be stated as follows:
Then, the MS image is interpolated at the scale of the PAN image. denotes the up-sampling MS image. (k = 1,…,N), which indicates the k-th band of the up-sampling MS image, contains r × r × m × n elements. r is the spatial resolution ratio between the PAN and MS image. Apparently, can be clustered in the same pattern with X, i.e,
According to the property of varimax rotation, there is a strong correlation among elements in cluster while the correlation between and () is weak. Therefor a simple PCA can extract the primary information in each cluster. denotes the first principal component of cluster . In the next step, the detailed information of PAN image is injected into ,27].
Finally, inverse PCA in every cluster () with the pansharpened first principal component yields the fused MS image, , where (k = 1,…, N), indicates the k-th band of the fused MS image.
Obviously, a precise image registration should be performed before the fusion process. A chess-board is viewed by the HR-SIS and a series of subpixel-level feature points can be obtained from the imaging branch and spectral branch by the combination of a quick detection algorithm  and detail refinement strategy . Image registration coefficients can be calculated via the corresponding feature points by using a least-squares fitting. The registration coefficients can be considered as a systematic parameter. Therefore, the primary amount of computations, involving searching feature points and calculation of registration coefficients, is saved.
4. Results and Discussion
The experimental prototype of the proposed system is illustrated as Fig. 3. The cage system is designed to provide precision axial alignment. Two commercial lens (Canon 50mm f/1.4 USM) are utilized as the OL and CL. A broadband polarizing beam splitter (PBS) and an achromatic half wave-plate (HWP1) are between these two lenses. A 1 × Telecentric Lens (TEC-M1065MP, Computar) is used as IL in the reflection optical path of the PBS. A 17 × 22 lenslet array (f = 10.9mm) made from silica glass with a pitch size of 1 mm × 1 mm is arranged after the CL. Meanwhile, the BPI, which is installed in front of the FPA2, comprises an achromatic half wave-plate (HWP2) with fast axis oriented at 45° with respect to the x-axis, two Nomarski prisms (γ = 86.1°, δ = 75°, which are identified in Fig. 1(b)) and an analyzer (A). Two FPAs, FPA1 and FPA2, are used in the system. Specifically, the FPA1 in imaging branch is a 2456 × 2058 monochromatic camera (AM-500 GE, JAI), while the FPA2 (BM-800 GE, JAI) in spectral branch processes 3296 × 2472 active pixels. Note that only part of the pixels are utilized in the FPA1 which can be replaced by a smaller sensor.
To evaluate the performance of the HR-SIS, several experiments had been implemented. Firstly, the spectral accuracy was demonstrated by imaging a commercial color checker. The experiment can only illustrate the spectral accuracy of the HR-SIS since the color checker is not varying rapidly spatially. Secondly, the spectral resolution was measured by using a He-Ne laser as light source. Thirdly, a test chart was imaged by the HR-SIS to illustrate the spatial improving capability. Then, the property of insensitivity to metamerism was demonstrated by distinguishing between a green leaf and artifacts. Finally, to evaluate the performance of the proposed fusion algorithm several colorful targets were captured by the HR-SIS and comparison between several state-of-the-art fusion algorithms and the proposed algorithm was carried out.
4.1 Spectral accuracy
To evaluate the spectral accuracy, an Xrite color checker Passport consisting 24 different color blocks, which is shown in Fig. 4(a), was imaged and a halogen lamp (MI-150, Edmund) was used as illumination. After post-processing, an MS image with spatial size of 170 × 170 pixels was achieved from the original interferogram acquired by the spectral branch. The imaging branch captured a high spatial resolution PAN image with 680 × 680 pixels. The proposed fusion procedure, which is described in Sec. 3, was implemented. The threshold in Eq. (8) is 0.95. As a result, an MS image with a high spatial size of 680 × 680 pixels was produced in a single snapshot.
The average spectra of 30 measurements by a commercial fiber spectrometer (AvaSpec-ULS2048-USB2, Avantes, spectral resolution 1.15nm) was utilized as ground truth. The integrate time of each measurement is 10 ms. The normalized root mean square error (NRMSE) of spectra in 24 color-block areas were calculated. The calculated area of color-block No. 21 is shown in Fig. 4(a) with a red dash square. Each area contains 65 × 65 pixels. The average and standard deviation of NRMSEs for each color-block are depicted in Fig. 4(b). The comparison between the ground truth and data of HR-SIS are further shown in Fig. 4(c).The data of HR-SIS is averaged from 9 × 9 pixels cross the color-block area. It is worth noting that the spectra of each color block are normalized individually. This is based on two factors. On one hand the measured light intensities by the fiber spectrometer are highly impacted by the orientation of the fiber probe. And the spectrum of each color block is measured successively. Therefore, we cannot ensure that all the measurements are under the same condition. On the other hand, the light source used in the experiment is not highly stable. While the measurements by the HR-SIS and fiber spectrometer are not simultaneous. Therefore, the HR-SIS and fiber spectrometer may measure the target under different illumination. Based on the above factors, a uniform normalization factor for all the color blocks cannot be obtained.
Obviously, the NRMSEs of most color blocks are smaller than 10%, expect for blocks with No. 8, No. 13, No. 18, No. 19 and No. 24 (the average NRMSEs are 18.7%, 16.5%, 9.3%, 10.4% and 9.3% respectively). The performance in blue blocks of No. 8, No. 13 and No. 18 is mainly caused by the low radiance of halogen lamp in short-wavelength range. To overcome this problem, an LED (IF803, IFIRE) was added as illumination. The results are shown in Fig. 5. One can see that the NRMSEs of blocks No. 8, 13, and 18 all fall below 10%. Saturated and insufficient reflectance of block No. 19 and No. 24 lead to high NRMSEs.
4.2 Spectral resolution
In the prototype, 13 × 18 sub-lenses were employed. According to the apex angle of the NPs (γ = 86.1°, which is identified in Fig. 1(b)), the sample interval of OPD between the adjacent sub-lenses is approximate to 0.2 μm. Therefore, the spectral resolution of the HR-SIS is better than 250 cm−1, which is approximately 10 nm at 632.8 nm. To demonstrate the spectral resolution, an integrating sphere (GAF-030, NMERRY) irradiated by a He-Ne laser (25-LHP-925-230, Melles Griot) is used as an objective and the full width at half maximum (FWHM) of obtained spectrum was used to characterize the spectral resolution of the HR-SIS. Figure 6 is the spectrum of the center point in the field of view (FOV). The standard deviation of fitted Gaussian curve (the red dotted line) is 4.1 nm which yields a FWHM of 9.7 nm. The average FWHM of 140 × 140 points across the FOV is 9.6 nm.
4.3 Spatial resolution
A test chart (Negative USAF1951 test target, Thorlabs) was captured by the proposed system to demonstrate the improvement of spatial resolution through the fusion algorithm. The composite images of the test chart before and after fusion are shown in Fig. 7. The composite image after image fusion contains much more spatial details. According to the resolving power lookup table of the test chart (as shown in Table 1) the spatial resolution was improved from 0.891 lines/mm (Group −1, element 6) to 3.56 lines/mm (Group 1, element 6).
4.4 Insensitivity to metamerism
Unlike other snapshot spectral imaging techniques based on information fusion [18–20], the proposed HR-SIS is insensitive to metamerism because of the monochrome sensors and high spectral resolution. As shown in Fig. 8(a), a leaf was attached to a green artificial background. Four different-sized green artifacts were cut into the word ‘HIT’ attached on the leaf, which is illuminated by a halogen lamp (MI-150, Edmund). As shown in Fig. 8(a), the artifacts are hardly distinguished from the RGB image obtained by a color digital camera since they appear the same color with the leaf, which is a typical example of metamerism. The spectra of two different points on the leaf and the artifact are shown in Fig. 8(b). The spectral difference is because of the absorbing property of chlorophyll. Two bands of the MS image at 655.2 nm and 713.2 nm are shown in Figs. 8(c) and 8(d), respectively. The artifacts can be clearly identified in the spectral band of 713.2 nm.
4.5 Performance of the fusion algorithm based on GPCA
To simplify the description, some notations, which are listed in Table 2, are used in the following. Meanwhile, the relationships among the notations are further depicted in Fig. 9. The proposed algorithm relies on the property of the HR-SIS, that the spectral response of the two sensors acquiring the MS and PAN image cover the same range. In this case, the correlation coefficient between and is theoretically equal to 1. The property is demonstrated in the following.
Twelve colorful maps with abundant spatial details, as shown in Fig. 10(a), were measured by our HR-SIS. A halogen lamp (MI-150, Edmund) and an LED (IF803, IFIRE) were used as illumination. MS images with spatial size of 170 × 170 pixels were obtained by the spectral branch while the imaging branch captured PAN images with spatial size of 680 × 680 pixels. The correlation coefficients between the and are plotted in Fig. 10(d). All correlation coefficients of the maps are not less than 0.9. The synthetic MS image and downsampled PAN image of the map No. 2, whose correlation coefficient is 0.9, are shown in Figs. 10(b) and 10(c) respectively. The differences between the two images are mainly caused by the defocusing error in the spectral branch.
To evaluate the performance of the proposed fusion algorithm based on GPCA, several state-of-the-art fusion algorithms, including PCA , adaptive Gram–Schmidt (GSA)  and modulation transfer functions & generalized Laplacian pyramid (MTF-GLP) , are compared with GPCA. Considering the absence of the reference MS image with high spatial and spectral resolution, the assessment of the fusion results follows the Wald’s protocol  which states that the fusion results must hold three properties, i.e.,
- (i) should be as close as possible to ;
- (ii) should be as close as possible to ;
- (iii) The spectral set of should be as close as possible to that of
To characterize the similarity between two MS images several metrics have been raised in the past decades . In this paper, the CC, SAM, and ERGAS were utilized. These metrics are defined below. Assume that and denote the reference MS image and fused MS image, respectively.
1) CC: The cross correlation (CC) defined below indicates the spatial distortion of the fused MS image,
2) SAM: The spectral angle mapper (SAM) characterizes the spectral distortion between MS images and can be defined as,
3) ERGAS: The erreur relative globale adimensionnelle de synthèse (ERGAS) is a global fusion quality indicator, which is defined as,
The CC and SAM between and were calculated to characterize the spatial and spectral distortions of , respectively. Meanwhile, the ERGAS between and was used to indicate the global deviation of . The evaluating results of different fusion algorithms on the 12 maps in Fig. 10(a) are shown in Fig. 11. The proposed algorithm based on GPCA performs competitively with other state-of-the-art algorithms. Additionally, the fused MS images by the proposed algorithm were compared with the upsampled MS images without fusion. The CC and SAM between and fused by the proposed algorithm are plotted as red lines, while the CC and SAM between and are plotted as blue lines in Figs. 12(a) and 12(b). Note that the improvements in the SAM by the fusion are not obvious when the target images don’t have large color variation such as images No. 5-12. Because interpolations introduce less deviations when the data changes more gently.
Meanwhile, the local accuracy between and the fused by the proposed fusion algorithm was further evaluated. The NRMSEs between the and at the pixels along the red dotted line, as shown in Fig. 13(a), are plotted in Fig. 13(b). The cross section profile of is also plotted in Fig. 13(b). It can be seen that, the NRMSE is higher near the edges of the color-blocks than that in the interior areas. This is mainly caused by the blur introduced during the up-sample of the unfused MS image.
The computing time of the reconstruction and fusion algorithm was also evaluated. The whole post-process procedure was implemented 30 times in Matlab R2012b with an Intel Core i5-4200H CPU. There are slight differences in computing time between each repetition. It is mainly caused by the fact that the condition of the computer is changing slightly over time. The average and standard deviation of the computing times are shown in Fig. 14. The computing time will severely limit the acquisition speed of the HR-SIS. However, it can be further reduced by utilizing a graphics processing unit (GPU).
In this paper, the high resolution snapshot imaging spectrometer (HR-SIS) was proposed and experimentally demonstrated. Meanwhile a matching fusion algorithm based on GPCA was developed. A high spectral resolution MS image with low spatial resolution and a high spatial resolution PAN image were acquired by the spectral branch and imaging branch of the HR-SIS, respectively. The fusion procedure based on GPCA was carried out to produce a high spatial-spectral resolution MS image. The HR-SIS can acquire an MS image with spatial size of 680 × 680 pixels and the spectral resolution of 250 cm−1 in a snapshot. The spectral accuracy, spatial resolution and insensitivity to metamerism of the HR-SIS were experimentally demonstrated. The NRMSE of the spectral accuracy can achieve around 10% under proper illumination. The spectral resolution of the HR-SIS is better than 10 nm at 632.8nm and the spatial resolution of the system is 3.56 lines/mm. The HR-SIS is compact and precise in spatial-spectral domain. Meanwhile, in contrast to other snapshot imaging spectrometer based on information fusion [18–20], the HR-SIS is insensitive to metamerism. Because the snapshot imaging spectrometers described in [18–20] all utilize RGB cameras while two monochromatic cameras are used in the HR-SIS. The proposed fusion algorithm was evaluated by observing twelve colorful maps. The property of the HR-SIS, on which the proposed fusion algorithm relies, was highlighted by an experiment. The proposed GPCA algorithm was compared with other state-of-the-art fusion algorithms. Under the Wald’s protocol, the GPCA algorithm is competitively with others on the ERGAS, CC and SAM. Meanwhile, the fusion algorithm also can be implemented in many other snapshot imaging spectrometers, such as techniques described in [7,15]. The HR-SIS can be applied in biomedical imaging, microscopy, endoscopy and air surveillance in concert with an unmanned aerial vehicle (UAV).
National High Technology Research and Development Program of China (2015AA042401).
References and links
3. D. J. Mulla, “Twenty five years of remote sensing in precision agriculture: key advances and remaining knowledge gaps,” Biosystems Eng. 114(4), 358–371 (2013). [CrossRef]
4. F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012). [CrossRef]
6. N. Bedard, R. A. Schwarz, A. Hu, V. Bhattar, J. Howe, M. D. Williams, A. M. Gillenwater, R. Richards-Kortum, and T. S. Tkaczyk, “Multimodal snapshot spectral imaging for oral cancer diagnostics: a pilot study,” Biomed. Opt. Express 4(6), 938–949 (2013). [CrossRef] [PubMed]
7. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). [CrossRef] [PubMed]
9. A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994). [CrossRef]
13. N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013). [CrossRef]
17. A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989). [CrossRef]
18. C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014). [CrossRef]
21. M. C. Abrams, S. P. Davis and J. W. Brault, Fourier Transform Spectrometry (Academic, 2001), Chap. 8.
22. H. Ghassemian, “A review of remote sensing image fusion methods,” Inf. Fusion 32, 75–89 (2016). [CrossRef]
23. L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015). [CrossRef]
24. G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015). [CrossRef]
25. V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogramm. Eng. Remote Sensing 58(5), 561–567 (1992).
26. C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008). [CrossRef]
27. H. F. Kaiser, “The varimax criterion for analytic rotation in factor analysis,” Psychometrika 23(3), 187–200 (1958). [CrossRef]
28. X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005). [CrossRef]
29. S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).
30. Signal and Image Processing Institute, University of Southern California, “The USC-SIPI Image Database,” http://sipi.usc.edu/database/
31. S. K. Mitra and L. Lucchese, “Using saddle points for subpixel feature derection in camera calibattion targets,” in Proceedings of IEEE Asia-Pacific Conference on Circuits and Systems (IEEE, 2002), pp. 191–195.
32. B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007). [CrossRef]
33. B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006). [CrossRef]
34. L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).