Abstract

We reported a high resolution snapshot imaging spectrometer (HR-SIS) and a fusion algorithm based on the properties of the HR-SIS. The system consists of an imaging branch and a spectral branch. The imaging branch captures a high spatial resolution panchromatic image with 680 × 680 pixels, while the spectral branch acquires a low spatial resolution spectral image with spectral resolution of 250 cm−1. By using a fusion algorithm base on grouping principal component analysis, the spectral image is highly improved in spatial resolution. Experimental results demonstrated that the performance of the proposed algorithm is competitive with other state-of-the-art algorithms. The computing time for a single frame is less than 1 min with an Intel Core i5-4200H CPU, which can be further reduced by utilizing a graphics processing unit (GPU).

© 2016 Optical Society of America

1. Introduction

An imaging spectrometer is capable of capturing a three-dimensional (3D) datacube (x, y, λ) of an object scene. In recent decades, imaging spectrometers have been wildly applied in remote sensing and biomedical imaging [1–4]. In some specific applications, such as cellular dynamics research and tissue classification in vivo, imaging spectrometers are required to capture real-time scenes [5,6]. However, most traditional imaging spectrometers suffer from low temporal resolution caused by long scanning time which causes serious blur for dynamic scenes.

To overcome such drawbacks, snapshot imaging spectrometers, which can capture the 3D datacube in a single snapshot, were developed [7–14]. Gao et al. developed a snapshot Image Mapping Spectrometer (IMS) based on the dispersion of image zones by an image mapper, a lens array and a prism array [7]. A snapshot hyperspectral imaging Fourier transform (SHIFT) spectrometer was derived from a Multiple-image Fourier Transform Spectrometer (MFTS) [8,9]. The computed tomographic imaging spectrometer (CTIS) utilized a two-dimensional (2D) disperser to project the object scene onto a detector and the 3D datacube was recovered by tomographic reconstruction techniques [10]. The coded aperture snapshot spectral imaging (CASSI) uses a binary-coded mask to build a transmission pattern on the detector and estimated the 3D datacube during post-processing [11]. Image replicating imaging spectrometer (IRIS) is compact and available in real-time applications because its direct acquisition of a 3D datacube is based on a Lyot Filter [12].

In general, some of these techniques acquire all voxels of the 3D datacube simultaneously by dividing it into multiple 2D elements. However, limited by the number of detector pixels, this mechanism causes a serious trade-off problem between spatial resolution and spectral resolution. Hegyi et al developed a hyperspectral imaging system based on a liquid crystal polarization interferometer [15]. In this system a series of small images are acquired to reconstruct a 3D datacube. Trade-off among the spectral resolution, imaging speed, and spatial resolution can be implemented in software. However, the system only makes the trade-off selectable and it does not migrate it. The trade-off problem in CASSI was reduced by using compressive sensing theory and Wang et al. proposed a dual-camera CASSI [16], which utilized a grayscale camera to improve the reconstruction quality, though overall computational burden remains high.

Information fusion technique was first proposed in military survey field to produce an informative image from multiple sources [17]. By applying the information fusion technique, the measurement systems can achieve high resolution both in spatial and spectral domain by merging a multispectral (MS) image and a high spatial resolution image. Therefore, the trade-off problem is reduced, since the high resolution in spatial and spectral domain is not required simultaneously. Recently, the information fusion technique was applied to spectral imaging. Ma et al. achieved content-adaptive high resolution hyperspectral video system based on a trilateral interpolation filtering approach [18,19]. A hybrid-resolution spectral imaging (HRSI) was proposed to combine a high resolution RGB image and a low resolution spectral image by utilizing piecewise Wiener estimation [20]. For scenes with high sparsity in spectral domain, these techniques can achieve high spatial-spectral resolution. However, for the low sparsity situation, these techniques are unavailable because of their low sampling rate in the low spatial resolution hyperspectral image. Moreover, these techniques are very sensitive to metamerism, which is a phenomenon that different spectra appear as the same color to RGB cameras and human eyes.

Here, we report a high resolution snapshot imaging spectrometer (HR-SIS) and a matching fusion algorithm. In the proposed HR-SIS, a polarization beam splitter is used to separate the system into a spectral branch and an imaging branch. The spectral branch, which consists a spectral imager based on the SHIFT spectrometer, is compact and robust. A low spatial resolution MS image (which is also called 3D datacube. To make it explicit, we insisted on using “MS image” in the following sections.) can be acquired in the spectral branch while the imaging branch captures a high spatial resolution panchromatic (PAN) image. These images are merged by using of the fusion algorithm based on grouping principal component analysis (GPCA) with a high fusion accuracy. Meanwhile, the HR-SIS is insensitive to metamerism because there is no RGB camera in the system. However, it is worth noting that the fusion algorithm assumes that the object scene can be clustered into several groups per the correlation between each spectral band. In some extreme cases, such as a target with high frequency spatial variation and very different spectral lines for different spatial areas, the fusion algorithm will introduce significant errors.

2. Principle of HR-SIS

The layout of the proposed HR-SIS is shown in Fig. 1. It contains the imaging branch and the spectral branch based on a SHIFT spectrometer. The target is first imaged by the objective lens (OL) onto the field stop (FS). The polarizing beam splitter (PBS) behind the FS splits the object beam into two orthogonally polarized components. The reflected component is imaged by the imaging lens (IL) onto the first focal plane array (FPA1). The polarization orientation of the transmitted component is rotated by the first half-wave plate (HWP1). Then, the transmitted component enters into the spectral branch after collimation by the collimating lens (CL). In the spectral branch, the lenslet array (LA) forms the multiple images of the object scene. The Nomarski prisms (NP1 & NP2), the second half-wave plate (HWP2) and the analyzer (A) compose the birefringent polarization interferometer (BPI). Compared to the SHIFT spectrometer, the HR-SIS contains several extra components, including a PBS, an IL, a half-wave plate (HWP1) and a focal plane array (FPA1). The FPA1 can be much smaller than the FPA2.

 figure: Fig. 1

Fig. 1 (a) Schematic of HR-SIS. (b) Polarization optical elements with optical axes indicated by arrows. Acronyms: Objective Lens (OL), Field Stop (FS), Polarizing Beam Splitter (PBS), Imaging Lens (IL), Focal Plane Array (FPA), Half-Wave Plate (HWP), Collimating Lens (CL), Lenslet Array (LA), Birefringent Polarization Interferometer (BPI) Nomarski Prism (NP), Analyzer (A).

Download Full Size | PPT Slide | PDF

In HR-SIS, the PBS acts as orthogonal polarizers for the reflected and transmitted beam. The corresponding polarization directions are 90° and 0° with respect to the x-axis. Their Jones matrices are denoted as JR and JT, respectively. Assuming that the Jones vector of incident light is EI=[ExEy]T, the Jones vector of the transmitted beam and the reflected beam are calculated by,

ET=JAJBPIJHWP1JTEI
ΕR=JREI
where JA, JBPI and JHWP1 are the Jones matrices of the analyzer, BPI and first HWP, respectively. According to the optical axes of the polarization elements depicted in Fig. 1(b), Eq. (1) and (2) can be transformed to,
ΕΤ=12[1111][100ei2πσΔ]22[1111][1000][ExEy]=24Ex(1+ei2πσΔ)[11]
ER=[0001][ExEy]=[0Ey]
where Δ is the optical path difference (OPD) introduced by the BPI. In [8], specific calculation of Δ is discussed in detail. The detected intensity of each FPA can be derived as,
IFPA2=Ix(σ)2[1+cos(2πσΔ)]
IFPA1=Iy(σ)
where Ix(σ) and Iy(σ) are the x- and y-component of the input spectra, respectively.

According to Eq. (5), an MS image can be reconstructed by the reconstruction method described in [8]. Meanwhile, a PAN image is obtained from the FPA1. There is only 25% energy lost in the HR-SIS since only one polarizer is installed in the transmitted branch of the PBS. On the other hand, there are two polarizers which cause a 75% energy lost in the SHIFT spectrometer. Therefore, theoretical optical throughput of the HR-SIS is triple that of the SHIFT spectrometer. However, the signal-to-noise rate (SNR) of the system is a more complicated issue [21], which is outside the scope of this article.

3. Fusion algorithm based on GPCA

The spatial resolution of the unfused MS image is low due to the trade-off problem, while the PAN image processes high spatial resolution but no spectral information. To solve the dilemma, the fusion technique is applied to achieve high spatial-spectral resolution by merging the MS image and PAN image.

Information fusion technique can synthetize an informative image from images of multiple sources. It is widely used in the remote sensing field [22]. The image with a high spatial-spectral resolution can be generated by combining a high spectral resolution MS image and a high spatial resolution PAN image. This process, also called pansharpening, can reduce the required storage capacity on satellites and data transmission rates. Since 1980s, several effective image fusion techniques have been developed [22–24]. Component substitution (CS) and multiresolution analysis (MRA) are the two most popular classes in these techniques. CS techniques, which rely on the substitution of a component of the MS image by the PAN image, are fast and simple. However, serious spectral distortion is introduced when the MS image and the PAN image don’t share the same spectral bandwidth. MRA techniques can mitigate the spectral distortion and the computational burden depends on the method used to decompose the PAN image. In the snapshot spectral imaging field, the fusion process is required to be fast and accurate. CS techniques are suitable except for the spectral distortion issue. To settle the problem in CS techniques, several essential requirements are necessary [25],

  • (i) High correlation between the replacement component of the MS and the PAN image, where the replacement component is the component replaced by the PAN image.
  • (ii) High percentage of variance accounted by the replacement component.
  • (iii) Equal contribution from each MS band in the substitution component.

In the remote sensing field, the spectral responses of the MS image are not always perfectly overlapped with the bandwidth of the PAN image [25] which causes the loss of Requirement (i). In contrast, two visible light sensors, whose spectral responses cover the same range, are used in spectral branch and imaging branch of the HS-SIS. The arrangement ensures that the system can meet the Requirement (i). On the other hand, Requirement (ii) and (iii) can be met with high correlation among spectral bands of the MS image. The correlation is determined by the spectral complexity of the MS image. Higher spectral complexity lead to lower correlation. Fortunately, sparsity is widely present in nature. The spectral bands of the MS image always can be divided into several groups in which the correlation among group members is high.

Based on the property of the HR-SIS, we proposed a novel fusion algorithm based on grouping principle component analysis (GPCA) to fuse the MS and PAN image. The framework of the proposed fusion algorithm based on GPCA is shown in Fig. 2. The MS image is firstly clustered into several groups according to the factor loading matrix. Then a modified principle component analysis (PCA) fusion process is carried out in each group.

 figure: Fig. 2

Fig. 2 Framework of proposed fusion algorithm based on GPCA.

Download Full Size | PPT Slide | PDF

Firstly, a factor analysis based on PCA is operated to the MS image,

X=AF+ε
where XT=[X1,,XN] denotes the MS image. N is the number of spectral bands. Xk (k = 1,…,N), which indicates the k-th band of the MS image, contains m × n elements. m × n is the number of pixels in a single band of the MS image. A is the factor loading matrix with size of N×M, where M is the number of common factors. FT=[F1,,FM] are the common factors of X. M is determined when the accumulated contribution ratio of the principal components of X exceed a threshold value,
i=1Mλii=1Nλithreshold
where λi (λ1>λ2>>λN) is the i-th eigenvalue of the covariance matrix of X.

During the factor analysis, A is rotated based on the varimax criterion [26]. This procedure, also called as varimax rotation, pushes the elements of A towards ± 1 and 0. According to the rotated factor loading matrix A, a clustering of X,  π(X), can be stated as follows:

π(X)={C1,C2,...,C|π(X)|}
where Ci (1i|π(X)|) is a single cluster of  π(X), and

X=i=1|π(X)|Ci

Then, the MS image is interpolated at the scale of the PAN image. X˜T=[X˜1,,X˜N] denotes the up-sampling MS image. X˜k (k = 1,…,N), which indicates the k-th band of the up-sampling MS image, contains r × r × m × n elements. r is the spatial resolution ratio between the PAN and MS image. Apparently, X˜ can be clustered in the same pattern with X, i.e,

π(X˜)={C˜1,C˜2,...,C˜|π(X˜)|}

According to the property of varimax rotation, there is a strong correlation among elements in cluster C˜i while the correlation between C˜i and C˜j (ij) is weak. Therefor a simple PCA can extract the primary information in each cluster. PC˜1i(i=1,,|π(X˜)|) denotes the first principal component of cluster C˜i. In the next step, the detailed information of PAN image is injected into PC˜1i,

PC^1i=PC˜1i+α(PPL)
where PC˜1i is the pansharpened version of PC˜1i. P denotes the PAN image with r × r × m × n elements. PL is a low-pass version of P. α is the gain factor calculated by,
α=σ(PC˜1i)σ(P)PC˜1ij=1|π(X˜)|PC˜1j
where σ(PC˜1i) and σ(P) are the standard deviation of PC˜1i and P respectively. The ratio of PC˜1i and Σj=1|π(X˜)|PC˜1j is based on the consideration of physical properties of the objective [27].

Finally, inverse PCA in every cluster C˜i (1i|π(Χ~)|) with the pansharpened first principal component PC˜1i yields the fused MS image, X^T=[X^1,,X^N], where X^k (k = 1,…, N), indicates the k-th band of the fused MS image.

Obviously, a precise image registration should be performed before the fusion process. A chess-board is viewed by the HR-SIS and a series of subpixel-level feature points can be obtained from the imaging branch and spectral branch by the combination of a quick detection algorithm [28] and detail refinement strategy [29]. Image registration coefficients can be calculated via the corresponding feature points by using a least-squares fitting. The registration coefficients can be considered as a systematic parameter. Therefore, the primary amount of computations, involving searching feature points and calculation of registration coefficients, is saved.

4. Results and Discussion

The experimental prototype of the proposed system is illustrated as Fig. 3. The cage system is designed to provide precision axial alignment. Two commercial lens (Canon 50mm f/1.4 USM) are utilized as the OL and CL. A broadband polarizing beam splitter (PBS) and an achromatic half wave-plate (HWP1) are between these two lenses. A 1 × Telecentric Lens (TEC-M1065MP, Computar) is used as IL in the reflection optical path of the PBS. A 17 × 22 lenslet array (f = 10.9mm) made from silica glass with a pitch size of 1 mm × 1 mm is arranged after the CL. Meanwhile, the BPI, which is installed in front of the FPA2, comprises an achromatic half wave-plate (HWP2) with fast axis oriented at 45° with respect to the x-axis, two Nomarski prisms (γ = 86.1°, δ = 75°, which are identified in Fig. 1(b)) and an analyzer (A). Two FPAs, FPA1 and FPA2, are used in the system. Specifically, the FPA1 in imaging branch is a 2456 × 2058 monochromatic camera (AM-500 GE, JAI), while the FPA2 (BM-800 GE, JAI) in spectral branch processes 3296 × 2472 active pixels. Note that only part of the pixels are utilized in the FPA1 which can be replaced by a smaller sensor.

 figure: Fig. 3

Fig. 3 Experimental prototype of the proposed system. Acronyms: Objective Lens (OL), Field Stop (FS), Polarizing Beam Splitter (PBS), Half-Wave Plate (HWP), Imaging Lens (IL), Focal Plane Array (FPA), Collimating Lens (CL), Lenslet Array (LA), Birefringent Polarization Interferometer (BPI).

Download Full Size | PPT Slide | PDF

To evaluate the performance of the HR-SIS, several experiments had been implemented. Firstly, the spectral accuracy was demonstrated by imaging a commercial color checker. The experiment can only illustrate the spectral accuracy of the HR-SIS since the color checker is not varying rapidly spatially. Secondly, the spectral resolution was measured by using a He-Ne laser as light source. Thirdly, a test chart was imaged by the HR-SIS to illustrate the spatial improving capability. Then, the property of insensitivity to metamerism was demonstrated by distinguishing between a green leaf and artifacts. Finally, to evaluate the performance of the proposed fusion algorithm several colorful targets were captured by the HR-SIS and comparison between several state-of-the-art fusion algorithms and the proposed algorithm was carried out.

4.1 Spectral accuracy

To evaluate the spectral accuracy, an Xrite color checker Passport consisting 24 different color blocks, which is shown in Fig. 4(a), was imaged and a halogen lamp (MI-150, Edmund) was used as illumination. After post-processing, an MS image with spatial size of 170 × 170 pixels was achieved from the original interferogram acquired by the spectral branch. The imaging branch captured a high spatial resolution PAN image with 680 × 680 pixels. The proposed fusion procedure, which is described in Sec. 3, was implemented. The threshold in Eq. (8) is 0.95. As a result, an MS image with a high spatial size of 680 × 680 pixels was produced in a single snapshot.

 figure: Fig. 4

Fig. 4 (a) Xrite color checker. (b) Normalized root mean square error (NRMSE) in 24 color-block areas. (c) Spectral curves of color blocks from HR-SIS and AvaSpec (ground truth).

Download Full Size | PPT Slide | PDF

The average spectra of 30 measurements by a commercial fiber spectrometer (AvaSpec-ULS2048-USB2, Avantes, spectral resolution 1.15nm) was utilized as ground truth. The integrate time of each measurement is 10 ms. The normalized root mean square error (NRMSE) of spectra in 24 color-block areas were calculated. The calculated area of color-block No. 21 is shown in Fig. 4(a) with a red dash square. Each area contains 65 × 65 pixels. The average and standard deviation of NRMSEs for each color-block are depicted in Fig. 4(b). The comparison between the ground truth and data of HR-SIS are further shown in Fig. 4(c).The data of HR-SIS is averaged from 9 × 9 pixels cross the color-block area. It is worth noting that the spectra of each color block are normalized individually. This is based on two factors. On one hand the measured light intensities by the fiber spectrometer are highly impacted by the orientation of the fiber probe. And the spectrum of each color block is measured successively. Therefore, we cannot ensure that all the measurements are under the same condition. On the other hand, the light source used in the experiment is not highly stable. While the measurements by the HR-SIS and fiber spectrometer are not simultaneous. Therefore, the HR-SIS and fiber spectrometer may measure the target under different illumination. Based on the above factors, a uniform normalization factor for all the color blocks cannot be obtained.

Obviously, the NRMSEs of most color blocks are smaller than 10%, expect for blocks with No. 8, No. 13, No. 18, No. 19 and No. 24 (the average NRMSEs are 18.7%, 16.5%, 9.3%, 10.4% and 9.3% respectively). The performance in blue blocks of No. 8, No. 13 and No. 18 is mainly caused by the low radiance of halogen lamp in short-wavelength range. To overcome this problem, an LED (IF803, IFIRE) was added as illumination. The results are shown in Fig. 5. One can see that the NRMSEs of blocks No. 8, 13, and 18 all fall below 10%. Saturated and insufficient reflectance of block No. 19 and No. 24 lead to high NRMSEs.

 figure: Fig. 5

Fig. 5 Spectral curves of color blocks (a) No.8, (b) No. 13, and (c) No. 18 from HR-SIS and AvaSpec (ground truth). (d) NRMSEs in the three color-block areas.

Download Full Size | PPT Slide | PDF

4.2 Spectral resolution

In the prototype, 13 × 18 sub-lenses were employed. According to the apex angle of the NPs (γ = 86.1°, which is identified in Fig. 1(b)), the sample interval of OPD between the adjacent sub-lenses is approximate to 0.2 μm. Therefore, the spectral resolution of the HR-SIS is better than 250 cm−1, which is approximately 10 nm at 632.8 nm. To demonstrate the spectral resolution, an integrating sphere (GAF-030, NMERRY) irradiated by a He-Ne laser (25-LHP-925-230, Melles Griot) is used as an objective and the full width at half maximum (FWHM) of obtained spectrum was used to characterize the spectral resolution of the HR-SIS. Figure 6 is the spectrum of the center point in the field of view (FOV). The standard deviation of fitted Gaussian curve (the red dotted line) is 4.1 nm which yields a FWHM of 9.7 nm. The average FWHM of 140 × 140 points across the FOV is 9.6 nm.

 figure: Fig. 6

Fig. 6 Spectrum of center point of the FOV with a fitted Gaussian curve (red dashed line).

Download Full Size | PPT Slide | PDF

4.3 Spatial resolution

A test chart (Negative USAF1951 test target, Thorlabs) was captured by the proposed system to demonstrate the improvement of spatial resolution through the fusion algorithm. The composite images of the test chart before and after fusion are shown in Fig. 7. The composite image after image fusion contains much more spatial details. According to the resolving power lookup table of the test chart (as shown in Table 1) the spatial resolution was improved from 0.891 lines/mm (Group −1, element 6) to 3.56 lines/mm (Group 1, element 6).

 figure: Fig. 7

Fig. 7 (a) Composite image from the MS image before fusion. (b) Composite image from the MS image after fusion.

Download Full Size | PPT Slide | PDF

Tables Icon

Table 1. Number of Line Pairs / mm in USAF Resolving Power Test Target 1951

4.4 Insensitivity to metamerism

Unlike other snapshot spectral imaging techniques based on information fusion [18–20], the proposed HR-SIS is insensitive to metamerism because of the monochrome sensors and high spectral resolution. As shown in Fig. 8(a), a leaf was attached to a green artificial background. Four different-sized green artifacts were cut into the word ‘HIT’ attached on the leaf, which is illuminated by a halogen lamp (MI-150, Edmund). As shown in Fig. 8(a), the artifacts are hardly distinguished from the RGB image obtained by a color digital camera since they appear the same color with the leaf, which is a typical example of metamerism. The spectra of two different points on the leaf and the artifact are shown in Fig. 8(b). The spectral difference is because of the absorbing property of chlorophyll. Two bands of the MS image at 655.2 nm and 713.2 nm are shown in Figs. 8(c) and 8(d), respectively. The artifacts can be clearly identified in the spectral band of 713.2 nm.

 figure: Fig. 8

Fig. 8 (a) RGB picture of the scene (white balance was corrected in Photoshop). (b) Spectra of two points on the leaf and the green artifacts respectively. (c) Spectral band at 655.2nm. (d) Spectral band at 713.2nm.

Download Full Size | PPT Slide | PDF

4.5 Performance of the fusion algorithm based on GPCA

To simplify the description, some notations, which are listed in Table 2, are used in the following. Meanwhile, the relationships among the notations are further depicted in Fig. 9. The proposed algorithm relies on the property of the HR-SIS, that the spectral response of the two sensors acquiring the MS and PAN image cover the same range. In this case, the correlation coefficient between MSsum and PANl is theoretically equal to 1. The property is demonstrated in the following.

Tables Icon

Table 2. List of notation

 figure: Fig. 9

Fig. 9 The relationships among the notation in Table 2

Download Full Size | PPT Slide | PDF

Twelve colorful maps with abundant spatial details, as shown in Fig. 10(a), were measured by our HR-SIS. A halogen lamp (MI-150, Edmund) and an LED (IF803, IFIRE) were used as illumination. MS images with spatial size of 170 × 170 pixels were obtained by the spectral branch while the imaging branch captured PAN images with spatial size of 680 × 680 pixels. The correlation coefficients between the MSsum and PANl are plotted in Fig. 10(d). All correlation coefficients of the maps are not less than 0.9. The synthetic MS image and downsampled PAN image of the map No. 2, whose correlation coefficient is 0.9, are shown in Figs. 10(b) and 10(c) respectively. The differences between the two images are mainly caused by the defocusing error in the spectral branch.

 figure: Fig. 10

Fig. 10 (a) Colorful maps with abundant spatial details. Among them, image No. 1-3 were homemade, and image No. 5-12 were obtained from the online image database [30]. Image No. 4 is an Xrite color checker. These pictures, except image No. 4, were printed by color printers (Epson Stylus Photo 1400 for No. 2, 6 and 8; Xerox Workcentre 7346 for No. 1, 3, 5 and 7; Canon iP2780 for No. 8-12). All pictures were taken by Canon 550D color digital camera and white balance was corrected in Photoshop. (b) Synthetic MS image and (c) downsampled PAN image of the map No. 13. (d) Correlation coefficients between the downsampled PAN images and synthetic MS images.

Download Full Size | PPT Slide | PDF

To evaluate the performance of the proposed fusion algorithm based on GPCA, several state-of-the-art fusion algorithms, including PCA [31], adaptive Gram–Schmidt (GSA) [32] and modulation transfer functions & generalized Laplacian pyramid (MTF-GLP) [33], are compared with GPCA. Considering the absence of the reference MS image with high spatial and spectral resolution, the assessment of the fusion results follows the Wald’s protocol [34] which states that the fusion results must hold three properties, i.e.,

  • (i) MSd* should be as close as possible to MSo;
  • (ii) MSo* should be as close as possible to MSo;
  • (iii) The spectral set of MSo* should be as close as possible to that of MSo

To characterize the similarity between two MS images several metrics have been raised in the past decades [23]. In this paper, the CC, SAM, and ERGAS were utilized. These metrics are defined below. Assume that Xref and X^ denote the reference MS image and fused MS image, respectively.

1) CC: The cross correlation (CC) defined below indicates the spatial distortion of the fused MS image,

CC(X^,Xref)=1Ni=1NCCS(X^i,Xiref)
where N is the number of spectral bands. Xiref and X^i are the i-th band image of the reference and fused MS image, respectively. CCS is the cross correlation for between single-band images. Higher value of the CC indicates better performance.

2) SAM: The spectral angle mapper (SAM) characterizes the spectral distortion between MS images and can be defined as,

SAM(X^,Xref)=1m×ni=1mj=1narccos(<x^i,j,xi,jref>x^i,j2xi,jref2)
where m and n are the number of rows and columns in a single-band image. x^i,j and xi,jref are spectrum of X^ and Xref at the spatial location of (i, j). Lower value of the SAM indicates better performance.

3) ERGAS: The erreur relative globale adimensionnelle de synthèse (ERGAS) is a global fusion quality indicator, which is defined as,

ERGAS(X^,Xref)=100×1r1Ni=1N(N×RMSE(X^i,Xiref)j=1NXiref)2
where r is the spatial resolution ratio between the PAN and MS image. REMS is the root mean squared error, which is defined as,
RMSE(X^i,Xiref)=X^iXirefF2m×n
Lower value of the ERGAS indicates better performance.

The CC and SAM between MSo* and MSo were calculated to characterize the spatial and spectral distortions of MSo*, respectively. Meanwhile, the ERGAS between MSd* and MSo was used to indicate the global deviation of .MSd* The evaluating results of different fusion algorithms on the 12 maps in Fig. 10(a) are shown in Fig. 11. The proposed algorithm based on GPCA performs competitively with other state-of-the-art algorithms. Additionally, the fused MS images by the proposed algorithm were compared with the upsampled MS images without fusion. The CC and SAM between MSo and MSo* fused by the proposed algorithm are plotted as red lines, while the CC and SAM between MSo and MSu* are plotted as blue lines in Figs. 12(a) and 12(b). Note that the improvements in the SAM by the fusion are not obvious when the target images don’t have large color variation such as images No. 5-12. Because interpolations introduce less deviations when the data changes more gently.

 figure: Fig. 11

Fig. 11 Evaluating results of different fusion algorithms. (a) ERGAS between MSd* and MSo. (b) CC and (c) SAM between MSo* and MSo. Note that the ideal values of the ERGAS, CC and SAM are 0, 1 and 0, respectively.

Download Full Size | PPT Slide | PDF

 figure: Fig. 12

Fig. 12 Comparison between fused MS images and upsampled MS images

Download Full Size | PPT Slide | PDF

Meanwhile, the local accuracy between MSo and the MSo* fused by the proposed fusion algorithm was further evaluated. The NRMSEs between the MSo* and MSo at the pixels along the red dotted line, as shown in Fig. 13(a), are plotted in Fig. 13(b). The cross section profile of MSsum is also plotted in Fig. 13(b). It can be seen that, the NRMSE is higher near the edges of the color-blocks than that in the interior areas. This is mainly caused by the blur introduced during the up-sample of the unfused MS image.

 figure: Fig. 13

Fig. 13 (a) RGB picture of the color checker obtained by a commercial color camera. (b) The red line indicates the NRMSE between MSo* and MSo at the pixels along the red dotted line, while the blue line is the cross section profile of MSsum.

Download Full Size | PPT Slide | PDF

The computing time of the reconstruction and fusion algorithm was also evaluated. The whole post-process procedure was implemented 30 times in Matlab R2012b with an Intel Core i5-4200H CPU. There are slight differences in computing time between each repetition. It is mainly caused by the fact that the condition of the computer is changing slightly over time. The average and standard deviation of the computing times are shown in Fig. 14. The computing time will severely limit the acquisition speed of the HR-SIS. However, it can be further reduced by utilizing a graphics processing unit (GPU).

 figure: Fig. 14

Fig. 14 Computing time of the reconstruction and fusion algorithm.

Download Full Size | PPT Slide | PDF

5. Conclusions

In this paper, the high resolution snapshot imaging spectrometer (HR-SIS) was proposed and experimentally demonstrated. Meanwhile a matching fusion algorithm based on GPCA was developed. A high spectral resolution MS image with low spatial resolution and a high spatial resolution PAN image were acquired by the spectral branch and imaging branch of the HR-SIS, respectively. The fusion procedure based on GPCA was carried out to produce a high spatial-spectral resolution MS image. The HR-SIS can acquire an MS image with spatial size of 680 × 680 pixels and the spectral resolution of 250 cm−1 in a snapshot. The spectral accuracy, spatial resolution and insensitivity to metamerism of the HR-SIS were experimentally demonstrated. The NRMSE of the spectral accuracy can achieve around 10% under proper illumination. The spectral resolution of the HR-SIS is better than 10 nm at 632.8nm and the spatial resolution of the system is 3.56 lines/mm. The HR-SIS is compact and precise in spatial-spectral domain. Meanwhile, in contrast to other snapshot imaging spectrometer based on information fusion [18–20], the HR-SIS is insensitive to metamerism. Because the snapshot imaging spectrometers described in [18–20] all utilize RGB cameras while two monochromatic cameras are used in the HR-SIS. The proposed fusion algorithm was evaluated by observing twelve colorful maps. The property of the HR-SIS, on which the proposed fusion algorithm relies, was highlighted by an experiment. The proposed GPCA algorithm was compared with other state-of-the-art fusion algorithms. Under the Wald’s protocol, the GPCA algorithm is competitively with others on the ERGAS, CC and SAM. Meanwhile, the fusion algorithm also can be implemented in many other snapshot imaging spectrometers, such as techniques described in [7,15]. The HR-SIS can be applied in biomedical imaging, microscopy, endoscopy and air surveillance in concert with an unmanned aerial vehicle (UAV).

Funding

National High Technology Research and Development Program of China (2015AA042401).

References and links

1. D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007). [CrossRef]   [PubMed]  

2. R. M. Levenson and J. R. Mansfield, “Multispectral imaging in biology and medicine: slices of life,” Cytometry A 69(8), 748–758 (2006). [CrossRef]   [PubMed]  

3. D. J. Mulla, “Twenty five years of remote sensing in precision agriculture: key advances and remaining knowledge gaps,” Biosystems Eng. 114(4), 358–371 (2013). [CrossRef]  

4. F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012). [CrossRef]  

5. G. Yang, “Bioimage informatics for understanding spatiotemporal dynamics of cellular processes,” Wiley Interdiscip. Rev. Syst. Biol. Med. 5(3), 367–380 (2013). [CrossRef]   [PubMed]  

6. N. Bedard, R. A. Schwarz, A. Hu, V. Bhattar, J. Howe, M. D. Williams, A. M. Gillenwater, R. Richards-Kortum, and T. S. Tkaczyk, “Multimodal snapshot spectral imaging for oral cancer diagnostics: a pilot study,” Biomed. Opt. Express 4(6), 938–949 (2013). [CrossRef]   [PubMed]  

7. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). [CrossRef]   [PubMed]  

8. M. W. Kudenov and E. L. Dereniak, “Compact real-time birefringent imaging spectrometer,” Opt. Express 20(16), 17973–17986 (2012). [CrossRef]   [PubMed]  

9. A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994). [CrossRef]  

10. B. Ford, M. Descour, and R. Lynch, “Large-image-format computed tomography imaging spectrometer for fluorescence microscopy,” Opt. Express 9(9), 444–453 (2001). [CrossRef]   [PubMed]  

11. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008). [CrossRef]   [PubMed]  

12. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010). [CrossRef]   [PubMed]  

13. N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013). [CrossRef]  

14. L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016). [CrossRef]   [PubMed]  

15. A. Hegyi and J. Martini, “Hyperspectral imaging with a liquid crystal polarization interferometer,” Opt. Express 23(22), 28742–28754 (2015). [CrossRef]   [PubMed]  

16. L. Wang, Z. Xiong, D. Gao, G. Shi, and F. Wu, “Dual-camera design for coded aperture snapshot spectral imaging,” Appl. Opt. 54(4), 848–858 (2015). [CrossRef]   [PubMed]  

17. A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989). [CrossRef]  

18. C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014). [CrossRef]  

19. C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014). [CrossRef]   [PubMed]  

20. Y. Murakami, K. Nakazaki, and M. Yamaguchi, “Hybrid-resolution spectral video system using low-resolution spectral sensor,” Opt. Express 22(17), 20311–20325 (2014). [CrossRef]   [PubMed]  

21. M. C. Abrams, S. P. Davis and J. W. Brault, Fourier Transform Spectrometry (Academic, 2001), Chap. 8.

22. H. Ghassemian, “A review of remote sensing image fusion methods,” Inf. Fusion 32, 75–89 (2016). [CrossRef]  

23. L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015). [CrossRef]  

24. G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015). [CrossRef]  

25. V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogramm. Eng. Remote Sensing 58(5), 561–567 (1992).

26. C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008). [CrossRef]  

27. H. F. Kaiser, “The varimax criterion for analytic rotation in factor analysis,” Psychometrika 23(3), 187–200 (1958). [CrossRef]  

28. X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005). [CrossRef]  

29. S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).

30. Signal and Image Processing Institute, University of Southern California, “The USC-SIPI Image Database,” http://sipi.usc.edu/database/

31. S. K. Mitra and L. Lucchese, “Using saddle points for subpixel feature derection in camera calibattion targets,” in Proceedings of IEEE Asia-Pacific Conference on Circuits and Systems (IEEE, 2002), pp. 191–195.

32. B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007). [CrossRef]  

33. B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006). [CrossRef]  

34. L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

References

  • View by:

  1. D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007).
    [Crossref] [PubMed]
  2. R. M. Levenson and J. R. Mansfield, “Multispectral imaging in biology and medicine: slices of life,” Cytometry A 69(8), 748–758 (2006).
    [Crossref] [PubMed]
  3. D. J. Mulla, “Twenty five years of remote sensing in precision agriculture: key advances and remaining knowledge gaps,” Biosystems Eng. 114(4), 358–371 (2013).
    [Crossref]
  4. F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
    [Crossref]
  5. G. Yang, “Bioimage informatics for understanding spatiotemporal dynamics of cellular processes,” Wiley Interdiscip. Rev. Syst. Biol. Med. 5(3), 367–380 (2013).
    [Crossref] [PubMed]
  6. N. Bedard, R. A. Schwarz, A. Hu, V. Bhattar, J. Howe, M. D. Williams, A. M. Gillenwater, R. Richards-Kortum, and T. S. Tkaczyk, “Multimodal snapshot spectral imaging for oral cancer diagnostics: a pilot study,” Biomed. Opt. Express 4(6), 938–949 (2013).
    [Crossref] [PubMed]
  7. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010).
    [Crossref] [PubMed]
  8. M. W. Kudenov and E. L. Dereniak, “Compact real-time birefringent imaging spectrometer,” Opt. Express 20(16), 17973–17986 (2012).
    [Crossref] [PubMed]
  9. A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
    [Crossref]
  10. B. Ford, M. Descour, and R. Lynch, “Large-image-format computed tomography imaging spectrometer for fluorescence microscopy,” Opt. Express 9(9), 444–453 (2001).
    [Crossref] [PubMed]
  11. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008).
    [Crossref] [PubMed]
  12. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010).
    [Crossref] [PubMed]
  13. N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013).
    [Crossref]
  14. L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016).
    [Crossref] [PubMed]
  15. A. Hegyi and J. Martini, “Hyperspectral imaging with a liquid crystal polarization interferometer,” Opt. Express 23(22), 28742–28754 (2015).
    [Crossref] [PubMed]
  16. L. Wang, Z. Xiong, D. Gao, G. Shi, and F. Wu, “Dual-camera design for coded aperture snapshot spectral imaging,” Appl. Opt. 54(4), 848–858 (2015).
    [Crossref] [PubMed]
  17. A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989).
    [Crossref]
  18. C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
    [Crossref]
  19. C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014).
    [Crossref] [PubMed]
  20. Y. Murakami, K. Nakazaki, and M. Yamaguchi, “Hybrid-resolution spectral video system using low-resolution spectral sensor,” Opt. Express 22(17), 20311–20325 (2014).
    [Crossref] [PubMed]
  21. M. C. Abrams, S. P. Davis and J. W. Brault, Fourier Transform Spectrometry (Academic, 2001), Chap. 8.
  22. H. Ghassemian, “A review of remote sensing image fusion methods,” Inf. Fusion 32, 75–89 (2016).
    [Crossref]
  23. L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
    [Crossref]
  24. G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
    [Crossref]
  25. V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogramm. Eng. Remote Sensing 58(5), 561–567 (1992).
  26. C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
    [Crossref]
  27. H. F. Kaiser, “The varimax criterion for analytic rotation in factor analysis,” Psychometrika 23(3), 187–200 (1958).
    [Crossref]
  28. X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
    [Crossref]
  29. S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).
  30. Signal and Image Processing Institute, University of Southern California, “The USC-SIPI Image Database,” http://sipi.usc.edu/database/
  31. S. K. Mitra and L. Lucchese, “Using saddle points for subpixel feature derection in camera calibattion targets,” in Proceedings of IEEE Asia-Pacific Conference on Circuits and Systems (IEEE, 2002), pp. 191–195.
  32. B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007).
    [Crossref]
  33. B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
    [Crossref]
  34. L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

2016 (2)

L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016).
[Crossref] [PubMed]

H. Ghassemian, “A review of remote sensing image fusion methods,” Inf. Fusion 32, 75–89 (2016).
[Crossref]

2015 (4)

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

A. Hegyi and J. Martini, “Hyperspectral imaging with a liquid crystal polarization interferometer,” Opt. Express 23(22), 28742–28754 (2015).
[Crossref] [PubMed]

L. Wang, Z. Xiong, D. Gao, G. Shi, and F. Wu, “Dual-camera design for coded aperture snapshot spectral imaging,” Appl. Opt. 54(4), 848–858 (2015).
[Crossref] [PubMed]

2014 (4)

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014).
[Crossref] [PubMed]

Y. Murakami, K. Nakazaki, and M. Yamaguchi, “Hybrid-resolution spectral video system using low-resolution spectral sensor,” Opt. Express 22(17), 20311–20325 (2014).
[Crossref] [PubMed]

S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).

2013 (4)

N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013).
[Crossref]

D. J. Mulla, “Twenty five years of remote sensing in precision agriculture: key advances and remaining knowledge gaps,” Biosystems Eng. 114(4), 358–371 (2013).
[Crossref]

G. Yang, “Bioimage informatics for understanding spatiotemporal dynamics of cellular processes,” Wiley Interdiscip. Rev. Syst. Biol. Med. 5(3), 367–380 (2013).
[Crossref] [PubMed]

N. Bedard, R. A. Schwarz, A. Hu, V. Bhattar, J. Howe, M. D. Williams, A. M. Gillenwater, R. Richards-Kortum, and T. S. Tkaczyk, “Multimodal snapshot spectral imaging for oral cancer diagnostics: a pilot study,” Biomed. Opt. Express 4(6), 938–949 (2013).
[Crossref] [PubMed]

2012 (2)

M. W. Kudenov and E. L. Dereniak, “Compact real-time birefringent imaging spectrometer,” Opt. Express 20(16), 17973–17986 (2012).
[Crossref] [PubMed]

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

2010 (2)

2008 (2)

A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008).
[Crossref] [PubMed]

C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
[Crossref]

2007 (2)

B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007).
[Crossref]

D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007).
[Crossref] [PubMed]

2006 (2)

R. M. Levenson and J. R. Mansfield, “Multispectral imaging in biology and medicine: slices of life,” Cytometry A 69(8), 748–758 (2006).
[Crossref] [PubMed]

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

2005 (1)

X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
[Crossref]

2001 (1)

1997 (1)

L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

1994 (1)

A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
[Crossref]

1992 (1)

V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogramm. Eng. Remote Sensing 58(5), 561–567 (1992).

1989 (1)

A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989).
[Crossref]

1958 (1)

H. F. Kaiser, “The varimax criterion for analytic rotation in factor analysis,” Psychometrika 23(3), 187–200 (1958).
[Crossref]

Aiazzi, B.

B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007).
[Crossref]

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

Almeida, L. B.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Alparone, L.

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

Audicana, M. G.

X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
[Crossref]

Bakker, W. H.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Baronti, S.

B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007).
[Crossref]

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

Bedard, N.

Bennett, S.

S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).

Bhattar, V.

Bioucas-Dias, J. M.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Brady, D.

Briottet, X.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Cao, X.

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014).
[Crossref] [PubMed]

Carranza, E. J. M.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Chanussot, J.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
[Crossref]

Dai, Q.

C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014).
[Crossref] [PubMed]

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

Dereniak, E. L.

Descour, M.

Dicker, D. T.

D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007).
[Crossref] [PubMed]

Dobigeon, N.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

El-Deiry, W. S.

D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007).
[Crossref] [PubMed]

Fabre, S.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Fletcher-Holmes, D. W.

Ford, B.

Fors, O.

X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
[Crossref]

Gao, D.

Gao, L.

Garzelli, A.

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

Ghassemian, H.

H. Ghassemian, “A review of remote sensing image fusion methods,” Inf. Fusion 32, 75–89 (2016).
[Crossref]

Gillenwater, A. M.

Gorman, A.

Hagen, N.

Harvey, A. R.

Hecker, C. A.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Hegyi, A.

Hirai, A.

A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
[Crossref]

Howe, J.

Hu, A.

Ichioka, Y.

A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
[Crossref]

Inoue, T.

A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
[Crossref]

Itoh, K.

A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
[Crossref]

John, R.

Kaiser, H. F.

H. F. Kaiser, “The varimax criterion for analytic rotation in factor analysis,” Psychometrika 23(3), 187–200 (1958).
[Crossref]

Kester, R. T.

Kudenov, M. W.

N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013).
[Crossref]

M. W. Kudenov and E. L. Dereniak, “Compact real-time birefringent imaging spectrometer,” Opt. Express 20(16), 17973–17986 (2012).
[Crossref] [PubMed]

Lasenby, J.

S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).

Lerner, J. M.

D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007).
[Crossref] [PubMed]

Levenson, R. M.

R. M. Levenson and J. R. Mansfield, “Multispectral imaging in biology and medicine: slices of life,” Cytometry A 69(8), 748–758 (2006).
[Crossref] [PubMed]

Licciardi, G. A.

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Lin, S.

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

Loncan, L.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Lucchese, L.

S. K. Mitra and L. Lucchese, “Using saddle points for subpixel feature derection in camera calibattion targets,” in Proceedings of IEEE Asia-Pacific Conference on Circuits and Systems (IEEE, 2002), pp. 191–195.

Lynch, R.

Ma, C.

C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014).
[Crossref] [PubMed]

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

Mangolini, M.

L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

Mansfield, J. R.

R. M. Levenson and J. R. Mansfield, “Multispectral imaging in biology and medicine: slices of life,” Cytometry A 69(8), 748–758 (2006).
[Crossref] [PubMed]

Martini, J.

Mitra, S. K.

S. K. Mitra and L. Lucchese, “Using saddle points for subpixel feature derection in camera calibattion targets,” in Proceedings of IEEE Asia-Pacific Conference on Circuits and Systems (IEEE, 2002), pp. 191–195.

Mulla, D. J.

D. J. Mulla, “Twenty five years of remote sensing in precision agriculture: key advances and remaining knowledge gaps,” Biosystems Eng. 114(4), 358–371 (2013).
[Crossref]

Mura, M. D.

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

Murakami, Y.

Nakazaki, K.

Noomen, M. F.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Nunez, J.

X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
[Crossref]

Otazu, X.

X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
[Crossref]

Qi, W.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Ranchin, T.

C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
[Crossref]

L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

Restaino, R.

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

Richards-Kortum, R.

Schwarz, R. A.

Selva, M.

B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007).
[Crossref]

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

Shettigara, V. K.

V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogramm. Eng. Remote Sensing 58(5), 561–567 (1992).

Shi, G.

Simo, M.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Smeth, J. B.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Thomas, C.

C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
[Crossref]

Tkaczyk, T. S.

Toet, A.

A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989).
[Crossref]

Tong, X.

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

Tourneret, J. Y.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Valeton, J. M.

A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989).
[Crossref]

van der Meer, F. D.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

van der Meijde, M.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

van der Werff, H. M. A.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

van Ruitenbeek, F. J. A.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

van Ruyven, L. J.

A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989).
[Crossref]

Veganzones, M. A.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Vivone, G.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

Wagadarikar, A.

Wald, L.

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
[Crossref]

L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

Wang, L.

Wang, L. V.

L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016).
[Crossref] [PubMed]

Wenzhi, L.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Willett, R.

Williams, M. D.

Woldai, T.

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Wu, F.

Wu, R.

Xiong, Z.

Yamaguchi, M.

Yang, G.

G. Yang, “Bioimage informatics for understanding spatiotemporal dynamics of cellular processes,” Wiley Interdiscip. Rev. Syst. Biol. Med. 5(3), 367–380 (2013).
[Crossref] [PubMed]

Yokoya, N.

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

Appl. Opt. (2)

Biomed. Opt. Express (1)

Biosystems Eng. (1)

D. J. Mulla, “Twenty five years of remote sensing in precision agriculture: key advances and remaining knowledge gaps,” Biosystems Eng. 114(4), 358–371 (2013).
[Crossref]

Cell Cycle (1)

D. T. Dicker, J. M. Lerner, and W. S. El-Deiry, “Hyperspectral image analysis of live cells in various cell cycle stages,” Cell Cycle 6(20), 2563–2570 (2007).
[Crossref] [PubMed]

Comput. Vis. Image Un. (1)

S. Bennett and J. Lasenby, “ChESS – Quick and robust detection of chess-board features,” Comput. Vis. Image Un. 118, 197–210 (2014).

Cytometry A (1)

R. M. Levenson and J. R. Mansfield, “Multispectral imaging in biology and medicine: slices of life,” Cytometry A 69(8), 748–758 (2006).
[Crossref] [PubMed]

IEEE Geosci. Remote S. (1)

L. Loncan, S. Fabre, L. B. Almeida, J. M. Bioucas-Dias, L. Wenzhi, X. Briottet, G. A. Licciardi, J. Chanussot, M. Simo, N. Dobigeon, J. Y. Tourneret, M. A. Veganzones, W. Qi, G. Vivone, and N. Yokoya, “Hyperspectral pansharpening: a review,” IEEE Geosci. Remote S. 3(3), 27–46 (2015).
[Crossref]

IEEE T. Geosci. Remote (1)

C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, “Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics,” IEEE T. Geosci. Remote 46(5), 1301–1312 (2008).
[Crossref]

IEEE Trans. Geosci. Remote (3)

G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A Critical Comparison Among Pansharpening Algorithms,” IEEE Trans. Geosci. Remote 53(5), 2565–2586 (2015).
[Crossref]

X. Otazu, M. G. Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods,” IEEE Trans. Geosci. Remote 43(10), 2376–2385 (2005).
[Crossref]

B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS +Pan Data,” IEEE Trans. Geosci. Remote 45(10), 3230–3239 (2007).
[Crossref]

Inf. Fusion (1)

H. Ghassemian, “A review of remote sensing image fusion methods,” Inf. Fusion 32, 75–89 (2016).
[Crossref]

Int. J. Appl. Earth Obs. (1)

F. D. van der Meer, H. M. A. van der Werff, F. J. A. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. Smeth, and T. Woldai, “Multi- and hyperspectral geologic remote sensing: A review,” Int. J. Appl. Earth Obs. 14(1), 112–128 (2012).
[Crossref]

Int. J. Comput. Vis. (1)

C. Ma, X. Cao, X. Tong, Q. Dai, and S. Lin, “Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System,” Int. J. Comput. Vis. 110(2), 141–155 (2014).
[Crossref]

Opt. Eng. (2)

A. Toet, L. J. van Ruyven, and J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28(7), 789–792 (1989).
[Crossref]

N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013).
[Crossref]

Opt. Express (6)

Opt. Lett. (1)

Opt. Rev. (1)

A. Hirai, T. Inoue, K. Itoh, and Y. Ichioka, “Application of multiple-image fourier transform spectral imaging to measurement of fast phenomena,” Opt. Rev. 1(2), 205–207 (1994).
[Crossref]

Photogramm. Eng. Remote Sensing (3)

V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogramm. Eng. Remote Sensing 58(5), 561–567 (1992).

B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, “MTF-tailored multiscale fusion of high-resolution MS and pan imagery,” Photogramm. Eng. Remote Sensing 72(5), 591–596 (2006).
[Crossref]

L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sensing 63(6), 691–699 (1997).

Phys. Rep. (1)

L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016).
[Crossref] [PubMed]

Psychometrika (1)

H. F. Kaiser, “The varimax criterion for analytic rotation in factor analysis,” Psychometrika 23(3), 187–200 (1958).
[Crossref]

Wiley Interdiscip. Rev. Syst. Biol. Med. (1)

G. Yang, “Bioimage informatics for understanding spatiotemporal dynamics of cellular processes,” Wiley Interdiscip. Rev. Syst. Biol. Med. 5(3), 367–380 (2013).
[Crossref] [PubMed]

Other (3)

M. C. Abrams, S. P. Davis and J. W. Brault, Fourier Transform Spectrometry (Academic, 2001), Chap. 8.

Signal and Image Processing Institute, University of Southern California, “The USC-SIPI Image Database,” http://sipi.usc.edu/database/

S. K. Mitra and L. Lucchese, “Using saddle points for subpixel feature derection in camera calibattion targets,” in Proceedings of IEEE Asia-Pacific Conference on Circuits and Systems (IEEE, 2002), pp. 191–195.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1 (a) Schematic of HR-SIS. (b) Polarization optical elements with optical axes indicated by arrows. Acronyms: Objective Lens (OL), Field Stop (FS), Polarizing Beam Splitter (PBS), Imaging Lens (IL), Focal Plane Array (FPA), Half-Wave Plate (HWP), Collimating Lens (CL), Lenslet Array (LA), Birefringent Polarization Interferometer (BPI) Nomarski Prism (NP), Analyzer (A).
Fig. 2
Fig. 2 Framework of proposed fusion algorithm based on GPCA.
Fig. 3
Fig. 3 Experimental prototype of the proposed system. Acronyms: Objective Lens (OL), Field Stop (FS), Polarizing Beam Splitter (PBS), Half-Wave Plate (HWP), Imaging Lens (IL), Focal Plane Array (FPA), Collimating Lens (CL), Lenslet Array (LA), Birefringent Polarization Interferometer (BPI).
Fig. 4
Fig. 4 (a) Xrite color checker. (b) Normalized root mean square error (NRMSE) in 24 color-block areas. (c) Spectral curves of color blocks from HR-SIS and AvaSpec (ground truth).
Fig. 5
Fig. 5 Spectral curves of color blocks (a) No.8, (b) No. 13, and (c) No. 18 from HR-SIS and AvaSpec (ground truth). (d) NRMSEs in the three color-block areas.
Fig. 6
Fig. 6 Spectrum of center point of the FOV with a fitted Gaussian curve (red dashed line).
Fig. 7
Fig. 7 (a) Composite image from the MS image before fusion. (b) Composite image from the MS image after fusion.
Fig. 8
Fig. 8 (a) RGB picture of the scene (white balance was corrected in Photoshop). (b) Spectra of two points on the leaf and the green artifacts respectively. (c) Spectral band at 655.2nm. (d) Spectral band at 713.2nm.
Fig. 9
Fig. 9 The relationships among the notation in Table 2
Fig. 10
Fig. 10 (a) Colorful maps with abundant spatial details. Among them, image No. 1-3 were homemade, and image No. 5-12 were obtained from the online image database [30]. Image No. 4 is an Xrite color checker. These pictures, except image No. 4, were printed by color printers (Epson Stylus Photo 1400 for No. 2, 6 and 8; Xerox Workcentre 7346 for No. 1, 3, 5 and 7; Canon iP2780 for No. 8-12). All pictures were taken by Canon 550D color digital camera and white balance was corrected in Photoshop. (b) Synthetic MS image and (c) downsampled PAN image of the map No. 13. (d) Correlation coefficients between the downsampled PAN images and synthetic MS images.
Fig. 11
Fig. 11 Evaluating results of different fusion algorithms. (a) ERGAS between M S d * and M S o . (b) CC and (c) SAM between M S o * and M S o . Note that the ideal values of the ERGAS, CC and SAM are 0, 1 and 0, respectively.
Fig. 12
Fig. 12 Comparison between fused MS images and upsampled MS images
Fig. 13
Fig. 13 (a) RGB picture of the color checker obtained by a commercial color camera. (b) The red line indicates the NRMSE between M S o * and M S o at the pixels along the red dotted line, while the blue line is the cross section profile of M S sum .
Fig. 14
Fig. 14 Computing time of the reconstruction and fusion algorithm.

Tables (2)

Tables Icon

Table 1 Number of Line Pairs / mm in USAF Resolving Power Test Target 1951

Tables Icon

Table 2 List of notation

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

E T = J A J BPI J HW P 1 J T E I
Ε R = J R E I
Ε Τ = 1 2 [ 1 1 1 1 ][ 1 0 0 e i2πσΔ ] 2 2 [ 1 1 1 1 ][ 1 0 0 0 ][ E x E y ]= 2 4 E x (1+ e i2πσΔ )[ 1 1 ]
E R =[ 0 0 0 1 ][ E x E y ]=[ 0 E y ]
I FP A 2 = I x (σ) 2 [ 1+cos( 2πσΔ ) ]
I FP A 1 = I y (σ)
X=AF+ε
i=1 M λ i i=1 N λ i threshold
π(X)={ C 1 , C 2 ,..., C | π(X) | }
X= i=1 | π(X) | C i
π( X ˜ )={ C ˜ 1 , C ˜ 2 ,..., C ˜ | π( X ˜ ) | }
P C ^ 1 i =P C ˜ 1 i +α(P P L )
α= σ(P C ˜ 1 i ) σ(P) P C ˜ 1 i j=1 | π( X ˜ ) | P C ˜ 1 j
CC( X ^ , X ref )= 1 N i=1 N CCS( X ^ i , X i ref )
SAM( X ^ , X ref )= 1 m×n i=1 m j=1 n arccos( < x ^ i,j , x i,j ref > x ^ i,j 2 x i,j ref 2 )
ERGAS( X ^ , X ref )=100× 1 r 1 N i=1 N ( N×RMSE( X ^ i , X i ref ) j=1 N X i ref ) 2
RMSE( X ^ i , X i ref )= X ^ i X i ref F 2 m×n

Metrics