Spectral cameras with integrated thin-film Fabry–Perot filters enable many different applications. Some applications require the detection of spectral features that are only visible at specific wavelengths, and some need to quantify small spectral differences that are undetectable with RGB color cameras. One factor that influences the central wavelength of thin-film filters is the angle of incidence. Therefore, when light is focused from an imaging lens onto the filter array, undesirable shifts in the measured spectra are observed. These shifts limit the use of the sensor in applications that require fast lenses or lenses with large chief ray angles. To increase flexibility and enable new applications, we derive an analytical model that explains and can correct the observed shifts in measured spectra. The model includes the size of the aperture and physical position of each filter on the sensor. We experimentally validate the model with two spectral cameras: one in the visible and near-infrared region and one in the short wave infrared region.
© 2018 Optical Society of America
Spectral imaging is a technique that combines photography and spectroscopy. It allows one to capture an image of a scene and simultaneously sample the electromagnetic spectrum, providing a “fingerprint” for each point in the scene.
Some applications require the detection of spectral features that are only visible at specific wavelengths, and some require the quantification of small spectral differences that are undetectable with RGB color cameras . Spectral cameras can therefore achieve higher selectivity in machine-learning applications and allow for physical interpretation of the data (e.g., material identification) [2,3].
In spectral camera designs with integrated thin-film Fabry–Perot filters, focused light from a finite aperture widens the transmittance peaks and shifts them toward shorter wavelengths [4–6]. The problem of assembling an accurate spectrum from the filter bank is complicated by the fact that the filters are patterned across the sensor. This makes the shift depend on both the physical position of each filter and the size of the aperture.
The shifts in wavelength limit the use of the sensor in applications that require lenses with large chief ray angles or require fast lenses (i.e., small f-number). The latter is often the case, since spectral filters are in general narrowband, which implies that more light or longer integration times are required. In this paper, we therefore study the effect of the f-number and the chief ray angle on the measured spectrum and derive a method to correct the spectral shift.
The main contribution of this paper is the use of the concept of the “effective refractive index”  and a convolution model of the effect of the aperture  in the context of spectral cameras with integrated thin-film filters that are patterned across an imaging sensor. We derive and use a compact equation to correct the spectral shifts observed in measurements. The presented method thus enables new applications and offers an increased flexibility for the sensor to be used in systems with different optical requirements.
The remainder of this paper is structured as follows. In Section 2, we discuss related work. In Section 3, we discuss the underlying theory and derive a method for wavelength correction. In Section 4, we validate the model experimentally, and in Section 5 we discuss the results and their limitations.
2. RELATED WORK
The presence of a finite aperture or, more generally, a varying angle of incidence, poses a challenge for many spectral imager designs [9,10]. Grating- and prism-based systems require careful alignment and good control of the angularity. This usually requires additional optical components (e.g., collimator). Spectral cameras with linear variable filters also need to take into account the angle of incidence. In Ref. , the authors analyze the effect of an aperture on their filter. The authors did not attempt to use their model to correct shifts in actual measurements. The authors of Ref.  briefly study the effect of an angle of incidence on interference filter-based designs. While they mention the concept of effective refractive index, they do not provide any practical results to describe the behavior for an actual aperture. The idea of using a convolution model for the effect of an aperture on the spectrum measured with a Fabry–Perot etalon is not new [8,13,14]. However, earlier results did not include thin-film Fabry–Perot filters placed behind an objective lens, but rather a Fabry–Perot spectrometer with an objective lens behind the etalon with the aperture in the imaging plane.
To the best of the authors’ knowledge, this is the first time that a practical method is proposed that corrects the shift in thin-film filters caused by an aperture, and that also takes into account the patterning on the spectral sensor such that the output of each filter is appropriately corrected and a more accurate spectrum can be assembled.
3. THEORY AND METHODS
In this section, we first introduce a model of a spectral imaging system with integrated thin-film filters (Section 3.A). We then discuss how the model of an ideal Fabry–Perot filter can be used to describe the effect of an angle of incidence on thin-film filters (Section 3.B). This is used to model the effect of a focused beam of light as a convolution of the transmittance with a kernel that depends on the f-number and the chief ray angle (Section 3.C). From this model a compact formula is derived that, for each filter on the sensor, can correct the shift in central wavelength (Section 3.D).
A. Spectral Imaging System Model
The imaging system used in this paper consists of an objective lens with a finite aperture (or more precisely, the exit pupil) that focuses the light onto pixels of an imaging sensor with integrated thin-film interference filters (Fig. 1). The output DN in digital numbers of a pixel with integrated filters can be modeled as15]. The limits of the integral describe the bandpass range of the spectral camera.
For a given f-number and chief ray angle , the effect of the aperture on the filter with central wavelength is modeled as a convolution of with a kernel , where the half-cone angle characterizes the f-number:3.C.
Below in Table 1, the most important symbols are summarized.
B. Collimated Light at Oblique Incidence
1. Ideal Fabry–Perot Model
The ideal Fabry–Perot etalon is a device that consists of two highly reflecting parallel mirrors such that reflected rays can interfere. By tuning the distance between the plates, selection of narrow wavelength bands becomes possible (Fig. 2).
The Fabry–Perot has specific transmittance characteristics that are described by the Airy function .
For an angle of incidence , the transmittance peak centered at (at orthogonal incidence) shifts by16].
In practice, the Fabry–Perot design can be approximated using multilayer thin-film structures  with Bragg mirrors that are more complex, and which are discussed in the next section.
2. Ideal Fabry–Perot Model Applied to an All-Dielectric Single-Cavity Filter
A Fabry–Perot etalon approximated with a thin-film structure can be partially understood in terms of an ideal Fabry–Perot model. It can be proven that the shift in central wavelength for small angles is asymptotically equivalent to the shift of the transmittance of an ideal Fabry–Perot etalon with a cavity material with effective refractive index [7,16]. It has been shown that the small-angle approximation can remain valid up to 40° .
Therefore, for collimated light at an incident angle , the shift () in central wavelength of a transmittance peak of a thin-film Fabry–Perot filter is
C. Focused Light from a Finite Circular Aperture
Focused light from a circular aperture can be interpreted as a distribution of incident angles. This distribution becomes more skewed for larger chief ray angles (Fig. 1). For each chief ray angle, the skewed cone of light can be decomposed in contributions that have identical angles of incidence (Fig. 3). This insight allows the problem to be treated as a weighted sum of contributions of the form Eq. (5).
The decomposition can be formalized in the following way. Let be the distance of the focusing plane to the aperture plane (or exit pupil), let be the radius of the aperture for a given f-number, and let the chief ray angle be defined as , with being the off-center distance (Fig. 4).
The resulting transmittance is the sum of the transmittance functions at different tilt angles [Eq. (5)]. The weight of each contribution is the infinitesimal area of the aperture that contributes to identical angles of incidence (Fig. 4). Therefore,4): 3). This is because , for .
We proceed by rewriting the integral in terms of the angle of incidence by substituting such that11) is the angular distribution of incident angles that was illustrated in Fig. 3.3.4) (See also Appendix A).
The shape of the convolution kernel models how the central wavelength and full width at half-maximum (FWHM) of the transmittance will change. For increasing and , the kernel shifts and becomes wider (Fig. 5), and thus so does the transmittance (Fig. 6).
To verify the accuracy of the model, the effect of applying the kernel is compared to a reference simulation of a thin-film stack using TFCalc , which is based on the transfer-matrix method. The transfer-matrix method requires the thickness and refractive index for each layer of the thin-film stack. In contrast, the kernel requires only one parameter: the effective refractive index. The simulation with TFCalc shows a very good match with the kernel model (Fig. 6).
D. Method for Central Wavelength Correction
The kernel models the effect of the aperture on a filter. This allows system designers to predict the effective position and FWHM of the transmittance and simulate the impact on real-life measurements with a camera. For practical use a simplified equation is derived.
To quantify the shift in central wavelength, the mean of the kernel (when interpreted as a distribution) is used:B), Eq. (16) becomes C.
To calculate the new central wavelength of a filter placed behind an aperture, the mean is subtracted from the original central wavelength such that
For practical use, we will reformulate Eq. (18) in terms of the working f-number and the off-center distance of the filter. Assuming that ,
4. EXPERIMENTAL VALIDATION
In this section, Eq. (18) is used to correct the shifts in the spectrum measured with a representative experimental setup.
Two camera systems are used: imec’s Short Wave InfraRed (SWIR) and Visible Near InfraRed (VNIR) Snapscan cameras. We have selected commercial off-the-shelf lenses that allow for f-numbers lower than 2 and where the exit pupil is not farther than 40 mm from the focal plane so we can test for significant chief ray angles (up to 15°). We tested the lenses in regimes where there is no significant vignetting. For the VNIR lens used in the experiments, this assumption is only valid when it is used at a high f-number (i.e., ).
The effectiveness of the correction is demonstrated in three experiments. In the first experiment, the sample is centered on the optical axis (). The reflectance is measured at different f-numbers. In the second experiment, the transmittance of a color filter is measured at different chief ray angles at a high f-number. In the third experiment, the sample is positioned off-axis and scanned at different f-numbers. In all three cases, the central wavelength of the filters needs to be corrected.
A. Experimental Setup
For the VNIR region, imec’s VNIR Snapscan system (150 spectral bands)  is used with an Edmund Optics 16 mm C Series VIS-NIR fixed focal length lens. For the SWIR region, imec’s SWIR Snapscan system (128 spectral bands)  is used with an Optec 16 mm SWIR lens. The properties of the lenses in the setup are listed in Table 2.
Without loss of generality, the Snapscan system is chosen because it is a system that allows control of the chief ray angle. This is because the sensor is positioned behind the lens, and its position can be controlled using an integrated translation stage. The method remains applicable to other classes of spectral cameras, even with different patterning of the filters on the sensor.
The reflectance of the sample is measured using a flat-fielding approach. This involves scanning the same scene twice, i.e., once with the sample and once with a Spectralon white reference tile. For each pixel, an estimate of the reflectance is obtained by dividing the measurement of the sample by the measurement of the white tile after subtracting the dark image. In the case of a transmittance measurement, an image of a white tile with a transmission filter on the front side of the lens is used instead of a sample.
In terms of the spectral imaging model [Eq. (2)], the reflectance estimation can be formulated for each pixel as7). For the SWIR camera, the reflectance of a Spectralon calibrated multicomponent wavelength standard is measured. For the VNIR camera, the transmittance of a Thorlabs FGB67S colored glass filter is measured. A transmittance filter enables the convenient measurement of the same spectrum at different off-axis distances. It is important that the filter itself is not angle-dependent; therefore, colored glass is used instead of an interference filter.
B. Determining the Model Parameters
To apply the correction, first all model parameters need to be known. These parameters are: the effective refractive index, the working f-number, and the distance to the exit pupil.
Assuming that the distance to the exit pupil is known (Table 2), we briefly discuss the working f-number and different methods to estimate the effective refractive index.
The working f-number is calculated as ()
The effective refractive index can be estimated in multiple ways. A first approach is to use the theoretical result for the first resonance mode of Fabry–Perot-like designs with a low-index cavity :
If one does not know the material parameters, there is another method. The alternative is that one can characterize the sensor under collimated light conditions at different wavelengths . By repeating this procedure at different tilt angles, the effective index follows from the shifts in central wavelength. From this procedure, we then estimated for the VNIR sensor and for the SWIR sensor. These are the values that we will use for correction.
C. Data Analysis
In this section, we explain how the measurements from the experiments will be analyzed and how the graphs should be interpreted.
For each filter, the measured reflectance is plotted at its central wavelength in collimated light conditions at normal incidence. Therefore, without correction, a shift of the filters towards shorter wavelengths creates the illusion that the spectrum moves toward longer wavelengths (Fig. 8). The reflectance is normalized by its peak value.
To correct the shift in the data, the central wavelengths at which the reflectance is plotted are updated using Eq. (18). Note that the angles in Eq. (18) are in radians, while we will often refer to the angles in degrees.
To better explain the effect of the aperture on the measurements, we will also compare the real measurements to simulations.
For comparison with future work, two quantitative error measures are used: the correlation coefficient and the maximum error between the most shifted spectrum and the reference. The correlation and maximum error are referred to as “corr” and “maxerr,” respectively, in Figs. 8–11. The correlation coefficient measures the improvement in shift, and the maximum error will better capture the losses in detail (Fig. 10). To enable comparison, the uncorrected and corrected spectra were resampled on a common grid of wavelengths.
D. Experiment 1: Sample in Center (SWIR)
The sample is centered at the optical axis of the lens. In this experiment, the chief ray angle is equal to zero such that21) is used. In this example, .
The reflectance of the same sample is measured at different f-numbers, and Eq. (23) is used to correct the shifts in wavelength. After applying the correction, the spread between the measurements is significantly smaller (Fig. 8).
E. Experiment 2: Sample at Different Positions (VNIR)
Using the VNIR Snapscan, the transmittance spectrum is measured at different off-axis distances at . At this high f-number, . The selected distances from the center are 0, 2.75, 4.4, and 5.5 mm, which correspond to chief ray angles of 0°, 7.5°, 12°, and 15°, respectively.
Assuming that , then9).
We show the spatial image () at a wavelength of 721 nm (Fig. 12). Because at 721 nm the spectrum has a strong edge, shifts cause a significant gradient in intensity. After correction, as desired, the image is spatially uniform.
F. Experiment 3: Sample Off-Center (SWIR)
The multicomponent sample is placed at an off-axis position such that the chief ray angle is about 10.2°. This adds an additional offset on top of the spread caused by the f-number. Therefore both effects need to be taken into account:21) is used.
We observe three important effects in the measurements (Fig. 10). First, there is still the effect of the cone angle causing a spread in the measurements. Second, there is an offset shift caused by the chief ray angle. Last, there is an increased loss of detail compared to Fig. 8. This loss is due to an increased bandwidth of the filters, which has a smoothing effect and cannot be corrected by simply shifting the central wavelengths. This loss in detail is modeled by the increased width of the kernel, which has a smoothing effect.
The observed effects can also be simulated using Eq. (2) (Fig. 11). For this purpose, the sensor used in the experiment was characterized under collimated light conditions at normal incidence . In essence, for each filter in Eq. (1) was measured. The effect of the aperture is then simulated using Eq. (2), combined with the calibrated reflectance spectrum of the Spectralon calibrated multicomponent tile.
The experimental results show that the analytical model can be successfully applied to correct the spectral shifts in the different experiments with minimal effort in terms of calibration.
The presented correction method takes into account the physical position of each individual filter such that, for each point in the scene, a more accurate spectrum can be assembled. There are many applications for which this contribution is very relevant. In material classification, for example, the method could help to reduce the within-class variation and improve the between-class separation. The method could also help to reduce errors in parameter quantification (e.g., agriculture indices ).
In our experiments, we realigned the spectra by plotting the data points at their corrected wavelengths. An alternative would be to interpolate the data points at their corrected wavelengths and resample at the original uncorrected wavelengths. From an end-user perspective, this makes more sense because the corrected output of each filter can always be interpreted at the same wavelength.
The method can correct the shifts in wavelength but not the loss in detail at larger apertures. For larger apertures, the filters have an increased bandwidth, which inherently implies a loss of detail. This only becomes an issue when the spectra contain high-frequency components that will be significantly smoothed.
We demonstrated that the derived approximation is sufficiently accurate to correct the shifts for the incident angles that were tested. To push the limits of the method, one could attempt to expand the asymptotic series to higher orders or use numerical approximations of the mean (See Appendix C).
The kernel can also be used as an alternative to investigate the effect of an aperture on filters instead of the transfer-matrix method. The advantage is that the kernel can be applied to the transmittance spectrum as it is measured, while the transfer-matrix method first requires a fitting of model parameters. Assuming that the user does not know what thin-film filters are deposited, the fitting problem becomes even harder to solve: the number of layers, the materials used, and the thickness of each layer need be fitted. In contrast, the effective refractive index is a single parameter that needs to be measured once and requires no further knowledge of the stack.
While assumed constant, the effective refractive index is in principle wavelength-dependent and could vary because of process tolerances. The effective refractive index is also different for harmonics of the cavity . So while Eq. (22) might provide a good estimate, it might be even better to characterize the effective refractive index experimentally by tilting the sensor under collimated light with a monochromator. This characterization only needs to be done once.
The presented model does not take vignetting into account. While there are many machine vision applications (e.g., medical) that require high-quality lenses, this remains an important limitation, since vignetting implies that part of the light beam is cut off, which changes the distribution of incident angles. This will be investigated in future work.
Some applications require the detection of spectral features that are only visible at specific wavelengths and some require quantifying small spectral differences that are undetectable with RGB color cameras. The use of the spectral sensor in these applications is limited because with integrated thin-film optical filters an aperture causes undesired shifts in the measured spectra.
Taking into account the physical position of each filter in the filter bank, the observed shifts of the measured spectra can be corrected using a compact formula. This was demonstrated experimentally with two realistic spectral imaging setups.
Our approach is of high practical relevance because it enables new applications and offers increased flexibility for using the sensor in different environments.
APPENDIX A: CLOSED-FORM SOLUTION OF THE KERNEL
In this section, we derive a closed-form solution of the kernel and derive the analytical approximation given in Eq. (15).
The kernel was defined as
If we define , it follows thatA4), we now replace using Eq. (A10) and substitute with . Then becomes . Therefore the result can be simplified by employing the sampling property of the Dirac distribution on the integrand. The integral is then equal to its integrand evaluated at such that 12) and the function containing all remaining terms. By applying trigonometric identities can be simplified such that
This approximation is plotted in Fig. 5. Notice that for , the kernel is asymptotic to a rectangular window.
APPENDIX B: MEAN OF THE KERNEL
We use the mean value of the kernel, when interpreted as a distribution, as a measure of the shift in central wavelength. The mean is defined as
For collimated light at zero chief ray angle, there is no shift. Thus should be 0. By construction, the area of the kernel is constant:B1) then implies that .
To calculate the coefficients of the monomials in , one can calculate that for , in Eq. (A12), and . The integral then becomes14)], we first substitute such that C.
Disregarding higher orders, the formula for the shift in central wavelength is approximated by
APPENDIX C: VALIDITY OF THE APPROXIMATION
For experiment 3, the error between the exact shift and approximation is small (1.6 nm at ) compared to the total correction of 18 nm.
The approximation error grows proportional to . This suggests that in the power series expansion an important higher-order term was truncated.
Based on direct comparison with the numerical solution, we propose the following approximation:14) and even holds up well for much larger values of and (Fig. 15).
We thank Joren Vanherck for his valuable comments and discussions.
1. P. Shippert, “Why use hyperspectral imagery?” Earth Sci. 70, 377–379 (2004).
2. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19, 010901 (2014). [CrossRef]
3. A. Gowen, C. O’Donnell, P. Cullen, G. Downey, and J. Frias, “Hyperspectral imaging—an emerging process analytical tool for food quality and safety control,” Trends Food Sci. Technol. 18, 590–598 (2007). [CrossRef]
4. N. Tack, A. Lambrechts, P. Soussan, and L. Haspeslagh, “A compact, high-speed, and low-cost hyperspectral imager,” Proc. SPIE 8266, 82660Q (2012). [CrossRef]
5. B. Geelen, N. Tack, and A. Lambrechts, “A compact snapshot multispectral imager with a monolithically integrated per-pixel filter mosaic,” Proc. SPIE 8974, 89740L (2014). [CrossRef]
6. B. Geelen, C. Blanch, P. Gonzalez, N. Tack, and A. Lambrechts, “A tiny VIS-NIR snapshot multispectral camera,” Proc. SPIE 9374, 937414 (2015). [CrossRef]
7. C. R. Pidgeon and S. D. Smith, “Resolving power of multilayer filters in nonparallel light,” J. Opt. Soc. Am. 54, 1459–1466 (1964). [CrossRef]
8. P. A. Wilksch, “Instrument function of the Fabry-Perot spectrometer,” Appl. Opt. 24, 1502–1511 (1985). [CrossRef]
9. N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013). [CrossRef]
10. N. Gat, “Imaging spectroscopy using tunable filters: a review,” Proc. SPIE 4056, 50–64 (2000). [CrossRef]
11. I. G. E. Renhorn, D. Bergström, J. Hedborg, D. Letalick, and S. Möller, “High spatial resolution hyperspectral camera based on a linear variable filter,” Opt. Eng. 55, 114105 (2016). [CrossRef]
12. T. Skauli, H. E. Torkildsen, S. Nicolas, T. Opsahl, T. Haavardsholm, I. Kåsen, and A. Rognmo, “Compact camera for multispectral and conventional imaging based on patterned filters,” Appl. Opt. 53, C64–C74 (2014). [CrossRef]
13. G. Hernandez, “Analytical description of a Fabry-Perot photoelectric spectrometer,” Appl. Opt. 5, 1745–1748 (1966). [CrossRef]
14. G. Hernandez, “Analytical description of a Fabry-Perot spectrometer. 3: off-axis behavior and interference filters: erratum,” Appl. Opt. 18, 3364–3365 (1979). [CrossRef]
15. J. R. Janesick, Photon Transfer (SPIE, 2007).
16. H. A. Macleod, Thin-Film Optical Filters (CRC Press, 2001).
17. Software Spectra, “Thin film design software for Windows, Version 3.5.15,” Software Spectra, 2009, http://sspectra.com/files/win_demo/manual.pdf.
18. J. Pichette, W. Charle, and A. Lambrechts, “Fast and compact internal scanning CMOS-based hyperspectral camera: the Snapscan,” Proc. SPIE 10110, 1011014 (2017). [CrossRef]
19. P. Gonzalez, J. Pichette, B. Vereecke, B. Masschelein, L. Krasovitski, L. Bikov, and A. Lambrechts, “An extremely compact and high-speed line-scan hyperspectral imager covering the SWIR range,” Proc. SPIE 10656, 106560L (2018). [CrossRef]
20. P. Agrawal, K. Tack, B. Geelen, B. Masschelein, P. Mateo, A. Moran, A. Lambrechts, and M. Jayapala, “Characterization of VNIR hyperspectral sensors with monolithically integrated optical filters,” in Image Sensors and Imaging Systems (Society for Imaging Science and Technology, 2016), pp. 1–7.
21. R. Kingslake, Optics in Photography, Vol. 6 of SPIE Press Monograph (SPIE, 1992).
22. P. Thenkabail, R. Smith, and E. De Pauw, “Hyperspectral vegetation indices and their relationships with agricultural crop characteristics,” Remote Sens. Environ. 71, 158–182 (2000). [CrossRef]