Abstract

Current high-resolution push-broom hyperspectral cameras introduce keystone errors to the captured data. Efforts to correct these errors in hardware severely limit the optical design, in particular with respect to light throughput and spatial resolution, while at the same time the residual keystone often remains large. The mixel camera solves this problem by combining a hardware component – an array of light mixing chambers – with a mathematical method that restores the hyperspectral data to its keystone-free form, based on the data that was recorded onto the sensor with large keystone. A Virtual Camera software, that was developed specifically for this purpose, was used to compare the performance of the mixel camera to traditional cameras that correct keystone in hardware. The mixel camera can collect at least four times more light than most current high-resolution hyperspectral cameras, and simulations have shown that the mixel camera will be photon-noise limited – even in bright light – with a significantly improved signal-to-noise ratio compared to traditional cameras. A prototype has been built and is being tested.

© 2013 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt.39(13), 2210–2220 (2000).
    [CrossRef] [PubMed]
  2. P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
    [CrossRef]
  3. P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
    [CrossRef]
  4. P. Mouroulis, R. O. Green, and D. W. Wilson, “Optical design of a coastal ocean imaging spectrometer,” Opt. Express16(12), 9087–9096 (2008).
    [CrossRef] [PubMed]
  5. G. Høye and A. Fridman, “Hyperspektralt kamera og metode for å ta opp hyperspektrale data,” Norwegian patent application number 20111001.
  6. G. Høye and A. Fridman, “Hyperspectral camera and method for acquiring hyperspectral data,” PCT international patent application number PCT/NO2012/050132.
  7. G. Høye and A. Fridman, “A method for restoring data in a hyperspectral imaging system with large keystone without loss of spatial resolution,” FFI-rapport 2009/01351 (2009), declassified on January 28th 2013.
  8. A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published).
  9. M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
    [CrossRef]
  10. H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE86(8), 1586–1593 (1998).
    [CrossRef]
  11. P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE4829, 1048–1049 (2003).
    [CrossRef]
  12. http://www.hyspex.no/products/hyspex/vnir1600.php
  13. R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE7812, 78120M, 78120M-13 (2010).
    [CrossRef]
  14. G. Høye and A. Fridman, “Performance analysis of the proposed new restoring camera for hyperspectral imaging,” FFI-rapport 2010/02383 (2010), to be declassified.
  15. Technical specifications on CIS 2521F (last accessed 20.04.2013), http://www.fairchildimaging.com/catalog/focal-plane-arrays/scmos/cis-2521f
  16. B. E. A. Salech and M. C. Teich, Fundamentals of Photonics (John Wiley & Sons Inc., 1991).

2012

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

2011

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

2010

R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE7812, 78120M, 78120M-13 (2010).
[CrossRef]

2008

2006

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

2003

P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE4829, 1048–1049 (2003).
[CrossRef]

2000

1998

H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE86(8), 1586–1593 (1998).
[CrossRef]

Boardman, J.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Chrien, T. G.

Eastwood, M.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Feldman, M.

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

Fisher, J.

R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE7812, 78120M, 78120M-13 (2010).
[CrossRef]

Franklin, B. D.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Fridman, A.

A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published).

Gao, B. C.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Green, R. O.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

P. Mouroulis, R. O. Green, and D. W. Wilson, “Optical design of a coastal ocean imaging spectrometer,” Opt. Express16(12), 9087–9096 (2008).
[CrossRef] [PubMed]

P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE4829, 1048–1049 (2003).
[CrossRef]

P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt.39(13), 2210–2220 (2000).
[CrossRef] [PubMed]

Guckel, H.

H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE86(8), 1586–1593 (1998).
[CrossRef]

Hebert, D.

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

Hoffmann, H. D.

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Høye, G.

A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published).

Løke, T.

A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published).

Loosen, P.

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Lucke, R.

R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE7812, 78120M, 78120M-13 (2010).
[CrossRef]

Mouroulis, P.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

P. Mouroulis, R. O. Green, and D. W. Wilson, “Optical design of a coastal ocean imaging spectrometer,” Opt. Express16(12), 9087–9096 (2008).
[CrossRef] [PubMed]

P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE4829, 1048–1049 (2003).
[CrossRef]

P. Mouroulis, R. O. Green, and T. G. Chrien, “Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information,” Appl. Opt.39(13), 2210–2220 (2000).
[CrossRef] [PubMed]

Mumolo, J. M.

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

Plum, H. D.

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Poprawe, R.

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Richardson, B. S.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Rodriguez, J. I.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Traub, M.

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Urquiza, E.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Van Gorp, B.

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

Van Gorp, B. E.

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

White, V. E.

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

Wieching, K.

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Wilson, D. W.

Appl. Opt.

Opt. Express

Proc. IEEE

H. Guckel, “High-aspect-ratio micromachining via deep X-ray lithography,” Proc. IEEE86(8), 1586–1593 (1998).
[CrossRef]

Proc. SPIE

P. Mouroulis and R. O. Green, “Optical design for high fidelity imaging spectrometry,” Proc. SPIE4829, 1048–1049 (2003).
[CrossRef]

R. Lucke and J. Fisher, “The Schmidt-Dyson: a fast space-borne wide-field hyperspectral imager,” Proc. SPIE7812, 78120M, 78120M-13 (2010).
[CrossRef]

P. Mouroulis, B. E. Van Gorp, V. E. White, J. M. Mumolo, D. Hebert, and M. Feldman, “A compact, fast, wide-field imaging spectrometer system,” Proc. SPIE8032, 80320U, 80320U-12 (2011).
[CrossRef]

P. Mouroulis, B. Van Gorp, R. O. Green, M. Eastwood, J. Boardman, B. S. Richardson, J. I. Rodriguez, E. Urquiza, B. D. Franklin, and B. C. Gao, “Portable remote imaging spectrometer (PRISM): laboratory and field calibrations,” Proc. SPIE8515, 85150F, 85150F-10 (2012).
[CrossRef]

A. Fridman, G. Høye, and T. Løke, “Resampling in hyperspectral cameras as an alternative to correcting keystone in hardware, with focus on benefits for the optical design and data quality,” Proc. SPIE (to be published).

M. Traub, H. D. Hoffmann, H. D. Plum, K. Wieching, P. Loosen, and R. Poprawe, “Homogenization of high power diode laser beams for pumping and direct applications,” Proc. SPIE6104, 61040Q, 61040Q-10 (2006).
[CrossRef]

Other

G. Høye and A. Fridman, “Hyperspektralt kamera og metode for å ta opp hyperspektrale data,” Norwegian patent application number 20111001.

G. Høye and A. Fridman, “Hyperspectral camera and method for acquiring hyperspectral data,” PCT international patent application number PCT/NO2012/050132.

G. Høye and A. Fridman, “A method for restoring data in a hyperspectral imaging system with large keystone without loss of spatial resolution,” FFI-rapport 2009/01351 (2009), declassified on January 28th 2013.

G. Høye and A. Fridman, “Performance analysis of the proposed new restoring camera for hyperspectral imaging,” FFI-rapport 2010/02383 (2010), to be declassified.

Technical specifications on CIS 2521F (last accessed 20.04.2013), http://www.fairchildimaging.com/catalog/focal-plane-arrays/scmos/cis-2521f

B. E. A. Salech and M. C. Teich, Fundamentals of Photonics (John Wiley & Sons Inc., 1991).

http://www.hyspex.no/products/hyspex/vnir1600.php

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (21)

Fig. 1
Fig. 1

Light from the scene (a) is focused by the foreoptics (b) onto the slit plane (c). The slit blocks most of the scene, leaving only a narrow horizontal portion (d) of the scene visible. The relay optics (e) forms an image of the scene with superimposed slit onto the sensor (f). Because of the presence of a dispersive element in the relay optics, each point of the narrow horizontal line (d) is stretched (dispersed) in the vertical direction. The image on the sensor (g) contains spectra for each small area of the scene (d). A 2-dimensional image is obtained by scanning in the vertical direction.

Fig. 2
Fig. 2

The light from a small scene area is dispersed in the vertical direction, creating an image of the spectrum in the sensor plane. When the keystone is large (a), some wavelengths (particularly red, in this example) are partially projected onto the neighboring pixels on the sensor. As a result, the spectrum, captured by one column of sensor pixels, may contain large errors. A perfect keystone-free optics would project the same spectrum onto the sensor as shown in (b). Then, the captured spectrum would be correct.

Fig. 3
Fig. 3

Datacube. Hyperspectral data should preferably be completely keystone-free so that the spectral information for each spatial pixel is correct.

Fig. 4
Fig. 4

The figure shows (a) scene pixels with known values and corresponding recorded sensor pixels, and (b) recorded sensor pixels with known values and corresponding scene pixels with unknown values.

Fig. 5
Fig. 5

Scene pixels and corresponding recorded sensor pixels for the general case.

Fig. 6
Fig. 6

The light mixing chambers. The light from the scene (green curve) is mixed in the chambers so that the light distribution at the output of each chamber (red curve) is as uniform as possible. The key is to obtain a known light distribution at the output that is independent of the light distribution at the input of the chambers. The picture of the chambers is for illustration purposes only. The dimensions of the real chambers will depend on the sensor pixel size and the optics.

Fig. 7
Fig. 7

Distribution of rays within a light mixing chamber. In this example, the F-number is F2.8 and the width of the chamber (x-axis) is 1. The corresponding length of the chamber (y-axis) is 5.6, as calculated from Eq. (5) with k = 2. The backface of the chamber is marked by a horizontal black line (approximately at the middle of the figure). (a) Rays are launched from all five areas on the front surface (bottom horizontal line). (b)-(d) Rays are launched from different single areas on the front surface. Note the different scales on the x- and y-axes. The aperture angle (10° in this example) therefore appears larger than it actually is.

Fig. 8
Fig. 8

Performance of the light mixing chambers.

Fig. 9
Fig. 9

An example of a possible relay system for the mixel camera. The slit with the mixing chambers is shown to the left in the figure. Different colors correspond to different field points. The direction of the dispersion is perpendicular to the drawing plane. The dispersion is therefore not visible in this figure.

Fig. 10
Fig. 10

Example of a possible foreoptics for the mixel camera. The location of the mixel array in the image plane is marked with red color. The direction of the mixel array is perpendicular to the figure plane.

Fig. 11
Fig. 11

The reference scene consisting of 320 scene pixels. The blue curve shows the photon number density, while the corresponding scene pixel values are shown in red.

Fig. 12
Fig. 12

Relative error (1σ) due to photon noise as a function of the number of photons in the signal.

Fig. 13
Fig. 13

Misregistration errors for (a) the HW corrected camera with 0.1 pixel keystone and (b) the mixel camera. The standard deviation of the error is marked by a dashed red line. Photon and readout noise are not included.

Fig. 14
Fig. 14

Camera performance when photon and readout noise are included. The figures show the relative error for (a) a HW corrected camera with 0.1 pixel keystone, (b) a HW corrected camera with 0.3 pixel keystone, (c) a mixel camera that collects the same amount of light as the HW corrected cameras and (d) a mixel camera that collects four times more light. The standard deviation of the error is marked by a dashed red line.

Fig. 15
Fig. 15

Camera performance in bright light when photon and readout noise are included. (a) Relative error for a HW corrected camera with 0.1 pixel keystone, (b) relative error for a mixel camera that collects the same amount of light as the HW corrected camera and (c) relative error for a mixel camera that collects four times more light. The standard deviation of the error is marked by a dashed red line.

Fig. 16
Fig. 16

Example of a third order polynomial transition (red).

Fig. 17
Fig. 17

Mixel camera with (a) 50% transitions and (b) 40% transitions. The data are restored assuming 35% transitions. Photon and readout noise are not included.

Fig. 18
Fig. 18

Relative error for the mixel camera due to (a) 0.06 pixel and (b) 0.01 pixel misalignment between the mixel array and the sensor pixels. The standard deviation of the error is marked by a dashed red line. Photon and readout noise are not included.

Fig. 19
Fig. 19

The mixel array with one single mixel at the left end of the slit.

Fig. 20
Fig. 20

Intensity distribution (blue curve) of light coming from a single mixel onto the sensor pixels. The shape of the curve is determined by the size of the mixel and the point spread function of the relay optics.

Fig. 21
Fig. 21

The mixel array with a single mixel at the end and a second mixel array below.

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

E 1 R = 4 5 E 1 = 4 5 10=8, E 2 R = 1 5 E 1 + 3 5 E 2 = 1 5 10+ 3 5 30=20, E 3 R = 2 5 E 2 + 2 5 E 3 = 2 5 30+ 2 5 100=52, E 4 R = 3 5 E 3 + 1 5 E 4 = 3 5 100+ 1 5 50=70, E 5 R = 4 5 E 4 = 4 5 50=40,
4 5 E 1 = E 1 R =8, 1 5 E 1 + 3 5 E 2 = E 2 R =20, 2 5 E 2 + 2 5 E 3 = E 3 R =52, 3 5 E 3 + 1 5 E 4 = E 4 R =70, 4 5 E 4 = E 5 R =40.
E m R = n=1 N q mn E n , m=1,2,...,m,...,M1,M,
( q 11 q 21 q 22 q m(n1) q mn q (M1)(N1) q (M1)N q MN )( E 1 E n E N )=( E 1 R E m R E M R ),
L=kFw,
σ= 1 E .
dE= ( E final E init ) E init ,
y=a (x x 0 ) 3 +c(x x 0 )+ E 0 ,
a= 4c 3 ( x 2 x 1 ) 2 , c= 3( E 2 E 1 ) 2( x 2 x 1 ) .

Metrics