Abstract

The ACTIVE-EYES (adaptive control for thermal imagers via electro-optic elements to yield an enhanced sensor) architecture, an adaptive image-segmentation and processing architecture, based on digital micromirror (DMD) array technology, is described. The concept provides efficient front-end processing of multispectral image data by adaptively segmenting and routing portions of the scene data concurrently to an imager and a spectrometer. The goal is to provide a large reduction in the amount of data required to be sensed in a multispectral imager by means of preprocessing the data to extract the most useful spatial and spectral information during detection. The DMD array provides the flexibility to perform a wide range of spatial and spectral analyses on the scene data. The spatial and spectral processing for different portions of the input scene can be tailored in real time to achieve a variety of preprocessing functions. Since the detected intensity of individual pixels may be controlled, the spatial image can be analyzed with gain varied on a pixel-by-pixel basis to enhance dynamic range. Coarse or fine spectral resolution can be achieved in the spectrometer by use of dynamically controllable or addressable dispersion elements. An experimental prototype, which demonstrated the segmentation between an imager and a grating spectrometer, was demonstrated and shown to achieve programmable pixelated intensity control. An information theoretic analysis of the dynamic-range control aspect was conducted to predict the performance enhancements that might be achieved with this architecture. The results indicate that, with a properly configured algorithm, the concept achieves the greatest relative information recovery from a detected image when the scene is made up of a relatively large area of moderate-dynamic-range pixels and a relatively smaller area of strong pixels that would tend to saturate a conventional sensor.

© 2002 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Castracane, M. Gutin, “DMD-based bloom control for intensified imaging systems,” in Diffractive and Holographic Technologies, Systems, and Spatial Light Modulators IV, I. Cindrich, S. H. Lee, R. L. Sutherland, eds., Proc. SPIE3633, 234–242 (1999).
    [CrossRef]
  2. K. Kearney, M. Corio, Z. Ninkov, “Imaging spectroscopy with digital micromirrors,” in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, N. Sampat, G. M. Williams, T. Yeh, eds., Proc. SPIE3965, 11–20 (2000).
    [CrossRef]
  3. K. Kearney, Z. Ninkov, “Characterization of a digital micromirror device for use as an optical mask in imaging and spectroscopy,” in Spatial Light Modulators, R. L. Sutherland, ed., Proc. SPIE3292, 81–92 (1998).
    [CrossRef]
  4. Rochester Microsystems, 400 Air Park Drive, Suite 60, Rochester, New York.
  5. P. B. Fellgett, E. H. Linfoot, “On the assessment of optical images,” Philos. Trans. R. Soc. London 247, 269–407 (1955).
    [CrossRef]
  6. F. O. Huck, C. L. Fales, Z. Rahman, “An information theory of visual communication,” Philos. Trans. R. Soc. London Ser. A 354, 2193–2248 (1996).
    [CrossRef]
  7. V. M. Brajovic, R. Miyagawa, T. Kanade, “Temporal photoreception for adaptive dynamic range image sensing and encoding,” Neural Netw. 11, 1149–1158 (1998).
    [CrossRef]

1998

V. M. Brajovic, R. Miyagawa, T. Kanade, “Temporal photoreception for adaptive dynamic range image sensing and encoding,” Neural Netw. 11, 1149–1158 (1998).
[CrossRef]

1996

F. O. Huck, C. L. Fales, Z. Rahman, “An information theory of visual communication,” Philos. Trans. R. Soc. London Ser. A 354, 2193–2248 (1996).
[CrossRef]

1955

P. B. Fellgett, E. H. Linfoot, “On the assessment of optical images,” Philos. Trans. R. Soc. London 247, 269–407 (1955).
[CrossRef]

Brajovic, V. M.

V. M. Brajovic, R. Miyagawa, T. Kanade, “Temporal photoreception for adaptive dynamic range image sensing and encoding,” Neural Netw. 11, 1149–1158 (1998).
[CrossRef]

Castracane, J.

J. Castracane, M. Gutin, “DMD-based bloom control for intensified imaging systems,” in Diffractive and Holographic Technologies, Systems, and Spatial Light Modulators IV, I. Cindrich, S. H. Lee, R. L. Sutherland, eds., Proc. SPIE3633, 234–242 (1999).
[CrossRef]

Corio, M.

K. Kearney, M. Corio, Z. Ninkov, “Imaging spectroscopy with digital micromirrors,” in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, N. Sampat, G. M. Williams, T. Yeh, eds., Proc. SPIE3965, 11–20 (2000).
[CrossRef]

Fales, C. L.

F. O. Huck, C. L. Fales, Z. Rahman, “An information theory of visual communication,” Philos. Trans. R. Soc. London Ser. A 354, 2193–2248 (1996).
[CrossRef]

Fellgett, P. B.

P. B. Fellgett, E. H. Linfoot, “On the assessment of optical images,” Philos. Trans. R. Soc. London 247, 269–407 (1955).
[CrossRef]

Gutin, M.

J. Castracane, M. Gutin, “DMD-based bloom control for intensified imaging systems,” in Diffractive and Holographic Technologies, Systems, and Spatial Light Modulators IV, I. Cindrich, S. H. Lee, R. L. Sutherland, eds., Proc. SPIE3633, 234–242 (1999).
[CrossRef]

Huck, F. O.

F. O. Huck, C. L. Fales, Z. Rahman, “An information theory of visual communication,” Philos. Trans. R. Soc. London Ser. A 354, 2193–2248 (1996).
[CrossRef]

Kanade, T.

V. M. Brajovic, R. Miyagawa, T. Kanade, “Temporal photoreception for adaptive dynamic range image sensing and encoding,” Neural Netw. 11, 1149–1158 (1998).
[CrossRef]

Kearney, K.

K. Kearney, M. Corio, Z. Ninkov, “Imaging spectroscopy with digital micromirrors,” in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, N. Sampat, G. M. Williams, T. Yeh, eds., Proc. SPIE3965, 11–20 (2000).
[CrossRef]

K. Kearney, Z. Ninkov, “Characterization of a digital micromirror device for use as an optical mask in imaging and spectroscopy,” in Spatial Light Modulators, R. L. Sutherland, ed., Proc. SPIE3292, 81–92 (1998).
[CrossRef]

Linfoot, E. H.

P. B. Fellgett, E. H. Linfoot, “On the assessment of optical images,” Philos. Trans. R. Soc. London 247, 269–407 (1955).
[CrossRef]

Miyagawa, R.

V. M. Brajovic, R. Miyagawa, T. Kanade, “Temporal photoreception for adaptive dynamic range image sensing and encoding,” Neural Netw. 11, 1149–1158 (1998).
[CrossRef]

Ninkov, Z.

K. Kearney, Z. Ninkov, “Characterization of a digital micromirror device for use as an optical mask in imaging and spectroscopy,” in Spatial Light Modulators, R. L. Sutherland, ed., Proc. SPIE3292, 81–92 (1998).
[CrossRef]

K. Kearney, M. Corio, Z. Ninkov, “Imaging spectroscopy with digital micromirrors,” in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, N. Sampat, G. M. Williams, T. Yeh, eds., Proc. SPIE3965, 11–20 (2000).
[CrossRef]

Rahman, Z.

F. O. Huck, C. L. Fales, Z. Rahman, “An information theory of visual communication,” Philos. Trans. R. Soc. London Ser. A 354, 2193–2248 (1996).
[CrossRef]

Neural Netw.

V. M. Brajovic, R. Miyagawa, T. Kanade, “Temporal photoreception for adaptive dynamic range image sensing and encoding,” Neural Netw. 11, 1149–1158 (1998).
[CrossRef]

Philos. Trans. R. Soc. London

P. B. Fellgett, E. H. Linfoot, “On the assessment of optical images,” Philos. Trans. R. Soc. London 247, 269–407 (1955).
[CrossRef]

Philos. Trans. R. Soc. London Ser. A

F. O. Huck, C. L. Fales, Z. Rahman, “An information theory of visual communication,” Philos. Trans. R. Soc. London Ser. A 354, 2193–2248 (1996).
[CrossRef]

Other

J. Castracane, M. Gutin, “DMD-based bloom control for intensified imaging systems,” in Diffractive and Holographic Technologies, Systems, and Spatial Light Modulators IV, I. Cindrich, S. H. Lee, R. L. Sutherland, eds., Proc. SPIE3633, 234–242 (1999).
[CrossRef]

K. Kearney, M. Corio, Z. Ninkov, “Imaging spectroscopy with digital micromirrors,” in Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, N. Sampat, G. M. Williams, T. Yeh, eds., Proc. SPIE3965, 11–20 (2000).
[CrossRef]

K. Kearney, Z. Ninkov, “Characterization of a digital micromirror device for use as an optical mask in imaging and spectroscopy,” in Spatial Light Modulators, R. L. Sutherland, ed., Proc. SPIE3292, 81–92 (1998).
[CrossRef]

Rochester Microsystems, 400 Air Park Drive, Suite 60, Rochester, New York.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1

Schematic diagram of ACTIVE-EYES architecture concept. Reflective sensor optics image the scene on the DMD device, which in turn segments the image into one of two legs. The imaging leg (shown on the top) relays the image to a broadband sensor, whereas the spectrometer leg introduces a grating to effect a starfield spectrometer.

Fig. 2
Fig. 2

Schematic depiction of ACTIVE-EYES experimental setup showing incoming light directed into one of two legs: one for broadband imaging and one for spectral analysis.

Fig. 3
Fig. 3

Photograph of the ACTIVE-EYES experimental setup.

Fig. 4
Fig. 4

Example test image as it appears in its original bitmapped form and as displayed on the DMD.

Fig. 5
Fig. 5

Flow chart describing algorithm for eliminating saturation in the sensor by iterative reduction of the DMD duty cycle on a pixel-by-pixel basis.

Fig. 6
Fig. 6

Example of a highly saturated scene with all micromirrors at 100% duty cycle.

Fig. 7
Fig. 7

Screen capture from starfield spectrometer leg showing the spectra from two glints from the trucks scene shown in Fig. 6.

Fig. 8
Fig. 8

Example screen capture from the input scene shown in Fig. 6 with micromirror duty cycles adjusted on a pixel-by-pixel basis to extend the dynamic range of the CCD (the interlacing of the CCD camera is evident in the stripes in the saturated areas).

Fig. 9
Fig. 9

Plot of DMD sensor contrast enhancement showing SNR for input pixels requiring four effective doublings of the maximum detectable light level. This graph assumes a priori knowledge of the required attenuation.

Fig. 10
Fig. 10

Plot of DMD sensor contrast enhancement showing SNR for pixel measurements assuming an iterative approach beginning with maximum attenuation and decreasing pixels that are beneath the noise floor on each successive frame for five frames. This is compared with a global attenuation of 1/16 with five integrated frames.

Fig. 11
Fig. 11

Plot of DMD sensor contrast enhancement showing SNR for pixel measurements assuming an iterative approach beginning with minimum attenuation and increasing attenuation for pixels that are saturated on each successive frame for five frames. This is compared with a global attenuation of 1/16 with five integrated frames.

Fig. 12
Fig. 12

Plot of DMD sensor contrast enhancement showing SNR for pixel measurements assuming an iterative approach beginning with an intermediate attenuation (appropriate for pixel values between 512 and 1024) and decreasing attenuation for pixels that are beneath the noise floor and increasing attenuation for pixels above saturation on each successive frame for five frames. This is compared to a global attenuation of 1/16 with five integrated frames.

Fig. 13
Fig. 13

(a) Bimodal scene histogram for Example 1. The dashed curve corresponds to pixels that will appear saturated when imaging with an 8-bit camera. (b) Modified scene histogram for Example 1 resulting from local attenuation by a factor of 2. Previously saturated pixels are now within the dynamic range of the 8-bit camera, but their SNR has been lowered; previously unsaturated pixels do not experience a reduction in SNR. (c) Modified scene histogram for Example 1 resulting from global attenuation by a factor of 2. Previously saturated pixels are now within the dynamic range of the 8-bit camera, but all pixels in the image now have a reduced SNR.

Fig. 14
Fig. 14

(a) Bimodal scene histogram for Example 2. The dashed curve represents pixels that will appear saturated when imaging with an 8-bit camera. (b) Modified scene histogram for Example 2 resulting from local attenuation by a factor of 16. Previously saturated pixels are now within the dynamic range of the 8-bit camera, but their SNR has been lowered; previously unsaturated pixels do not experience a reduction in SNR. (c) Modified scene histogram for Example 2 resulting from global attenuation by a factor of 16. Previously saturated pixels are now within the dynamic range of the 8-bit camera, but all pixels in the image now have a reduced SNR.

Tables (1)

Tables Icon

Table 1 Statistical Pixel and Scene Information Data from Quantitative Examples

Metrics