Abstract

We propose an ultra-thin imaging system which is based on the neural superposition compound eye of insects. Multiple light sensitive pixels in the footprint of each lenslet of this multi-channel configuration enable the parallel imaging of the individual object points. Together with the digital superposition of related signals this multiple sampling enables advanced functionalities for artificial compound eyes. Using this technique, color imaging and a circumvention for the trade-off between resolution and sensitivity of ultra-compact camera devices have been demonstrated in this article. The optical design and layout of such a system is discussed in detail. Experimental results are shown which indicate the attractiveness of microoptical artificial compound eyes for applications in the field of machine vision, surveillance or automotive imaging.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |

  1. M. F. Land and D.-E. Nilsson, Animal Eyes, Oxford Animal Biology Series (Oxford University Press, Oxford, 2002).
  2. J. Duparre, P. Dannberg, P. Schreiber, A. Bruer and A. Tunnermann, "Thin compound-eye camera," Appl. Opt. 44,2949-2956 (2005).
    [CrossRef] [PubMed]
  3. J. Duparre, F. Wippermann, P. Dannberg and A. Reimann, "Chirped arrays of refractive ellipsoidal microlenses for aberration correction under oblique incidence," Opt. Express 13,10539-10551 (2005).
    [CrossRef] [PubMed]
  4. K. Kirschfeld and N. Franceschini, "Optical characteristics of ommatidia in the complex eye of Musca," Kybernetik 5,47-52 (1968).
    [CrossRef] [PubMed]
  5. H. Tian, B. Fowler and A. E. Gamal, "Analysis of temporal noise in CMOS photodiode active pixel sensor," IEEE J. Solid-State Circuits 36,92-101 (2001).Q1
    [CrossRef]
  6. A. E. Gamal, B. Fowler, H. Min and X. Liu, "Modeling and estimation of FPN components in CMOS image sensors," in Solid State Sensor Arrays: Development and Applications II, M. M. Blouke, ed., Proc. SPIE 3301, 168-177 (1998).
  7. I. N. Bronstein, K. A. Semendjajew, G. Musiol and H. Muhlig, "Taschenbuch der Mathematik," (Harri Deutsch, Frankfurt am Main, 2001).
  8. B. E. Bayer, "Color Imaging Array," U.S. Patent 3,971,065, 1976.
  9. J. Adams, K. Parulski and K. Spaulding, "Color processing in digital cameras," IEEE Micro 18,20-30 (1998).
    [CrossRef]
  10. R. F. Lyon and P. M. Hubel, "Eyeing the camera: into the next century," in Proc. IS&T/TSID 10th Color Imaging Conf., (Scottsdale, AZ, USA, 2002), pp. 349-355.
  11. J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto and S. Miyatake, "Color imaging with an integrated compound imaging system," Opt. Express 11,2109-2117 (2003).
    [CrossRef] [PubMed]
  12. R. Shogenji, Y. Kitamura, K. Yamada, S. Miyatake and J. Tanida, "Multispectral imaging using compact compound optics," Opt. Express 12,1643-1655 (2004).
    [CrossRef] [PubMed]

2005

J. Duparre, P. Dannberg, P. Schreiber, A. Bruer and A. Tunnermann, "Thin compound-eye camera," Appl. Opt. 44,2949-2956 (2005).
[CrossRef] [PubMed]

J. Duparre, F. Wippermann, P. Dannberg and A. Reimann, "Chirped arrays of refractive ellipsoidal microlenses for aberration correction under oblique incidence," Opt. Express 13,10539-10551 (2005).
[CrossRef] [PubMed]

2004

2003

2001

H. Tian, B. Fowler and A. E. Gamal, "Analysis of temporal noise in CMOS photodiode active pixel sensor," IEEE J. Solid-State Circuits 36,92-101 (2001).Q1
[CrossRef]

1998

J. Adams, K. Parulski and K. Spaulding, "Color processing in digital cameras," IEEE Micro 18,20-30 (1998).
[CrossRef]

1968

K. Kirschfeld and N. Franceschini, "Optical characteristics of ommatidia in the complex eye of Musca," Kybernetik 5,47-52 (1968).
[CrossRef] [PubMed]

Appl. Opt.

IEEE J. Solid-State Circuits

H. Tian, B. Fowler and A. E. Gamal, "Analysis of temporal noise in CMOS photodiode active pixel sensor," IEEE J. Solid-State Circuits 36,92-101 (2001).Q1
[CrossRef]

IEEE Micro

J. Adams, K. Parulski and K. Spaulding, "Color processing in digital cameras," IEEE Micro 18,20-30 (1998).
[CrossRef]

Kybernetik

K. Kirschfeld and N. Franceschini, "Optical characteristics of ommatidia in the complex eye of Musca," Kybernetik 5,47-52 (1968).
[CrossRef] [PubMed]

Opt. Express

Other

R. F. Lyon and P. M. Hubel, "Eyeing the camera: into the next century," in Proc. IS&T/TSID 10th Color Imaging Conf., (Scottsdale, AZ, USA, 2002), pp. 349-355.

M. F. Land and D.-E. Nilsson, Animal Eyes, Oxford Animal Biology Series (Oxford University Press, Oxford, 2002).

A. E. Gamal, B. Fowler, H. Min and X. Liu, "Modeling and estimation of FPN components in CMOS image sensors," in Solid State Sensor Arrays: Development and Applications II, M. M. Blouke, ed., Proc. SPIE 3301, 168-177 (1998).

I. N. Bronstein, K. A. Semendjajew, G. Musiol and H. Muhlig, "Taschenbuch der Mathematik," (Harri Deutsch, Frankfurt am Main, 2001).

B. E. Bayer, "Color Imaging Array," U.S. Patent 3,971,065, 1976.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1.

(a): Schematic section of an ultra-thin artificial apposition compound eye camera. The object space is sampled by Δϕ which is the angle between two adjacent optical axes (sampling angle). The acceptance angle of one channel is denoted by Δφ. (b): Trade-off between resolution (red, dotted) and sensitivity (black line) in dependency on the pinhole diameter.

Fig. 2.
Fig. 2.

(a): Working principle of a neural superposition eye as found in nature. Note the neural wiring of different photo-receptors with parallel optical axes. (b): Its artificial counterpart - an artificial apposition compound eye with multiple pixels in each channel.

Fig. 3.
Fig. 3.

(a): Bayer color filter array on image sensor (adapted from [8]). (b): Layout of the artificial neural superposition eye with integrated polymer color filters to acquire color images. The case of N=3 is shown here. Each point in object space is imaged through a red, green and blue color filter in different channels. Note that axes with the same line style are parallel.

Fig. 4.
Fig. 4.

Left: Transmission curves of the used polymer color filter array for a thickness of 1.5 µm (source: Brewer Science Inc.). Right: Microscopic view on the pinhole group array with front side illumination. Distance between two pinholes of one group and diameters are 10.6 µm and 3 µm, respectively.

Fig. 5.
Fig. 5.

Left: Artificial neural superposition eye objective for color imaging (red box) directly attached to CMOS image sensor (model Saentis, ZMD). The microlens array with integrated color filters is shown in detail (black box). Right: Experimental setup for the assembly and characterization of the artificial neural superposition eye.

Fig. 6.
Fig. 6.

Images taken with the artificial neural superposition eye: Left: Siemens radial star target, Middle: an image of Carl Zeiss and Right: “Image Processing Lena”. (a),(b) and (c): Each image pixel recorded from one pixel per channel. (d), (e) and (f): Each image pixel is an average of nine pixels out of different channels. In (b) and (e) the fixed-pattern noise (FPN) has not been subtracted. The image resolution is 70×53 pixels without and 65×43 pixels with digital superposition.

Fig. 7.
Fig. 7.

Schematic algorithm for calculating the signal-to-noise ratio (SNR) from an image sequence.

Fig. 8.
Fig. 8.

(a): The measured signal-to-noise ratio (SNR) as a function of the integration time. (b): The ratio of the SNR for nine pixels per channel and for one pixel per channel gives the increase of the SNR as a function of integration time.

Fig. 9.
Fig. 9.

Measured and simulated MTF (polychromatic, spatial frequency normalized to Nyquist frequency ν n =1.28 cycles/degree) for either one pixel per channel and nine pixels per channel mode. Both measured curves result from a measurement of an average MTF across all channels which approaches the tangential off-axis simulation rather than the paraxial one. Note that there is no severe difference between both MTFs.

Fig. 10.
Fig. 10.

Images taken with the color imaging artificial neural superposition eye: From left to right: Macbeth color checker, the principle author, “Image Processing Lena”, a color mixing circle and a Siemens radial star pattern. The image resolution is 52×43 pixels except the first image which has 70×39 pixels.

Fig. 11.
Fig. 11.

Comparison of measured and simulated MTF for the artificial compound eye color camera (demo 1 with a Nyquist frequency of ν n =1.28 cyc/deg). The simulations are done for the three wavelengths: λ 1=455nm for blue, λ 2=530nm for green and λ 3=680nm for red. The off-axis simulation was carried out for a field angle of 13.6 degrees tangential.

Tables (1)

Tables Icon

Table 1. Parameters of selected demonstration systems and alignment tolerances.

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

Δ ϕ = arctan ( f p L p K ) .
Δ ϕ px = arctan ( f p px ) .
Δ ϕ Δ ϕ px = m 0 .
A i , k = S i , k + n i , k + o i , k .
A a , b = l = 1 m ( S a , b l + n a , b l ) .
σ 2 ( l = 1 m n a , b l ) = l = 1 m σ 2 ( n a , b l ) = m · σ 2 ( n a , b 1 ) .
σ tot = σ ( l = 1 m n a , b l ) = m · σ ( n a , b 1 ) .
SNR = S σ ,
SNR increase = SNR super position SNR standard ,

Metrics