Abstract

We propose a microoptical approach to ultra-compact optics for real-time vision systems that are inspired by the compound eyes of insects. The demonstrated module achieves approx. VGA resolution with a total track length of 1.4 mm which is about two times shorter than comparable single-aperture optics on images sensors of the same pixel pitch. The partial images that are separately recorded in different optical channels are stitched together to form a final image of the whole field of view by means of image processing. A software correction is applied to each partial image so that the final image is made free of distortion. The microlens arrays are realized by state of the art microoptical fabrication techniques on wafer-level which are suitable for a potential application in high volume e.g. for consumer electronic products.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. S. Mathur, M. Okincha, and M. Walters, "What Camera Manufacturers Want," 2007 International Image Sensor Workshop, Ogunquit Maine, USA, June 7.-10., 2007.
  2. Z. Popovic, R. Sprague, and G. Connell, "Technique for monolithic fabrication of microlens arrays," Appl. Opt. 7, 1281-1284 (1988).
    [CrossRef] [PubMed]
  3. D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
    [CrossRef]
  4. J. Duparré, P. Dannberg, P. Schreiber, A. Bräuer, and A. Tünnermann, "Thin compound-eye camera," Appl. Opt. 44, 2949-2956 (2005).
    [CrossRef] [PubMed]
  5. A. Bruckner, J. Duparré, F. Wippermann, P. Dannberg, and A. Bräuer, "Microoptical Artificial Compound Eyes," in Flying Insects and Robots, D. Floreano, J.-C. Zufferey, M. V. Srinivasan and C. Ellington, eds., (Springer, 2009), pp. 127-142.
    [CrossRef]
  6. A. Bruckner, J. Duparré, P. Dannberg, A. Bräuer, and A. Tünnermann, "Artificial neural superposition eye," Opt. Express 15, 11922-11933 (2007).
    [CrossRef] [PubMed]
  7. E. Buschbeck, B. Ehmer, and R. Hoy, "Chunk versus Point Sampling: Visual Imaging in a Small Insect," Science 286, 1178-1179 (1999).
    [CrossRef] [PubMed]
  8. W. Pix, J. M. Zanker, and J. Zeil, "The Optomotor Response and Spatial Resolution of the Visual System in Male Xenos Vesparum (Strepsiptera)," J. Exp. Biol. 203, 3397-3409 (2000).
    [PubMed]
  9. M. Shankar, R. Willett, N. Pitsianis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carriere, C. Chen, D. Prather, and D. Brady, "Thin infrared imaging systems through multichannel sampling," Appl. Opt. 47, B1-B10 (2008).
    [CrossRef] [PubMed]
  10. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, "Reconstruction of a High-Resolution Image on a Compound-Eye Image-Capturing System," Appl. Opt. 43, 1719-1727 (2004).
    [CrossRef] [PubMed]
  11. K. Nitta, R. Shogenji, S. Miyatake, and J. Tanida, "Image reconstruction for thin observation module by bound optics by using the iterative back projection method," Appl. Opt. 45, 2893-2900 (2006).
    [CrossRef] [PubMed]
  12. R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
    [CrossRef]
  13. G. Druart, N. Guérineau, R. Haidar, S. Thétas, J. Taboury, S. Rommeluère, J. Primot, and M. Fendler, "Demonstration of an infrared microcamera inspired by Xenos peckii vision," Appl. Opt. 48, 3368-3374 (2009).
    [CrossRef] [PubMed]
  14. T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
    [CrossRef]
  15. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
    [CrossRef]
  16. A. Lohmann, "Scaling laws for lens systems," Appl. Opt. 28, 4996-4998 (1989).
    [CrossRef] [PubMed]
  17. R. Kinglake, A History of the Photographic Lens (Academic Press, 1989).
  18. J. Duparré, F. Wippermann, P. Dannberg, and A. Reimann, "Chirped arrays of refractive ellipsoidal microlenses for aberration correction under oblique incidence," Opt. Express 13, 10539-10551 (2005).
    [CrossRef] [PubMed]
  19. P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
    [CrossRef]
  20. P. Dannberg, F. Wippermann, A. Bruckner, A. Matthes, P. Schreiber, and A. Bräuer, Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745 Jena, Germany, are preparing a manuscript to be called "Wafer-Level hybrid integration of complex micro-optical modules."

2009 (1)

2008 (1)

2007 (2)

A. Bruckner, J. Duparré, P. Dannberg, A. Bräuer, and A. Tünnermann, "Artificial neural superposition eye," Opt. Express 15, 11922-11933 (2007).
[CrossRef] [PubMed]

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

2006 (1)

2005 (2)

2004 (2)

2000 (2)

W. Pix, J. M. Zanker, and J. Zeil, "The Optomotor Response and Spatial Resolution of the Visual System in Male Xenos Vesparum (Strepsiptera)," J. Exp. Biol. 203, 3397-3409 (2000).
[PubMed]

T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
[CrossRef]

1999 (2)

E. Buschbeck, B. Ehmer, and R. Hoy, "Chunk versus Point Sampling: Visual Imaging in a Small Insect," Science 286, 1178-1179 (1999).
[CrossRef] [PubMed]

P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
[CrossRef]

1990 (1)

D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
[CrossRef]

1989 (1)

1988 (1)

Bierbaum, R.

P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
[CrossRef]

Brady, D.

Bräuer, A.

Bruckner, A.

Buschbeck, E.

E. Buschbeck, B. Ehmer, and R. Hoy, "Chunk versus Point Sampling: Visual Imaging in a Small Insect," Science 286, 1178-1179 (1999).
[CrossRef] [PubMed]

Carriere, J.

Catrysse, P.

T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
[CrossRef]

Chen, C.

Chen, T.

T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
[CrossRef]

Connell, G.

Daly, D.

D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
[CrossRef]

Dannberg, P.

Davies, N.

D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
[CrossRef]

Druart, G.

Duparré, J.

Ehmer, B.

E. Buschbeck, B. Ehmer, and R. Hoy, "Chunk versus Point Sampling: Visual Imaging in a Small Insect," Science 286, 1178-1179 (1999).
[CrossRef] [PubMed]

Elad, M.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Erdmann, L.

P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
[CrossRef]

Farsiu, S.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Fendler, M.

Gamal, A. E.

T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
[CrossRef]

Gibbons, R.

Guérineau, N.

Haidar, R.

Horisaki, R.

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

Hoy, R.

E. Buschbeck, B. Ehmer, and R. Hoy, "Chunk versus Point Sampling: Visual Imaging in a Small Insect," Science 286, 1178-1179 (1999).
[CrossRef] [PubMed]

Hutley, M. C.

D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
[CrossRef]

Ichioka, Y.

Irie, S.

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

Kitamura, Y.

Kley, E. B.

P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
[CrossRef]

Kolste, R. T.

Kondou, N.

Krehl, A.

P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
[CrossRef]

Lohmann, A.

Masaki, Y.

Milanfar, P.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Miyamoto, M.

Miyatake, S.

Miyazaki, D.

Morimoto, T.

Nakao, Y.

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

Nitta, K.

Ogura, Y.

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

Pitsianis, N.

Pix, W.

W. Pix, J. M. Zanker, and J. Zeil, "The Optomotor Response and Spatial Resolution of the Visual System in Male Xenos Vesparum (Strepsiptera)," J. Exp. Biol. 203, 3397-3409 (2000).
[PubMed]

Popovic, Z.

Prather, D.

Primot, J.

Reimann, A.

Robinson, D.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Rommeluère, S.

Schreiber, P.

Schulz, T.

Shankar, M.

Shogenji, R.

Sprague, R.

Stevens, R. F.

D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
[CrossRef]

Taboury, J.

Tanida, J.

Thétas, S.

Toyoda, T.

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

Tünnermann, A.

Wandell, B.

T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
[CrossRef]

Willett, R.

Wippermann, F.

Yamada, K.

Zanker, J. M.

W. Pix, J. M. Zanker, and J. Zeil, "The Optomotor Response and Spatial Resolution of the Visual System in Male Xenos Vesparum (Strepsiptera)," J. Exp. Biol. 203, 3397-3409 (2000).
[PubMed]

Zeil, J.

W. Pix, J. M. Zanker, and J. Zeil, "The Optomotor Response and Spatial Resolution of the Visual System in Male Xenos Vesparum (Strepsiptera)," J. Exp. Biol. 203, 3397-3409 (2000).
[PubMed]

Appl. Opt. (7)

Int. J. Imaging Syst. Technol. (1)

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

J. Exp. Biol. (1)

W. Pix, J. M. Zanker, and J. Zeil, "The Optomotor Response and Spatial Resolution of the Visual System in Male Xenos Vesparum (Strepsiptera)," J. Exp. Biol. 203, 3397-3409 (2000).
[PubMed]

Meas. Sci. Technol. (1)

D. Daly, R. F. Stevens, M. C. Hutley, and N. Davies, "The manufacture of microlenses by melting photoresist," Meas. Sci. Technol. 1, 759-766 (1990).
[CrossRef]

Microsyst. Technol. (1)

P. Dannberg, L. Erdmann, R. Bierbaum, A. Krehl, A. Bräuer, and E. B. Kley, "Micro-optical elements and their integration to glass and optoelectronic wafers," Microsyst. Technol. 6, 41-47 (1999).
[CrossRef]

Opt. Express (2)

Proc. SPIE (2)

T. Chen, P. Catrysse, A. E. Gamal, and B. Wandell, "How small should pixel size be?" Proc. SPIE 3965, 451-459 (2000).
[CrossRef]

R. Horisaki, S. Irie, Y. Nakao, Y. Ogura, T. Toyoda, Y. Masaki, and J. Tanida, "3D information acquisition using a compound imaging system," Proc. SPIE 6695, 66950F (2007).
[CrossRef]

Science (1)

E. Buschbeck, B. Ehmer, and R. Hoy, "Chunk versus Point Sampling: Visual Imaging in a Small Insect," Science 286, 1178-1179 (1999).
[CrossRef] [PubMed]

Other (4)

S. Mathur, M. Okincha, and M. Walters, "What Camera Manufacturers Want," 2007 International Image Sensor Workshop, Ogunquit Maine, USA, June 7.-10., 2007.

A. Bruckner, J. Duparré, F. Wippermann, P. Dannberg, and A. Bräuer, "Microoptical Artificial Compound Eyes," in Flying Insects and Robots, D. Floreano, J.-C. Zufferey, M. V. Srinivasan and C. Ellington, eds., (Springer, 2009), pp. 127-142.
[CrossRef]

R. Kinglake, A History of the Photographic Lens (Academic Press, 1989).

P. Dannberg, F. Wippermann, A. Bruckner, A. Matthes, P. Schreiber, and A. Bräuer, Fraunhofer Institute for Applied Optics and Precision Engineering, Albert-Einstein-Str. 7, D-07745 Jena, Germany, are preparing a manuscript to be called "Wafer-Level hybrid integration of complex micro-optical modules."

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

(a) Schematic working principle of the compound eye of Xenos Peckii (after [7]). Different extended parts of the object space (illustrated by letter sequence) are imaged onto the retina of each individual eyelet. The signals of each retina are rotated in the early neural layers. Finally, the partial images are linked up to a real image of the object space in the lamina. (b) Schematic section demonstrating the working principle of an electronic cluster eye (eCley).

Fig. 2
Fig. 2

Visualization of the basic parameters which characterize the FOV of a camera system. The number of image pixels (Nx, Ny) directly correspond to the number of object sampling intervals. The object distance is denoted by OD.

Fig. 3
Fig. 3

Principle of braided sampling of the object space by adjacent optical channels with k = 2. Colors are used for visualization only. Equal colored items refer to the same optical channel.

Fig. 4
Fig. 4

Lateral scaling behavior of the proposed eCley when using commercial image sensors. The fill factor Γ (ratio of partial image size and pitch of the partial images, black solid lines) as well as sensor format size in pixels along one dimension (red solid lines) are shown as functions of the final image resolution (x axis) and the number of pixels per channel (y axis). The bold black line marks the case of a maximum fill factor of Γ = 1. A fixed FOV of 70 degrees, a F/# of 2.8 and k = 2 are assumed.

Fig. 5
Fig. 5

Schematic cross section of the optical layout of the electronic cluster eye prototype. The optical module consists of different layers and is directly attached to the image sensor (shown in gray). Glass substrates are indicated in light blue. Dark blue and green color is used for the Ormocomp material (Micro Resist Tech.) of the microlenses and the polymer spacer, respectively.

Fig. 6
Fig. 6

Simulated images of the eCley using a non-sequential ray tracing method and a two-dimensional representation with lambertian illumination for the target object. (a): CCTV test chart. (b): An image of an array of spoke targets. The image resolution is 700 x 550 pixels in both cases.

Fig. 7
Fig. 7

(a): Diced wafer with several optics modules of electronic cluster eyes. (b): Photograph showing the size comparison between the eCley with VGA resolution on the image sensor array and one cent coin.

Fig. 8
Fig. 8

Image of a CCTV test chart which has been acquired by the prototype of the eCley. (a): Image as it is recorded by the image sensor array with a full resolution of 3MP. The inset shows a magnified section of the 17×13 partial images. (b): Final image after the application of the image processing pipeline. The image resolution is 700×550 pixels.

Fig. 9
Fig. 9

(a) View on the Fraunhofer-Instiute IOF as it has been captured with the eCley prototype system. The image resolution is 700×550 pixels. (b) Comparison between different polychromatic on-axis MTF curves. Solid red line: simulated MTF. Black squares: MTF measured within the center of the image plane of the optical module with relay lens. Green circles: MTF measured within the unprocessed image of the central channel of the fully assembled prototype. Blue triangles: MTF measured in the center of the final image of the prototype (optics + image sensor) including all image processing. The gray dashed line shows the on-axis MTF of a commercial WLO camera.

Fig. 10
Fig. 10

(a) Image of a square grating target. Color has been transformed to grayscale in order to measure distortion. The line scan for the grayscale profile in (b) is shown in the dashed black line. (b) Line scan through image of the grid. The measured center of each grid line is marked by a tick label on the x-axis in order to characterize distortion.

Tables (2)

Tables Icon

Table 1 Parameters of the VGA eCley demonstration systems

Tables Icon

Table 2 Fabrication steps and their achieved accuracy

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

f eff = I 2 tan ( α ) .
Δ ϕ x = arctan { tan α x N x / 2 } ; Δ ϕ y = arctan { tan α y N y / 2 } .
tan α x tan α y = N x N y
tan 2 α = tan 2 α x + tan 2 α y .
f s a = p p x tan ( Δ ϕ ) ,
Δ ϕ x = Δ ϕ y = Δ ϕ .
Δ p k = p L p k ,
Δ p eff = p p x k ,
tan ( Δ ϕ eff ) = Δ p e f f f = p p x k f .
tan ( Δ ϕ eff ) tan ( Δ ϕ ) = f sa k f .
f sa k f = ! 1
f = f sa k .
Δ p k = N g p p x k ,
Γ = N g p p x p k .
K x , y = N x , y n x , y .

Metrics