Wafer-level optics is considered as a cost-effective approach to miniaturized cameras, because fabrication and assembly are carried out for thousands of lenses in parallel. However, in most cases the micro-optical fabrication process is not mature enough to reach the required accuracy of the optical elements, which may have complex profiles and sags in the mm-scale. Contrary, the creation of microlens arrays is well controllable so that we propose a multi aperture system called ”Optical Cluster Eye” which is based on conventional micro-optical fabrication techniques. The proposed multi aperture camera consists of many optical channels each transmitting a segment of the whole field of view. The design of the system provides the stitching of the partial images, so that a seamless image is formed and a commercially available image sensor can be used. The system can be fabricated on wafer-level with high yield due to small aperture diameters and low sags. The realized optics has a lateral size of 2.2 × 2.9 mm 2, a total track length of 1.86 mm, and captures images at VGA video resolution.
© 2011 OSA
The fabrication method of UV-molding has become more important for the fabrication of miniaturized camera modules during the last years. Being able to fabricate a high number of lenses simultaneously on wafer-level is the main advantage of this technique. Additionally, the assembly process can be done on wafer-level as well. However, there are still several problems concerning the fabrication process. First the shrinkage of the lenses is not sufficiently controllable and the complex aspherical lens shapes cannot be accomplished within the given tolerances. Second, the required vertical thickness uniformity cannot be guaranteed for all optical components that are necessary to build up one camera module. These problems are caused by the lens diameters and sags which are too large for state of the art micro-optical fabrication processes.
We propose a multi aperture approach named ”Optical Cluster Eye” (oCLEY) which applies a segmentation of the field of view by using microlens arrays (MLA) and aperture arrays. The mold masters of all MLA can be fabricated on wafer-level. Furthermore, the shrinkage during the UV-molding process is negligible, due to low sags and comparatively small microlens diameters. Consequently, the micro-optical fabrication process is much easier to handle than that of single aperture systems and can be completely carried out on wafer-level.
2. Prior art
Imaging of objects with multiple lenses, arranged side by side, is commonly known from scanner optics, copying machines or facsimile. Therefore, the object is usually scanned using GRIN lenses which generate an erect image with unity magnification. The partial images of different lenses superimpose at the image plane. A configuration of lenses in two or three rows, arranged in an offset pattern, is advantageous to achieve a uniform illumination of the image [1–3].
One of the first multi-channel systems with non-unity magnification was patented by Dennis Gabor in 1940, who described an optical setup consisting of two microlens arrays with slightly different pitches that are separated by the sum of their focal lengths . The system can be considered as a moiré magnifier, where an array of intermediate images is situated in the focal plane of the second MLA, which samples the intermediate image array with a period depending on the pitch difference of the two lens arrays. Additionally, the magnification as well as the superposition area of the images depend on this pitch difference .
To our knowledge, the first realized system using this superposition principle was shown by Hembd-Sölner et al. . Recent research on those superposition systems can be found in [7, 8]. The oCLEY system is no superposition system, but uses a special sampling frequency and aperture layout in order to achieve the stitching of the partial images without overlap. First steps in the development of such a system were made by Völkel et al.  and Duparré et al. . An oCLEY was realized that consists of three MLAs and aperture arrays. Unfortunately, a tradeoff between focused partial images and a good image stitching occurred due to tolerances in the fabrication process . In opposite to the optical image stitching in the oCLEY, there are several multi-aperture systems using a post-processing in order to stitch the partial images to achieve a complete image [11, 12]. Another multi aperture approach captures the whole field of view with every optical channel. Afterwards, those multiple images are processed in a way, that a final image with a much higher resolution than that of the single partial images is computed.
3. Optical Cluster Eye principle
The oCLEY is an imaging system based on a multi aperture architecture. The working principle is shown in Fig. 1. Different parts of the total field of view are transmitted by separated optical channels and are focused on the image plane as erect partial images. The optics is designed in a way, that all partial images are stitched together and reconstruct a complete conventional image at the sensor without overlap of the partial images.
The system consists of four microlens arrays (MLA) with different pitches. Thus, an optical channel consists of four lenslets along the optical path. The optical axes of the different channels are tilted to each other in order to transmit a wide field of view. The first microlens array focuses the rays into the intermediate image plane. The intermediate images are relayed by the third and fourth MLA onto the image sensor plane. The second MLA does not contribute to the focusing power of the system but acts as a field lens array. It redirects the rays in order to minimize vignetting and to decrease the lens diameters at the subsequent arrays [13, 14]. In comparison to the prior systems of Voelkel et al.  and Duparre et al.  we use four instead of three MLAs. The extra array tilts the optical axes in order to minimize the angle of incidence on the image plane and is positioned between the field lens array and the last lens array. The tilt leads to an increase of irradiance at the image sensor and a decreased crosstalk between adjacent pixels, both improving the signal-to-noise ratio. Hence, for the first time, it is possible to use a conventional image sensor to capture images with an oCLEY system of large FOV. The tilt leads to an increase of irradiance at the image plane and a decreased crosstalk between adjacent pixels, both improving the signal-to-noise ratio.
4. Design and simulation
4.1. Optical design
The oCLEY system was described by a one-dimensional paraxial model in order to achieve a basic understanding of the relationships between the different parameters. For this purpose the 3x3-matrix formalism was used which extends the 2x2 formalism by taking into account a shift Δx and/or a tilt Δφ of the optical elements . The system matrix Msys was computed in a way that is analog to that in . Knowing the paraxial input height hin and angle αin of one ray, its paraxial output height hout and angle αout can be computed as a function of the system parameters by Eq. (1).Eq. (1), the parameters of a starting system for the subsequent optical design are obtained. A system of equations is worked out, which is based on specific conditions that result from the oCLEY’s optical principle as shown in Section 3. For example, two of them are given in the following:
- Focusing condition:
An object in infinity should be focused on the image plane which implies that rays with the same field angle have to converge to a common focus. Therefore, hout has to be independent of hin, which leads to:
- Image stitching condition:
An object point, imaged with different channels, has to be focused on the same point at the image plane to yield an optimal stitching of the partial images. This means, that hout has to be independent of the channel number N for a constant value of αin. Since M 12 is independent of N, while all terms of M 13 are depending on N, this yields:
Conditions concerning the magnification, the number and size of partial images, the f-number (F/#), the centration of the second and fourth lens array, the tilt of the optical axes by the third MLA, and the numerical aperture (NA) of the first lens have been defined by using similar considerations. In this way, a system of ten equations has been revealed. The input parameters are: the total track length L, the size of the field of view αmax, the f-number F/#, the numerical aperture of the first lens NA 1, and the diameter of the image sensor D (see Fig. 2 for the notation of the parameters). Hence, the pitches and the focal lengths of all lens arrays as well as the total number of channels Ntot can be computed. However, there are still four undefined parameters left, since conditions could not be specified for computing all system parameters:
- the telecentric coefficient TC (defined in Fig. 2)
- the half angle of the field of view per channel
- the distance t between the first and third microlens array
- the distance d between the first and fourth microlens array
The parameters TC, , and t build an independent set of variables and are determined in an iterative procedure. First, an appropriate set of those three is chosen, followed by a plotting of the numerical apertures of the last three MLAs against d. The numerical aperture is limited to a maximum of about 0.4, due to the fabrication of the lenses by a reflow process. The value of d is chosen that gives the lowest values of the numerical apertures. In case that no d can be found for that condition, a new set of TC, , and t must be chosen, until the computed numerical apertures are below the limit.
In this way, a set of parameters was determined to build up a starting lens file using optical raytracing software. In further simulation steps, the paraxial lenses were gradually substituted by real lenses. Furthermore, glass substrates were inserted between the lenses in order to yield a technologically realizable system. Generally, these changes introduce aberrations which lead to a decrease in optical performance. An adjustment of the radii of curvature (ROC) of the microlenses, the spaces between the arrays, and the pitches was carried out by numerical optimization with a sequential raytracing software. A customized merit function used for this optimization controls both focusing and stitching of the partial images.
4.2. Optical layout
A layout of the oCLEY prototype is shown in Fig. 3. The system is built up by two glass substrates, each carrying MLAs on both sides. An aperture array is located on the substrate below each MLA. Except the second one, the aperture layers have been introduced to avoid optical crosstalk between different channels. Therefore, those apertures do not cause vignetting of the intended ray paths to the individual partial images. Thus, they do not influence the light sensitivity of the system. The second diaphragm array acts as field aperture and defines the shape and size of each partial image. It cuts out that part of the FOV which shall be imaged in the respective channel. The light sensitivity of the system is mainly determined by the F-number. The hexagonal arrangement of the optical channels provides the highest possible fill factor. Thus, the partial images and hence the field apertures are hexagonally shaped. A top view of the non-central apertures and lenses can be seen in Fig. 3 on the right side. The radii of curvature of the first and fourth MLA vary across different channels. Additionally, these arrays consist of toroidal lenses, which means that their ROC differs for the sagittal and tangential plane. As a result, each channel can be adapted to its central viewing direction, leading to a correction of astigmatism . The microlenses of the second and third array are rotationally symmetric, but differ in size. Radii of curvature and lenslet sags are specified in Table 1.
4.3. Simulation results
The designed oCLEY system was analyzed and image simulations were carried out in order to determine the optical performance. One of these simulated images is shown in Fig. 4(a). A brighter region can be observed between adjacent partial images due to a slight overlap. The field apertures have been designed with a small overlap in order to allow some tolerances for the fabrication process. Furthermore, the focal spot size, the focal depth, and the modulation transfer function (MTF) of the simulated system were analyzed to predict the resolution. The simulated MTF was plotted against the field angle in tangential and sagittal direction for half of the Nyquist frequency (111 cycles/mm) and for a quarter of the Nyquist frequency (55.5 cycles/mm) (see Fig. 4(b)).
5. Technological realization
The oCLEY is a perfect candidate for micro-optical resist reflow fabrication on wafer-level due to the low lens sags (see Table 1). In a first production step, chromium masks were generated by electron beam lithography. Subsequently, aperture arrays were structured on both sides of a precise thin glass substrate. The fabrication of the aperture arrays is carried out by photolithography. The first aperture array was etched in black chromium and the second in titanium, the third and fourth aperture arrays are made out of a black matrix polymer. The fabrication processes used for the MLAs and the accuracy during each step are shown in Fig. 5. The microlens arrays were UV-molded in a polymer (ORMOCOMP, Micro Resist Technologies) directly on the respective aperture layers on wafer-level. For this purpose a mask aligner (MA6, SUSS MicroTec) was used, which was modified for UV-molding. Please refer to [17–19] for detailed information about the fabrication process. After optics fabrication, the wafers were diced and the two components of the oCLEY system were assembled on die-level. For this purpose, alignment marks were used, which have been structured in the second and third aperture layers. Glass spacers with the required thickness were placed in between to stack the two components of the oCLEY in the correct distance. The alignment of the two components can be done on wafer-level as well and the entire stack can be diced subsequent to assembly, if additional spacer wafers are fabricated. Figure 6 shows the composite system in its final size in comparison to a pin.
The assembled oCLEY was mounted in front of a CMOS image sensor (model MT9M032, Aptina) on a demo-board and 640 × 480 pixels were read out. Active alignment was used for the assembly process in order to avoid tilt and to achieve the correct distance between the image sensor and the optics.
6. Experimental characterization
For the experimental characterization, test targets were displayed on a TFT-monitor and imaged with the optical system mounted on the image sensor. The test targets were imaged in different distances in order to examine the MTF as a function of the object distance. The nominal distance between the oCLEY and the monitor was about 36 cm.
6.1. MTF performance
A slanted edge test target was imaged with the prototype system for determining the MTF performance of the realized oCLEY system according to ISO 12233. Measurements have been performed for an on-axis position and a position at about 70 % of the field of view (FOV), which corresponds to a field angle of about 23°. For comparison, the MTF was simulated by an optical raytracing program which also takes account of diffraction effects. In Fig. 7(a), both the simulated and the experimental MTF are plotted against the spatial frequency for two positions of the field of view. The spatial frequency axis has a maximum value of 222 cycles/mm, which is defined by the physical Nyquist frequency of the image sensor (pixel pitch: 2.25 μm). The on-axis MTF for a quarter of the Nyquist frequency is theoretically 73 % and experimentally 60 %. At 70 % of the FOV it is 61 % in the simulated case and 45 % in the experimental case. The difference between simulations and experimental results arises mainly from tolerances in the fabrication process, in particular from thickness tolerances of both substrates and tolerances of the ROCs of the first array. The highest frequency which can be transmitted by the prototype is about 155 cycles/mm which leads to a modulation of 10 %.
The system is designed for infinite object distance, which means a parallel incidence of rays. Practically, this condition is not fulfilled in the prototype until a certain object distance is achieved. Therefore, the dependence of the MTF on the object distance has been characterized. The experimental and simulated MTF for half (111 cycles/mm) and for one quarter of the Nyquist frequency (55.5 cycles/mm) is plotted against the object distance in Fig. 7(b). It is obvious that the simulated as well as the measured plots converge for an object distance of about 40 cm, implying that a focused image can be obtained for distances greater than this.
6.2. Partial image stitching
Each partial image has to fit together with its adjacent ones in order to seamlessly stitch to a total image of the full field of view. Otherwise, an offset of image details occurs at the point of intersection of two partial images. Test targets of an edge were imaged with the prototype to verify the stitching between adjacent partial images. The image was scanned along a line perpendicular to the edge. This scan was interpolated to a tenth fraction of a pixel. Afterwards, the gradient of the profile was calculated and the position of the extremum was plotted for all scans. The offset between two partial images could be determined from this. The maximum offset was measured for an edge lying in the outer regions of the partial images as it is shown in Fig. 8(a). It was determined to be 0.2 pixel in the center of the field of view and 0.4 pixel at a location around 70 % of the field of view. This result shows that the stitching between adjacent partial images achieves a sub-pixel-accuracy.
6.3. Relative illumination
The optical cluster eye prototype was also characterized in terms of its relative illumination along the image plane. A white test chart was imaged in order to analyze the vignetting in the prototype. Such an unprocessed image of a white test target is shown in Fig. 8(b). The normalized illumination distribution, which is plotted in Fig. 9, has been derived from the marked cross section for three different configurations. For the theoretical analysis, a white image was simulated considering diffraction, aberrations, and relative illumination. The theoretical illumination distribution is given in Fig. 9 (curve A). The peaks in regular intervals arise due to the slight overlap of the partial images both, in the simulated and in the captured images. Two experimental illumination distributions are shown in curves B and C in Fig. 9. For curve B, a white diffuse LED panel was imaged by the oCLEY mounted on the image sensor. A significant deviation is found between the simulated A and experimental case B. Additionally, the outer partial images in the experimental configuration B are brighter at higher field angles and darker at lower field angles (see also Fig. 8(b)).
The image of the same test target as curve B was relayed on a high-resolution CCD imager by using a microscope objective (NA= 0.4) for the investigation of the influence of the image sensor. The detected relative illumination is plotted in curve C in Fig. 9. A much better agreement between the experimental curve C and the simulated curve A has been achieved in this case. Therefore, the deviation between the image captured with the image sensor and the simulated image can be traced back to the angular sensitivity of the used image sensor. As a reason the fill factor enhancing microlens array can be found, which is located on the pixels and adapted to a conventional ray angle distribution of single aperture lenses. However, the oCLEY optics creates a completely different chief ray angle distribution. Figure 3 shows, that the incidence angle on the image sensor plane is larger for ray bundles with higher field angles than the incidence angle for ray bundles with lower field angles. In contrast, for single aperture systems the incidence angle becomes continuously smaller with increasing field angles. This explains the fact that within single partial images a darkening for small field angles occurs.
Conventional methods which measure the distortion as a function of the field position yield no meaningful result for the oCLEY. Due to the multi-channel design, the distortion does not depend only on the field position, but also on the position in each partial image. The distortion increases towards the edge of the partial image in each channel. Three partial images with different field positions have been simulated using a constant object distance of 40 cm and the distortion was determined. Subsequently, the distortion has been measured for the same set of partial images in the prototype system. A good agreement with the simulation has been found. Table 2 compares the maximum detected and the simulated distortion for these three different partial images.
6.5. Example images and summary table
We judged the image quality by capturing different test images. For this purpose, the oCLEY system was mounted on the image sensor on a demoboard. In Fig. 10(a) an unprocessed image of a star pattern array is shown, whereas in Fig. 10(b) the image was processed using a flat field correction (FFC). In this way the partial image pattern is suppressed and the relative illumination is corrected at the same time. However, the signal-to-noise-ratio is low due to strong vignetting effects in the outer regions of the image. This leads to problems with the FFC, because of a decreased dynamic range. Figures 10(c), 10(d), and 10(e) show color images, which also have been processed using a FFC. A summary of different experimentally obtained parameters of the oCLEY is shown in Table 3. In order to compare the optical performance and the size of the system Table 3 also lists the parameters of the predecessor cluster eye system  and of a commercially available wafer-level camera system (OVM7690 CameraCube™, OmniVision, ). The listed values of the spatial and angular resolution of the CameraCube™ were measured by our group using the same methods as described in section 6.1. The oCLEY achieves a higher angular resolution, a smaller thickness, and a smaller lateral size than the CameraCube™. In comparison to the predecessor cluster eye system, we reach improvements with respect to all parameter values. The sensitivity was increased by a factor of 1.5 and the spatial resolution was raised more than twice. In the predecessor system gaps between the focussed partial images occurred whereas in the oCLEY a complete image without gaps was generated. The measured partial image offset of the oCLEY is at least four times smaller than that found in the predecessor cluster eye system .
A multi aperture imaging system (oCLEY) has been designed, which was subsequently fabricated and experimentally characterized. The prototype system consists of 175 optical channels built up similar to miniaturized telescopes. Different parts of the object are transmitted by different channels in a way that they stitch together on the image plane in order to form a conventional seamless image. The segmentation of the FOV in multi aperture systems leads to the advantage that the optical components of each channel are relatively simple, i.e. all lenses have small sags and spherical profiles. This means that the whole system achieves a shorter total track length as a comparable single aperture system. Additionally, by transmitting only small parts of the FOV in every channel the aberrations are much smaller than in single aperture systems. Furthermore, the used microstructures can be fabricated completely by state of the art microoptics technologies.
The demonstrated oCLEY system reaches a VGA resolution. To our knowledge, this is the highest resolution ever reached by a multi aperture optics camera which generates a conventional image by optical image stitching. In further steps the relative illumination in the image plane and the light sensitivity have to be improved. However, in comparison to the predecessor system , the sensitivity was improved by a factor of 1.5 and the spatial resolution was increased more than twice. The trade-off between the focal plane and the image stitching plane as it can be found in the predecessor system was minimized by precise fabrication methods. This enabled a focused image that is created by optical partial image stitching with sub-pixel accuracy. By implementing an extra MLA which tilts the optical axes of the channels it was possible to use a conventional image sensor. It turned out that the distribution of angles of incidence for strongly off-axis channels is still problematic due to the angular sensitivity of the used image sensor with integrated fill factor enhancing microlenses. This leads to the conclusion that for the oCLEY an image sensor either without a fill factor enhancing MLA or with an adapted MLA has to be used.
The demonstrated camera system has an extreme small overall size of 2.2 × 2.9 × 1.9 mm 3 (W×H×L), which enables the use in very small spaces as necessary for example in electronic devices in ICT, in medical imaging devices, and in machine vision.
We appreciate the funding by the German Federal Ministry of Education and Research (BMBF) for the project Insect inspired imaging, ( FKZ: 01RB0705A) within the BIONA initiative. Furthermore, the authors would like to thank Bernd Höfer for his contributions to the assembly of the Optical Cluster Eye.
References and links
1. R. H. Anderson, “Close-up imaging of documents and displays with lens arrays,” Appl. Opt. 18, 477–484 (1997). [CrossRef]
4. D. Gabor, UK Patent 541753 (1940).
5. M. C. Hutley, R. Hunt, R. F. Stevens, and P. Savander, “The moiré magnifier,” Pure Appl. Opt. 3, 133–142 (1994). [CrossRef]
6. C. Hembd-Sölner, R. F. Stevens, and M. C. Hutley, “Imaging properties of the gabor superlens,” J. Opt. A: Pure Appl. Opt. 1, 94–102 (1998). [CrossRef]
7. K. Stollberg, A. Brückner, J. Duparré, P. Dannberg, A. Bräuer, and A. Tünnermann, “The gabor superlens as an alternative waferlevel camera approach inspired by superposition compound eyes of nocturnal insects,” Opt. Express 17, 15747–15759 (2009). [CrossRef] [PubMed]
8. A. Garza-Rivera and F. J. Renero-Carrillo, “Design of an ultra-thin objective lens based on superposition compound eye,” Proc. SPIE 7930, 79300D (2011). [CrossRef]
9. R. Völkel, M. Eisner, and K. J. Weible, “Miniaturized imaging systems,” Microelectron. Eng. 67–68, 461–472 (2003). [CrossRef]
10. J. Duparré, P. Schreiber, A. Matthes, E. Pshenay-Severin, A. Bräuer, A. Tünnermann, R. Völkel, M. Eisner, and T. Scharf, “Microoptical telescope compound eye,” Opt. Express 13, 889–903 (2005). [CrossRef] [PubMed]
11. G. Druart, N. Guérineau, R. Haïdar, S. Thétas, J. Taboury, S. Rommeluère, J. Primot, and M. Fendler, “Demonstration of an infrared microcamera inspired by xenos peckii vision,” Appl. Opt. 48, 3368–3374 (2009). [CrossRef] [PubMed]
12. A. Brückner, J. Duparré, R. Leitel, P. Dannberg, A. Bräuer, and A. Tünnermann, “Thin wafer-level camera lenses inspired by insect compound eyes,” Opt. Express 18, 24379–24394 (2010). [CrossRef] [PubMed]
13. W. J. Smith, Modern Optical Engineering: The Design of Optical Systems, 2nd ed. (McGraw-Hill, 1990).
15. N. Lindlein, “Simulation of micro-optical systems including microlens arrays,” J. Opt. A: Pure Appl. Opt. 4, 1–9 (2002). [CrossRef]
16. J. Duparré, F. Wippermann, P. Dannberg, and A. Reimann, “Chirped arrays of refractive ellipsoidal microlenses for aberration correction under oblique incidence,” Opt. Express 13, 10539–10551 (2005). [CrossRef] [PubMed]
17. P. Dannberg, G. Mann, L. Wagner, and A. Bräuer, “Polymer UV-moulding for micro-optical systems and OlE-integration,” Proc. SPIE 4179, 137–145 (2000). [CrossRef]
19. S. Haselbeck, H. Schreiber, J. Schwider, and N. Streibl, “Microlenses fabricated by melting a photoresist on a base layer,” Opt. Eng. 32, 1322–1324 (1993). [CrossRef]
20. OmniVision: OVM7690; 640 × 480 CameraCube™ device; product brief, Version 1.0 (September 2010).