Depth of field of the integral imaging microscope is studied. In the integral imaging microscope, 3-D information is encoded as a form of elemental images Distance between intermediate plane and object point decides the number of elemental image and depth of field of integral imaging microscope. From the analysis, it is found that depth of field of the reconstructed depth plane image by computational integral imaging reconstruction is longer than depth of field of optical microscope. From analyzed relationship, experiment using integral imaging microscopy and conventional microscopy is also performed to confirm enhanced depth of field of integral imaging microscopy.
© 2012 OSA
Three dimensional (3-D) information capture techniques including pickup, display, and image processing have been widely studied in the fields of science, industry, the military, broadcasting, and bio-medicine. Especially, 3-D information capture techniques of microscopic objects have become a main concern of microscopy. The objective lens of the optical microscope has a shallow depth of field (DOF) so that a thin section of the object can be imaged [1–5]. Hence, a stack of focal images are required to reconstruct the 3-D information of an object [6,7]. A representative 3-D information capture technique of microscopy is confocal microscopy. Confocal microscopy is a technique whereby a researcher captures different depth images of an object by axial scanning. Then these image sets are synthesized in depth plane order. A synthesized depth plane image set has extended DOF. Another common technique of 3-D microscopy is digital holographic microscopy (DHM) . DHM offers full volumetric information of the object by numerical processing. However, it has been limited by difficulties in practical use, such as those encountered when capturing full color images and requiring coherent illumination. Integral imaging microscopy is the 3-D microscopy that provides different depth plane information; with it you view images and full-color information of the object by numerical processing. An integral imaging microscope (IIM) acquires 3-D information in an elemental form using a micro lens array. Due to the micro lens array, the DOF of the IIM can be extended to capture a thick object. Acquired 3-D information, i.e. elemental images, of the object is encoded as a form of the disparity between the elemental images, and simple pixel mapping methods are used to generate views and depth plane images. Thus, generated depth plane images are free from off-axis distortion caused by mechanical movements [9,10].
In this paper, we analyze the extended DOF of the IIM compared with conventional optical microscopes. Each depth plane image reconstructed by the computational integral imaging reconstruction method (CIIR) in the IIM is compared to the sequentially-captured 2-D images by a conventional optical microscope. Also, profiles of the reconstructed depth plane images are generated to verify efficient 3-D visualization. In Section 2, the principle of lateral resolution in the IIM is explained. In Section 3, capturing elemental image conditions related with the DOF of the IIM and simulation results are explained and presented. In Section 4, the experimental results of the DOF image comparison between the IIM and conventional optical microscope are presented. In section 5, we explain our conclusions.
2. Lateral resolution in integral imaging microscope
In conventional IIM, orthographic view image reconstruction (OVIR) and the CIIR method are used to generate multiple view images and different depth plane images . To analyze the DOF in IIM, it is necessary to consider two factors, lateral and axial resolution of the elemental image and the numerical aperture (NA) of the micro lens array. The 3-D information including different views and depth planes is encoded as a disparity between the elemental images. Hence, the axial resolution is related to the lateral resolution.
The IIM is composed of the objective lens, tube lens, micro lens array, and charge coupled device (CCD). The structure of the IIM is shown in Fig. 1 .
In Fig. 1, f1 and f2 are the focal length of the objective lens and tube lens, respectively; d is the distance between the intermediate plane to the micro lens array; g is the gap between the micro lens array and the CCD sensor; φ is the pitch size of the micro lens array; and e is the pixel size of the CCD. In the conventional optical microscope, lateral resolution is given byEq. (1). In the CCD sensor plane, lateral resolution is related to magnification of the optical microscope, g, d, and NA of the micro lens array. The minimum resolvable spot in the object plane is given byEq. (2), the minimum resolvable spot in the intermediate plane is derived byEq. (2) and Eq. (3), the minimum resolvable spot in the elemental image plane is derived byFig. 2 .
In Fig. 2 fla is the focal length of the micro lens array. To capture more views of the object, third case has to be satisfied.
3. DOF in IIM
Axial resolution is often related to DOF in the microscope. To explain DOF in a microscope, there are two formulas: geometrical optics and wave optics, both shown in Eq. (5).Fig. 3 .
DOF by lens array around the intermediate image plane can be express asEq. (4) and Eq. (7), both lateral and axial resolutions are determined by NAla. Note that extended DOF of the IIM compared to conventional optical microscope means that area of focused depth plane image is extended by micro lens array. The resolution of reconstructed images in the extended DOF has low quality than conventional optical microscope. The IIM still has resolution tradeoff by micro lens array .
Figure 4 shows simulation results of the calculated DOF of a conventional optical microscope and IIM from Eq. (5) and Eq. (7), where NA = 0.2, λ = 532nm, M = 10, fla = 2.4mm, e = 6μm, and φ = 125μm. In Fig. 4(a), DOF of the IIM is longer than DOF of the conventional optical microscope when d = 3.3mm. As mentioned above in section 2, NAla<NA/M, to form an elemental image with multiple view information, d is longer than 3.125mm. In the IIM, the depth information of the object is encoded as a disparity between the elemental images. Hence, the minimum depth deviation which causes a barely detectable change of the disparity between the elemental images can be regarded as a measure of axial resolution. In Fig. 4(b), if d is 3.5mm, then g is 7.6mm. Numerically calculated DOF of the IIM exceed DOF of the conventional optical microscope. In this condition, Rele is 30.5μm and each object point is imaged to four elemental lens. Four view images can be reconstructed via the OVIR method. However, these images are blurred. Rele is larger than pixel size of the CCD e. Hence, effective range of d has to be set.
The relationship between the DOF of the IIM and a conventional optical microscope related with Eq. (5) and Eq. (7) is shown in Fig. 5 . The IIM captures a wide range of objects, then the CIIR method separates each depth slice image computationally. Hence, the DOF of the conventional optical microscope is included in the DOF of the IIM.
4. Experiments for generating extended DOF images and comparison with conventional extended depth of field images
The experimental setup was composed of the Olympus BX-51 and a micro lens array module as shown in Fig. 6(a) . The micro lens array module is composed of the micro lens array, focal lens, and CCD. Each part of micro lens array module can be separable as shown in Fig. 6(b). The NA of the objective lens was 0.2, the wavelength was 532nm, the magnification of the objective lens was 10, the focal length of the micro lens array was 2.4mm, the pixel pitch size of the CCD was 6μm, the pitch size of the micro lens was 125μm and d is 10mm.
Figure 7(a) shows the 2D image of the object used in the experiment. The micro register height is about 140μm. From Eq. (7), calculated D, Do, and Rele are 180μm, 18μm, and 12.6μm, respectively. Figure 7(b) shows the elemental image of Fig. 7(a). The pixel size and number of pixels of the elemental image are 946 × 946 and 11 × 11, respectively. Calculated DOF of the IIM is 18μm, and then 8 images are required to compare the DOF of a conventional optical microscope shown as Fig. 8 . From Fig. 7(b), depth slice images were generated at a reconstructed distance with z = −3mm to −6mm, and 3mm to 6mm, shown as Fig. 9 . In conventional CIIR method, for a given depth plane, specific depth plane can be generated. CIIR method used in this paper is only effective within integer depth plane . If g is same to focal length of the micro lens array, errors occurs for coding. Each local position of pixels is remapped on certain depth plane. In this case, focal length of the micro lens array is 2.4mm, minimum condition to generate depth plane of center region of the object is 3mm vice versa. In our experiment, 3mm to 6mm and −3mm to −6mm on depth plane of CIIR cover the range of 140μm.
In order to verify that the reconstructed images of IIM are equivalent to the 2-D images of conventional microscopy, we first compared PSNR of Figs. 8 and 9. The average PSNR difference between the two images is 11.75dB, and the maximum difference is 2.2dB, shown as Fig. 10 . Reconstructed depth plane images using CIIR has low resolution and noise which is caused by a boundary between each lens of the micro lens array. Results show that depth plane images using the CIIR method have similar DOF with sequentially-captured 2-D images of conventional optical microscopes. Second, we analyzed image intensities both of Fig. 8 and Fig. 9. Distribution of maximum pixel intensity of Fig. 11(a) and Fig. 11(b) are 95th, 205th, and 380th, and of Fig. 11(c) and Fig. 11(d) are 89th, 218th, and 376th. The difference of maximum intensity between both images is about 40. It is also affected by boundary distortion of the micro lens array while CIIR is processing. Results show that two image sets have a similar profile.
IIM can form elemental images in a unique way. The relationship between lateral resolution and DOF was analyzed to explain the capability to surface reconstructed depth plane images of an object. Corresponding distance from the intermediate plane to the micro lens array and object point around the focal plane of the microscope are main factors in forming an elemental image. These factors determine the disparity, that is, 3-D information of the object. The reconstructed depth plane image has low resolution, but it includes the DOF of the conventional optical microscope. In future work, a resolution-enhancing method such as an intermediate view reconstruction technique will be used for our research. It will be a useful device to acquire 3-D information of the microscope object without mechanical movements.
This research was supported by the Leaders in INdustry-university Cooperation(LINC) Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education, Science and Technology (2012-B-0013-010116).
References and links
5. C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988). [CrossRef]
6. S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.
7. Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.
9. Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE 7237, 72371Q, 72371Q-12 (2009). [CrossRef]
11. D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008). [CrossRef]