Abstract

Depth of field of the integral imaging microscope is studied. In the integral imaging microscope, 3-D information is encoded as a form of elemental images Distance between intermediate plane and object point decides the number of elemental image and depth of field of integral imaging microscope. From the analysis, it is found that depth of field of the reconstructed depth plane image by computational integral imaging reconstruction is longer than depth of field of optical microscope. From analyzed relationship, experiment using integral imaging microscopy and conventional microscopy is also performed to confirm enhanced depth of field of integral imaging microscopy.

© 2012 OSA

1. Introduction

Three dimensional (3-D) information capture techniques including pickup, display, and image processing have been widely studied in the fields of science, industry, the military, broadcasting, and bio-medicine. Especially, 3-D information capture techniques of microscopic objects have become a main concern of microscopy. The objective lens of the optical microscope has a shallow depth of field (DOF) so that a thin section of the object can be imaged [15]. Hence, a stack of focal images are required to reconstruct the 3-D information of an object [6,7]. A representative 3-D information capture technique of microscopy is confocal microscopy. Confocal microscopy is a technique whereby a researcher captures different depth images of an object by axial scanning. Then these image sets are synthesized in depth plane order. A synthesized depth plane image set has extended DOF. Another common technique of 3-D microscopy is digital holographic microscopy (DHM) [8]. DHM offers full volumetric information of the object by numerical processing. However, it has been limited by difficulties in practical use, such as those encountered when capturing full color images and requiring coherent illumination. Integral imaging microscopy is the 3-D microscopy that provides different depth plane information; with it you view images and full-color information of the object by numerical processing. An integral imaging microscope (IIM) acquires 3-D information in an elemental form using a micro lens array. Due to the micro lens array, the DOF of the IIM can be extended to capture a thick object. Acquired 3-D information, i.e. elemental images, of the object is encoded as a form of the disparity between the elemental images, and simple pixel mapping methods are used to generate views and depth plane images. Thus, generated depth plane images are free from off-axis distortion caused by mechanical movements [9,10].

In this paper, we analyze the extended DOF of the IIM compared with conventional optical microscopes. Each depth plane image reconstructed by the computational integral imaging reconstruction method (CIIR) in the IIM is compared to the sequentially-captured 2-D images by a conventional optical microscope. Also, profiles of the reconstructed depth plane images are generated to verify efficient 3-D visualization. In Section 2, the principle of lateral resolution in the IIM is explained. In Section 3, capturing elemental image conditions related with the DOF of the IIM and simulation results are explained and presented. In Section 4, the experimental results of the DOF image comparison between the IIM and conventional optical microscope are presented. In section 5, we explain our conclusions.

2. Lateral resolution in integral imaging microscope

In conventional IIM, orthographic view image reconstruction (OVIR) and the CIIR method are used to generate multiple view images and different depth plane images [11]. To analyze the DOF in IIM, it is necessary to consider two factors, lateral and axial resolution of the elemental image and the numerical aperture (NA) of the micro lens array. The 3-D information including different views and depth planes is encoded as a disparity between the elemental images. Hence, the axial resolution is related to the lateral resolution.

The IIM is composed of the objective lens, tube lens, micro lens array, and charge coupled device (CCD). The structure of the IIM is shown in Fig. 1 .

 

Fig. 1 Structure of the integral imaging microscope.

Download Full Size | PPT Slide | PDF

In Fig. 1, f1 and f2 are the focal length of the objective lens and tube lens, respectively; d is the distance between the intermediate plane to the micro lens array; g is the gap between the micro lens array and the CCD sensor; φ is the pitch size of the micro lens array; and e is the pixel size of the CCD. In the conventional optical microscope, lateral resolution is given by

R=0.47λNA,
where R is lateral resolution, λ is wavelength, and NA is the NA of the objective lens. In the object plane of the IIM, the minimum resolvable spot is same as a conventional optical microscope in Eq. (1). In the CCD sensor plane, lateral resolution is related to magnification of the optical microscope, g, d, and NA of the micro lens array. The minimum resolvable spot in the object plane is given by
R=0.47λMNAla,
where M is the magnification of the optical microscope and NAla is the NA of the micro les array. From Eq. (2), the minimum resolvable spot in the intermediate plane is derived by
Rint=MR=0.47λNAM.
From Eq. (2) and Eq. (3), the minimum resolvable spot in the elemental image plane is derived by
Rele=0.47λNAlagd.
If the Rele is bigger than the pixel pitch of the CCD, a blurred image will be captured. Resolution of the reconstructed view images and depth plane images are also blurred. There are three types of imaging cases of IIM. First, if the NA of the micro lens array, NAla, is the same as NA/M, all rays from the objective lens are collected by the micro lens array. In that case, loss of resolution does not occur due to the micro lens array. In this case, each object point is imaged to only one or two elemental lenses. Hence, the reconstructed view image has one or two views. Second, NAla is bigger than NA/M. In this case, each object point is imaged to only one or two elemental lenses, but the partial aperture of the micro lens array is used. Third, NAla is smaller than NA/M. In this case, some rays from the objective lens are not collected by a micro lens. Captured elemental image set by CCD is limited by number of elemental lens. In this case, total pixel count of reconstructed view images and depth plane images by using OVIR and CIIR method are limited by NAla. This tradeoff is traditional disadvantage of the integral imaging, however, it is necessary to generate 3-D information of the object. Conditions to form an elemental image are shown in Fig. 2 .

 

Fig. 2 Relationship between numerical aperture of the microscope lens array and intermediate plane when forming an elemental image.

Download Full Size | PPT Slide | PDF

In Fig. 2 fla is the focal length of the micro lens array. To capture more views of the object, third case has to be satisfied.

3. DOF in IIM

Axial resolution is often related to DOF in the microscope. To explain DOF in a microscope, there are two formulas: geometrical optics and wave optics, both shown in Eq. (5).

Do=Dwave+Dgeom=λnNA2+nMNAe,
where Do is the total DOF of the conventional optical microscope, Dwave is the DOF with wave optics, Dgeom is DOF with geometrical optics, and e is the pixel size of the CCD. As mentioned above, NA of the single micro lens, NAla, is determined by d and g as shown in Fig. 3 .

 

Fig. 3 Depth of field in an integral imaging microscope.

Download Full Size | PPT Slide | PDF

DOF by lens array around the intermediate image plane can be express as

Dla=λNAla2+dgNAlae.
It is magnified as 1/M2 by the objective and tube lenses. DOF around the object plane is given by
D=1M2Dla=λM2NAla2+dM2gNAlae.
From Eq. (4) and Eq. (7), both lateral and axial resolutions are determined by NAla. Note that extended DOF of the IIM compared to conventional optical microscope means that area of focused depth plane image is extended by micro lens array. The resolution of reconstructed images in the extended DOF has low quality than conventional optical microscope. The IIM still has resolution tradeoff by micro lens array [10].

Figure 4 shows simulation results of the calculated DOF of a conventional optical microscope and IIM from Eq. (5) and Eq. (7), where NA = 0.2, λ = 532nm, M = 10, fla = 2.4mm, e = 6μm, and φ = 125μm. In Fig. 4(a), DOF of the IIM is longer than DOF of the conventional optical microscope when d = 3.3mm. As mentioned above in section 2, NAla<NA/M, to form an elemental image with multiple view information, d is longer than 3.125mm. In the IIM, the depth information of the object is encoded as a disparity between the elemental images. Hence, the minimum depth deviation which causes a barely detectable change of the disparity between the elemental images can be regarded as a measure of axial resolution. In Fig. 4(b), if d is 3.5mm, then g is 7.6mm. Numerically calculated DOF of the IIM exceed DOF of the conventional optical microscope. In this condition, Rele is 30.5μm and each object point is imaged to four elemental lens. Four view images can be reconstructed via the OVIR method. However, these images are blurred. Rele is larger than pixel size of the CCD e. Hence, effective range of d has to be set.

 

Fig. 4 Simulation result: (a) comparison of DOF with a conventional optical microscope and integral imaging microscope; (b) calculated DOF after changing d and g.

Download Full Size | PPT Slide | PDF

The relationship between the DOF of the IIM and a conventional optical microscope related with Eq. (5) and Eq. (7) is shown in Fig. 5 . The IIM captures a wide range of objects, then the CIIR method separates each depth slice image computationally. Hence, the DOF of the conventional optical microscope is included in the DOF of the IIM.

 

Fig. 5 DOF region comparison.

Download Full Size | PPT Slide | PDF

4. Experiments for generating extended DOF images and comparison with conventional extended depth of field images

The experimental setup was composed of the Olympus BX-51 and a micro lens array module as shown in Fig. 6(a) . The micro lens array module is composed of the micro lens array, focal lens, and CCD. Each part of micro lens array module can be separable as shown in Fig. 6(b). The NA of the objective lens was 0.2, the wavelength was 532nm, the magnification of the objective lens was 10, the focal length of the micro lens array was 2.4mm, the pixel pitch size of the CCD was 6μm, the pitch size of the micro lens was 125μm and d is 10mm.

 

Fig. 6 Experimental setup: (a) captured experimental setup; (b) diagram of the micro lens array module.

Download Full Size | PPT Slide | PDF

Figure 7(a) shows the 2D image of the object used in the experiment. The micro register height is about 140μm. From Eq. (7), calculated D, Do, and Rele are 180μm, 18μm, and 12.6μm, respectively. Figure 7(b) shows the elemental image of Fig. 7(a). The pixel size and number of pixels of the elemental image are 946 × 946 and 11 × 11, respectively. Calculated DOF of the IIM is 18μm, and then 8 images are required to compare the DOF of a conventional optical microscope shown as Fig. 8 . From Fig. 7(b), depth slice images were generated at a reconstructed distance with z = −3mm to −6mm, and 3mm to 6mm, shown as Fig. 9 . In conventional CIIR method, for a given depth plane, specific depth plane can be generated. CIIR method used in this paper is only effective within integer depth plane . If g is same to focal length of the micro lens array, errors occurs for coding. Each local position of pixels is remapped on certain depth plane. In this case, focal length of the micro lens array is 2.4mm, minimum condition to generate depth plane of center region of the object is 3mm vice versa. In our experiment, 3mm to 6mm and −3mm to −6mm on depth plane of CIIR cover the range of 140μm.

 

Fig. 7 (a) 2-D micro scale object image and (b) elemental image.

Download Full Size | PPT Slide | PDF

 

Fig. 8 Sequentially-captured 2D microscopic image of Fig. 7(a).

Download Full Size | PPT Slide | PDF

 

Fig. 9 Depth plane image generated from Fig. 7(b) using CIIR.

Download Full Size | PPT Slide | PDF

In order to verify that the reconstructed images of IIM are equivalent to the 2-D images of conventional microscopy, we first compared PSNR of Figs. 8 and 9. The average PSNR difference between the two images is 11.75dB, and the maximum difference is 2.2dB, shown as Fig. 10 . Reconstructed depth plane images using CIIR has low resolution and noise which is caused by a boundary between each lens of the micro lens array. Results show that depth plane images using the CIIR method have similar DOF with sequentially-captured 2-D images of conventional optical microscopes. Second, we analyzed image intensities both of Fig. 8 and Fig. 9. Distribution of maximum pixel intensity of Fig. 11(a) and Fig. 11(b) are 95th, 205th, and 380th, and of Fig. 11(c) and Fig. 11(d) are 89th, 218th, and 376th. The difference of maximum intensity between both images is about 40. It is also affected by boundary distortion of the micro lens array while CIIR is processing. Results show that two image sets have a similar profile.

 

Fig. 10 PSNR of each depth plane image between 2-D and IIM.

Download Full Size | PPT Slide | PDF

 

Fig. 11 Intensities of pixels: (a) z = 3 to 6mm by CIIR, (b) z = 16.6 to 66.4μm by 2-D, (c) z = −3 to −6mm by CIIR, (d) z = −16.6 to −66.4μm by 2-D.

Download Full Size | PPT Slide | PDF

5. Conclusion

IIM can form elemental images in a unique way. The relationship between lateral resolution and DOF was analyzed to explain the capability to surface reconstructed depth plane images of an object. Corresponding distance from the intermediate plane to the micro lens array and object point around the focal plane of the microscope are main factors in forming an elemental image. These factors determine the disparity, that is, 3-D information of the object. The reconstructed depth plane image has low resolution, but it includes the DOF of the conventional optical microscope. In future work, a resolution-enhancing method such as an intermediate view reconstruction technique will be used for our research. It will be a useful device to acquire 3-D information of the microscope object without mechanical movements.

Acknowledgments

This research was supported by the Leaders in INdustry-university Cooperation(LINC) Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education, Science and Technology (2012-B-0013-010116).

References and links

1. D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984). [CrossRef]   [PubMed]  

2. J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas Jr., “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A 11(3), 1056–1067 (1994). [CrossRef]   [PubMed]  

3. S. Liu and H. Hua, “Extended depth-of-field microscopic imaging with a variable focus microscope objective,” Opt. Express 19(1), 353–362 (2011). [CrossRef]   [PubMed]  

4. J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005). [CrossRef]   [PubMed]  

5. C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988). [CrossRef]  

6. S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.

7. Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.

8. J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45(16), 3893–3901 (2006). [CrossRef]   [PubMed]  

9. Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE 7237, 72371Q, 72371Q-12 (2009). [CrossRef]  

10. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009). [CrossRef]   [PubMed]  

11. D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008). [CrossRef]  

References

  • View by:
  • |
  • |
  • |

  1. D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984).
    [Crossref] [PubMed]
  2. J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas., “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A 11(3), 1056–1067 (1994).
    [Crossref] [PubMed]
  3. S. Liu and H. Hua, “Extended depth-of-field microscopic imaging with a variable focus microscope objective,” Opt. Express 19(1), 353–362 (2011).
    [Crossref] [PubMed]
  4. J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005).
    [Crossref] [PubMed]
  5. C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988).
    [Crossref]
  6. S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.
  7. Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.
  8. J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45(16), 3893–3901 (2006).
    [Crossref] [PubMed]
  9. Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE 7237, 72371Q, 72371Q-12 (2009).
    [Crossref]
  10. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009).
    [Crossref] [PubMed]
  11. D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008).
    [Crossref]

2011 (1)

2009 (2)

2008 (1)

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008).
[Crossref]

2006 (1)

2005 (1)

J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005).
[Crossref] [PubMed]

1994 (1)

1988 (1)

C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988).
[Crossref]

1984 (1)

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984).
[Crossref] [PubMed]

Agard, D. A.

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984).
[Crossref] [PubMed]

Conchello, J. A.

Hua, H.

Katz, J.

Kim, E.-S.

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008).
[Crossref]

Kim, N.

Kwon, K.-C.

Lichtman, J. W.

J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005).
[Crossref] [PubMed]

Lim, Y.-T.

Liu, S.

Malkiel, E.

Mao, X. Q.

C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988).
[Crossref]

McNally, J. G.

Park, J.-H.

Preza, C.

Sheng, J.

Sheppard, C. J. R.

C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988).
[Crossref]

Shin, D.-H.

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008).
[Crossref]

Thomas, L. J.

Annu. Rev. Biophys. Bioeng. (1)

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984).
[Crossref] [PubMed]

Appl. Opt. (1)

J. Mod. Opt. (1)

C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988).
[Crossref]

J. Opt. Soc. Am. A (1)

J. Opt. Soc. Kor. (1)

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008).
[Crossref]

Nat. Methods (1)

J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005).
[Crossref] [PubMed]

Opt. Express (2)

Proc. SPIE (1)

Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE 7237, 72371Q, 72371Q-12 (2009).
[Crossref]

Other (2)

S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.

Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Structure of the integral imaging microscope.

Fig. 2
Fig. 2

Relationship between numerical aperture of the microscope lens array and intermediate plane when forming an elemental image.

Fig. 3
Fig. 3

Depth of field in an integral imaging microscope.

Fig. 4
Fig. 4

Simulation result: (a) comparison of DOF with a conventional optical microscope and integral imaging microscope; (b) calculated DOF after changing d and g.

Fig. 5
Fig. 5

DOF region comparison.

Fig. 6
Fig. 6

Experimental setup: (a) captured experimental setup; (b) diagram of the micro lens array module.

Fig. 7
Fig. 7

(a) 2-D micro scale object image and (b) elemental image.

Fig. 8
Fig. 8

Sequentially-captured 2D microscopic image of Fig. 7(a).

Fig. 9
Fig. 9

Depth plane image generated from Fig. 7(b) using CIIR.

Fig. 10
Fig. 10

PSNR of each depth plane image between 2-D and IIM.

Fig. 11
Fig. 11

Intensities of pixels: (a) z = 3 to 6mm by CIIR, (b) z = 16.6 to 66.4μm by 2-D, (c) z = −3 to −6mm by CIIR, (d) z = −16.6 to −66.4μm by 2-D.

Equations (7)

Equations on this page are rendered with MathJax. Learn more.

R= 0.47λ NA ,
R= 0.47λ MN A la ,
R int =MR= 0.47λ NA M.
R ele = 0.47λ N A la g d .
D o = D wave + D geom = λn N A 2 + n MNA e,
D la = λ N A la 2 + d gN A la e.
D= 1 M 2 D la = λ M 2 N A la 2 + d M 2 gN A la e.

Metrics