Video-rate quantitative phase analysis by a DIC microscope using a polarization camera

Shuhei Shibata; Wataru Takano; Nathan Hagen; Masaru Matsuda; Yukitoshi Otani

doi:10.1364/BOE.10.001273

1. Introduction

Differential interference contrast (DIC) microscopy [1] is a widely used method for observing tissues or cells in vivo without the need for applying florescent dyes. DIC microscopy allows analysis of detailed structures, high sensitivity in detecting phase information, and has a strong capability for optical sectioning [2]. Many researchers have proposed methods for the quantitative phase measurement of volumes using transmission DIC microscopy [3,4], and surface profile measurement using reflection DIC microscopy [5–7]. Ishiwata et al. proposed a retardation-modulated DIC (RM-DIC) microscope for quantitative measurement of planar microstructure at a given focal depth [8,9], and to measure 3D phase volume by optical sectioning [2]. However, these quantitative reconstruction methods [2–9] can require significant computation for each phase image because of the need to acquire multiple images for use with the phase shift technique.

We show below that it is possible to obtain a quantitative phase image from a single raw frame, so that real-time video is possible for a DIC microscope using a polarization camera – a camera for which a micropolarizer array has been attached to a detector array [10–12]. While Fabre et al. have previously shown how to make use of a photonic crystal polarization camera [11] with a DIC microscope, they used scanning to collect multiple raw images in order to obtain a single phase image [13]. Thus, they were unable to demonstrate video imaging of dynamic phase objects. We demonstrate that it is possible to capture measurements at the native frame rate of the polarization camera, and that through deconvolution with the DIC system’s inverse MTF, we can convert the raw intensity data into quantitative phase images.

In Section 2 below, we introduce how to calculate a quantitative phase image from one image captured by a polarization camera. In Section 3, we evaluate the proposed method for getting accurate phase measurement using a sample of glass beads inside a lacquer medium. In order to demonstrate the advantages of our method, we also show a video-rate phase measurement of rapid motion in a living medaka egg.

2. Snapshot phase analysis method

We start with a simple one-dimensional model consisting of a pure phase object imaged by the DIC microscope, generating an intensity distribution I(x) at the detector plane. The microscope consists of a light source, illumination optics, the phase object, imaging optics, and a detector array (Fig. 1).

Fig. 1 Simple model of a transmission microscope.

Download Full Size | PDF

According to Hopkins’ theory [14] and the partial coherence theory of a microscope [15], the image intensity distribution I(x) on the detector array can be defined as

I (x) = \int \int_{- \infty}^{\infty} R (f_{x}, {f^{'}}_{x}) O (f_{x}) O^{*} ({f^{'}}_{x}) e x p {- 2 π i (f_{x} - {f^{'}}_{x}) x} d f_{x} d {f^{'}}_{x},

where O(f_x) is the Fourier transform of phase object o(x), and R(f_x, f'_x) is the complex-valued transmission cross-coefficient (TCC) given by

R (f_{x}, {f^{'}}_{x}) = \int_{- \infty}^{\infty} Q (ξ) p (ξ + f_{x}) p^{*} (ξ + {f^{'}}_{x}) d ξ,

for Q(ξ) the intensity distribution of the illuminating system, and p(ξ) the complex-valued pupil function.

The DIC microscope utilizes lateral shearing interference between two orthogonally polarized beams produced by two Nomarski prisms, in which the beams pass through slightly different areas (shear distance Δ) of a specimen. The two beams are then brought back together by a second pair of Nomarski prisms that cancel the shear. The resulting interferogram observed at the image plane encodes the spatial gradient of the object phase along the shear direction.

Figure 2 shows our proposed optical layout for the DIC microscope using a polarization camera. The DIC microscope is set to generate π/2 of retardation along the shear axis between the two Nomarski prisms because the microstructure phase is emphasized best at this setting. The polarization camera can detect four kinds of intensities with each polarizer azimuthal angle (0°, 45°, 90°, and −45°) by one image. The intensities detected using the pixels oriented at 0° and 90° of the micropolarizer array do not produce interference because these axes are aligned to the shearing direction. Therefore we use only the two intensities detected using the pixels oriented at 45° and −45° and ignore the 0° and 90° data.

Fig. 2 The optical setup of the DIC microscope using a polarization camera.

Download Full Size | PDF

The polarization elements are defined with the following Jones matrices:

\begin{array}{l} P (45^{\circ}) = [\begin{matrix} \sqrt{2} & 0 \\ 0 & \sqrt{2} \end{matrix}], M P A (\pm 45^{\circ}) = [\begin{matrix} \pm \sqrt{2} & 0 \\ 0 & \sqrt{2} \end{matrix}], \\ N_{1} (Δ, ξ) = [\begin{matrix} e x p {i π (ξ Δ + 1) / 2} & e x p (- i π ξ Δ / 2) \\ 0 & 0 \end{matrix}], \\ N_{2} (Δ, ξ) = [\begin{matrix} e x p (- i π ξ Δ / 2) & e x p (i π ξ Δ / 2) \\ 0 & 0 \end{matrix}] . \end{array}

where, P(45°) and MPA(θ) are Jones matrices of the polarizer and the micro polarizer array oriented at θ = 0°, 45°, 90°, and −45°, and N(Δ, ξ) is Jones matrix of the Nomarski prism.

The TCC R(f_x,f'_x) in the case of a polarization camera is rewritten using Eqs. (2) and (3) as

R (f_{x}, {f^{'}}_{x}) = \int_{- \infty}^{\infty} Q (ξ) A p (ξ + f_{x}) A^{*} p^{*} (ξ + {f^{'}}_{x}) d ξ,

where

A = M P A (\pm 45^{\circ}) N_{1} (Δ, ξ) N_{2} (Δ, ξ) P (45^{\circ}) .

According to Ishiwata’s theory [8,9], we will assume that the object phase can be approximated as a small phase distribution atop a (potentially large) spatially-constant phase. Therefore, phase object o(x) can be approximated by its Taylor expansion about the constant phase value as

o (x) = C e x p {i ϕ (x)} \approx C e x p (i ϕ_{0}) {1 + i ϕ (x) - ϕ {(x)}^{2}},

for transmission C and spatially-constant phase φ₀. The Fourier transform of phase object o(x) can be expressed as

O (f_{x}) = C e x p (i ϕ_{0}) {δ (f_{x}) + i Φ (f_{x}) - \frac{1}{2} Φ (f_{x}) \otimes Φ^{*} (f_{x})} .

where δ(f_x) is the Dirac delta function, and Φ(f_x) is the Fourier transform of phase distribution φ(x). Here

\otimes

is the convolution operator. Assuming that the pupil is independent of any polarization, the intensity distribution I(x,θ) is rewritten by substituting Eqs. (4) and (7) to Eq. (1). Since the introduction of the polarization calculus has converted R(f_x,f'_x) from a scalar to a vector, we also need to convert the complex conjugate in Eq. (1) to an adjoint operator (i.e. conjugate transpose), producing

\begin{matrix} \begin{array}{l} I (x, \pm 45^{\circ}) = \frac{C}{2} [M (0) - \int_{- \infty}^{\infty} \cos (π f_{x} Δ) M (f_{x}) Φ (f_{x}) \otimes Φ^{*} (f_{x}) e x p (- 2 π i f_{x} x) d f_{x} \\ + \int_{- \infty}^{\infty} {1 + \sin (2 π f_{x} Δ)} m_{0} (f_{x}) Φ (f_{x}) Φ^{*} (f_{x}) d f_{x} \end{array}} (a) \\ \pm 2 i \int_{- \infty}^{\infty} \sin (π f_{x} Δ) M (f_{x}) Φ (f_{x}) e x p (- 2 π i f_{x} x) d f_{x} (b) \\ \pm \int_{- \infty}^{\infty} \cos (2 π f_{x} Δ) m_{d} (f_{x}) Φ (f_{x}) Φ^{*} (- f_{x}) e x p (- 4 π i f_{x} x) d f_{x}, (c) \end{matrix}

where

\begin{array}{l} M (0) = \int_{- \infty}^{\infty} Q (ξ) p (ξ) p^{*} (ξ) d ξ, M (f_{x}) = \int_{- \infty}^{\infty} Q (ξ) p (ξ + f_{x}) p^{*} (ξ) d ξ, \\ m_{0} (f_{x}) = \int_{- \infty}^{\infty} Q (ξ) p (ξ + f_{x}) p^{*} (ξ + f_{x}) d ξ, m_{d} (f_{x}) = \int_{- \infty}^{\infty} Q (ξ) p (ξ + f_{x}) p^{*} (ξ - f_{x}) d ξ . \end{array}

Here M(0), M(f_x), m₀(f_x), and m_d(f_x) describe the illumination profiles in terms of the two sheared beams illuminating the sample. We see that the intensity distribution I(x,θ) has five terms comprising three components (a), (b), and (c). The background component (a) consists of M(0) plus an integral that evaluates to a constant – together these represent the DC part of the illumination and object transmission – plus an integral that expresses the illumination profile due to the pair of illumination beams sheared by separation Δ. The second component (b) is the primary item of interest – the object phase gradient given by taking the difference between the two shifted versions of the object phase. The difference operation is due to the Fourier transform of the sine function being a pair of Dirac delta functions of opposite sign. Finally, the third component (c) is the transmitted intensity profile dependent on the squared object phase.

If we approximate the object as having a weak phase variation, the convolved phase $Φ (f_{x}) \otimes Φ^{*} (f_{x})$ , the absolute value $Φ (f_{x}) Φ^{*} (f_{x}) = {| Φ (f_{x}) |}^{2}$ and the squared object phase $Φ (f_{x}) Φ^{*} (- f_{x})$ all become approximately zero. As a result, the intensity images (Eq. (8)) captured by the polarization camera depend only on the DC part of background M(0) and the object phase gradient component (b). Thus, in the weak phase variation approximation Eq. (8) becomes:

I (x, \pm 45^{\circ}) = \frac{C}{2} [M (0) \pm 2 i \int_{- \infty}^{\infty} \sin (π f_{x} Δ) M (f_{x}) Φ (f_{x}) e x p (- 2 π i f_{x} x) d f_{x}] .

The object phase gradient (the differential phase distribution) can be obtained by

\begin{matrix} \frac{I (x, + 45^{\circ}) - I (x, - 45^{\circ})}{I (x, + 45^{\circ}) + I (x, - 45^{\circ})} = \frac{2 i}{M (0)} \int_{- \infty}^{\infty} \sin (π f_{x} Δ) M (f_{x}) Φ (f_{x}) e x p (- 2 π i f_{x} x) d f_{x} \\ = 2 i \int_{- \infty}^{\infty} M T F (f_{x}) Φ (f_{x}) e x p (- 2 π i f_{x} x) d f_{x}, \end{matrix}

where

M T F (f_{x}) = \frac{\sin (π f_{x} Δ) M (f_{x})}{M (0)}

represents the Modulation Transfer Function (MTF) of the DIC microscope.

The phase distribution φ(x) of the observed object can be obtained using the MTF(f_x) of the DIC microscope with

ϕ (x) = \frac{1}{2 i} ℑ^{- 1} [\frac{1}{M T F (f_{x})} ℑ {\frac{I (x, + 45^{\circ}) - I (x, - 45^{\circ})}{I (x, + 45^{\circ}) + I (x, - 45^{\circ})}}],

where

ℑ ()

is the Fourier transform function and

ℑ^{- 1} ()

is the inverse Fourier transform function.

By using the polarization camera, the image phase distribution is measured using two different pixel positions on the polarization camera by Eq. (13). The spatial resolution of the measured phase is increased (worsened) by a factor of 2 in each of the two spatial directions. Therefore, the polarization camera can experience aliasing error due to insufficient spatial sampling. To overcome this problem, Tyo et al. proposed a Fourier-domain pixel interpolation method [16]. This method can reconstruct a spatially band-limited polarization image without instantaneous field of view (IFOV) error by filtering in the frequency domain. We apply this method to the raw captured image in order to extract, I(x, 45°) and I(x, −45°) prior to applying them in Eq. (13). In order to simplify the description of Tyo’s method, we switch from the Jones to the Stokes-Mueller calculus, and expand from one-dimension (x) to two-dimensions (x, y) while maintaining the same image model. At the DIC microscope image plane the Stokes parameters are given by

s_{0} (x, y) = I (x, y, 0^{\circ}) + I (x, y, 90^{\circ}) = I (x, y, 45^{\circ}) + I (x, y, - 45^{\circ}),

s_{1} (x, y) = I (x, y, 0^{\circ}) - I (x, y, 90^{\circ}),

s_{2} (x, y) = I (x, y, 45^{\circ}) - I (x, y, - 45^{\circ}),

where I(x, y, θ) is the image captured by the polarization camera for a pixel whose micropolarizer is oriented at angle θ. While we do not make use of the 0° and 90° pixel orientations in the phase imaging analysis, we keep them in the discussion here in order to keep the explanation of the filtering method as clear as possible.

In the frequency domain, (ζ, η) are Fourier spatial frequency axes corresponding to (x, y). We write the Fourier transforms of Stokes parameters s₀(x, y), s₁(x, y), and s₂(x, y) as S₀(ζ, η), S₁(ζ, η), and S₂(ζ, η). Using the discrete-space Fourier transform (DSFT) of the image I(x, y, θ), we can separately filter the intensity component S₀(ζ, η), the {S₁(ζ, η) + S₂(ζ, η)} component, and the {S₁(ζ, η)−S₂(ζ, η)} component such as in Fig. 3. The interpolated images of s₀(x, y), s₁(x, y) and s₂(x, y) can be reconstructed by 4 arithmetic operations and an inverse DSFT after using a low-pass or high-pass filter.

Fig. 3 The flowchart for the Fourier-domain pixel interpolation method of the polarization camera image.

Download Full Size | PDF

After the pixel interpolation calibration, Eq. (13) becomes

ϕ (x, y) = \frac{1}{2 i} ℑ^{- 1} [\frac{1}{M T F (f_{x})} ℑ {\frac{s_{2} (x, y)}{s_{0} (x, y)}}] .

This Fourier-domain implementation for interpolating the polarization camera’s pixels provides the ideal filtering method from the standpoint of sampling theory. As a result, polarization error caused by edges of object features (i.e. instantaneous field of view errors) and aliasing errors are minimized.

3. Evaluation of quantitative analysis

A DIC microscope (Olympus IX70) using a 4D Technology PolarCam (V model) polarization camera is shown in Fig. 4. The light from a halogen lamp is transmitted through a 650 ± 25 nm bandpass filter into the DIC unit. The polarization camera exhibits pixel extinction ratios of ~30 across the visible spectral range, the pixel size is 7.5 × 7.5 μm, the overall number of pixels is 648 × 460, and the maximum camera frame rate is 135 Hz.

Fig. 4 The DIC microscope using a polarization camera.

Download Full Size | PDF

In our analysis, we use MTF(f_x) calculated by Eq. (12) using the illumination aperture, wavelength, shear distance Δ, NA and magnification of the objective lens, then we calculate 1/MTF(f_x) using a Wiener filter. Figure 5 shows the inverse MTF for our system, showing that the system suppresses low and medium spatial frequencies (f_x < 0.4μm⁻¹ and f_x > 0.6μm⁻¹, for the 20 × lens) while maintaining the spatial frequencies near 0.5 μm⁻¹.

Fig. 5 The reciprocal MTF(f_x) of our DIC microscope. The red curves are for our 30 × objective lens (NA = 0.40), while the black curves are for our 20 × objective lens (NA = 0.40). The solid curve is the MTF at maximum aperture, while the dashed curve is the MTF at half aperture.

Download Full Size | PDF

In order to test our approach quantitatively, we measure the phase profile of glass beads (refractive index n₁ = 1.56) embedded in a lacquer medium (refractive index n₂ = 1.54) to determine how well the system can estimate quantitative phase using the proposed analysis. The diameter d of the glass beads is 2 μm. Figure 6 shows the measurement results of the differential phase on a region of our sample before the inverse Fourier transform (calculated by Eq. (11), shown on the left side of the figure) and the quantitative phase after inverse Fourier transform (calculated by Eq. (14), shown on the right side of the figure). The image shown in the figure is sampled at 0.25 μm / pixel

Fig. 6 The measurement results of the glass beads (n₁ = 1.56) embedded in lacquer (n₂ = 1.54). We use an NA = 0.4 objective lens, (a), (c) and (e) are the differential phase results, while (b), (d), and (f) are the quantitative phase results. (c) and (d) are the enlarged images of red square of (a) and (b), respectively. (e) and (f) are cross-sections taken at the center of the glass beads (red dished lines of (c) and (d)). The black line of (f) is the calculated theoretical value.

Download Full Size | PDF

We can see the well-known halo artifact indicated by negative phase values on both sides of the glass beads. The theoretical curve is calculated from the difference of refractive index (n₁ − n₂ = 0.02), the bead diameter (d = 2 μm), and the wavelength (650 nm).

The halo artifact visible in Fig. 6(f) is caused by insufficient spatial coherence of the illumination system. Nguyen et al. have shown that they were able to successfully remove the artifact by using an external interferometric unit [17], but our system currently does not have a similar hardware setup allowing for removal of the halo. We obtain an absolute maximum phase difference of 0.05 rad between our measurement curve and theoretical curve at the glass bead area without the halo artifact.

4. Video-rate quantitative analysis result

In biology, zebrafish and medaka are often used for investigating the tissue formation process and for the observation of abnormal and healthy cells. Therefore, in order to demonstrate video-rate quantitative phase measurement, we measure a living medaka egg (5~6 days after spawning) (see Fig. 7). Figure 8 shows a 20 Hz video measurement of a medaka heart’s pumping and blood flow [18]. In this measurement, the spatial sampling is 0.25 μm/pixel. The conventional method of quantitative phase measurement (not video-rate measurement) cannot take detailed phase images because of the fast motion, while our method (video-rate measurement) using the polarization camera can obtain clear phase images of the working valve between the atrium and the ventricle, and movement of red blood cells even during the heart’s pumping action.

Fig. 7 A living medaka egg as sample for video-rate measurement.

Download Full Size | PDF

Fig. 8 Two frames from a quantitative phase video of the pumping of a medaka heart, captured at a 20 Hz frame rate [18]. (Visualization 1).

Download Full Size | PDF

5. Conclusion

Adding a polarization camera to a DIC microscope allows video-rate quantitative phase measurement of transparent samples. We can extract the differential phase component from the DIC images captured by the polarization camera, and reconstruct the quantitative phase distribution from the differential phase by our proposed analysis. Our method can measure the depth-integrated phase to an accuracy of better than 0.05 rad. Conventional quantitative phase microscopy without using a polarization camera cannot measure dynamic phase objects or object that move quickly during measurement. Our method, however, can achieve accurate measurement of these dynamic objects despite their movement. As a result, we can observe the structures of the living tissue without staining.

Funding

MEXT.

Acknowledgements

We would like to thank Prof. Toyohiko Yatagai, Dr. Wataru Ishiwata and Olympus Corporation for useful discussions, for providing the DIC microscope, and for the DIC MTF information for this work. This work was supported by the MEXT Opto-Bio Project.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

References

1. G. Nomarski, “Differential microinterferometer with polarized waves,” J. Phys. Radium 16, 9S (1955).

2. A. Noguchi, H. Ishiwata, M. Itoh, and T. Yatagai, “Optical sectioning in differential interference contrast microscopy,” Opt. Commun. 282(16), 3223–3230 (2009). [CrossRef]

3. M. R. Arnison, K. G. Larkin, C. J. R. Sheppard, N. I. Smith, and C. J. Cogswell, “Linear phase imaging using differential interference contrast microscopy,” J. Microsc. 214(1), 7–12 (2004). [CrossRef] [PubMed]

4. S. V. King, A. Libertun, R. Piestun, C. J. Cogswell, and C. Preza, “Quantitative phase microscopy through differential interference imaging,” J. Biomed. Opt. 13(2), 024020 (2008). [CrossRef] [PubMed]

5. D. L. Lessor, J. S. Hartman, and R. L. Gordon, “Quantitative surface topography determination by Nomarski reflection microscopy. I. Theory,” JOSA 69(2), 357–366 (1979). [CrossRef]

6. J. S. Hartman, R. L. Gordon, and D. L. Lessor, “Quantitative surface topography determination by Nomarski reflection microscopy. 2: Microscope modification, calibration, and planar sample experiments,” Appl. Opt. 19(17), 2998–3009 (1980). [CrossRef] [PubMed]

7. J. S. Hartman, R. L. Gordon, and D. L. Lessor, “Nomarski differential interference contrast microscopy for surface slope measurements: an examination of techniques,” Appl. Opt. 20(15), 2665–2669 (1981). [CrossRef] [PubMed]

8. H. Ishiwata, M. Itoh, and T. Yatagai, “Retardation modulated differential interference microscope and its application to 3-D shape measurement,” Proc. SPIE 2873, 21–24 (1996). [CrossRef]

9. H. Ishiwata, M. Itoh, and T. Yatagai, “A new method of three-dimensional measurement by differential interference contrast microscope,” Opt. Commun. 260(1), 117–126 (2006). [CrossRef]

10. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010). [CrossRef] [PubMed]

11. S. Kawakami, T. Kawashima, Y. Inoue, Y. Homma, T. Sato, S. Ota, S. Nagashima, and T. Aoki, “Polarization imaging device utilizing photonic crystal polarizer,” The Transactions of the Institute of Electronics Information and Communication Engineers. C, J90–C17–24, (2007) (Japanese).

12. Y. Maruyama, T. Terada, T. Yamazaki, Y. Uesaka, M. Nakamura, Y. Matoba, K. Komori, Y. Ohba, S. Arakawa, Y. Hirasawa, Y. Kondo, J. Murayama, K. Akiyama, Y. Oike, S. Sato, and T. Ezaki, “3.2-MP back-illuminated polarization image sensor with four-directional air-gap wire grid and 2.5-μm pixels,” IEEE Trans. Electron Dev. 65(6), 2544–2551 (2018). [CrossRef]

13. L. Fabre, Y. Inoue, T. Aoki, and S. Kawakami, “Differential interference contrast microscope using photonic crystals for phase imaging and three-dimensional shape reconstruction,” Appl. Opt. 48(7), 1347–1357 (2009). [CrossRef] [PubMed]

14. H. H. Hopkins, “On the diffraction theory of optical images,” Proc. R. Soc. Lond. A Math. Phys. Sci. 217, 408–432 (1953).

15. R. S. Sirohi and G. S. Bhatnagar, “Effect of partial coherence on the resolution of a microscope,” Opt. Acta (Lond.) 17(11), 839–842 (1970). [CrossRef]

16. J. S. Tyo, C. F. LaCasse, and B. M. Ratliff, “Total elimination of sampling errors in polarization imagery obtained with integrated microgrid polarimeters,” Opt. Lett. 34(20), 3187–3189 (2009). [CrossRef] [PubMed]

17. T. H. Nguyen, M. Kandel, H. M. Shakir, C. Best-Popescu, J. Arikkath, M. N. Do, and G. Popescu, “Halo-free phase contrast microscopy,” Sci. Rep. 7(1), 44034 (2017). [CrossRef] [PubMed]

18. While the camera itself is capable of capturing images at frame rates of up to 100 Hz, the camera interface software available to us for this experiment could only run at frame rates up to 20 Hz.

Video-rate quantitative phase analysis by a DIC microscope using a polarization camera

Abstract

Corrections

1. Introduction

2. Snapshot phase analysis method

3. Evaluation of quantitative analysis

4. Video-rate quantitative analysis result

5. Conclusion

Funding

Acknowledgements

Disclosures

References

Supplementary Material (1)

Cited By

Figures (8)

Equations (17)

Biomedical Optics Express