The Rayleigh-Sommerfeld back-propagation method is a fast and highly flexible volume reconstruction scheme for digital holographic microscopy. We present a new method for 3D localization of weakly scattering objects using this technique. A well-known aspect of classical optics (the Gouy phase shift) can be used to discriminate between objects lying on either side of the holographic image plane. This results in an unambiguous, model-free measurement of the axial coordinate of microscopic samples, and is demonstrated both on an individual colloidal sphere, and on a more complex object — a layer of such particles in close contact.
© 2012 OSA
Digital holographic microscopy (DHM) shows great promise as a tool for investigating soft matter [1,2] and biological systems [3–6]. Although 2D tracking using digital video microscopy has given valuable new insight into the structure and dynamics of soft matter systems from colloids [7, 8] to surface-associated bacteria , tracking particles in the third dimension is more challenging. Confocal microscopy is widely used for probing three dimensional structures , but its time resolution is limited by the mechanical nature of its scanning components. DHM has the comparative advantage of very high time resolution; modern CMOS and CCD cameras can comfortably reach several hundred frames per second at megapixel resolution. Previous studies of colloidal dynamics have used results from Mie scattering theory [1, 2] to fit holograms, accurately giving not only particle coordinates, but also particle refractive index and radius. Despite its successes this method is computationally expensive, and only applicable in situations where a model of the scatterer is already established. This is the case for only a handful of simple objects such as spheres or cylinders.
In comparison, the Rayleigh-Sommerfeld back-propagation scheme [11, 12] is highly flexible and fast but has not been widely used, in part because a rigorous criterion for localizing an object’s position in three dimensions has not been established. Various localization schemes have been put forward for use with DHM (both with Rayleigh-Sommerfeld and other reconstruction methods). Most methods so far employed are heuristic in some sense, finding peaks in the axial intensity profile [13–15], or using in-plane Laplacian derivatives to locate the edges of objects . Instead, our approach is based on a well-known physical principle – the Gouy phase shift – and Rayleigh-Gans (RG) scattering theory. It is applicable to a wide variety of weakly scattering microscopic objects such as colloidal suspensions or swimming planktonic bacteria.
2. Theory and methods
Various optical arrangements have been used to create holograms (several of which are reviewed in references  and ), but the in-line version is often favored due to its simplicity — indeed, it is fundamentally similar to the original holographic setup proposed by Gabor . In this arrangement, the object wave and reference wave are approximately collinear. Our setup (see cartoon, Fig. 1(a)) is of this type, and we assume this layout throughout. Furthermore, we restrict our discussion to weakly scattering objects — those amenable to RG scattering theory. Strictly, in this regime a scatterer’s refractive index relative to its surroundings (m = np/nm) and characteristic dimension, d, should obey the following conditions:19]. In essence, RG theory assumes that the scatterer is composed of independent dipole oscillators whose individual contributions can be summed to find the total scattered field . The RG approximation is particularly appropriate for many biological specimens, which have a refractive index close to that of their surroundings; they are effectively phase objects, exhibiting negligible absorption. Although such objects are practically invisible when located in the focal plane of a bright field imaging system, a small amount of defocus renders them visible again; this effect has been used by previous authors to study fluctuations in cell membranes [21, 22] or investigate the propagation of complex optical fields through the transport of intensity equation .
In fact this increased contrast is inverted on opposite sides of the focal plane. For example, consider a small polystyrene sphere in water (m > 1), as viewed by an inverted microscope. The center of its image appears light when the particle is just above the focal plane (i.e. the particle’s axial coordinate zp < 0), nearly invisible in focus, then dark when below focus (zp > 0). This phenomenon has been noted in passing by previous authors studying DHM , but not investigated in detail. We provide an explanation of this effect on a rigorous footing, in terms of the Gouy phase anomaly . This anomaly is observed when a converging spherical wavefront passes through its geometrical focus; the converging wavefronts are retarded by a net phase of π radians compared to a plane wave of the same frequency and wavelength, propagating in the same direction. This effect has been measured with high precision in the case of focused microwaves , and has been exploited in measuring the position of an optically trapped particle, with nanometer precision [26–28]. In our case, we have a phase object acting as a source of secondary wavelets. Rayleigh-Sommerfeld back-propagation allows us to numerically refocus these scattered wavelets at a particular position in the reconstructed volume. By examining their interference with the unscattered (plane wave) reference field around the reconstructed focal region, we observe this phase anomaly as a contrast inversion about the object’s position. We will use this effect to pinpoint the location of small, weakly scattering particles in 3D.
We first consider a small spherical particle, with radius a, in a typical DHM setup (Fig. 1(b)). A plane wave with amplitude E0 is incident on a particle located at the origin. The scattered field Es interferes with the unscattered light E0, resulting in a total intensity at a point r :19]. The incident, unpolarized field may be represented as the superposition of two orthogonally polarized beams. This leads to a scattering pattern which is azimuthally symmetric about the z-axis, 20], 29]. We use Eq. (4) to calculate the RG hologram of a spherical particle as shown in Fig. 1(c). The hologram was calculated on a plane located 40μm from the scatterer (i.e. the scatterer was located at zp = −40μm in our coordinate system, as we consider the center of the hologram plane to be the origin), and measures 512 pixels in each dimension. The other relevant parameters were λ = 0.505μm, a = 0.1μm, the particle and surrounding medium refractive indices np = 1.55 and nm = 1.33 respectively, and the sampling frequency in the x, y plane, 10 pixels/μm. Based on this simulated data, we reconstruct the light field associated with the particle using the Rayleigh-Sommerfeld scheme. This method has been described in detail elsewhere [11, 12], but briefly, we recover the electric field at a height −z′ with the use of the Rayleigh-Sommerfeld propagator
The central region of the plane y′ = 0 (i.e. through the center of the scatterer) is shown in Fig. 2(a). Note the small bright region below the midpoint and the small dark region just above the midpoint; this feature is due to the phase shift on passing through the geometrical focus. This image was then convolved with a kernel based on the Sobel filter  to extract the intensity gradient in the negative z′-direction only, which we denote g(x′, z′):Eq. (8) to be valid, the optical field must vary slowly compared to the sampling frequency in the axial direction. This condition is easily satisfied, as Δz′ can be made arbitrarily small. For clarity, we retain only the values of g(x′, z′) greater than zero, setting all negative values to zero. The result of this process can be seen in Fig. 2(b). This gradient operation provides a very efficient method of locating the center of the scatterer, with sub-pixel resolution. To demonstrate this, we extracted the intensity profiles along lines through the center vertical (red) and horizontal (black) directions of each of the images. These plots are shown in Fig. 2(c) and Fig. 2(d). In both panels, the axial intensity profile is shifted by +40μm, to place the scatterer’s center at the origin. Note also that this method has yielded the center position of a scatterer whose z-coordinate is negative; the field, imaged onto the camera sensor is diverging. If a particle’s z-coordinate is positive, the field is converging when it arrives at the image sensor and has not yet passed through its geometrical focus. This allows us to separate objects lying on different sides of the hologram plane unambiguously. Whether the particles have a physical position zp above or below the focal plane, they have twin images at ±z′p in the reconstructed space. However, the twin images are not identical. If we only reconstruct the half-space ‘upstream’ of the focal plane (i.e. z′ < 0), we find that g(z′p) is a maximum for particles with z < 0, and a minimum for those with z > 0. As a check of this method, we also generated a hologram using the more accurate Mie scattering solution for the scattered field [19, 29], using the parameters outlined above. The reconstructed axial coordinate obtained by analyzing the Mie hologram was z = −39.92μm, an error of 0.2%.
We tested this method experimentally, using a standard microscopy setup. Our experiment (see cartoon, Fig. 1(a)) consisted of a Nikon Ti-E inverted microscope with a 60× oil immersion objective lens. The sample illumination was provided by a λ = 0.505μm LED (Thorlabs M505L2) placed directly behind the condenser iris. The iris itself was closed as far as possible, with a remaining clear aperture of ∼ 1.5mm. The resulting quasi-plane wave illumination in the sample plane was sufficient to generate the interference rings seen in the defocused particle images (see results, Fig. 3). Although collimated lasers are often used as a light source [6,12,15], LEDs carry several distinct advantages as outlined in previous work [31–33]. Our main motivation was to remove unwanted interference fringes caused by multiple reflections in various parts of the optical train (e.g. From the surfaces of the sample chamber). These fringes can easily swamp the signal from weakly scattering particles, complicating analysis. The partial coherence of our illumination means that we are limited to reconstructing the positions of objects within a volume extending ∼ 100μm either side of the the focal plane; objects outside this volume are effectively invisible.
Imaging was performed using a Mikrotron MC-1362 camera attached to an EPIX PIXCI E4 frame grabber, with an exposure time of 19.9 ms per frame. Sample chambers were constructed from microscope coverslips with internal dimensions 0.2 × 5 × 20mm. One such chamber was filled by capillary action, with an aqueous suspension of polystyrene particles (Bangs Labs, manufacturer-specified diameter 535 nm). This sample was allowed to dry, leaving particles adhered to the bottom surface of the chamber. The chamber was then refilled with immersion oil (Nikon ‘type A’, nd = 1.515 at 23°C). This was done to minimize the effects of aberrations incurred when imaging through water into glass and to limit any multiple reflections between the particles and the wall of the sample chamber. The stepper motor used to position the objective lens turret was used to defocus the image of a single isolated particle by a defined amount. Although the calibrated motor readout implied a precision of ±25 nm, this did not accurately reflect our ability to reproducibly place a particle in ‘perfect focus’. The slight flexibility of the sample chamber wall and the viscosity of the immersion oil (which causes the coverslip to deform as the objective lens is moved) place a more conservative uncertainty on the true z position of the particle. Furthermore, the selection of a perfectly in-focus plane (i.e. placing a particle at z = 0μm) was performed by eye, and was hence a little subjective. Based on several cycles of defocusing then refocusing the image of a layer of particles, we estimate the total uncertainty in z to be ±150 nm.
Once a suitable particle had been selected, a stack of images was acquired by moving the focus of the objective lens in steps of 1μm across a range of defocus, giving a range of axial particle coordinates from zp = −50μm to +50μm (zp = 0μm implies that the particle lies in the focal plane). Next, a patch of closely-spaced particles was found. Images of this particle layer were acquired at different defocus distances. This second experiment demonstrates the technique’s use for measuring the z-coordinate of a flat, extended object. In both cases, the filter Sz′ was convolved with every reconstructed x′z′ plane. Due to the chosen form of the convolution kernel, an identical stack of gradient images g(x′, y′, z′) would be obtained by convolution with the y′z′ planes. In both the single- and multiple-particle cases, images were taken with and without the sample in place. This allowed the removal of unwanted background artifacts caused by imperfections in the optical system .
Figure 3(a) shows an xz slice through a stack of images of a single particle, created by manually refocusing the stage at Δz = 1μm intervals (a similar process was implemented by Elliot and Poon  who compared different schemes for imaging colloids: Phase contrast, DIC etc.) If the particle was in focus in the microscope (z′ = z = 0), this would be a map of the intensity close to the camera’s image sensor. Figure 3(b) and Fig. 3(e) show example xy images of an isolated particle at z = 9μm and z = −9μm respectively. From each of these images, a stack similar to that in Fig. 3(a) can be reconstructed (Figs. 3(c) and 3(f)), using Eq.s (3) and (7).
Note that the pixel values of each calculated stack have been rescaled to fill 256 gray levels for clarity; slight differences in background illumination level lead to the apparent intensity difference between the two x′z′ images, but the important qualitative features are the same. In both cases, the particle is located in the center of the panel. Figure 3(d) shows g(x′, z′) < 0 from Fig. 3(b), and Fig. 3(g) shows g(x′, z′) > 0 from Fig. 3(f), clearly showing the position of the particle in both cases. These intensity features may be fitted in three dimensions in order to accurately localize the particle. We find z = 8.9±0.15μm for Fig. 3(d) and z = −9.1±0.15μm for Fig. 3(g), with uncertainties dominated by the ±150 nm instrumental error.
Next, we used our method to reconstruct the axial position of an extended object. To do this, we imaged a layer of particles resting on a coverslip, as described above. Images were acquired at a range of defocus distances (positions relative to the focal plane) from z = 0μm to z = −80μm in steps of Δz = −4μm. Similarly accurate results were obtained from data in the range z = 0μm → 80μm (data not shown). From each of these stage displacements, a full stack of images was calculated, and from these we obtained the gradient g(x′, y′, z′) as before. As opposed to fitting subregions corresponding to individual particles, we integrated over all x′, y′ values at constant z′, for each actual stage displacement z. This yielded a single value which is a function of actual (z) and reconstructed (z′) defocus, and which we denote G(z, z′) = ∫ (g z;x′, y′, z′)dx′dy′. These one-dimensional quantities are plotted in Fig. 4, where each row of colors corresponds to an actual defocus distance z. The horizontal axis shows reconstructed defocus distance, and the colors indicate the magnitude of G(z, z′). The data have been rescaled such that the maximum value of G(z, z′) is unity for each actual defocus z. Although the peaks in G(z, z′) broaden considerably as a function of z, the peak values are well-defined and easy to fit in all cases.
The results of this fitting procedure as well as some sample images of the 2D layer of particles can be seen in Fig. 5. The first panel, Fig. 5(a) shows the layer of particles with a defocus of z = −4μm; Fig. 5(b) shows the layer with a defocus of −12μm. Even though the individual particle images are indistinct at −12μm, the reconstruction accurately reproduces the optical field in the plane of the particles. As stated in the introduction, we consider each particle to be composed of independent scattering centers; we therefore find the distance between particles to be unimportant. The maximum value of G(z, z′) indicates the axial position of the layer of particles. We plot these reconstructed defocus values as a function of actual defocus in Fig. 5(c), where excellent correspondence is found. Figure 5(d) shows the absolute error as a function of physical stage displacement. There is a systematic error of around −150 nm in the reconstructed particle positions. This seems most likely to be dominated by uncertainty in the z = 0 (‘in-focus’) position, which was chosen by eye, and would lead to the same offset in all measurements. The scatter of points around this value is more likely due to the mechanical considerations outlined above, as some tuning of the focus position was required. Lastly, although we have not investigated the matter in detail here, preliminary results show that this method is applicable to particles located at close lateral (x′, y′) positions, but separated axially (data not shown). This implies that the tracking of multiple weakly-scattering objects (such as colloids or bacteria) in 3D should be reasonably straightforward, at least in the dilute limit. Special treatment may be required to uniquely identify objects in very close proximity — a limitation shared by conventional 2D video tracking.
We have outlined a novel method for 3D object localization in digital holographic microscopy. This technique draws on a well-known aspect of classical wave optics, the Gouy phase anomaly, to localize objects in three dimensions with high precision. The Rayleigh-Gans theory that describes the scattering of light by phase objects is underpinned by the concept that small volumes of the same body can be treated as independent scattering centers. Each scattering center is the source of secondary spherical wavelets which, when numerically refocused, demonstrate the Gouy phase shift. The interference between scattered and unscattered fields results in a contrast inversion about their geometrical focus. We anticipate that this technique will find many applications in microbiology, where the subjects are well described as weak phase objects. Not only can single, small scatterers be tracked with high resolution in space and time, but the structure of larger objects, which may be considered as a distributed collection of scattering centers, can also be studied.
References and links
1. S. Lee, Y. Roichman, G.-R. Yi, S.-H. Kim, S.-M. Yang, A. van Blaaderen, P. van Oostrum, and D. G. Grier, “Characterizing and tracking single colloidal particles with video holographic microscopy,” Opt. Express 15, 18275–18282 (2007). [CrossRef] [PubMed]
2. J. Fung, K. E. Martin, R. W. Perry, D. M. Katz, R. McGorty, and V. N. Manoharan, “Measuring translational, rotational, and vibrational dynamics in colloids with digital holographic microscopy,” Opt. Express 19, 8051–8065 (2011). [CrossRef] [PubMed]
5. J. Sheng, E. Malkiel, J. Katz, J. Adolf, R. B. amnd, and A. R. Place, “Digital holographic microscopy reveals prey-induced changes in swimming behavior of predatory dinoflagellates,” Proc. Natl. Acad. Sci. USA 104, 17512–17517 (2007). [CrossRef] [PubMed]
6. Z. Frentz, S. Kuehn, D. Hekstra, and S. Leiber, “Microbial population dynamics by digital in-line holographic microscopy,” Rev. Sci. Inst. 81, 084301 (2010). [CrossRef]
7. J. C. Crocker and D. G. Grier, “Methods of digital video microscopy for colloidal studies,” J. Colloid Interface Sci. 179, 298–310 (1996). [CrossRef]
9. J. Conrad, M. Gibiansky, F. Jin, V. Gordon, D. Motto, M. Mathewson, W. Stopka, D. Zelasko, J. Shrout, and G. Wong, “Flagella and pili-mediated near-surface single-cell motility mechanisms in p. aeruginosa,” Biophys. J. 100, 1608–1616 (2011). [CrossRef] [PubMed]
10. A. van Blaaderen and P. Wiltzius, “Real-space structure of colloidal hard-sphere glasses,” Science 270, 1177–1179 (1995). [CrossRef]
11. J. W. Goodman, Introduction to Fourier Optics3rd Ed., (Roberts & Company, 2005).
14. C. Fournier, C. Ducottet, and T. Fournel, “Digital in-line holography: influence of the reconstruction function on the axial profile of a reconstructed particle image,” Meas. Sci. Technol. 15, 686–693 (2004). [CrossRef]
16. M. Kim, “Principles and techniques of digital holographic microscopy,” SPIE Rev. 1, 018005 (2010). [CrossRef]
17. J. Katz and J. Sheng, “Applications of holography in fluid mechanics and particle dynamics,” Ann. Rev. Fluid Mech. 42, 531–555 (2010). [CrossRef]
18. D. Gabor, “A new microscopic principle,” Nature 161, 18275–18282 (1948). [CrossRef]
19. C. Bohren and D. Huffman, Absorption and Scattering of Light by Small Particles (John Wiley & Sons., 1983).
20. B. Berne and R. Pecora, Dynamic Light Scattering (John Wiley & Sons, Inc., 1976).
21. U. Agero, C. Monken, C. Ropert, R. Gazzinelli, and O. Mesquita, “Cell surface fluctuations studied with defocusing microscopy,” Phys. Rev. E 67, 051904 (2003). [CrossRef]
22. L. Mesquita, U. Agero, and O. Mesquita, “Defocusing microscopy: An approach for red blood cell optics,” Appl. Phys. Lett. 88, 133901 (2006). [CrossRef]
24. M. Born and E. Wolf, Principles of Optics6th Ed., (Cambridge University Press, 1998).
25. G. Farnell, “Measured phase distribution in the image space of a microwave lens,” Can. J. Phys. 36, 935–943 (1958). [CrossRef]
26. A. Pralle, M. Prummer, E.-L. Florin, E. Stelzer, and J. Hörber, “Three-dimensional high-resolution particle tracking for optical tweezers by forward scattered light,” Microsc. Res. Tech. 44, 378–86 (1999). [CrossRef] [PubMed]
27. A. Rohrbach and E. Stelzer, “Three-dimensional position detection of optically trapped dielectric particles,” J. Appl. Phys. 91, 5474 (2002). [CrossRef]
30. E. L. Hall, Computer Image Processing and Recognition (Academic Press, 1979).
32. B. Kemper, S. Stürwald, C. Remmersmann, P. Langehanenberg, and G. von Bally, “Characterisation of light emitting diodes (leds) for application in digital holographic microscopy for inspection of micro and nanostructured surfaces,” Opt. Laser. Eng. 46, 499–507 (2008). [CrossRef]
33. B. Kemper, S. Kosmeier, P. Langehanenberg, S. Przibilla, C. Remmersmann, S. Stürwald, and G. von Bally, “Application of 3d tracking, led illumination and multi-wavelength techniques for quantitative cell analysis in digital holographic microscopy,” Proc. SPIE 7184, 71840R–1–71840R–12 (2009).
34. M. Elliot and W. Poon, “Conventional optical microscopy of colloidal suspensions,” Adv. Coll. Interf. Sci. 92, 133–194 (2001). [CrossRef]