In-line holographic optical imaging has the unique capability of high speed imaging in three dimensions at rates limited only by the imaging rate of the camera used. In this technique the 3D data is recorded on the detector in a form of a hologram generated by diffraction between the scattered and unscattered light passing through the sample. For dilute samples of single particles or a small cluster of particles, this technique was shown to result in particle tracking with spatial positioning accuracy of a few nanometers. For dense suspension only approximate reconstruction were achieved with systematic axial positioning errors. We propose a scheme to extend accurate holographic microscopy to dense suspensions, by calibrating the Rayleigh-Sommerfeld reconstruction algorithm against Lorentz-Mie scattering theory. We perform this calibration both numerically and experimentally and define the parameter space in which accurate imaging is achieved, and in which numerical calibration holds. We demonstrate the validity of our approach by imaging two attached particles and measuring the distance between their centers with 36 nm accuracy. A difference of 50 nm in particle diameter is easily measured.
© 2013 OSA
Fast three dimensional imaging of dense microscopic colloidal suspensions is greatly desired for many applications, such as 3D fluid velocity field characterization, microrheology of complex media, and characterization of active motion in live cells. The most common microscopy technique used for three dimensional imaging is confocal microscopy, in which fluorescently labeled tracer particles are followed and tracked. The rate of imaging of the full field of view is determined by the camera capture rate, the fluorescence efficiency, and the number of planes imaged in the axial direction. In-line digital holographic microscopy is an imaging technique offering three dimensional imaging of unlabeled particle suspensions at a single camera exposure, capable of reaching kHz imaging rates in 3D [1, 2, 3]. However, accurate rendering of particle location within the sample, done in the framework of the Lorentz-Mie scattering theory, is restricted to very dilute samples having non-ovelapping diffraction patterns. Here we demonstrate how to extend accurate in-line holographic 3D positioning to denser suspensions.
Digital holographic microscopy (DHM) is a 3D imaging technique based on recording the interference pattern between a reference beam and a beam diffracted by a sample. This interference pattern constitutes a hologram that encodes the three dimensional image of the sample. In-line digital holography is a variant of the technique that uses the light that does not scatter by the sample as a reference beam. Since the reference beam and the scattered beam travel through exactly the same optical path except for diffracting from an imaged particle, the method is less sensitive to variations in refractive index,thickness, and composition. The implementation of in-line holographic microscopy involves replacing the bright field illumination of an optical microscope by collimated laser light. Two different approaches can be used to extract 3D particle positions from the obtained holograms, one based on reconstructing the hologram digitally, using Fresnel or Rayleigh-Sommerfeld diffraction integrals [2, 4, 5], and the other, by fitting the acquired diffraction pattern to the theoretical pattern calculated using Mie scattering and T-Matrix theory [6, 1, 7]. The most accurate method to date is the latter, based on fitting of the scattering data to the Mie scattering pattern of a single sphere , or to a superposition of them , achieving positioning accuracy as good as 7 nm laterally and 10 nm axially . The normalized intensity on a plane perpendicular to the optical axis in such a hologram, is a function of the sphere’s 3D position, size, refractive index and the refractive index of its environment . When determining the laser’s wavelength and the effective magnification, this fitting process yields a precise measure to the sphere’s parameters and reveals more information than do hologram reconstruction algorithms. Once the 3D position of the particle is determined standard tracking algorithms  can be applied to characterize its motion. The main caveat of this fitting method is its sensitivity to imaging noise, overlapping diffraction patterns prevent its convergence. As a result, accurate extraction of the three dimensional positions of particles is limited to very dilute suspension.
The Rayleigh-Sommerfeld reconstruction algorithm recreates the three dimensional light field producing the hologram without requiring prior knowledge of the sample shape and composition, enabling imaging of denser suspensions [6, 10, 4]. However, previous studies have shown that the extracted axial positions of beads in this method suffer from systematic errors whose extent depends on particle size, composition, and height . In addition, axial positioning accuracy is reduced compared to Mie scattering fitting algorithms. Nontheless Rayleigh-Sommerfeld reconstruction has been used to follow diffusion of colloidal particles of various shapes [8, 4]. The more involved fitting procedure of the Lorentz-Mie approach requires intensive computation as compared with the straight forward back propagation of the Rayleigh-Sommerfeld reconstruction algorithm, which means that the Rayleigh-Sommerfeld scheme is easier to implement and faster to run by a factor depending on the specific implementation. This is not a major drawback however, since new implementations of Lorentz-Mie fitting, based on GPU accelerated protocols, enable real time analysis of in-line holographic data .
We propose a calibration scheme to correct for these axial positioning errors in order to achieve accurate 3D position rendering of dense colloidal suspensions within the Rayleigh-Sommerfeld reconstruction framework.In this study we identify the main sources of the systematic errors in the Rayleigh-Sommerfeld reconstruction algorithm and correct for them by calibrating the Rayleigh-Sommerfeld scheme against the Lorentz-Mie approach on simulated data, and on experimental data. As a result we achieve improved accuracy of 3D particle positioning using Rayleigh-Sommerfeld reconstruction. We proceed to demonstrate two scenarios in which the calibrated Rayleigh-Sommerfeld reconstruction produces better 3D particle positioning than the Lorentz-Mie approach; particles near a partially reflective wall, and two attached particles with overlapping diffraction patterns.
2. Rayleigh-Sommerfeld reconstruction
Rayleigh-Sommerfeld reconstruction of an in-line holographic microscope image involves three steps: Background subtraction, image back propagation, and image deconvolution . Once the three dimensional image is obtained standard image analysis algorithms are used to identify feature locations, from which particle location is then interpreted. Let us describe the reference beam and scattered light in In-line microscopy following the notation of [6, 4] as:6]. Dividing the recorded hologram I (r⃗) by the illumination’s intensity distribution then yields: 12, 4]. Conventional accuracies measured in this manner range up to 20 nm laterally and 60 nm axially . Once the hologram is recorded and normalized by its background, the scattered field at height z above the focal plane may be reconstructed from the complex field at the focal plane by convolving ES (r⃗, 0) with the the Rayleigh–Sommerfeld propagator h (r⃗, −z) : 4]. Following Dixon et al. we now deconvolve our volumetric reconstruction with the Rayleigh-Sommerfeld point spread function itself, KR (r⃗, z − zp) = |h (r⃗, z − zp)|2, in Fourier space: 4], resulting in a twofold improvement in positioning accuracy. A transparent colloidal particle subjected to plane wave illumination acts as an imperfect lens and focuses the light to a focal spot further away. The bright feature observed in the 3D light pattern reconstructed from a hologram is then identified as that focal spot. For large particles, of the order of 30 μm in diameter the distance between the focal spot and the particle can be calculated analytically, and is in accord with experimental measurements . However, previous studies have shown that the axial position of smaller colloidal particles, of the order of 1 μm, calculated by the Rayleigh-Sommerfeld reconstruction algorithm is shifted form the one obtained by fitting to Lorentz-Mie theory by an amount which depends not only on the refractive index of bead and suspension, and on bead size, but also on the bead’s distance from the focal plane . In the following section we will describe a calibration scheme that corrects for these errors rendering Rayleigh-Sommerfeld reconstruction accurate.
3. Calibrating Rayleigh-Sommerfeld reconstruction
3.1. Numerical calibration
We consider now the incident plane-wave and the scattered light from a spherical particle, E⃗0 and E⃗s, as defined in Eq. (1). We describe the scattered field at position r⃗ in the focal plane due to an object at r⃗p, relative to the center of the focal plane, as :6]: 6]. With these definitions, the normalized hologram can be written as:
In order to calibrate the Rayleigh-Sommerfeld scheme for hologram reconstruction, we first simulated theoretical calibration graphs by computing digital holograms of spheres using Lorentz-Mie theory as described above, and calculating the axial shift, ΔzLM−RS, between the height of the particle according to Rayleigh-Sommerfeld reconstruction ZRS, and Lorentz-Mie fitting ZLM, with different values of particle size and height above the focal plane. Since Lorentz-Mie fitting recovers exactly the input position of the particles, Rayleigh-Sommerfeld reconstruction of the simulated holograms reveals the effective focal length of the particle, ΔzLM−RS = zLM − zRS. Figure 1 shows the measured ΔzLM−RS value from computer-generated holograms of polystyrene spheres in TE buffer. The result indicates clearly that zRS depends not only on the particle’s size, but also on its height above the focal plane. It has already been proven  that minor deviations exist in the value of ΔzLM−RS when a diffusing particle changes its height above the focal plane. Figure 1(a), however, proves these deviations are minor only when small particles (ap ∼ 0.5 μm) are taken into account and cannot be ignored for larger particles. The reason for the increased error in axial position of particle location obtained by Rayleigh-Sommerfeld reconstruction can be explained by a simple argument. The diffraction from each particle results in a pattern of rings with different locations and intensities, the most accurate reconstruction would use all diffraction rings. However, practical consideration limit the number of diffraction rings used for reconstruction, such as experimental optical noise obscuring the weaker rings, or the limited field of view chosen around the particle center for reconstruction. As a result, if too few diffraction rings are used a systematic error in location is created. From our simulations we can asses at which condition this systematic error can be avoided, and at which conditions a linear correction can be applied to correctly asses relative motion in the axial direction. It is important to state that the Lorentz-Mie fitting approach is more robust, since it can converge even when a smaller number of diffraction rings are fitted. To determine a range within which Rayleigh-Sommerfeld reconstruction is accurate at an experimentally available field-of-view we calculated ΔzLM−RS as a function of window size for different particle heights (see Fig. 1(b)). From Fig. 1 we learn that window size should be over 30 μm to reach a linear shift for particles between ap ∼ 0.5 – 1 μm, which we used for our experimental data reconstruction. For small particles (ap ∼ 0.5 μm), in standard imaging conditions, axial position obtained by Rayleigh-Sommerfeld reconstruction is insensitive to the distance from the microscope’s focal length, at window sizes greater than 20 μm. Moreover, the accurate axial position of such a bead is about 0.8 μm above the bright feature in the rendered three dimensional image. In this way, we can produce height calibration curves for any given particle with known composition and size.
3.2. Experimental calibration
To test our calibration method we assembled an in-line holographic imaging microscope equipped with holographic optical tweezers (HOTs) . Figure 2 shows a schematic representation of the integrated system, which is based on an inverted optical microscope (Olympus IX71) working at a magnification of 100x with an oil immersion objective lens (NA = 1.4). This lens is used both to project holographic optical traps, and also to collect in-line holographic images of trapped objects. Holographic traps are powered by a CW Ti:Shapire laser (Coherent) operating at a wavelength of 750 nm. The collimated laser light is imprinted with a phase pattern by a liquid crystal spatial light modulator (SLM, Hamamatsu X-10468-07) encoding optical trap location in the sample plane. Holographic imaging is achieved by illuminating the sample with laser light at 532 nm (DPSS, Coherent) and filtering out the trapping laser light. A CMOS camera (Gazelle, Point Grey Research) records the holographic images at a rate of 70 fps. Polystyrene colloids, ap = 1.025 μm, were trapped at three different heights and allowed to diffuse within the trap. Trap height was measured in three techniques: by volumetric imaging of the light on a reflecting mirror , by Rayleigh-Sommerfeld reconstruction, and by fitting to Lorentz-Mie theory. Figure 3(a) shows the trajectory of the particle according to Lorentz-Mie fitting (blue dots), the average height according to Lorentz-Mie fitting (black line), zLM, the average height according to Rayleigh-Sommerfeld reconstruction (red line, no calibration, no deconvolution), zRS, and the height according to the volumetric imaging of the beam (green line), zmirror. The shift between the axial position of the trap according to Lorentz-Mie fitting and volumetric imaging, changes with depth, and depends on the error in the assessment of the objective’s focal length by volumetric imaging and on spherical abberations due to refractive index mismatch between the glass slide and TE buffer. The shift between zLM, and zRS is indeed not constant even for a displacement of 0.3 μm, however, zLM, and zRS are linearly dependent in this range (see Fig. 3(b)). The distribution of zLM−RS − < zLM−RS > shown in Fig. 3(c) indicates that without calibration and deconvolution Rayleigh-Sommerfeld reconstruction has an axial positioning accuracy of about 55 nm. In order to use numerically obtained calibration data, we wish to determine the parameter range in which it coincides with experiment. Specifically, for known particle size and sample composition we would like to compare the linear relation between zLM, and zRS (as measured in Fig. 3(b)), to an equivalent, numerically obtained, curve. To this end we performed a set of experiments tracking the diffusion of colloidal particles, creating calibration curves for them at two different heights and comparing them to numerically reconstructed calibration curves (Fig. 4). Theory and experiment agree well (Fig. 4(a)) at conditions in which simulations showed no dependence on window size, as the number of diffraction rings in the field-of-view decreases, a shift between simulation and experiment is observed (Fig. 4(b)). This shift remains constant for a range of at least 4 μm. In this range, therefore, numerical calibration curves suffice for determining the motion on the axial direction. Below we describe two experiment to which we applied our numerical calibration scheme.
4. Application to experiments
4.1. A particle diffusing near a wall
A single silica bead, suspended in an aqueous solution sediments to the sample cell floor due to gravity. It is repelled from the floor due to screened Coulomb interactions enhanced by plasma etching of the sample chamber. The height distribution of the sphere should be dominated at low heights by a sharp repulsion from the wall and decay exponentially as mgh/kBT far from the wall :Fig. 5. The trajectory of the particle was extracted using both reconstruction algorithms, applying the numerical calibration protocol to data from Rayleigh-Sommerfeld reconstruction (Fig. 5(a)). Having a glass wall near the diffusing particle distorts the diffraction pattern to some extent preventing the Lorent-Mie algorithms from correct convergence occasionally (see deviating green dots on Fig. 5(a)). While there is a small shift observed between the axial position according to both algorithms the height distributions coincide, well fitted by theoretical prediction (Fig. 5(b)), with T ≈ 28°C, κ ≈ 60nm, and Q = q(κap)/(1 +κap) with q ≈ 6500. For such application, where relative axial position is of interest, and boundary conditions distort the diffraction pattern, the calibrated Rayleigh-Sommerfeld reconstruction provides better reconstruction than the more elaborated Lorentz-Mie algorithm. This can be seen in two ways, the precision of particle positioning, which is extracted from the intercept of the mean square displacement plot  is comprable for both algorithms (εxy,RS = εxy,LM ∼ 1nm, εz,RS ∼ 2.5nm < εz,LM ∼ 3.5nm), and the Lorent-Mie algorithm results in locations below the glass wall, ocationally (see low green dots, Fig. 5(a)). Our goal however, is to use calibrated Rayleigh-Sommerfeld reconstruction to measure accurately axial positions of dense suspensions. To this end we must be able to measure the distance between two attached beads.
4.2. Two diffusing spheres
The main purpose of this study was to devise a method in which particle tracking in dense suspensions including aggregated particles could be done in 3D by in-line holographic imaging, with reliable and accurate positioning. The most challenging problem is to extract accurate location and distances between two attached particles of different size with highly overlapping diffraction patterns. To this end we created dilute suspension of colloidal particle pairs with different sizes, one with DNA coating and the other without. Our protocol involved mixing two suspensions, DIG-labeled 2 μm diameter (Spherotech) and NeutrAvidin-labeled 1 μm diameter (Invitrogen) Polystyrene spheres. Prior to mixing each suspension was diluted and washed three times by resuspension in TE buffer, centrifugation, and removal of the supernatant. The first set of experiments was performed after the two suspension were mixed and allowed to form aggregates. For the second set of experiments each suspension was incubated with λ-DNA (In-vitrogrn) after washing and prior to mixing. In all experiments the resulting dilute suspension consisted of aggregates of colloidal particles from both sizes, we chose six particle pairs of each batch and imaged them holographically while diffusing in three dimensions. Using deconvolution Rayleigh-Sommerfeld reconstruction we measured the center-to-center distance between particle pairs and averaged over all measured pairs. In Fig. 6 the probability distribution of this distance during acquisition is plotted before and after the use of our numerical calibration scheme. In the absence of DNA we expect the inter-particle distance to be d = 1.5 μm. Raw data show a wide distribution of distances due to the large difference in the effective focal length of the large and small particles, and due to its different dependence on height. However, after calibration the distribution collapses to a thin gaussian distribution centered around the expected value, d = 1.505 ± 0.038 μm. Suspension prepared with DNA show similar results, however a layer of approximately 100 nm is now coating each sphere.
The flexible algorithm of hologram reconstruction based on Rayleigh-Sommerfeld back propagation can be transformed into an accurate measuring technique by use of the numerical calibration we introduced here. Our approach results in axial localization accuracy of the order of 40 – 60 nm. As a result we were able to perform accurate three dimensional imaging at conditions which are challenging for the more accurate Lorentz-Mie algorithm, such as particles at the vicinity of scattering boundaries and for two attached particles with strongly overlapping diffraction patterns. Our calibration scheme opens new possible applications for in-line holographic imaging which will benefit from its capability for fast three dimensional imaging. For example, characterization of complex flow patterns in three dimensions, especially in microfluidic setups in which the scattering effect of the walls can be significant.
This work was supported in part by the James Franck German-Israeli Binational Program in Laser-Matter interactions, in part by the Marie Curie Reintegration Grant ( PIRG04-GA-2008-239378), and in part by the Israel Science Foundation (Grants Number 1271/08).
References and links
7. J. Fung, K. E. Martin, R. W. Perry, D. M. Kaz, R. McGorty, and V. N. Manoharan, “Measuring translational, rotational, and vibrational dynamics in colloids with digital holographic microscopy,” Opt. Express 19, 8051–8065 (2011) [CrossRef]
9. J. C. Crocker and D. G. Grier, “Methods of digital video microscopy for colloidal studies,” J. Colloid Interface Sci. 179, 298 – 310 (1996) [CrossRef]
10. Y.-S. Choi and S.-J. Lee, “High-accuracy three-dimensional position measurement of tens of micrometers size transparent microspheres using digital in-line holographic microscopy,” Opt. Lett. 36, 4167–4169 (2011) [CrossRef]
12. T. Savin and P. S. Doyle, “Role of a finite exposure time on measuring an elastic modulus using microrheology,” Phys. Rev. E 71, 041106 (2005) [CrossRef]
13. J. Goodman, Introduction to Fourier Optics (McGraw-Hill, 2005).
16. S. D. A. Russel, W. B. Schowalter, and W. R. Schowalter, Colloidal Dispersions (Cambridge University Press, 1989) [CrossRef]