Open Access
Add to BibSonomyAbstract
The video stream captured by an in-line holographic microscope can be analyzed on a frame-by-frame basis to track individual colloidal particles’ three-dimensional motions with nanometer resolution, and simultaneously to measure their sizes and refractive indexes. Through a combination of hardware acceleration and software optimization, this analysis can be carried out in near real time with off-the-shelf instrumentation. An efficient particle identification algorithm automates initial position estimation with sufficient accuracy to enable unattended holographic tracking and characterization. This technique’s resolution for particle size is fine enough to detect molecular-scale coatings on the surfaces of colloidal spheres, without requiring staining or fluorescent labeling. We demonstrate this approach to label-free holographic flow cytometry by detecting the binding of avidin to biotinylated polystyrene spheres.
©2009 Optical Society of America
1. Introduction
Colloidal spheres are pervasive in natural, industrial and biomedical applications. Here, we describe a rapid and precise method for measuring individual colloidal spheres’ radii and refractive indexes as they flow down a microfluidic channel, while simultaneously tracking their three dimensional positions with nanometer resolution. Based on quantitative analysis [1] of images obtained with holographic video microscopy [2, 3], this technique is exceptionally robust against motion blurring [4, 5] and offers near-real-time performance through hardware accelerated image analysis. Its subnanometer resolution for particle sizing is fine enough to detect the presence of molecular coatings on the surface of micrometer-scale colloidal beads and so can be used for label-free molecular binding assays. We demonstrate this by detecting the binding of neutravidin to biotinylated polystyrene spheres.
2. Holographic video microscopy
Our in-line holographic video microscope [1, 3], shown schematically in Fig. 1, is based on an inverted microscope (Zeiss Axiovert 100 STV) outfitted with a 100× numerical aperture 1.4 oil immersion objective (Zeiss S Plan Apo). The conventional incandescent illumination is replaced with the collimated coherent beam from a solid-state laser (Coherent Verdi 5W) operating at a vacuum wavelength of λ=532 nm. Individual colloidal spheres scatter a small proportion of the incident beam, and the scattered light interferes with the unscattered portion in the focal plane of the microscope’s objective lens. The microscope magnifies the interference pattern and projects it onto the face of a low-noise gray-scale video camera (NEC TI 324 IIA), with a total system magnification of 101 nm/pixel. The video stream is recorded as uncompressed digital video with a digital video recorder (Pioneer H520S) for subsequent analysis. The illuminating beam’s fluence is on the order of 1 nW/µm2, comparable to that of conventional microscope illumination.
The measured intensity at point r in the focal plane,
is the superposition of the incident plane wave, E 0(z), propagating along z and the scattered wave, ES(r−rp), that propagates from the particle at rp(t) to the point of observation, r. This scattered field is described by Lorenz-Mie theory [6] and depends not only on the particle’s position, but also on its radius, ap, and its refractive index, np relative to the refractive index, nm, of the surrounding medium. Consequently, measured images such as the example in Fig. 1 can be fit to Eq. (1) with rp(t), ap and np as adjustable parameters [1]. The computed uncertainties in the fit parameters are found to accurately assess each such measurement’s precision [1, 4, 5, 7]. This procedure routinely yields the three-dimensional position and radius of a micrometer-scale dielectric sphere to within a nanometer and its refractive index to within one part in a thousand [1].
Fig. 1. In-line holographic video microscope. A collimated laser beam illuminates the sample. Light scattered by the sample interferes with the unscattered portion of the beam in the focal plane of the objective lens. The interference pattern is magnified, recorded and then fit to predictions of Lorenz-Mie theory to obtain measurements of the particle’s position, its size, and its refractive index.
2.1. GPU acceleration
Fitting image data to the results of Lorenz-Mie theory is computationally intensive, and requires initial estimates for the adjustable parameters. Previous implementations were not suitable for high-speed automated analysis because each sphere had to be identified in each image by hand and each fit then required several seconds of computation [1, 8, 9, 10]. Fortunately, holographic fitting lends itself to parallel processing on the graphical processing unit (GPU) of a computer graphics card [11]. A sphere’s scattering pattern typically subtends tens of thousands of pixels and must be computed dozens of times in the course of each fit. Each scattering pattern, furthermore, is expressed as an expansion in special functions, each of which must be separately computed. Whereas conventional CPU-based implementations operate on each pixel in sequence, a GPU-enabled algorithm operates on multiple pixels simultaneously, and so is substantially faster.
We analyze holographic images with software developed in the IDL programming language (ITT Visual Information Solutions, Boulder, CO), taking advantage of the MPFIT suite of Levenberg-Marquardt nonlinear least-squares fitting routines [12]. We implemented a GPU-enabled computation of Es(r) using the GPUlib [13] (Tech-X Corp., Boulder, CO) extensions to IDL on an nVidia GTX 280 graphics card (nVidia Corp., Santa Clara, CA) installed in the host computer. GPUlib provides access to the underlying CUDA framework (http://www.nvidia.com/cuda/) for mathematical computation on GPUs. With these enhancements, our implementation computes a 200×200 pixel trial image in 3 ms, a factor of 20 faster than the equivalent calculation performed on the central processing unit (CPU) alone. Substantial further acceleration could be attained by performing the fits on the GPU and implementing the entire software suite in an optimized compiled programming language. When supported by a multi-core CPU, the GPU can process multiple computational threads simultaneously, yielding a proportional increase in processing speed. Our implementation uses three independent computational threads on a quad-core CPU to attain the equivalent of 8 frames/s. Adding a second graphics card nearly doubles the processing speed to 15 frames/s.
2.2. Unattended feature identification
Fig. 2. Detecting particle images in a video hologram. Original (a) and transformed (b) holographic images of three colloidal spheres. Superimposed line segments in (a) indicate the votes cast by three representative pixels. Intensity in (b) is scaled by the number of votes, with black representing 0 and white representing 800 votes. Superimposed surface plots illustrate the middle sphere’s transformation. Scale bar indicates 10 µm. Media 1 shows detection, identification and fitting of a moving particle in a holographic video sequence.
We automate GPU-accelerated fitting by first identifying particle images in the field of view and then estimating the fitting parameters at low precision through a combination of standard methods. Estimates for each identified particle then are refined to high precision with GPU-accelerated fits to Eq. (1).
Each sphere appears in a snapshot, such as the example in Fig. 2(a), as concentric bright and dark rings. The gradient of the intensity at each pixel therefore defines a line segment in the imaging plane along which a sphere’s center may lie. The intersection of such lines defines an estimate for the sphere’s centroid in the focal plane. We identify such intersections with a simplified variant of the circular Hough transform [14] in which each pixel in the original image casts “votes” for the pixels in the transformed image that might be centroids. The three line segments superimposed on Fig. 2(a) indicate votes cast by three representative pixels in the original image. Figure 2(b) shows how the image in Fig. 2(a) is transformed by accumulating all possible single-pixel votes. The inset surface plots demonstrate how the extended interference pattern due to a single sphere is transformed into a sharply defined peak, even if two or more spheres’ holographic images overlap.
Those pixels in the transformed image with the most votes are taken to indicate the positions of spheres. Initial estimates for their in-plane coordinates then are computed as the brightness-weighted center of brightness for each peak. This procedure typically identifies particles’ centroids to within a few tenths of a pixel [7], or a few dozen nanometers. We find in practice that this is sufficiently precise to ensure convergence of the subsequent Lorenz-Mie fit.
Given a sphere’s in-plane centroid, we estimate its axial position, radius and refractive index with low-resolution Monte Carlo fits over the anticipated range of parameters. This process can be slow for large, high-index particles whose error function has many local minima. For micrometer-scale latex spheres, on the other hand, it is both fast and robust. Furthermore, in cases where the refractive index and particle size are known to within a few percent, this boot-strapping process can be very fast. For the data presented here, automated preprocessing took up no more than ten percent of the processing time for each fit.
Fig. 3. Holographic particle-image velocimetry. (a) Measured three-dimensional trajectories of 500 colloidal spheres travelling down a microfluidic channel in a pressure-driven flow. Each sphere represents the position of a particle in one field of a holographic snapshot. Features from a sequence of fields are linked into trajectories that are colored by the particle’s measured speed. (b) Poiseuille flow profile along the vertical direction obtained from the data in (a). Particles are excluded from the shaded regions by their interactions with the upper and lower glass walls of the channel. The dashed curve is a fit to the anticipated parabolic flow profile.
Because each sphere’s image extends over a large number of pixels, reliable and accurate results may be obtained even when particles’ images overlap or intersect the edge of the field of view. Systematic studies of the limits of single-particle fitting based on sphere concentration, image truncation, and camera noise will be reported elsewhere. The present studies focus on colloidal dispersions whose concentrations are sufficiently low to avoid errors due to overlapping images.
The combination of hardware acceleration and rapid initialization transform quantitative holographic video microscopy into a suitable tool for large-scale automated colloidal tracking and characterization. We next demonstrate its performance on a model colloidal dispersion and then apply it to label-free detection of avidin binding to functionalized latex beads.
3. Results
3.1. Nanometer-resolution 3D particle-image velocimetry
Holographic particle tracking has immediate applications for three-dimensional particle-image velocimetry. Figure 3(a) shows the superimposed trajectories of 500 individual one-micrometer-diameter polystyrene spheres (Duke Scientific, catalog number 5100A) travelling down a 2 cm long microfluidic channel of 100 µm width and 17 µm depth. The spheres were dispersed in water at a volume fraction of 10-5, and were advected by a pressure-driven flow of water created by raising a reservoir against gravity. Images were obtained in a 50×70 µm2 area near the middle of the channel, with the focal plane set roughly 5 µm below the lower glass-water interface. Spheres’ locations in each snapshot are linked with a maximum-likelihood algorithm [7] into single-particle trajectories, rp(t), sampled at 1/60 s intervals. Not every time step is represented in each particle’s trace because faster-moving particles near the mid-plane of the flow occasionally obscure slower-moving particles near the walls. Figure 3(a) presents only those particle positions that were identified unambiguously. Even such incomplete time series can be used to estimate the particles’ instantaneous velocities. The traces in Fig. 3(a) are colored according to the trajectory-averaged speed.
Fig. 4. Holographic characterization of streaming particles. (a) Trajectory-averaged radii ap and refractive indexes np for a sample of commercial polystyrene spheres in water. Histograms show the distributions of observed sizes and refractive indexes, together with Gaussian fits. (b) Trajectory-averaged radius and refractive index as a function of mean speed.
These trajectories also are useful for mapping the three-dimensional flow field. Each point in Fig. 3(b) represents one particle’s speed as a function of its mean height, z, in the microfluidic channel. The superimposed results of 1000 such trajectories clearly show the parabolic flow profile expected for Poiseuille flow down a channel, the width of the cluster of data reflecting spatial variations across the long horizontal axis of the channel. The limits of the vertical axis indicate the positions of the channel’s upper and lower walls, with heights being reported relative to the microscope’s focal plane. The dashed horizontal lines represent the region of the flow into which particles cannot wander because of their hard-sphere interaction with the glass walls. The fit parabola shows the flow vanishing at the channel’s boundaries.
3.2. Holographically characterizing fast-moving particles
Each trajectory also yields trajectory-averaged measurements of the radius and refractive index for each particle individually. Combining multiple measurements on a single particle minimizes systematic errors due to inevitable position-dependent variations in the illumination. The results in Fig. 4(a) show the radii and refractive indexes of the spheres in a commercial sample of polystyrene microspheres dispersed in water. The mean radius of ap=0.4995 µm agrees with the manufacturer’s specification obtained by conventional light scattering, as does the measured 2.5 percent polydispersity in the radius. The mean refractive index of np=1.595 is consistent with independent measurements on polystyrene spheres [15].
Single-particle characterization is a substantial benefit of holographic characterization compared with bulk light-scattering measurements, which are the usual basis for analyzing particle dispersions. Building up distributions such as the example in Fig. 4(a) from single-particle measurements eliminates the need for population models, and thus affords more general insights into a sample’s composition. For example, the anticorrelation between the particles’ size and refractive index evident in Fig. 4(a) would not be apparent in light scattering data. No such anticorrelation is apparent in holographic analyses of homogeneous fluid droplets [10]. One interpretation of this observation is that the larger spheres in the emulsion polymerized sample are more porous, and consequently have lower refractive indexes.
Fig. 5. Detection of avidin binding to biotinylated polystyrene spheres. (a) Yellow circles show the probability distribution for the measured particle radii in stock biotinylated polystyrene spheres. Red circles show the corresponding distribution for a sample of these spheres after incubation with neutravidin. Dashed curves are guides to the eye. (b) Equivalent distributions for particles’ refractive indexes. Arrow indicates redistribution of probability from low density tail in the stock sample to the peak in the coated sample.
Simultaneously tracking and characterizing individual particles enables us to confirm our results’ freedom from motion-based artifacts. Colloidal particles’ images become blurred if they move during the period that the camera’s shutter is open. This blurring introduces substantial artifacts into conventional bright-field video microscopy data [4, 5]. As the results in Fig. 4(b) demonstrate, however, motion blurring has no discernible influence on values for the radii and refractive indexes obtained by holographic analysis for speeds as high as 500 µm/s. Additional measurements reveal deviations from the population average values only for peak flow speeds exceeding 700 µm/s.
This robustness is surprising at first blush because particles travelling at several hundred micrometers per second traverse several of our camera’s pixels during its 1 ms shutter period. The resulting incoherent average of the oscillatory scattering pattern serves primarily to reduce the contrast in the direction of motion, however, and so has little influence on the Lorenz-Mie fit. Even this amount of blurring could be reduced through the use of a faster shutter or a pulsed laser for illumination.
Being able to characterize individual colloidal particles as they travel down a microfluidic channel provides an effective basis for detecting molecular-scale coatings on functionalized beads. If the individual spheres’ radii were known to within a nanometer or so, then the presence of a molecular coating of similar refractive index could be discerned in the apparent increase in the radius. More generally, the characteristics of a treated sample can be compared with control measurements on untreated spheres.
3.3. Label-free molecular binding assays
Figure 5 shows one such comparative study of 2 µm diameter biotinylated polystyrene spheres before and after incubation with neutravidin. The biotinylated polystyrene spheres used in this study were obtained from Polysciences Inc (Warrington, PA) (catalog number 24172). Neutravidin was obtained from Invitrogen (Carlsbad, CA) (catalog number A2666). A neutravidin solution at a concentration of 1 mg/mL was prepared by adding 1 mg of neutravidin to 1 mL of phosphate buffer saline (PBS) (50 mM, [NaCl]=50 mM). The stock sample of beads was obtained by adding 10 µL of the as-delivered dispersion to 990 µL of PBS. The coated sample was prepared by adding 10 µL of the as-delivered dispersion to 990 µL of neutravidin solution. Particles were incubated and shaken at room temperature for 1 hr before they were introduced into the microfluidic channels by capillary action. Flow was induced by introducing a slip of absorbent paper into one end of the channel and images recorded until results were obtained for 1,000 spheres from each sample. Each data set consisted of roughly 5,000 holographic measurements, which were obtained over the course of roughly 5 min.
From these measurements, we determined that the untreated sample has a population-averaged radius of 0.996±0.015 µm, consistent with the manufacturer’s specification. The incubated population appears to some 6 nm larger, with an average radius of 1.002±0.015 µm. Even though the two size distributions plotted in Fig. 5(a) overlap substantially, a Wilcoxon rank-sum test demonstrates that their means differ with better than 99 percent certainty. This then constitutes a statistically significant detection of change in the treated sample’s radius, which can reasonably be ascribed to the presence of a molecular-scale coating. The coating’s thickness, in this case, is consistent with the size of a multi-domain avidin derivative.
Pronounced differences between the two samples also are evident in the measured distribution of refractive indexes, plotted in Fig. 5(b). The incubated sample’s distribution is significantly sharper, presumably because protein, whose refractive index is similar to that of polystyrene, displaces water in the spheres’ porous surfaces, and raises their effective refractive indexes. This would affect the more porous particles on the lower side of the refractive index distribution more than the denser particles on the high side, thereby sharpening the distribution. The arrow in Fig. 5(b) indicates this redistribution.
Similar analyses of random samples of the two data sets further confirm that the particles from the untreated sample all come from the same population, whose size and refractive index is consistent with the manufacturer’s specification. The treated samples, by contrast show more variability in size, possibly because the thickness and evenness of the bound avidin layer can vary from sphere to sphere.
These results demonstrate the utility of hardware-accelerated digital video microscopy for detecting molecular-scale coatings on functionalized colloidal spheres. Unlike conventional molecular binding assays, holographic analysis does not require fluorescent or radiological markers, and so eliminates the effort and expense ordinarily required to label molecules bound to beads.
4. Discussion
The proof-of-concept demonstration of holographic flow cytometry presented here can be improved upon in several respects. The substrate beads’ size and composition, selected here for convenience, can be optimized for sensitivity. Analysis of calculated scattering patterns suggests that somewhat smaller spheres made of a lower-index material such as silica would offer greater sensitivity to the presence of molecular-scale coatings. This added sensitivity would be useful for detecting smaller molecules, and could be sufficient to seek out nonuniformity in individual spheres’ surface coatings.
Comparison with similar GPU-based applications suggests that processing time, which presently stretches to several hours, can be reduced by another order of magnitude on existing computer hardware through rigorous software optimization. In that case, population-based molecular binding assays of the kind we have presented could be completed in several minutes.
Greater sensitivity and speed also could be attained through an optimized choice of laser wavelength. Simultaneous measurements in two or more wavelengths might even enable detection of molecular coatings on individual spheres while also providing spectroscopic information on the coatings’ composition.
Even in its present form, the method we have presented here offers precision, simplicity, generality and speed for colloidal tracking and characterization. Consequently, holographic analysis should prove useful in other application areas. For example, high-resolution single-particle characterization is superior to bulk light-scattering for probing the properties of mixed colloidal samples because it does not rely on models for those properties’ distributions. Holographic characterization therefore should be a useful adjunct for colloidal synthesis, and is rapid enough to be useful for process control. Nanometer-resolution three-dimensional holographic tracking data already has proved useful for microrheology [9] and research in statistical physics [8]. The method’s comparative simplicity and use of off-the-shelf components should encourage rapid adoption in areas that previously have been dominated by conventional light microscopy.
This work was supported by the National Science Foundation through Grant Number DMR-0606415 and by the Keck Foundation. B.S. acknowledges support of the Kessler Family Foundation.
References and links
1. S.-H. Lee, Y. Roichman, G.-R. Yi, S.-H. Kim, S.-M. Yang, A. van Blaaderen, P. van Oostrum, and D. G. Grier, “Characterizing and tracking single colloidal particles with video holographic microscopy,” Opt. Express 15, 18275–18282 (2007), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-26-18275. [CrossRef] [PubMed]
2. J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45(16), 3893–3901 (2006). [CrossRef]
3. S.-H. Lee and D. G. Grier, “Holographic microscopy of holographically trapped three-dimensional structures,” Opt. Express 15, 1505–1512 (2007), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-4-1505. [CrossRef] [PubMed]
4. T. Savin and P. S. Doyle, “Role of finite exposure time on measuring an elastic modulus using microrheology,” Phys. Rev. E 71, 041106 (2005). [CrossRef]
5. T. Savin and P. S. Doyle, “Static and dynamic errors in particle tracking microrheology,” Biophys. J. 88, 623–638 (2005). [CrossRef]
6. C. F. Bohren and D. R. Huffman, Absorption and Scattering of Light by Small Particles (Wiley Interscience, New York, 1983).
7. J. C. Crocker and D. G. Grier, “Methods of digital video microscopy for colloidal studies,” J. Colloid Interface Sci. 179, 298–310 (1996). [CrossRef]
8. Y. Roichman, B. Sun, A. Stolarski, and D. G. Grier, “Influence of non-conservative optical forces on the dynamics of optically trapped colloidal spheres: The fountain of probability,” Phys. Rev. Lett. 101, 128301 (2008). [PubMed]
9. F. C. Cheong, S. Duarte, S.-H. Lee, and D. G. Grier, “Holographic microrheology of polysaccharides from Streptococcus mutans biofilms,” Rheologica Acta 48, 109–115 (2009). [CrossRef]
10. F. C. Cheong, K. Xiao, and D. G. Grier, “Characterization of individual milk fat globules with holographic video microscopy,” J. Dairy Sci. 92, 95–99 (2009). [CrossRef]
11. J. D. Owens, D. Luebke, N. Govinadaraju, M. Harris, J. Kruger, A. E. Lefohn, and T. J. Purcell, “A survey of general-purpose computation on graphics hardware,” Comp. Graph. Forum 26, 80–113 (2007). [CrossRef]
12. C. B. Markwardt, “Non-linear least squares fitting in IDL with MPFIT,” in Astronomical Data Analysis Software and Systems XVIII, D. Bohlender, P. Dowler, and D. Durand, eds. (Astronomical Society of the Pacific, San Francisco, 2009), in press.
13. P. Messmer, P. J. Mullowney, and B. E. Granger, “GPULib: GPU computing in high-level languages,” Comp. Sci. Eng. 10, 70–73 (2008). [CrossRef]
14. R. O. Duda and P. E. Hart, “Use of the Hough transformation to detect lines and curves in pictures,” Commun. ACM 15, 11–15 (1972). [CrossRef]
15. X. Ma, J. Q. Lu, R. S. Brock, K. M. Jacobs, P. Yang, and X.-H. Hu, “Determination of complex refractive index of polystyrene microspheres from 370 to 1610 nm,” Phys. Med. Bio. 48, 4165–4172 (2003). [CrossRef]
