Laser sheet based microscopy has become widely accepted as an effective active illumination method for real time three-dimensional (3D) imaging of biological tissue samples. The light sheet geometry, where the camera is oriented perpendicular to the sheet itself, provides an effective method of eliminating some of the scattered light and minimizing the sample exposure to radiation. However, residual background noise still remains, limiting the contrast and visibility of potentially interesting features in the samples. In this article, we investigate additional structuring of the illumination for improved background rejection, and propose a new technique, “3D HiLo” where we combine two HiLo images processed from orthogonal directions to improve the condition of the 3D reconstruction. We present a comparative study of conventional structured illumination based demodulation methods, namely 3Phase and HiLo with a newly implemented 3D HiLo approach and demonstrate that the latter yields superior signal-to-background ratio in both lateral and axial dimensions, while simultaneously suppressing image processing artifacts.
©2012 Optical Society of America
Three-dimensional (3D) thick tissue imaging in real time is an invaluable tool for visualizing directly the mechanisms of different biological processes in developmental, immunological, or other contexts. However, in vivo or in situ imaging with single cell resolution remains a major challenge. Existing techniques are based on variants of confocal microscopy [1,2] and can provide good lateral and moderate axial resolution, but they are limited by high photobleaching and poor temporal resolution. Confocal techniques are based on scanning in 3D, so for large specimens (size ~1mm3), they become impractical. For the same reason, photobleaching is severe as the excitation occurs along the entire optical axis with variable beam waist as the sample is being scanned.
Recently, laser sheet based microscopic techniques [3–8] have emerged as a useful alternative for imaging biological tissue specimens of size larger than ~1mm3 and over long time periods with high speed, good resolution, and significantly reduced photo bleaching and phototoxic effects. Unlike confocal techniques, which use spot-like excitation, laser-sheet based methods attempt to excite an entire plane within the tissue sample; the illuminated plane is either formed at once using cylindrical lens (called selective plane illumination microscopy: SPIM) or by fast-scanning a pencil-like beam (called digital scan laser sheet microscopy: DSLM). The camera is oriented perpendicular to the illuminated plane, thus significantly speeding up the image acquisition time and, for the same reason, mitigating photobleaching. However, the propagation of a “plane” of light through large-thick tissue results in scattering artifacts in the final image.
Background “haze” is common even in conventional wide-field microscopy and structured illumination-based methods have been developed to reject it, with the aim of visualizing more clearly the in-focus fluorescent-plane [9,10]. By projecting a single spatial frequency grid pattern onto the sample, the microscope effectively images only parts of the grid that lie within its focal range, whereas the background is rejected by frequency mixing of in-focus information (in time-domain radar signal processing, similar approaches are often referred to as “phase locking”). Two approaches for background “haze” removal by demodulation have been proven to be particularly popular: the “3Phase” method [8,11,15] and the “HiLo” method [12,13].
The first method is based on recording images corresponding to the three different phases of the projected grid (0°, 120° and 240° Phase shifts) and was proposed first by Wilson and associates . More recently, Mertz and associates presented a new approach , which requires only one uniform and one structured image at each imaging plane for the reconstruction of optical sectioned image. The final reconstructed image combines two images: the in-focus high frequency content (called “Hi” image) extracted from a uniformly illuminated image; and the in-focus low frequency content (called “Lo” image) obtained under structured illumination. The linear combination of “Hi” and “Lo” images with a scaling factor ‘η’ (Hi + ηLo) is used to ensure a seamless transition between spatial frequencies of the object features; for that reason the final post-processed image is called the “HiLo,” from which term the technique derives its name as well.
Structured illumination methods have been combined with laser sheet microscopy recently for the suppression of background noise [8,16,17] but they were applied mainly to imaging of relatively thin and optically clean biological samples. Here we present a structured illumination based version of DSLM that is successful in imaging thick tissue samples and compare the new technique quantitatively with existing structured illumination-based methods (3Phase and HiLo). We refer to our new approach as “3D HiLo,” because it combines HiLo images from two orthogonal directions (excitation objective point of view and detection objective point of view) to reconstruct a 3D tissue image with subtracted background noise and low image processing artifacts.
The experimental apparatus for implementing conventional HiLo, 3D HiLo, and alternative structured illumination programs (such as 3Phase) is described in Section 2. In section 3, we describe the post-processing algorithms that we have devised to combine the HiLo images from two orthogonal directions. In section 4, we present numerical and experimental results of images acquired with these three alternate structured illumination programs and post-processing algorithms, and compare them quantitatively in terms of background rejection and image artifacts. Conclusions and some suggestions for follow-up work are presented in Section-5.
2. Digital scanned light sheet microscope with multiple programs of structured illumination
We modified the design of Digital Scanned Light-sheet Microscope (DSLM)  as shown in Fig. 1 to provide real time structured illumination to the sample according to the new 3D HiLo principle as well as older, alternative programs of structured illumination [4–8]. In our experimental apparatus, the illumination is provided by four laser lines: 405 nm (Coherent CUBE 403 nm), 488 nm (Coherent Sapphire 488 nm), 561 nm (Coherent Sapphire 561 nm), and 659 nm (Coherent CUBE 659 nm) so that we can excite different biological flurophores at will. All four laser lines are combined and passed through an excitation filter-wheel (Sutter instruments, Lambda 10-3) for excitation of biological samples, one color at a time. The selected laser beam then passes through a sequence of optics designed to produce the two orthogonal structured illumination images as follows: The first element, an acousto-optic-tunable filter, AOTF (AA optoelectronics, AOTFTnc-400-650-TN) adjusts the laser illumination power at any given wavelength, and also modulates the pencil beam sinusoidally to structure the illuminated “sheet.” After the AOTF, the beam goes through a beam expander (1.5X, Edmund optics) which fills the back aperture of the objective, and a 2 axis mechanical optical scanner (Cambridge Technology, MicroMax 673). Finally, the beam goes through the excitation objective (Olympus 4X, 0.13 NA in air). Depending on how we program the AOTF and scanner, we can obtain an almost arbitrary structured (modulated) laser-sheet at the focal point of the collection objective (Zeiss 10X, 0.3 NA water immersion).
The sample is mounted from the top onto a tightly sealed water chamber, on a 4-axis stage (X-Y-Z-θ) (Physik Instrumente, E-665, M-11k028, M-116DGH). The collection objective is inserted into the same sample chamber. We also used two piezo-stages (Physik Instrumente, P721.CLQ) for precise alignment of the illumination and collection objectives. The emission from the sample after the collection objective is passed through an electronically controlled emission filter-wheel and finally transduced by an Andor Clara camera. The whole system is computer controlled through Labview-2011 (National Instruments) using NI-DAQ SCB-68.
The combination of AOTF plus scanner allows us to implement different modes of structured illumination. A digitally controlled trigger signal is used to synchronize AOTF, scanner and camera so as to precisely reproduce the imprinted sinusoidal excitation pattern over different fluorescent planes, depending on the desired mode of structuring the illumination. For the 3D HiLo program, we capture two raw images. The first is a standard DSLM image where the pencil is scanned across the sample without any modulation. For the second raw image, we modulate the pencil so that the resulting three-dimensional (3D) structured illumination pattern from the combination of AOTF action and mechanical scanning consists of sheets oriented parallel to the XZ plane, as shown in Fig. 2 . The two collected images can either be processed individually at each XY plane to yield slice-wise 3D images of the object according to conventional HiLo; or the additional structure along the YZ plane can be exploited to yield even better background rejection, as we describe in detail in section 3.
Alternatively, to implement 3Phase, we expose the sample to sinusoidal patterns of light shifted by 0, 120, and 240 degrees in three consecutive laser scans on the plane of excitation XY; these three collected fluorescent raw images are then post-processed according to the 3Phase scheme [4,7] to reject the background. Due to the frame transfer rate of our camera, we are limited to capturing ~4 images (3 modes of structured illumination and one corresponding to uniform illumination) for each fluorescent plane in a time of less than 1 second. We wish to emphasize that we are camera limited, and not signal limited; based on the amount of fluorescence collected from the sample, we estimate that by using a low-noise CMOS camera such as Hamamatsu-Orca Flash-4, or Andor Neo s-CMOS, we could image more than 10 times faster.
We imaged the fluorescently-labeled membrane and nucleus of live zebrafish at 6 and 36 hours post fertilization (hpf) to demonstrate and quantitatively compare 3Phase, conventional HiLo, and 3D HiLo. We microinjected mRNA inside freshly laid zebrafish embryo (1ng) using an Eppendorf microinjector. Cell nuclei were labeled by expression H2B-mRFP (or H2B-eGFP, depending upon experiments) while membranes were labeled by expression of a membrane bound pCAG-mGFP.
3. Background removal in 3D HiLo
The combined action of AOTF and mechanical scanning in our experiment creates rich spatial structure on the sample, which may be exploited in several ways. For example, if one looks at a single slice parallel to the XY plane at a fixed axial position z, there is a sinusoidal illumination pattern cos(ky) imposed, where k is the spatial frequency. This raw structured illuminated image can be combined with the unstructured (uniformly illuminated) raw image of the same slice according to conventional HiLo to reject the background noise from this slice only. All the slices collected from different z locations can then be put together to create a 3D post-processed image IXY(x,y;z), which we will refer to as XY-HiLo. (This is, of course, identical to conventional HiLo as described by Ref-12, 13.)
Alternatively, if one looks at a single slice parallel to the YZ plane at a fixed lateral position x, there is again a sinusoidal pattern cos(ky) which can similarly combined with the unstructured raw image according to conventional HiLo. Thus, we can create an alternate a 3D post-processed image IYZ(y,z;x), which we will refer to as YZ-HiLo.
Even better, one can exploit the 3D structure of the modulated illumination to reject background more effectively than conventional HiLo with least artifacts by combining the XY and YZ versions. The combination is what we refer to as “3D HiLo.” In this paper, we limit our analysis to linear convex combinations of the IXY and IYZ images according to,
The disadvantage of the geometrically weighted mean method is that the choice of mixing factor α(x) is fixed a priori, without any opportunity to take into account the nature of the sample. For example, if the contrast of projected grid is poorer toward the illumination objective due to the structure of a particular highly inhomogeneous sample, this method will turn out to do exactly the opposite of what it should. This event is relatively rare, however, and can be further mitigated by judicious placement of the specimen in the sample holder.(3) Variance weighted method: To combat the potential problem with inhomogeneous samples mentioned just before, we may define the mixing factor to depend on the local image fluctuations rather than the geometry directly. One may think of this as ad hoc adaptive weighting: we give large weight to portions of the image with large variation (rich structure) and penalize those with small variation, assuming that the latter is an artifact of the structured illumination locally having limited contrast. The simplest way to implement variance weighting is to select the mixing factor as
The relative performance of the three methods, as well as comparisons with conventional DSLM, conventional 3Phase, and conventional HiLo are reported in section 4 below. It should be noted that nonlinear methods for combining the XY-HiLo and YZ-HiLo raw images, such as expectation maximization, may also be used, but they are outside the scope of the present paper.
To model and compare the performance of the different flavors of 3D HiLo presented in section 3, we conducted simulations and experiments image formation for a collection of fluorescence beads. In the simulation, the beads were assumed to be 12 μm and distributed in a 3D volume segmented into voxels. We assumed laser excitation wavelength and fluorescence emission wavelength 530nm. Uniform and structured excitation patterns were simulated for the laser sheet, with thickness, corresponding to the 150 axial planes. For structured illumination, the resulting spatial frequency of the sinusoidal structured pattern at the object plane was 392 lines 392. A microscope objective (water immersion) with NA 1.0 was used for collection of the fluorescence signal. The PSF of the raw (unprocessed) fluorescence image was computed assuming diffraction-limited optics. Quantitative analysis of reconstructed images was performed by plotting the normalized intensity cross-sections along the x-, y- and z-axes respectively to test the proposed method, and the results were compared with conventional DSLM (unstructured illumination).
The simulation results are shown in Fig. 3(a) . We find that the XY-HiLo image is closest to the original bead for both the lateral X and Y directions, but it does not yield any improvement along the Z direction. Similarly, YZ-HiLo yields significant improvement in the Z and Y profiles of the bead, but not in X. These results are intuitively satisfying and justify our motivation for combining the XY-HiLo and YZ-HiLo raw images into the final 3D HiLo image.
Figure 3(b) shows experimental data from a collection of 25 μm size fluorescent beads excited at 488 nm and emitting at 530nm with other conditions matching those of the simulations. Qualitatively, the experimental data clearly agree with the simulations of Fig. 3(a), as XY-HiLo processing shows clear improvement in X and Y axial profiles of the image, whereas YZ-HiLo shows improvement of Y and Z axis profiles.
To compare the quantitative effect of these image processing methods at different imaging depths, we imaged membrane-labeled Zebrafish-embryos imaged at 30 μm depth using uniform illumination, 3Phase structured illumination, conventional XY-HiLo, and 3D HiLo post-processing. The results are shown in Fig. 4 as both reconstructed images and intensity cross-sections. The latter were obtained from two different regions, near (line A) and far (line B) from the laser excitation direction. We observe that 3Phase image removes most fluorescent features, except in the extreme left position, which corresponds to the lowest laser penetration and hence best grid contrast. On the other hand, conventional HiLo (equivalent to XY-HiLo) is much better at preserving features at lower laser penetration [line A, Fig. 4(ii)] whereas deeper inside the tissue many object structures are removed [line B, Fig. 4(iii)]. 3D HiLo does a much better job at recovering the deep sample structure [also line B, Fig. 4(iii)].
Another important observation is derived from the region denoted with a red arrow in Fig. 4(iii-iv). Conventional HiLo (XY-HiLo) removes valid low-bandwidth features of the object (also visible by conventional DSLM) because it low-pass filters them out. In 3D HiLo, the orthogonal (YZ) image picks them up because it does not low pass filter on the XY plane. This is the main benefit of using two orthogonally processed images. 3Phase also performs poorly in that region because it employs a subtraction that removes useful features and amplifies noise when the grid contrast is poor.
The significance of grid contrast variation and recovery of corresponding object figures in the two orthogonal planes XY and YZ is illustrated in Fig. 4(iv). Confirming our intuition about the different ways of combining the two orthogonally processed methods, we can see that the Variance Weighted 3D HiLo provides a better contrast enhancement than Mean 3D HiLo, Geometrically Weighted 3D HiLo.
Figure 5 shows images acquired in the case of a representative difficult 3D imaging problem, the spine-structure of 36 hpf (hours post fertilization) larval zebrafish. These images were composed from stacking the 3D reconstructions from the respective methods and then performing a standard Maximum Intensity Projection (MIP) operation. At a deep fluorescent plane (Z = 110 μm), images post-processed with the different techniques clearly show that 3D HiLo can remove background in a thick tissue sample without removing any features of the object, whereas both XY HiLo and 3Phase-image remove some object features along with background noise.
Finally, Fig. 6 shows 3D images of zebrafish embryo (6 hpf) post-processed with the same suite of techniques as in Fig. 5 and displayed using MIP. This specimen contains both heterogeneous low-frequency structures (membranes) and discrete high-frequency objects (nuclei), which have been labeled with different fluorophores so they are observable separately. We observe that for high frequency objects the improvement of either 3Phase or HiLo over uniform (unstructured) DSLM illumination is minimal. However, 3D HiLo significantly improves the image quality of the membranes.
In addition to the obvious one of requiring two raw images instead of three, we have observed certain additional advantages of HiLo over 3Phase for thick-large tissue imaging. As 3Phase is a subtraction based method, its effectiveness is strongly contingent upon grid contrast. Therefore, at image portions where grid contrast is low, or image portions where the fluorescent signal is lower, 3Phase fails dramatically. In practice, such failures are common for thicker and deeper portions of the tissue. Moreover, the 3Phase method is very sensitive on the relative phase of the three raw images. Slight deviation from theoretical phase in the image can result in major image processing artifacts, e.g. line patterns in the post-processed image.
Even though we took great effort to minimize experimental errors in the relative phases, contrast-related artifacts are vividly displayed in the 3Phase result for the membrane in Fig. 6, where the left-hand side of the object appears darker even though the excitation there was stronger and the illumination grid contrast should be higher at that side of the object (recall that the excitation illumination is always coming from the left in our experimental results). The reason for the darker appearance on the left-hand side is that the fluorophore concentration was lower in that region. We also have found few severe line pattern artifacts due to phase mismatch, especially in deeper tissue portions. Hence, we conclude that 3Phase structured illumination works well for thinner and more transparent tissue samples, where one can imprint good-contrast structured imaging throughout the volume and the phase shift is precise; whereas for thicker samples, where grid contrast is liable to vary widely across the 3D sample size, 3D HiLo is preferable.
In summary, we have developed a novel digital scanned laser sheet microscope capable of capturing raw images with different programs of structured illumination and post-processing them according to algorithms optimized for the corresponding caveats of illumination grid contrast decay and scattering artifacts inside the sample. We have proposed a method of combining orthogonally processed HiLo images from two planes: one oriented as in conventional DSLM imaging (XY) and one perpendicular to it (YZ). We have shown quantitatively that 3D HiLo provides better image contrast than both conventional HiLo and 3Phase, without removing significant object features together with the background. We have also compared the different techniques in terms of local sample structure, e.g. discrete spatially confined objects such as nuclei versus heterogeneous extended low-frequency objects such as membranes.
Much work remains to be done in optimizing the programs of illumination. Here we presented only a simple sinusoidal raster scan, whereas obviously one could try alternate schemes of illumination, especially if object priors can be taken into account to design illumination programs tailored to the specific object or object class. Moreover, our simple linear convex method of combining the orthogonal HiLo images XY and YZ into a single 3D HiLo image can be extended to more sophisticated nonlinear or iterative methods, also taking object priors into account.
This research was supported by the National Research Foundation, Singapore through the Singapore MIT Alliance for Research and Technology's BioSystems and Micromechanics Inter-Disciplinary Research programme (015824-039) and Center for BioImaging Sciences, NUS (R-154-000-413-720), and the U.S. National Institutes of Health through grant (1R01CA134424-01A1).
References and links
2. J. B. Pawley, Handbook of Confocal Microscopy (Plenum: New York, 1995).
3. O. E. Olarte, J. Licea-Rodriguez, J. A. Palero, E. J. Gualda, D. Artigas, J. Mayer, J. Swoger, J. Sharpe, I. Rocha-Mendoza, R. Rangel-Rojo, and P. Loza-Alvarez, “Image formation by linear and nonlinear digital scanned light-sheet fluorescence microscopy with Gaussian and Bessel beam profiles,” Biomed. Opt. Express 3(7), 1492–1505 (2012). [CrossRef] [PubMed]
4. A. H. Voie, D. H. Burns, and F. A. Spelman, “Orthogonal-plane fluorescence optical sectioning: three-dimensional imaging of macroscopic biological specimens,” J. Microsc. 170(3), 229–236 (1993). [CrossRef] [PubMed]
5. D. McLachlan Jr., “Extreme focal depth in microscopy,” Appl. Opt. 3(9), 1009 (1964). [CrossRef]
6. R. R. Schram, “Light-scanning photomicrography,” Microscope 29, 13–17 (1981).
7. P. J. Keller, A. D. Schmidt, J. Wittbrodt, and E. H. K. Stelzer, “Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy,” Science 322(5904), 1065–1069 (2008). [CrossRef] [PubMed]
8. P. J. Keller, A. D. Schmidt, A. Santella, K. Khairy, Z. Bao, J. Wittbrodt, and E. H. K. Stelzer, “Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy,” Nat. Methods 7(8), 637–642 (2010). [CrossRef] [PubMed]
9. G. Best, R. Amberger, D. Baddeley, T. Ach, S. Dithmar, R. Heintzmann, and C. Cremer, “Structured illumination microscopy of autofluorescent aggregations in human tissue,” Micron 42(4), 330–335 (2011). [CrossRef] [PubMed]
10. L. Gao, N. Bedard, N. Hagen, R. T. Kester, and T. S. Tkaczyk, “Depth-resolved image mapping spectrometer (IMS) with structured illumination,” Opt. Express 19(18), 17439–17452 (2011). [CrossRef] [PubMed]
14. M. A. A. Neil, A. Squire, R. Juskaitis, R. Juskaitis, P. I. Bastiaens, and T. Wilson. “Wide-field optically sectioning fluorescence microscopy with laser illumination,” J. Microsc.-Oxford 197, 1–4 (2000).
17. T. J. Schröter, S. B. Johnson, K. John, and P. A. Santi, “Scanning thin-sheet laser imaging microscopy (sTSLIM) with structured illumination and HiLo background rejection,” Biomed. Opt. Express 3(1), 170–177 (2012). [CrossRef] [PubMed]