Traditional optical projection tomography (OPT) acquires a single image at each rotation angle, thereby suffering from limitations in CCD dynamic range; this conventional usage cannot resolve features in samples with highly heterogeneous absorption, such as in small animals with organs of varying size. We present a novel technique, applying multiple-exposure high dynamic range (HDR) imaging to OPT, and demonstrate its ability to resolve fine details in zebrafish embryos, without complicated chemical clearing. We implement the tomographic reconstruction algorithm on the GPU, yielding a performance increase of two orders of magnitude. These features give our method potential application in high-throughput, high-resolution in vivo 3D imaging.
© 2012 OSA
Tomography is a long-used technique to reconstruct the 3D structure of objects based on a series of 2D images. In projection tomography, developed for X-rays [1–3], a sample is illuminated from behind and rotated, so that the transmission at every angle is recorded, and forms the basis for 3D reconstruction. More recently, this technique has been developed with visible light, called optical projection tomography (OPT) , and is particularly well suited for reconstructing absorptive/fluorescent specimens with intermediate sizes [5–8]. In conventional usage, a single image is captured at each angle, with fixed camera exposure time. As a result, samples with highly heterogeneous absorption can be difficult to image. If exposure is set to observe features in transparent areas, the more opaque ones may appear completely solid; similarly, if the exposure is increased to image the less transparent regions, the more transparent ones can be washed out. Consequently, resolving the full 3D internal structure of many objects is difficult to impossible with conventional OPT.
Visualizing 3D structure is particularly important in morphogenesis and developmental genetics. In these fields, zebrafish play an important role as model organisms, from fertilized embryos [9, 10], through 1–3 days post fertilization (dpf) embryos, and eventually to adults with fully developed organs [11–13]. Early-stage development has been thoroughly investigated with a variety of optical microscopy techniques, including differential interference contrast (DIC), confocal, and single plane illumination microscopy (SPIM) [12, 13], techniques best suited for small samples up to a fraction of a millimeter. In particular, confocal microscopy can give a real-time 3D view of the evolution of fluorescently-tagged samples [14, 15]. However, for larger samples, such as macroscopic embryos and adults spanning several millimeters in size, microscopy is less convenient. Instead, tomographic approaches have been used more recently for these larger organisms, including mesoscopic fluorescence tomography and selective plane photoacoustic tomography [16,17]. Several techniques have been developed recently to enhance OPT imaging of developing organisms. “Flow-OPT” generates flow contrast images from many static images to extract the moving components ; this label-free method uniquely dissects and highlights the circulatory system in vivo, generating highly-resolved images with good contrast, but is limited to structures containing traceable movement. Fluorescence lifetime optical projection tomography (FLIM-OPT)  allows 3D reconstruction of fluorescently-labeled living organisms, such as zebrafish embryo; however, resolution is not high enough to finely resolve details of the organs, and the requirement of fluorescent labeling restricts the method to samples with ultra-low scatting and attenuation.
Tomography to reconstruct the full 3D morphology of live embryos has several challenges. Thick tissue can scatter light strongly, and labeled specimens can be too opaque due to strong absorption. Scattering can be reduced by first dehydrating tissue in ethanol, then clearing with BABB or Murray’s solution (a mixture of benzyl alcohol and benzyl benzoate in a 2:1 ratio), which matches the refractive index of the tissue; this method has been applied to zebrafish larvae for tomographic reconstruction . However the clearing process is time consuming, does not work with complete reliability, and entirely precludes imaging live organisms. What is needed is a method for 3D in vivo imaging of organisms that allows fine resolution of structures deep within highly heterogeneous tissue.
In this paper, we present a new method combining optical projection tomography (OPT) with high dynamic-range (HDR) imaging, a widely-used technique developed to preserve more complete information from the lightest and the darkest areas of an image, when the dynamic-range may exceed that able to be captured in a single exposure. We collect images at different rotation angles, as in standard OPT; however, at each angle, we collect a series of images with different exposures [21, 22], and combine them into a single HDR image, which forms the input 2D data for tomographic reconstruction. We accelerate the reconstruction by performing the calculation on the GPU, yielding an increase relative to CPU-based calculations of two orders of magnitude. We demonstrate the ability of HDR-OPT to resolve fine details deep inside the organs of 1–3 dpf zebrafish embryos, which in all cases exceed the quality, contrast and resolution of conventional OPT by several fold, without the need for any clearing—thereby opening up the possibility of in vivo 3D imaging of whole organisms.
2. Materials and methods
2.1. Staining of blood vessels
We select 3 dpf wild type and scotch tape zebrafish embryos for vessel staining. We anaesthetize the embryos in 0.02% tricaine, then fix them overnight in 4% paraformaldehyde at 4°C and stain them for endogenous alkaline phosphatase activity. We wash the embryos three times in phosphate buffered saline (PBS) and dehydrate them by immersing in 25, 50, 75 and 100% methanol in PBST (PBS + 0.1% tween-20) to permeabilize. Finally we rehydrate the embryos stepwise to 100% PBST. For staining, we equilibrate embryos in NTMT buffer (0.1 M Tris-HCl pH 9.5; 50 mM MgCl; 0.1 M NaCl; 0.1% tween 20) at room temperature for 5 min and repeat the equilibration step twice. We add 4.5 l of 75 mg/mL NBT and 3.5 l of 50 mg/mL BCIP per mL once the embryos are equilibrated in NTMT, After staining for 10 min, all the blood vessels in the fish embryo were labeled. We then add PBST to stop the staining reaction, and immerse the embryos in a solution of 5% formamide and 10% hydrogen peroxide in PBS for 20 min to remove endogenous melanin in the pigment cells and allow full visualization of the stained vessels (Fig. 1(b)). We store the embryos in PBST at 4°C refrigerator and cover with foil to protect from stray light.
2.2. Staining of cartilage
We select 5 dpf or older zebrafish embryos without pigments, fix them overnight in 4% paraformaldehyde at 4°C, and maintain in 100% methanol at −20°C until use. We wash the embryos several times in PBS and then transfer them into an Alcian blue solution (1% concentrated hydrochloric acid, 70% ethanol, 0.05% Alcian blue). After staining the embryos for 0.5–2 h, we rinse them 3 or 4 times with 1–1.5 mL acidic ethanol (5% concentrated hydrochloric acid, 70% ethanol, HCl-EtOH) and immerse them in it for around 20 minutes. We then rehydrate embryos by immersion in a series of 1.5 mL 75/25, 50/50, 25/75, 0/100 HCl-EtOH/H2O solutions for a total of 40 min. Finally, we digest the stained embryos in a 1% pancreatic enzymes solution to make their bodies translucent, and store in PBST at 4°C.
2.3. Embryo preparation for imaging
We embed each stained embryonic sample into a 0.4–0.6% agarose gel, which is packed in a circular glass tube. We chose glass tubes with thin walls (1 mm) and a large diameter (8 mm), which minimizes image distortion caused by the index mismatch at the water-glass and glass-agarose interfaces. During rotating process, we immerse the tube in a water solution to match the refractive indices. With this configuration, we observere that any refractive-index difference between glass and water/agrose is not sufficiently big to cause observable image distortion.
2.4. BABB clearing
We dehydrate the agarose gel containing the stained embryo by sequentially immersing the gel into 20, 40, 60, 80 and 100% ethanol at room temperature, for 2 hours per step. We then immerse the agarose gel with the stained embryo into BABB solution (1:2 Benzyl alcohol, Benzyl benzoate) for 24–48 h. After the cellular fluids inside the cell membranes are completely replaced by BABB solution, the refractive index matches the cell membranes well. This makes the embryos transpraent, and removes the effect of internal light scattering.
3. The HDR-OPT system
3.1. HDR-OPT setup
We show the experimental setup in Fig. 1(d). We illuminate with a commercial white light-source composed of a 5×6 LED lighting array and a slab diffuser (WorldView, Beijing, China), which provides uniform illumination over the entire sample. We create a precise, high-speed rotation stage by combining a 5-axis translation stage with a stepper motor triggered by a function generator (Tektronix 3020), which allows us to rotate the fixed zebrafish embryo along its vertical axis with high accuracy. We precisely control the axis and position of the projected image by displacing the camera and lens with a separate two-axis tilt platform and three-axis linear stage. We collect the light transmitted through the sample using a 1/2” CMOS monochrome camera (EO-5012M, Edmund Optics, New Jersey, USA) with a high density 2560 × 1920 imaging array and a 2X telecentric lens (REV 02, Edmund Optics, New Jersey, USA), which creates a geometrically correct projection of the transmission in a plane perpendicular to its optical axis. In contrast to a normal lens, where closer objects appear larger, in a telecentric lens, two equally-sized objects appear the same size irrespective of their distance from the lens. For each tomographic reconstruction, we collect 800 images each separated by a 0.45° rotational angle around the vertical axis. For the stained fish shown in Fig. 2(a), we collect nine sets of images at a resolution of 2560 × 1920 pixels and a frame rate at 4 fps for short-exposure shots, or 2 fps for long-exposure ones. The entire data set requires 36.7 minutes to acquire, which can be reduced by reducing the image resolution.
3.2. Image acquisition
For our vessel-stained fishes, the “normal” exposure time using our setup is 30 ms, and we set a stepsize of about 0.5 EV to adjust the exposure time. Two sets of both underexposed and overexposed images are the minimum requirement for HDR composition. We select 10 ms as the shortest exposure time to visualize the most transparent parts of the sample, and nine progressively greater exposure times to resolve the more opaque parts of fish body, which has highly heterogeneous optical absorption (Fig. 2(a)). Other samples may require different combination of exposure values to achieve the optimal results. We also vary between five and twelve different integration times, with different samples, with similarly positive results. Using nine exposures and 800 images per exposure value, as in Fig. 2, the total integration time of a whole data set is 768 seconds. Following data collection, we transfer images from the camera to a RAID 0 array of two 7200 RPM SATA hard drives (Western Digital RE4), then to a separate PC for analysis via gigabit ethernet. To maximize camera frame rate, we collect images in video mode, and adjusted the motor speed to synchronize the rotation with the camera acquisition frame rate. We store the data as one M-JPEG AVI file with a quality setting of 100. File sizes vary from hundreds of megabytes to gigabytes.
3.3. HDR algorithm
We extract AVI files to series of images using VirtualDub. To account for minor non-circular motion and tilt of rotation axis, we first select the group of images acquired with normal exposure to measure the actual orbit of the axis. We split the scans of K degrees and (180-K) degrees into 64 stripes along the rotation axis, then reverse the corresponding part of the (180-K) degrees scan around different centers of rotation and check its overlap with the K degrees stripe. We choose the rotation center with the maximum overlap as the real rotation center. We apply the geometric correction of the normal exposure group to all the other groups of images with different exposure. We also horizontally flip the scans of (180-K) degrees around the corrected center of rotation and add them to the scans of K degrees to optimize for data processing, yielding 400 final images at each exposure (Fig. 2(b), line 1).
At each angle, we first remove the overexposed parts of images corresponding to pixels with an intensity greater than a threshold value (Fig. 2(b), line 2), selected manually to avoid our camera’s nonlinear-response close to saturation (this step may not be needed in other systems where the camera’s response is known to be entirely linear). We then subtract from each image the minor exposed images, and divide the result by the difference in exposure times to generate images with proportional intensity (Fig. 2(b), line 3).
We composite different parts of the images to generate an HDR image. For example, we use all the properly-exposed parts of the image with 300 ms exposure time; in the image with 200 ms exposure time, we only use those normally exposed parts which correspond to overexposed areas in the images with 300 ms exposure time. We select the available parts of images with other exposure times to make the best use of information in the image (Fig. 2(b), line 4). By combining those images, we generate a 16-bit HDR image. For those pixels whose intensity is close to saturation in image group n+1, but low in image group n-1, the result may suffer from non-linearity (close to saturation) and noise (low intensity). To reduce this impact, we choose different threshold limit values to generate several HDR images and take the average of them as the final result. This method reduces noise and is computationally extremely simple and fast; another possibility that might achieve even lower noise involves weighting each pixel by a function that depends on intensity, but this involves substantially greater computation. We implemented our script for image adjustment and HDR image generation in MATLAB, with specific functions written in C to improve performance. We vary the transfer function between 16-bit HDR and standard 8-bit images, which allows us to highlight selectively regions of the sample with different transparency, as shown in Fig. 3.
3.4. GPU accelerated OPT reconstruction
We calculate the 2D slices from HDR images of different aspects using a filtered back-projection algorithm with a Hamming-Filter implemented together with an inverse Radon-transform. The reconstruction script was written in MATLAB, with specific functions for filtered projection and backprojection written in NVIDIA’s CUDA-C and executed on a GPU. During filtered projection, we optimize the algorithm by using shared memory of GPU streaming multiprocessors; in backprojection, the data is too large to be stored in shared memory, and instead we use texture memory to accelerate data transfer from global memory.
We test the single-precision performance of OPT reconstruction of several GPUs and CPUs, including dual NVIDIA GeForce GTX295 (4×GT200b GPU, 240 cores per GPU running at 1242 MHz, 112 GB/s memory bandwidth per GPU), NVIDIA Tesla C2050 (GF100 GPU, Fermi architecture, 448 cores running at 1150 MHz, 144 GB/s memory bandwidth) and Intel Core i7 (quad-core at 3GHz, 38.4 GB/s memory bandwidth). For the dual GTX295 setup, we divide the data into 4 parts and launch the reconstruction on 4 GPUs controlled by 4 host threads. Table 1 and Table 2 show the comparison of reconstruction speed between GPUs and CPUs. Overall, we see performance increase by more than two orders of magnitude when executed on the GPU.
3.5. 3D visualization
We save the section images in 16 bit grayscale tiff format. We reconstruct the 3D volume by sequentially stacking the 2D slices and viewing with Amira (Visage Imaging, Austria).
4. Results and discussion
For a cartilage-stained embryo with very low scattering and absorption, conventional OPT is sufficient, as most of the sample area is transparent and already transmits clearly (Fig. 1(b)). However, other types of staining lead to significant dye accumulation that prevents even transmission throughout the embryo. Staining the blood vessels, for example, leads to higher absorption and scattering in the anterior region of the embryo, relative to the posterior, due to a high concentration of blood vessels in the head. We clear the sample with BABB to better match the refractive indices of sample and surrounding fluid, which decreases scattering but does not reduce the effects of the strong absorption. Consequently, there is no way to image the entire structure (the CCD has insufficient dynamic range) with a single 2D image with one exposure time, precluding a correct 3D reconstruction with conventional OPT (Fig. 1(b)).
Instead, we combine images from the same position collected with different exposure times; each individual image is properly exposed for a certain set of features in a particular range of transparency. By combining these images, we composite the properly-exposed sample regions from each image, yielding an image where all parts of the sample are properly exposed. At each position, we collect nine images with exposure times ranging from 10 to 300 ms (Fig. 2(a)). The shortest exposure (10 ms) captures details in the most transparent parts of the sample; as exposure time increases, details of progressively thicker sample portions become properly exposed. We align all original images and remove any overexposed areas, then subtract from each image the minimum exposed image, and divide the result by the difference in exposure times; this yields images with proportional intensity. Combining these images yields an HDR image, which forms the input data for tomographic reconstruction (Fig. 2(b)).
We apply 5 different transfer functions to slices to emphasize different region of the specimen (Fig. 3(a)). The linear HDR keeps all the information in the image and shows all the details. HDR curve 1 loses most of the details in the head while HDR curves 2, 3 and 4 show more details in the head with a stronger contrast. By adjusting the transfer functions, we obtain an optimal visualization of the tomographic reconstruction. We select two orthographic projection planes inside a 3 dpf, blood vessel-stained zebrafish embryo (anterior part and trunk part; Fig. 3(b)) as regions of interest (ROI) and select the transfer function that preserves the most details. Compared with the linear HDR curve and HDR curve 1, HDR curves 3, 4 and 5 result in a reconstruction that far more clearly shows the eye balls and pharyngeal arches (Fig. 3(c)). Furthermore, we visualize the structure of a corner of the imaging volume, which selectively highlights the intersegment vessels (ISVs), arteries and veins (Fig. 3(d)).
Using both traditional OPT and HDR-OPT with a suitable transfer function, we reconstruct the full 3D structure of a 3 dpf blood vessel-stained zebrafish embryo, and choose five planes along which to slice the 3D data set. Planes 1, 2 and 3 represent sagittal and coronal planes sliced from the embryo’s anterior, posterior and trunk parts, respectively; slices in the transverse plane of the tail and mutant heart are shown in planes 4 and 5; the positions of all planes are shown in Fig. 4(a). For traditional OPT, we select different single exposure times, and show the results in Fig. 4(b). In all cases, choosing a single exposure value (ev) precludes clear observation of all features of the sample; that is, if the exposure is long enough to visualize the transparent parts of the head (+2 ev, plane 5), then the image of the tail is entirely saturated and no structure can be seen (+2 ev, plane 1). By contrast, when the exposure is short enough to capture the details of the tail (−3 ev, plane 1), then the head is entirely opaque, and only the outline is resolved (+2 ev, plane 5). In no case, even when using a higher bit-depth for collection, can a single exposure time allow resolution of all features in the embryo. Clearing the embryo with BABB decreases scattering but leaves dye absorption unchanged, somewhat improving contrast for some features (Fig. 4(b)), but the overall contrast limitation imposed by a single exposure time remains.
By contrast, the HDR combination of frames with multiple exposure times overcomes these limitations. The HDR-OPT images, performed on a sample without BABB clearing, show good contrast and resolved details in all parts of the embryo, giving a far more accurate picture of the internal structure (Fig. 4(b)). We highlight areas of particular interest with colored arrows, whose contrast are greatly improved, including labeled ISVs (yellow arrow, planes 1 and 3), pharyngeal arch area (purple arrow, plane 5), notochord and veins (red and cyan arrows, plane 2). We also compare the visualization of the embryos with thick slices, approximately 10 um deep, taken from the 3D volumes, in Fig. 5. Regions marked with red, green and blue rectangles show details of the embryos trunk and posterior section, including the ventral artery and ISVs. As in the previous case of 2D planar slices (Fig. 4), the HDR-OPT reconstruction shows clear and sharp detail in all parts of the embryo, no matter what the thickness. As before, the single-exposure conventional OPT entirely fails to show features in all parts of the image, whereas HDR-OPT offers the highest micron-scale resolving power and shows the most complete details.
The ability for HDR-OPT to resolve structures in all parts of the embryo may allow us to visualize morphological differences between different genetic variants. To investigate this possibility, we compare the OPT and HDR-OPT reconstructions of normal wild-type zebrafish, and of the scotch tape variant, a mutant with a narrow blood vessel pattern. Our 3D reconstructions (Fig. 6) show the difference between these two types clearly; we highlight the dosal aorta (DA) with green arrows and posterior cardinal vein (PCV) with red arrows, in both transverse and sagittal views, in Fig. 6. The difference between the wild type and scotch tape mutants, as viewed from the conventional OPT structures, is minor; by contrast, the difference between wild-type and scotch tape mutant embryos in the HDR-OPT reconstruction is dramatic. The improvement in contrast with HDR-OPT allows the clear observation of the narrow, irregular artery shapes in the scotch tape mutant, which is not apparent with conventional OPT, as shown in Fig. 6(b).
We develop and present a novel HDR-OPT technique that does not require specialized sample preparation, allows observation of finely-resolved features, and performs at high speed as a result of GPU acceleration. We are able to observe these features in millimeter-scale samples with widely-varying thicknesses, which is not possible with the single-exposure time in conventional OPT. We achieve far better imaging results without time-consuming dehydration and clearing, making possible in vivo imaging. One direction for future research would be to use multiple low-cost cameras without sample rotation, which opens up the possibility for true real-time tomography, of great potential interest to embryonic anatomy and developmental biology.
This work was funded by the National Natural Science Foundation of China (Grant No. 20890020 and 90913011), the Ministry of Science and Technology of China (Grant No. 2011CB809106 and 2010CB529503), the Fok Ying Tung Education Foundation, and the NVIDIA professor partnership program.
References and links
2. D. Kim, S. Park, J. H. Lee, Y. Y. Jeong, and S. Jon, “Antibiofouling polymer-coated gold nanoparticles as a contrast agent for in vivo X-ray computed tomography imaging,” J. Am. Chem. Soc. 129,7661–7665 (2007). [CrossRef] [PubMed]
4. J. Sharpe, U. Ahlgren, P. Perry, B. Hill, A. Ross, J. Hecksher-Sorensen, R. Baldock, and D. Davidson, “Optical projection tomography as a tool for 3D microscopy and gene expression studies,” Science 296, 541–545 (2002). [CrossRef] [PubMed]
5. T. Alanentalo, A. Asayesh, H. Morrison, C. E. Lorén, D. Holmberg, J. Sharpe, and U. Ahlgren, “Tomographic molecular imaging and 3D quantification within adult mouse organs,” Nat. Meth. 4, 31–33 (2006). [CrossRef]
6. M. J. Boot, C. H. Westerberg, J. Sanz-Ezquerro, J. Cotterell, R. Schweitzer, M. Torres, and J. Sharpe, “In vitro whole-organ imaging: 4D quantification of growing mouse limb buds,” Nat. Meth. 5, 609–612 (2008). [CrossRef]
7. C. Vinegoni, L. Fexon, P. F. Feruglio, M. Pivovarov, J. L. Figueiredo, M. Nahrendorf, A. Pozzo, A. Sbarbati, and R. Weissleder, “High throughput transmission optical projection tomography using low cost graphics processing unit,” Opt. Express 17, 22320–22332 (2009). [CrossRef]
9. B. B. Kirby, N. Takada, A. J. Latimer, J. Shin, T. J. Carney, R. N. Kelsh, and B. Appel, “In vivo time-lapse imaging shows dynamic oligodendrocyte progenitor behavior during zebrafish development,” Nat. Neurosci. 9, 1506–1511 (2006). [CrossRef] [PubMed]
11. B. D. Wilson, M. Ii, K. W. Park, A. Suli, L. K. Sorensen, F. Larrieu-Lahargue, L. D. Urness, W. Suh, J. Asai, G. A. H. Kock, T. Thorne, M. Silver, K. R. Thomas, C. B. Chien, D. W. Losordo, and D. Y. Li, “Netrins promote developmental and therapeutic angiogenesis,” Science 313, 640–644 (2006). [CrossRef] [PubMed]
13. C. Jopling, E. Sleep, M. Raya, M. Marti, A. Raya, and J. C. I. Belmonte, “Zebrafish heart regeneration occurs by cardiomyocyte dedifferentiation and proliferation,” Nature 464, 606–609 (2010). [CrossRef] [PubMed]
16. C. Vinegoni, C. Pitsouli, D. Razansky, N. Perrimon, and V. Ntziachristos, “In vivo imaging of Drosophila melanogaster pupae with mesoscopic fluorescence tomography,” Nat. Methods 5, 45–47 (2008). [CrossRef]
17. D. Razansky, M. Distel, C. Vinegoni, R. Ma, N. Perrimon, R. W. Koster, and V. Ntziachristos, “Multispectral opto-acoustic tomography of deep-seated fluorescent proteins in vivo,” Nat. Photonics 3, 412–417 (2009). [CrossRef]
18. A. Bassi, L. Fieramonti, C. D’Andrea, M. Mione, and G. Valentini, “In vivo label-free three-dimensional imaging of zebrafish vasculature with optical projection tomography,” J. Biomed. Opt. 16, 100502 (2011). [CrossRef] [PubMed]
19. J. McGinty, H. B. Taylor, L. Chen, L. Bugeon, J. R. Lamb, M. J. Dallman, and P. M. W. French, “In vivo fluorescence lifetime optical projection tomography,” Biomed. Opt. Express 2, 1340–1350 (2011). [CrossRef] [PubMed]
20. R. J. Bryson-Richardson and P. D. Currie, “Optical projection tomography for spatio-temporal analysis in the zebrafish,” in Methods in Cell BiologyM. W. H. William Detrich Iii and I. Z. Leonard, eds. (Academic, 2004), pp. 37–50. [CrossRef] [PubMed]
21. E. Reinhard, G. Ward, S. Pattanaik, P. Debevec, G. Ward, and K. Myszkowski, High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting (Morgan Kaufmann, 2005).
22. C. Bloch, The HDRI Handbook: High Dynamic Range Imaging for Photographers and CG Artists (Rocky Nook, 2007).