Open Access
Add to BibSonomyAbstract
Computational image restoration finds wide applicability for fluorescence intensity imaging. Relatively little work in this regard has been performed on FLIM images, which also suffer from diminished spatial resolution. In this work, we report two separate approaches to enhance FLIM image quality while maintaining lifetime accuracy. A 2D-image restoration algorithm was employed to improve resolution in gated intensity images of various samples including fluorescent beads, living cells and fixed tissue samples. The restoration approach improved lifetime image quality without significant variation in lifetime. Further, overlaying a restored-intensity image over the native lifetime image provided even better results, where the resulting lifetime map had spatial features similar to the intensity map. 2D and 3D image restoration also benefit from advances in computational power and hence holds potential for enhancing FLIM resolution, particularly in ICCD-based wide-field FLIM systems, without sacrificing vital quantitative information.
©2008 Optical Society of America
1. Introduction
Fluorescence lifetime is defined as the rate constant associated with an exponentially decaying fluorophore population [1]. Given the independence of fluorescence lifetimes from intensity associated artifacts such as concentration, scattering and absorption, FLIM is uniquely applicable for high resolution studies of biological systems and their milieu. Lifetime studies have hence been used to study physiological parameters such as (but not limited to) membrane dynamics, metabolism, oxygenation and molecular associations [2-5].
While theoretically independent of intensity, several fluorescence lifetime imaging microscopy (FLIM) techniques derive lifetime images via intensity image analysis [6]. As a result, quality of the lifetime map is directly affected by the acquisition of intensity images. It is commonly known that the imaging properties of any optical microscope give rise to distortions [7].
FLIM is increasingly applicable for studies of intrinsic fluorescence, with an eye on clinical applications. Naturally-occurring fluorophores such as NADH, collagen, keratin, flavins, porphyrins, etc. provide promise for non-invasive analysis of living cells and tissues. A key issue is that the low quantum yield of such fluorophores, coupled with the use of low energy and pulsed laser sources, results in weak, barely detectable signals [8]. This necessitates the use of signal amplification techniques, such as Intensified-CCD cameras, that are a key component of several wide-field FLIM systems [9-13]. The resulting images, however, suffer from haze and corresponding loss of spatial resolution. Figure 1 illustrates the effect of an intensifier on a high SNR biological image with almost no background, yet significant haziness or ‘lateral smearing’ is observed. The occurrence of haze due to image intensifiers is explained by several reasons, including low gating voltages for ultrafast intensifiers and degradation by the phosphor screen [14].
Fig. 1. Illustration of lateral smearing, or haze, with an image-intensified CCD camera. (a) Blue fluorescence from a fixed mouse intestine section imaged with a CCD alone. (b) Same region when imaged with an ICCD. The demagnification due to the lens-coupling between the intensifier and CCD (=2.17) is evident in the image. (c) Red rectangular region from (b) magnified to show the smearing effect. The excitation source for all images was a mercury lamp.
While haze can be qualified as unwanted signal or noise, it is inherently more difficult to remove while maintaining the quantitative relationships within the intensity image (for accurate lifetime estimation). Further, since the source of haze is the signal itself, it has a near-identical lifetime and is virtually indistinguishable in lifetime images. This leads to lateral smearing in lifetime maps as well, where the footprint of the fluorescent structure is larger in the lifetime image as compared to intensity images, leading to a loss of resolution.
Limited work has been performed towards computational image restoration in FLIM till date. Among optical methods, structured illumination has been used to improve spatial resolution of intensity and lifetime maps based on the Moiré effect, albeit at the loss of SNR, which is an issue when working with dim fluorophores [15]. Squire et al. illustrated the possibility of reconstructing 3D lifetime images from a frequency-domain FLIM system via the established Iterative Constrained Tikhonov-Miller (ICTM) algorithm [16]. Intensity-based image restoration, on the other hand, has seen extensive investment, both research and commercial. Approaches include prior knowledge of the object, known or estimated noise characteristics, known or estimated PSF, constrained methods, etc [17-21].
In this work, a commercial version of an iterative constrained 2D blind image restoration algorithm was applied to FLIM intensity images, and the effect on lifetime images and values were observed. A new approach to fluorescence lifetime representation was proposed. Utilization of current intensity-based image restoration techniques to FLIM provides promise for more lifetime-specific restoration algorithms in the future. With advances in desktop computing power, numerical methods provide increasingly better resolution without compromising SNR (sometimes improving it) or the need for additional optical hardware, and are readily applicable to previously acquired data as well.
For experimental purposes, a wide-field, time-domain fluorescence lifetime imaging microcopy (FLIM) system was utilized to probe fluorescence in microspheres, living cells and fixed tissues. The large temporal dynamic range (600 ps – infinity) of this system is unique and provides flexible application: UV excitation and an ultrafast gated camera enable imaging of sub-nanosecond lifetimes, while a low repetition rate laser source and large gates enable measuring lifetimes for long-lived dyes with lifetimes that are typically hundreds of nanoseconds.
2. Materials and methods
2.1 Sample Preparation
Fluorescent Microspheres – Yellow-green fluorescent microspheres were purchased in three sizes of 1 µm, 3 µm, 10 µm diameter (Polysciences, Warrington, PA) and each was diluted in PBS to a concentration of approximately 104 spheres/ml. 20 µl of each were dried between a glass slide and coverslip for imaging. All YG spheres had an excitation/emission maximum of 441/486nm as specified by the manufacturer.
RTDP stained cancer cells – RTDP (460/600 nm) incubation and temperature-controlled imaging of living Barrett’s adenocarcinoma columnar cells (SEGs) has been described previously [5].
Fixed Mouse Intestine section – A fixed cryostat mouse intestine section of approximately 16 µm thickness was purchased (Molecular Probes, Carlsbad, CA). The slide is multiply-stained; Alexa Fluor 350 wheat germ agglutinin (346/442 nm), a blue-fluorescent lectin, was used to stain the mucus of goblet cells. The filamentous actin prevalent in the brush border was stained with red-fluorescent Alexa Fluor 568 phalloidin (578/600). Finally, the nuclei were stained with SYTOX Green nucleic acid stain (504/523).
Fig. 2. FLIM Setup. Abbreviations: CCD=charge-coupled device; HRI=high rate imager; INT=intensifier; TTL I/O=TTL input/output card; OD=optical discriminator; BS=beam splitter; DC=dichroic mirror; FM=“flippable” mirror; L1, L2, L3, L4=quartz lenses; M=mirror. Thick solid lines=light path; thin solid line=electronic path.
2.2 Fluorescence Lifetime Imaging Microscopy (FLIM)
A schematic of the FLIM is shown in Fig. 2. Briefly, FLIM is configurable to use either the nitrogen laser (GL-3300, PTI, Lawrenceville, NJ) output for UV excitation at 337.1 nm or a tunable dye laser (GL-301, PTI) for excitation in the vis-NIR range (337-960 nm), depending on the setting(s) of the flippable mirror(s) (FM). Pulse energy from the nitrogen laser was 1.3 mJ (±2%) at the output and 60 µJ at the sample. Lifetime measurements were possible in the range 750 ps – 1 µs with a temporal resolution of 50 ps. A complete description of FLIM operation, including image acquisition and analysis, can be found elsewhere [11]. FLIM is capable of rapid image acquisition and lifetime analysis times (<15 seconds).
2.3 Image restoration
Intensity image restoration of each gated image was carried out by the Autoquant (Media Cybernetics, Bethesda, MD) software. The underlying Maximum Likelihood Estimation (MLE) algorithm, a mathematical optimization strategy that is generally used for producing estimates of quantities corrupted by some form of random noise, forms the basis of the 2D Blind approach and has been explained elsewhere [22, 23]. Briely, image restoration and/or deconvolution form a set of processes used to reverse the optical distortion that takes place in a microscope. It is usually assumed that the original image is distorted by an instrument response defined by the point spread function (PSF) of the imaging system. The PSF is interpreted as the image of an infinitely small point source, and can be estimated by either a) imaging a fluorescent bead of smaller size than the resolution of the microscope, or b) theoretically estimating the PSF by known parameters of the imaging system, the objective lens in particular. The latter method is known as the blind approach. Blind image restoration hence attempts to recover the original image by deconvolving the calculated PSF from the observed image.
Lifetime Image Restoration-For each image series (i.e. 4 gated intensity images), the first gated image was analyzed with the blind/derived PSF setting. The output was the restored image, as well as a PSF file. The PSF file was saved and used to analyze the next 3 gated images (i.e., non-blind). Each image was restored with 10 iterations and medium-high noise specification. The computationally-restored images were then analyzed for lifetime. Approximate restoration time for each image was 1 minute, or 4 minutes for four gated intensity images per lifetime image.
The optical parameters specified for each image are listed in Table 1. Note that the pixel spacing parameter accounted for demagnification due to the image intensifier:
Table 1. Parameter Setting for Image Restoration
Intensity-overlay Restoration-For this approach, the native lifetime map was generated, as was the restored intensity map (1st gate). The lifetime map was color coded (RGB), while the intensity map was in grayscale. Adobe Photoshop was used to combine the two images via multiplication to yield a single intensity-overlay lifetime map. Since only one image was restored, restoration time was 1 minute. The overlay procedure took minimal (<1s) processing time after the files were imported to Adobe Photoshop.
3. Results
3.1 Lifetime Image Restoration
Development of computational methods that preserve quantitative aspects of images hold promise for lifetime studies. The 2D Blind algorithm is constrained to preserve total photon counts during computational restoration (personal communication). This is especially useful for analyzing ICCD-generated images, where the haze, instead of being removed, is reassigned to the (apparent) source of origin. Hence, not only is the haze in the image greatly reduced, but the SNR of the fluorescent source is increased.
Figure 3 illustrates the effect of restoration on both intensity and lifetime images on 3 micron-diameter YG fluorescent microspheres. Overlap of haze from the 2 spheres seen in N(I) leads to a lifetime map where one is indistinguishable from the other in N(τ). As expected, the restored intensity map R(I) has improved SNR, spheres are brighter. The reduction in haze is evident in the lifetime map R(τ), where the 2 spheres are now clearly distinguishable. Important as well, there is only a 3% change in lifetime (from 1.65 to 1.60 ns). It is also worthy to note that the sphere diameter in the native intensity image is 4.81 µm, while it is 3.35 µm in the restored intensity image and a more accurate estimate of the actual sphere size.
3.2 Intensity-overlay Image Restoration
Both fluorophore intensity and lifetime provide complementary information; while lifetime is indicative of the milieu of the fluorescent molecule, intensity provides concentration, and sometimes morphological information. It has previously been suggested that whether a given pixel contributes to the information in an image should be weighted by both its lifetime and intensity content [24]. Such an approach attenuates haze to a great extent due to its low SNR, even though haze has virtually the same lifetime as the source. It is now possible to use advances in image restoration to significantly improve the quality of such weighted intensity-lifetime maps. By simply restoring the first gated image (highest SNR) alone and using it to weigh the native lifetime image, we can obtain a highly resolved lifetime distribution image for the fluorescent beads. A key strength of some of the robust intensity-based restoration algorithms (such as 2D Blind) is they seldom create structural artifacts within a given image. By using the native lifetime image, the lifetime/quantitative content has been fully conserved as well. In fact, due to de-emphasis of the edge pixels, the spiky lifetime values (red pixels) are missing from the restored I-τ maps, thereby yielding a more realistic lifetime distribution.
One approach to Intensity-overlay based restoration is illustrated and compared with direct lifetime restoration in Fig. 3 for the same 3-micron YG microsphere sample. The restored intensity image, R(I), can be applied for direct lifetime evaluation to yield R(τ) as described earlier with minimal change of lifetime values. Alternately, R(I) can be used to ‘weight’ the native lifetime image N(τ) to yield the intensity-overlay lifetime image OI(τ). It is worthy to note there is no change in lifetime with this approach.
Fig. 3. N(I)=Native fluorescence intensity image. N(τ)=Native fluorescence lifetime image. R(I)=Restored intensity image. R(τ)=Restored lifetime image. OI(τ)=Intensity-overlay lifetime image.
Figure 4 illustrates the comparative results of applying lifetime restoration and intensity-overlay restoration to fluorescence images of various sized microspheres, living cells and fixed tissue samples. The resolution of 1 µm sphere images is only marginally improved. This is logical, since 1 µm is the limiting resolution of our FLIM system. The star arrangement of 10 µm spheres is significantly deteriorated in N(τ) due to the combined haze from each sphere; The resolution is markedly improved in R(τ), and is not only restored but significantly enhanced in OI(τ). The average measured size of the spheres also improves from 12.48 µm on average to approximately 10.2 µm.
The living cell image provides a more relevant, biological application. The large nuclei associated with the cancer cell is mitigated in N(τ), but more accurate and evident in R(τ) and OI(τ). Applying a previously calculated RTDP lifetime calibration for oxygen sensitivity to this image will not affect the calculated oxygen levels, hence preserving the quantitative nature of the image.
Fig. 4. Native intensity, native lifetime, restored lifetime and intensity-overlay lifetime images for various samples. 1 µm and 10 µm spheres, living cells were imaged with a 100x objective.
The mouse intestine section (Fig. 5) illustrates how the I-τ approach can be applied to study only relevant content in biological systems without sacrificing SNR. The mucus of goblet cells is stained with Alexa Fluor 350, a blue-emitting fluorophore that is excited by UV light. Since most endogenous fluorophores also excite/emit in the UV/blue, a dim background fluorescence from the tissue is also observed. While it is barely detectable in the intensity image, it is clearly evident in the lifetime map and has comparable lifetime to the stained cells, making them difficult to visualize. R(τ) provides better resolution; crypts in the intestine and individual cells along the edges are more easily visible. The OI(τ) map, however, restored the focus on the lifetime of the brightly fluorescent cells.
Fig. 5. Native intensity, native lifetime, restored lifetime and intensity-overlay lifetime images for a fixed mouse intestine section imaged with a 10x objective.
4. Conclusion
Wide-field microscopes are routinely used to collect 2D data sets. Such images contain sharp in-focus features as well as blur due to signal originating above or below the plane of focus. With the use of ultrafast gated image intensifiers for FLIM, further in-plane smearing (haze) is observed, which leads to reduced resolution for intensity and especially for lifetime imaging. To recover true image heterogeneity, it is advantageous to not only eliminate some of the haze, but to be able to reassign it to its source, so as to improve SNR. This article reports results for resolution enhancement via image restoration in wide-field FLIM.
Direct lifetime restoration of gated intensity images is the most straightforward approach for improving resolution of FLIM images. Restoration techniques have been in development for >20 years now and form a mature foundation to base future work on FLIM-specific algorithms. Constrained approaches allow for selective retention of relevant quantitative information in a fluorescence image while improving resolution. Further, blind image restoration provides the convenience of not requiring the PSF of the optical system, but generating a reconstructed estimate instead, also with constraints [19]. PSF measurement via fluorescent beads, as is the most commonly used approach, suffers from potential photobleaching effects and statistical noise due to low SNR [25]. The PSF is also distorted by variations in refractive index in biological samples, which makes computationally-generated, adaptive PSF approaches more relevant. This work indicates that iterative, constrained 2D blind image restoration holds potential for improving spatial resolution in wide-field FLIM by using restored intensity images for evaluation fluorescence lifetimes via the RLD approach. The commercial availability of this algorithm makes it easy to adopt. Image restoration also leads to reduction in noise, an observation explained by the inherent assumption of quantum photon noise in MLE. Maintaining lifetime accuracy as well as improving estimates of spatial structures (fluorescent microspheres in this case) is possible with this approach. Future work in this regard could involve development of FLIM specific constraints that correct some of the observed structural artifacts associated with biological samples.
Fluorescence intensity-weighted lifetime images have been reported previously as a means to provide morphological (intensity) and functional (lifetime) information in the same image. A significant enhancement, however, is afforded by using computationally restored intensity images to balance the native lifetime image. Limited preliminary studies suggested that veracity of lifetime was also maintained when images contained fluorophores with significantly different lifetimes. Expanded studies on this and other systems will form the basis of future work.
It is worth mentioning that the direct CCD image (i.e. similar to Fig.1, with the intensifier removed) could be used to weight the native lifetime image instead. This would, however, require hardware modification that involves removal and reinstallation of the intensifier for each image series; it would also require additional processing to account for differences between the CCD and the ICCD configurations. The benefit of this approach is that not only quantitative algorithms like the 2D blind, but any restoration/deblurring algorithm can be used without sacrificing lifetime information. Note that all the data presented in this work is largely 2D, as is the analysis. We also assume the veracity of all lifetime information obtained. In reality, there is always the prospect of lifetime ‘blurring’ due to out of focus haze. Removal of haze due to the 3D nature of thicker samples (and its effect on fluorescence lifetime) has been reported in a limited sense so far and provides impetus for future work in the 3D blind image restoration-FLIM regime. Wide-field FLIM has its limitations for studying 3D samples, especially those with high background [26]. Restoration techniques hold promise for extending these limits without the use of additional optics and while increasing SNR.
Acknowledgments
The Authors would like to thank Ching-Wei Chang (University of Michigan) and Jonathon Girroir (Media Cybernetics) for helpful discussions, and Joe Delli (Nuhsbaum) for providing the Autoquant X software. This work was supported by a grant from the National Institutes of Health: NIH CA-114542 (to M.-A.M.).
References and links
1. C. W. Chang, D. Sud, and M. A. Mycek, “Fluorescense lifetime imagining microscopy,” Methods Cell Bio. 81, 495–524 (2007). [CrossRef]
2. Y. Chen and J. D. Mills, “Protein localization in living cells and tissues using FRET and FLIM,” Differentiation 71, 528–541 (2003). [CrossRef] [PubMed]
3. I. Konig, J. P. Schwarz, and K. I. Anderson, “Fluorescence lifetime imaging: Association of cortical actin with a PIP3-rich membrane compartment,” Eur. J. Cell. Biol. 87, 735–741 (2008). [CrossRef] [PubMed]
4. D. Schweitzer, S. Schenke, M. Hammer, F. Schweitzer, S. Jentsch, E. Birckner, W. Becker, and A. Bergmann, “Towards metabolic mapping of the human retina,” Microsc. Res. Tech. 70, 410–419 (2007). [CrossRef] [PubMed]
5. D. Sud, W. Zhong, D. G. Beer, and M. A. Mycek, “Time-resolved optical imaging provides a molecular snapshot of altered metabolic function in living human cancer cell models,” Opt. Express 14, 4412–4426 (2006). [CrossRef] [PubMed]
6. K. K. Sharman, A. Periasamy, H. Ashworth, J. N. Demas, and N. H. Snow, “Error analysis of the rapid lifetime determination method for double-exponential decays and new windowing schemes,” Anal. Chem. 71, 947–952 (1999). [CrossRef] [PubMed]
7. P. C. Goodwin, “Evaluating optical aberration using fluorescent microspheres: methods, analysis, and corrective actions,” Methods Cell Biol. 81, 397–413 (2007). [CrossRef] [PubMed]
8. H. R. Petty, “Fluorescence microscopy: established and emerging methods, experimental strategies, and applications in immunology,” Microsc. Res. Tech. 70, 687–709 (2007). [CrossRef] [PubMed]
9. M. J. Cole, J. Siegel, S. E. Webb, R. Jones, K. Dowling, P. M. French, M. J. Lever, L. O. Sucharov, M. A. Neil, R. Juskaitis, and T. Wilson, “Whole-field optically sectioned fluorescence lifetime imaging,” Opt. Lett. 25, 1361–1363 (2000). [CrossRef]
10. T. Uchimura, S. Kawanabe, Y. Maeda, and T. Imasaka, “Fluorescence lifetime imaging microscope consisting of a compact picosecond dye laser and a gated charge-coupled device camera for applications to living cells,” Anal. Sci. 22, 1291–1295 (2006). [CrossRef] [PubMed]
11. P. K. Urayama, W. Zhong, J. A. Beamish, F. K. Minn, R. D. Sloboda, K. H. Dragnev, E. Dmitrovsky, and M. A. Mycek, “A UV-visible fluorescence lifetime imaging microscope for laser-based biological sensing with picosecond resolution,” App. Phys. B: Lasers Opt. 76, 483–496 (2003). [CrossRef]
12. W. Zhong, P. Urayama, and M. A. Mycek, “Imaging fluorescence lifetime modulation of a ruthenium-based dye in living cells: the potential for oxygen sensing,” J. Phys. D-Appl. Phys. 36, 1689–1695 (2003). [CrossRef]
13. W. Zhong, M. Wu, C.-W. Chang, K. A. Merrick, S. D. Merajver, and M. A. Mycek, “Picosecond-resolution fluorescence lifetime imaging microscopy: a useful tool for sensing molecular interactions in vivo via FRET,” Opt. Express 15, 18220–18235 (2007). [CrossRef] [PubMed]
14. E. Pietka and H. K. Huang, “Correction of Aberration in Image-Intensifier Systems,” Computerized Medical Imaging and Graphics 16, 253–258 (1992). [CrossRef] [PubMed]
15. M. J. Cole, J. Siegel, S. E. Webb, R. Jones, K. Dowling, M. J. Dayel, D. Parsons-Karavassilis, P. M. French, M. J. Lever, L. O. Sucharov, M. A. Neil, R. Juskaitis, and T. Wilson, “Time-domain whole-field fluorescence lifetime imaging with optical sectioning,” J. Microsc. 203, 246–257 (2001). [CrossRef] [PubMed]
16. A. Squire and P. I. Bastiaens, “Three dimensional image restoration in fluorescence lifetime imaging microscopy,” J. Microsc. 193, 36–49 (1999). [CrossRef] [PubMed]
17. J. Boutet de Monvel, S. Le Calvez, and M. Ulfendahl, “Image restoration for confocal microscopy: improving the limits of deconvolution, with application to the visualization of the mammalian hearing organ,” Biophys. J. 80, 2455–2470 (2001). [CrossRef] [PubMed]
18. Y. Q. Guan, Y. Y. Cai, X. Zhang, Y. T. Lee, and M. Opas, “Adaptive correction technique for 3D reconstruction of fluorescence microscopy images,” Microsc. Res. Tech. 71, 146–157 (2008). [CrossRef]
19. P. Pankajakshan, B. Zhang, L. Blanc-Feraud, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Parametric blind deconvolution for confocal laser scanning microscopy,” Conf. Proc. IEEE Eng. Med. Biol. Soc. 2007, 6532–6535 (2007). [PubMed]
20. J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Eng. Biotechnol. 95, 201–243 (2005). [PubMed]
21. B. J. Vermolen, Y. Garini, and I. T. Young, “3D restoration with multiple images acquired by a modified conventional microscope,” Microsc. Res. Tech. 64, 113–125 (2004). [CrossRef] [PubMed]
22. J. Markham and J. A. Conchello, “Fast maximum-likelihood image-restoration algorithms for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A. Opt. Image Sci. Vis. 18, 1062–1071 (2001). [CrossRef] [PubMed]
23. P. J. Verveer, M. J. Gemkow, and T. M. Jovin, “A comparison of image restoration approaches applied to three-dimensional confocal and wide-field fluorescence microscopy,” J. Microsc. 193, 50–61 (1999). [CrossRef]
24. J. Siegel, D. S. Elson, S. E. Webb, K. C. Lee, A. Vlandas, G. L. Gambaruto, S. Leveque-Fort, M. J. Lever, P. J. Tadrous, G. W. Stamp, A. L. Wallace, A. Sandison, T. F. Watson, F. Alvarez, and P. M. French, “Studying biological tissue with fluorescence lifetime imaging: microscopy, endoscopy, and complex decay profiles,” Appl. Opt. 42, 2995–3004 (2003). [CrossRef] [PubMed]
25. J. B. de Monvel, E. Scarfone, S. Le Calvez, and M. Ulfendahl, “Image-adaptive deconvolution for three-dimensional deep biological imaging,” Biophys. J. 85, 3991–4001 (2003). [CrossRef] [PubMed]
26. J. M. Murray, P. L. Appleton, J. R. Swedlow, and J. C. Waters, “Evaluating performance in three-dimensional fluorescence microscopy,” J. Microsc. 228, 390–405 (2007). [CrossRef] [PubMed]
