Fluorophore excited state lifetime is a useful indicator of micro-environment in cellular optical molecular imaging. For quantitative sensing, precise lifetime determination is important, yet is often difficult to accomplish when using the experimental conditions favored by live cells. Here we report the first application of temporal optimization and spatial denoising methods to two-photon time-correlated single photon counting (TCSPC) fluorescence lifetime imaging microscopy (FLIM) to improve lifetime precision in live-cell images. The results demonstrated a greater than five-fold improvement in lifetime precision. This approach minimizes the adverse effects of excitation light on live cells and should benefit FLIM applications to high content analysis and bioimage informatics.
© 2010 OSA
Fluorophore excited state lifetime is an intrinsic property of fluorophores that is sensitive to micro-environmental conditions such as temperature, pH, and interactions with other molecules. Hence, it can be employed as an optical sensor to indicate, for example, Förster resonance energy transfer, oxygen levels, and the conformational state of endogenous / exogenous fluorophores in live-cell and in vivo studies [1–5]. Importantly, fluorescence lifetimes are relatively insensitive to the factors affecting intensity: variation in excitation source intensity, detection gain setting, optical loss in the optical path or sample, variation in sample fluorophore concentration, photobleaching, and microscope focusing .
However, in the applications of fluorescence lifetime, the precision of lifetime determination can be a crucial issue. For example, precise quantification of fluorescence lifetime can have an impact on high content analysis and bioimage informatics. High content analysis of images requires automated processing of a large amount of image data, and this technique has been demonstrated to have biomedical applications such as the characterization of cell cycle therapeutics , the analysis of neurite branching , and the quantification of caspase activation . It has been demonstrated that FLIM can be used for high content and high throughput screening , implying that the above applications as well as research in proteomics, cellomics, and drug discovery can potentially be facilitated by improvements in FLIM precision. Another related emerging area is bioimage informatics, whose applications include high-throughput / high-content phenotyping and atlas building for model organisms . Since bioimage informatics utilizes computational tools for the acquisition, visualization, and analysis of image data sets, it can also benefit from better quantification of fluorescence lifetime if fluorescence lifetime imaging is employed.
To improve the precision of microscopic fluorescence imaging, several “denoising” (noise removal) techniques have been proposed for fluorescence microscopy and FLIM. Wavelet analysis  has been used for denoising images from confocal and full-field frequency-domain FLIM [13,14]. Non-parametric regression method  and multiframe SURE-LET (Stein’s unbiased risk estimate -linear expansion of thresholds) denoising  have been reported for fluorescence microscopy image denoising. However, it has not been reported that denoising can be used in time-domain FLIM for improvements of lifetime determination.
In this study, we applied novel total variation (TV) denoising models in time-domain FLIM for precision improvement. TV models, based on local denoising algorithms, are very commonly used in medical imaging systems and even non-imaging technologies, because they perform selective smoothing and hence are edge-preserving [17,18]. For example, a variety of improved TV models [17,18] and related algorithms have been used with many other image processing techniques and medical imaging systems such as 3D confocal microscopy deconvolution , X-ray-computed tomography , deconvolution-based correction in positron emission tomography , image segmentation , and they can be applied to non-imaging medical technologies such as detecting and delineating genomic regions with biased gene expression in cancer .
We previously tested the novel TV denoising models developed in our laboratory with artificial images and with images of fluorescent beads and live-cells acquired from a wide-field time-gated FLIM system [24–26]. Tests with artificial images indicated that there was no lifetime bias in the fluorescence lifetime value, provided that the estimation of noise magnitude was accurate. Tests with fluorescent beads demonstrated that TV denoising could be combined with optimal gating to achieve lifetime precision improvement greater than 4-fold, with the lifetime values basically uniform inside the beads, as expected. Tests with live-cells indicated that the novel denoising models preserved the overall lifetime and amplitude values of the single-exponential decay model while improving local lifetime fitting.
In this study, we further focus on lifetime precision improvements in time-correlated single photon counting (TCSPC) FLIM, a widely used approach, where fluorescence decay curves are constructed by photon emission histograms , as in Fig. 1 . We demonstrate, for the first time, how temporal optimization (optimal virtual gating, see section 2.3) and spatial denoising methods can be used to improve the precision of lifetime determination in live-cell two-photon TCSPC FLIM.
2.1. Live-cell sample
LLC-PK1 live cells expressing mEmerald-EB3 and mCherry-H2B were kindly provided by Dr. Michael Davidson (Florida State University). The cells were cultured in Dulbecco’s minimal essential medium (DMEM) supplemented with 10% fetal bovine serum in humidified 37 °C incubator with 5% CO2. The cells were trypsinized and seeded in MetTek glass-bottom dishes with approximately 20% of confluence 12-24 hours before imaging.
2.2. TCSPC FLIM
Fluorescence lifetime images can be constructed by using raster-scanned time-correlated single photon counting (TCSPC) modules. With this technique, an entire exponential fluorescence decay curve can be constructed by a histogram of single photon collections at each pixel of the image (Fig. 1), and the lifetime value can be retrieved by curve fitting, such as nonlinear least squares fitting, of the data points constructing the detected decay curve.
In the two-photon TCSPC FLIM instrumentation, a Zeiss inverted LSM-510 laser scanning confocal system was used with two-photon excitation (ChameleonTM Vision, Coherent Inc.) and a Becker & Hickl TCSPC module (DCC software version 1.23 and SPCM software version 8.70).
The live-cell images were acquired with the following settings. A 100 x objective was used. mEmerald was excited at λex = 820 nm and the fluorescence was collected at λem = 500-550 nm. Data acquisition time was 100 seconds. The maximum total photon counts were about 2500.
2.3. Optimal “virtual” gating
TCSPC data can be “virtually gated” to form gated intensity images similar to those from time-gated FLIM. Given the values of the gate width, g, and the time interval between the starting points of two consecutive gates, dt (see Fig. 1), virtual gating of TCSPC data is a post-data-acquisition technique and is implemented by summing up the values of the data points within each virtual gate to form an intensity image. In this study, four virtual gates were used for its robustness , and a four-gate protocol was applied to determine the lifetime values by using Eq. (1) on a pixel-by-pixel basis [28–30]:
The cellular morphology pattern of the conventional TCSPC lifetime map, shown in Fig. 4(a) , was used as a reference for further parameter selection in lifetime mapping after virtual gating. Figure 4(a) was generated with “threshold” = 5 in conventional TCSPC lifetime mapping, meaning that the pixels with peak photon numbers lower than 5 would not be analyzed. The “reject” values, below which the intensities were set to zero in the virtually-gated images, were adjusted such that the morphology was approximately the same as in the reference image, Fig. 4(a). This was for better comparisons of the variations only in the morphology that we were interested in. Indeed, lowering the values of “reject” would change the apparent morphology and the relative standard deviation (RSD, defined as the standard deviation divided by the mean value, used as an indicator for the uncertainty of lifetime determination) values by bringing a certain amount of background to foreground. Changing the “threshold” value in TCSPC lifetime mapping had similar effects. Therefore, it was reasonable to adjust “reject” accordingly to match the selected “threshold” value.
Optimal virtual gating parameters can be determined. The RSD of the lifetime values calculated by using the four-gate protocol could be analytically determined by applying error propagation to Eq. (1), with the assumption of Poisson noise (see Eq. (2)). The optimal gating parameters then occurred at the minimal RSD values. The live-cell samples described in section 2.1 (approximate fluorescence lifetime value around 2.7 ns) were then determined to have the optimal gating scheme around dt = 2 ns and g ≥ 8 ns. For non-optimal gating, dt was chosen to be 20% of the optimal dt for consistency with our previous study , where the this change in dt was proven to have an impact on lifetime RSD values in a time-gated FLIM system. In fact, dt around 0.5 ns or 1 ns is commonly used in time-gated FLIM systems to measure fluorescence lifetimes of several nanoseconds. Therefore, dt = 0.4 ns was used in regular (non-optimal) virtual gating of TCSPC.
There are some advantages of virtual gating of TCSPC data. First, it greatly accelerates the lifetime retrieving steps when used with subsequent closed-form lifetime determination methods such as rapid lifetime determination and the four-gate protocol mentioned above. Although virtual gating itself with closed-form lifetime solutions might not improve the precision of lifetime determination, it enables image processing, such as image denoising (section 2.4), for each virtual gate, and this allows further improvements of lifetime imaging.
2.4. Total variation denoising
2.4.1. Denoising algorithms
In this study, two novel total variation (TV) based image denoising models were used to improve the lifetime precision of live-cell two-photon TCSPC FLIM. They can remove Poisson noise and can also be easily adapted for any forms of noise introduced by imaging systems and image processing procedures. These forms of noise can be intensity-dependent, lifetime-dependent, or even spatially-dependent. Therefore, they can provide an accurate estimation of noise magnitude and have been demonstrated to produce no lifetime bias in their applications to FLIM [24–26]. The first model we used was a general variance-weighted TV (VWTV) model:31], which requires the fidelity term (the second term on the right hand side of Eq. (3)) evaluated with f and the final u to be the same as that evaluated with f and the estimated uncorrupted image . Denoising was implemented through the minimization of energy (E), during which the processed image (u) evolved to a stable state that should be close to the original image without noise corruption. For the specific application of denoising virtually-gated intensity images, a novel f-weighted TV (FWTV) model  based on an f-weighted fidelity term was used. In this model, the Var(f) term in Eq. (3) was simply replaced by f, with the assumption that the intensity values followed Poisson distribution due to the single photon counting behavior in TCSPC.
2.4.2. Denoising procedures
Two procedures were used with TV denoising. In “lifetime denoising” (Fig. 2 (a) ), a lifetime map was first constructed, either by four-gate lifetime mapping (for virtually-gated TCSPC FLIM) or by TCSPC lifetime mapping (for regular TCSPC FLIM), before applying denoising directly on lifetime maps. Since the variance of lifetime was not proportional to the lifetime values, VWTV was used. The variance estimation of virtually-gated TCSPC lifetime maps, as a function of τ, g, dt, and total photon counts, was performed by solving analytically the error propagation of the four-gate lifetime mapping formula (Eq. (1), also see Eq. (2)), while the variance estimation of TCSPC lifetime map, as a function of τ and total photon counts, was performed by direct sampling of the uniform regions of lifetime maps (see Fig. 3 ). In “intensity denoising” (Fig. 2 (b)), each virtually-gated intensity image was denoised before four-gate lifetime mapping. In this case, TV denoising was performed with FWTV.
3. Results and discussion
3.1. Overall lifetime precision improvement
Figure 4 demonstrates that, for TCSPC FLIM, both virtual gating and our novel TV denoising techniques can remove uncertainties in lifetime maps (the major non-uniformity in the lifetime map should arise from noise, since the fluorescence lifetime should be nearly uniform with the fluorophores in similar environments; even if there was any intrinsic lifetime distribution, it would be preserved due to the edge-preserving property of the TV denoising techniques). Overall, the precision was improved by greater than five-fold (RSD from 18.8% to 3.7%, Fig. 4(a) and (f)) in the lifetime map. The remaining RSD of 3.7% could be attributed to the intrinsic lifetime distribution instead of noise. The improvement from the intensity denoising (1% in this case, from Fig. 4(b) to (g) and from Fig. 4(c) to (h)) was independent of that from optimal virtual gating (2.1% in this case, from Fig. 4(b) to (c)).
The overall precision improvement depends on several factors. Based on our previous studies with a time-gated FLIM system , optimal gating can reduce the RSD by a certain ratio (for example, reduction by 2/3 from 54% to 18%), which is independent of the geometry of objects in the image and the total photon counts (TC). On the other hand, denoising depends more on the starting RSD (starting with an already low RSD may cause negligible denoising effects) and also the geometry (see sections 3.3 and 3.4 below), but the reduction in the percentage of RSD is usually in the range of 1% to 10%, which also depends on the denoising procedures used (section 3.4). In theory, noise-related RSD is inversely proportional to the square root of TC. Therefore, for TC ≥ 104, although optimal gating is always recommended, the noise-related RSD could become 5% or less after optimal gating, and denoising may not be needed in this case. However, for TC ≤ 104 to as low as around 100, the combined approach of optimal gating and denoising is recommended and has been demonstrated to reduce the RSD multifold [24,25]. These principles obtained from time-gated FLIM should also apply to TCSPC FLIM, since TCSPC data have Poisson-distributed noise, and virtually gated TCSPC FLIM data are of the same form as time-gated FLIM data.
3.2. Lifetime precision improvement by virtual gating
After virtual gating was applied to the raw data, clearly and surprisingly, the virtual gating alone could greatly improve the precision. Even the non-optimal virtual gating could achieve a great deal of improvement (RSD = 9.0%, Fig. 4(b)), while the optimal virtual gating led to even greater improvement (RSD = 6.9%, Fig. 4(c)). Although this kind of improvement would be expected with time-gated FLIM based on our previous studies [24,25], the degree of (and whether there is) precision improvement with virtual gating in TCSPC FLIM may depend on total photon counts and will need further investigations.
3.3. Lifetime precision improvement by total variation denoising
Before virtual gating was applied, there were no gated intensity images for denoising and subsequent four-gate lifetime construction. Therefore, only lifetime denoising was applied and this resulted in an RSD reduction from 18.8% to 8.9% (Fig. 4(a) and (d)).
For either non-optimally or optimally virtually-gated intensity maps, both lifetime denoising and intensity denoising can be applied to achieve even better precision. With a lower RSD to start with after optimal virtual gating, the denoising predictably achieved better precision (RSD = 3.7% and 5.9%, Fig. 4(f) and (h), for lifetime denoising and intensity denoising, respectively) compared to the non-optimal virtual gating (RSD = 4.3% and 8.0%, Fig. 4(e) and (g) for lifetime denoising and intensity denoising, respectively).
3.4. Lifetime denoising versus intensity denoising
For the live-cell images shown in Fig. 4, lifetime denoising had greater influence on the precision of lifetime determination than intensity denoising. Merely lifetime denoising on the original curve-fit lifetime map could produce much more than two-fold precision improvement. This may be attributed to the geometry of the features in the images, since our previous studies demonstrated that lifetime denoising appeared to be worse for removing the irregularities in the geometry of objects arising from noise but better for smoothing off-edge, internal pixels for pattern revealing. In the live-cell images (Fig. 4), the irregularities in the geometry arose mostly from the cellular structures but not from noise. Therefore, we presume that the irregular edges, which lifetime denoising preserved, did not cause imprecision after lifetime denoising, and in this case lifetime denoising could improve the precision better. Alternatively, the better performance of lifetime denoising could also be attributed to the medium total photon counts (≤ 2500), in the sense that higher total photon counts also removed noisy edges in lifetime maps. The full characterization of the differences in the performances of lifetime denoising and intensity denoising requires further investigations.
In this study, we applied optimal virtual gating and total variation image denoising to live-cell two-photon TCSPC FLIM images to remove the uncertainties and improve the precision of lifetime determination by greater than five-fold (relative standard deviation, or RSD, from 18.8% to 3.7%; see Fig. 4(a) and (f)) in the lifetime maps. This approach is in principle applicable to single-photon TCSPC FLIM and time-gated FLIM, and allows low-light live-cell imaging with high precision and minimizes the adverse effects of excitation light on live cells. Therefore, our techniques can help avoid unnecessary high-intensity excitation of biological samples, possible sample damage / photobleaching, and unwanted detection of sample movement with long acquisition time. Improvements in FLIM precision can also influence other applications of fluorescence lifetime imaging, such as high content analysis and bioimage informatics.
This work was supported in part by a research grant from the National Institutes of Health (NIH)CA-114542. The authors thank Mr. Louis Kerr of the Central Microscopy Facility at the Marine Biological Laboratory and Dr. Anna Krzywicka-Racka of the University of Massachusetts Medical School for technical assistance, and Dr. Michael Davidson of Florida State University for providing access to living cells.
References and links
1. S. Bloch, F. Lesage, L. McIntosh, A. Gandjbakhche, K. X. Liang, and S. Achilefu, “Whole-body fluorescence lifetime imaging of a tumor-targeted near-infrared molecular probe in mice,” J. Biomed. Opt. 10(5), 054003 (2005). [CrossRef] [PubMed]
2. S. Pelet, M. J. R. Previte, D. Kim, K. H. Kim, T. T. J. Su, and P. T. C. So, “Frequency domain lifetime and spectral imaging microscopy,” Microsc. Res. Tech. 69(11), 861–874 (2006). [CrossRef] [PubMed]
3. C. W. Chang, M. Wu, S. D. Merajver, and M. A. Mycek, “Physiological fluorescence lifetime imaging microscopy improves Förster resonance energy transfer detection in living cells,” J. Biomed. Opt. 14(6), 060502 (2009). [CrossRef]
5. D. Sud, W. Zhong, D. G. Beer, and M. A. Mycek, “Time-resolved optical imaging provides a molecular snapshot of altered metabolic function in living human cancer cell models,” Opt. Express 14(10), 4412–4426 (2006). [CrossRef] [PubMed]
7. J. Low, S. Huang, W. Blosser, M. Dowless, J. Burch, B. Neubauer, and L. Stancato, “High-content imaging characterization of cell cycle therapeutics through in vitro and in vivo subpopulation analysis,” Mol. Cancer Ther. 7(8), 2455–2463 (2008). [CrossRef] [PubMed]
8. P. Vallotton, R. Lagerstrom, C. Sun, M. Buckley, D. D. Wang, M. De Silva, S. S. Tan, and J. A. Gunnersen, “Automated analysis of neurite branching in cultured cortical neurons using HCA-Vision,” Cytometry A 71A(10), 889–895 (2007). [CrossRef]
10. C. B. Talbot, J. McGinty, D. M. Grant, E. J. McGhee, D. M. Owen, W. Zhang, T. D. Bunney, I. Munro, B. Isherwood, R. Eagle, A. Hargreaves, C. Dunsby, M. A. A. Neil, and P. M. W. French, “High speed unsupervised fluorescence lifetime imaging confocal multiwell plate reader for high content analysis,” J. Biophotonics 1(6), 514–521 (2008). [CrossRef]
11. J. R. Swedlow, I. G. Goldberg, K. W. EliceiriJ. R. SwedlowI. G. GoldbergK. W. EliceiriOME Consortium, “Bioimage informatics for experimental biology,” Annu. Rev. Biophys. 38(1), 327–346 (2009). [CrossRef] [PubMed]
12. C. Vonesch, “Fast and automated wavelet-regularized image restoration in fluorescence microscopy,” Ph.D. thesis, École Polytechnique Fédérale De Lausanne (2009).
13. C. Buranachai, D. Kamiyama, A. Chiba, B. D. Williams, and R. M. Clegg, “Rapid frequency-domain FLIM spinning disk confocal microscope: lifetime resolution, image improvement and wavelet analysis,” J. Fluoresc. 18(5), 929–942 (2008). [CrossRef] [PubMed]
14. B. Q. Spring and R. M. Clegg, “Image analysis for denoising full-field frequency-domain fluorescence lifetime images,” J. Microsc. (Oxford) 235(2), 221–237 (2009). [CrossRef]
15. J. Boulanger, J. B. Sibarita, C. Kervrann, and P. Bouthemy, “Non-parametric regression for patch-based fluorescence microscopy image sequence denoising,” 2008 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Vols 1–4, 748–751 (2008).
16. S. Delpretti, F. Luisier, S. Ramani, T. Blu, and M. Unser, “Multiframe SURE-LET denoising of timelapse fluorescence microscopy images,” 2008 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Vols 1–4, 149–152 (2008).
17. S. Osher, M. Burger, D. Goldfarb, J. J. Xu, and W. T. Yin, “An iterative regularization method for total variation-based image restoration,” Multiscale Model. Simul. 4(2), 460–489 (2005). [CrossRef]
18. E. Tadmor, S. Nezzar, and L. Vese, “A multiscale image representation using hierarchical (BV, L2) decompositions,” Multiscale Model. Simul. 2(4), 554–579 (2004). [CrossRef]
19. N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution,” Microsc. Res. Tech. 69(4), 260–266 (2006). [CrossRef] [PubMed]
20. C. N. Wu, Y. Cheng, M. L. Liu, and Y. Jin, “Measurement of axisymmetric two-phase flows by an improved x-ray-computed tomography technique,” Ind. Eng. Chem. Res. 47(6), 2063–2074 (2008). [CrossRef]
21. J. Tohka and A. Reilhac, “Deconvolution-based partial volume correction in Raclopride-PET and Monte Carlo comparison to MR-based method,” Neuroimage 39(4), 1570–1584 (2008). [CrossRef]
23. B. Nilsson, M. Johansson, A. Heyden, S. Nelander, and T. Fioretos, “An improved method for detecting and delineating genomic regions with altered gene expression in cancer,” Genome Biol. 9(1), R13 (2008). [CrossRef] [PubMed]
24. C. W. Chang, “Improving Accuracy and Precision in Biological Applications of Fluorescence Lifetime Imaging Microscopy,” Ph.D. thesis, University of Michigan (2009).
25. C. W. Chang and M. A. Mycek, “Improving precision in time-gated FLIM for low-light live-cell imaging,” Proc. SPIE 7370, 7370091–7370096 (2009).
26. C. W. Chang and M. A. Mycek, “Increasing precision of lifetime determination in fluorescence lifetime imaging,” Proc. SPIE 7570, 757007 (2010). [CrossRef]
27. H. C. Gerritsen, M. A. H. Asselbergs, A. V. Agronskaia, and W. G. J. H. M. Van Sark, “Fluorescence lifetime imaging in scanning microscopes: acquisition speed, photon economy and lifetime resolution,” J. Microsc. (Oxford) 206(3), 218–224 (2002). [CrossRef]
28. I. Bugiel, K. König, and H. Wabnitz, “Investigation of cell by fluorescence laser scanning microscopy with subnanosecond time resolution,” Lasers Life Sci. 3, 47–53 (1989).
29. X. F. Wang, T. Uchida, D. M. Coleman, and S. Minami, “A two-dimensional fluorescence lifetime imaging system using a gated image intensifier,” Appl. Spectrosc. 45(3), 360–366 (1991). [CrossRef]
30. K. K. Sharman, A. Periasamy, H. Ashworth, J. N. Demas, and N. H. Snow, “Error analysis of the rapid lifetime determination method for double-exponential decays and new windowing schemes,” Anal. Chem. 71(5), 947–952 (1999). [CrossRef] [PubMed]
31. T. Le, R. Chartrand, and T. J. Asaki, “A variational approach to reconstructing images corrupted by poisson noise,” J. Math. Imaging Vis. 27(3), 257–263 (2007). [CrossRef]