1. Introduction

Fluorescence lifetime imaging (FLIM) is an important part of spectrometry dealing with photoluminescence (PL) dynamics. It is used mainly in biology [1,2], chemical physics [3,4], as well as in materials engineering [5–8].

FLIM can be achieved by means of several methods. The best-known are time-correlated single-photon counting (TCSPC) [9], streak camera [10], gated photon counting [11], and the analog time-domain or frequency-domain techniques [12,13]. However, each method has its principle limitations dictating its scope and, as a result, it may be simultaneously perfect for a certain application and completely unsuitable for another [14].

After decades of development, recent years have brought some innovative improvements in FLIM. They focus on obtaining measurements and evaluations in real time, aiming primarily to investigate biological and medical samples [15,16]. However, real-time FLIM acquisition depends on using pulsed lasers, digitizers, GPUs, or 2D arrays of single photon avalanche diodes, which considerably increase the cost of such setups. The same issue applies to the new FLIM alternatives, such as phasor spectral FLIM (Phasor S-FLIM) [17]. Another competing concept, the analog mean delay method (ADM) [18], can be used for real-time in vivo measurements but it is not suitable for determining the actual shape of PL decay.

Therefore, FLIM research has also developed in the direction of using compressed sensing (CS), where the most typical application is the single-pixel camera (SPC) experiment, which makes the FLIM optical setup more affordable [19]. SPC experiments are very useful in terms of reducing the cost of the experimental setup but they are typically not suited for real-time in vivo imaging—both due to the sequential signal acquisition and to the consequent data retrieval. However, new post-processing approaches to SPC application in FLIM are still being developed. A notable example is the use of deep learning, which significantly shortens reconstruction time [20] although it requires the neural network to be trained properly in order to obtain the required performance over the necessary time range of the investigated lifetimes.

Most of the improvements in CS are applied to standard time-resolved methods which, however, still retain their principle limitations. For instance, TCSPC achieves excellent results for nanosecond or sub-nanosecond decays but requires measurement times on the order of hours for decay measurement on the order of hundreds of nanoseconds (field of material engineering).

Consequently, it is essential to continue developing new methods of analyzing PL dynamics such as the RAndom Temporal Signals (RATS) method, which is one of the novel approaches to FLIM. It is based on the excitation of the measured sample by an intensity randomly fluctuating in time, which makes it possible to obtain a complete PL decay from a single measured dataset. Owing to the RATS principle of continuous random excitation, it is not necessary to excite samples by short pulses with a temporal gap allowing for complete PL decay. Hence, a higher signal-to-noise ratio (SNR) is obtained compared to conventional methods, and the choice of proper repetition rate for excitation pulses does not have to be resolved. The RATS and 2D-RATS methods have proven to be valuable, straightforward, and low-cost alternatives to the commonly used FLIM approaches, although their previous implementation had limited their use in real-life experiments.

The initial proof-of-principle measurements of PL decay via RATS were based on a simple generator of a random excitation signal, where we focused a beam on a rotary diffuser and cropped the generated field of speckles with a suitable iris aperture [21]. This procedure enables the generation of an analog random excitation in the microsecond and sub-microsecond region, which is well suited for long-lived luminophores or nanoporous silicon [3,22]. However, reaching faster timescales is not realistic in this configuration.

At the same time, employing a diffuser limited the overall efficiency of the excitation laser because only a fraction of the diffused light was used. This fact was even more pronounced in our proof-of-principle implementation of the RATS method in FLIM (2D-RATS), which demonstrated that it is possible to map PL dynamics using an optical setup based on two diffusers [23].

Finally, the initially used post-processing consists of the reconstruction of PL maps at each delay after excitation and subsequent fitting to obtain a lifetime in a given image pixel. This procedure is highly computationally demanding.

This article presents a new implementation of the 2D-RATS method, i.e., the FLIM experiment, which overcomes all the above-stated shortcomings. This was achieved by introducing three fundamental modifications to the RATS experiment. First, we introduced a new digital mode of random intensity fluctuation by using direct laser modulation, which enabled us to reach the temporal resolution of 6 ns in the PL decay measurement. Secondly, we employed a digital micromirror device (DMD) to invoke random spatial masks, which dramatically increased the efficiency of the setup and enabled us to reach intensity values a hundred times higher than the proof-of-principle experiments [23]. The last change involves a new approach to PL data analysis, in which we first identify the significant PL lifetime components and, subsequently, we computationally extract an amplitude map of each component. As a result, the post-processing time for a typical analysis (a bi-exponential fit of 35 × 35 map with a compression ratio of 0.4 and 100 timepoints of PL decay) becomes about 10 times shorter compared to the approach used in our previous work [23].

Overall, the presented implementation shifts the potential of the 2D-RATS method to a different level, which we demonstrate below on synthetic as well as experimental data. The ability to retrieve PL dynamics on the order of tens of nanoseconds in combination with the ability to cover long-lived PL decay make the method very useful in the field of material engineering [7,8]. At the same time, its current implementation remains a low-cost solution compared to standard FLIM setups.

2. Overview of the RATS method

2.1 0D-RATS

As mentioned in the introduction, the principle of the method is to excite the measured sample with a random excitation signal I_EXC. The generated photoluminescent I_PL signal, which is a convolution of the excitation signal and the PL decay of the measured sample I_D, also has a random character as a result of the random excitation:

The PL decay I_D curve can be calculated by using deconvolution. In the presented results, the Tikhonov regularization was applied to avoid ill-conditioned problems [24]:

Moreover, the Tikhonov regularization parameter ε can also be used to suppress the noise effect of the system [25]. The single-point measurement approach (0D-RATS) is described in more detail in our previous works [21,23].

2.2 2D-RATS

In the case of 2D measurements, a straightforward option is to use the 0D-RATS method in raster mode, which can be very effective, for example, using a mirror with a Micro Electro Mechanical System (MEMS) [26]. Nevertheless, the use of compressive sampling in the configuration of the single-pixel camera ensures the acquisition of the desired dataset by a lower number of measurements compared to the raster mode, where the number of measurements corresponds to the number of pixels of an image [27].

The principle of the so-called 2D-RATS measurement lies in illuminating the scene with a set of M random masks of N pixels, where the ratio M/N indicates the compression ratio k [23]. The individual masks have a random character in space, and their intensity varies in time according to I_EXC—see Fig. 1(A). The randomness of the mask ensures that a different n illuminated segment of the sample is excited in each measurement. Therefore, Eq. (1) can be rewritten to the sum of the PL from each segment:

As a result, we always attain a unique I_PL for each of the M masks. By using the I_PL dataset, it is possible to reconstruct the individual I_DA decay curves corresponding to the given mask:

 

Fig. 1. (A) Scheme of principles of the 2D-RATS method. The sample is illuminated with masks following a temporal fluctuation of excitation signal I_EXC, which generate the corresponding PL signal I_PL. (B) Set of calculated I_DA (solid lines in the main graph) corresponding to the set of masks can be recalculated into the I_SPC signals (inset graphs for several delays t). By using Eq. (5) we attain the corresponding reconstructions of the PL maps. (C) Scheme of a 3D datacube of reconstructed PL maps and indicated fitting. (D) Example of a final FLIM spectrogram as a map of PL lifetimes.

Download Full Size | PDF

Thus, each random mask is represented by one I_DA.

The measured set of I_DA curves then needs to be converted into the temporal slides of PL decay. To do this, we need to determine the PL intensity I_SPC at a selected delay for all masks—see Fig. 1(B). The I_SPC dataset is simply extracted from the I_DA curves for the studied delay after excitation t. By using the knowledge of the random masks and the I_SPC, the PL map for each delay after excitation m(x,y,t) can be determined using standard compressed sensing algorithms employed for the single-pixel camera experiment. We aim at solving an undetermined system using Eq. (5), where the set of vectorized masks is stated as B and TV is stated for total variation and the PL map for a given delay after excitation is declared just as m(x’, t) because it is vectorized:

The presented approach uses a well-established image retrieval procedure from the single-pixel camera experiment. A significant drawback is that it requires reconstructing PL maps for all delays after excitation and creating a 3D datacube to obtain complete information—see Fig. 1(C). This typically implies carrying out tens of iterative solutions of Eq. (5), which is computationally very demanding. Finally, to attain the FLIM spectrogram (Fig. 1(D)), it is necessary to fit the third dimension (temporal dependence) of the 3D datacube in each pixel with a suitable function—typically a multi-exponential fit—and determine the lifetime $\tau$. For better understanding, refer to Fig. 1, which summarizes the whole procedure.

Compared to standard FLIM approaches, such as TCSPC or gated photon counting, the 2D-RATS method provides very robust PL retrieval. Its only special requirement is for the setup to feature a sufficiently low width of the impulse response function (IRF) compared to the measured lifetimes, i.e., a sufficiently fast random signal. In contrast, the suitability of the setting parameters in TCSPC must be checked with respect to the repetition rate of excitation pulses or PL intensity. In the case of using gated photon counting, the user should choose the appropriate number of gates, length of gate, and other parameters.

3. Digital RATS modulation with direct PL lifetime map reconstruction

3.1 Digital random signal generation

The previously published proof-of-principle RATS experiment was based on generating a random excitation pattern by using a rotary diffuser. This leads to a rapidly fluctuating signal, which we denote as a random analog signal, as the light intensity continuously randomly varies in time. While this approach is useful owing to its simplicity and negligible cost, it limits the use of the RATS setup with respect to the reachable temporal resolution.

Therefore, we implemented an advanced mode of random signal generation with higher efficiency by using a modulated laser. One of the lasers that makes this possible is the Cobolt S06-01 (405 nm), which can be digitally modulated to up to 150 MHz in a random fashion. The result is a rectangular signal with a randomly distributed duty cycle, which we denote as a random digital modulation.

We compare the digital and analog generation of a random signal in 0D-RATS on simulated data, while the experimental demonstration can be found in Section 4. An example of a simulated digital signal can be seen in Fig. 2 (upper left panel), where the character of the signal is more apparent in a zoomed part of the signal from 0 to 0.5 ms (see inset).

 

Fig. 2. Simulated measurement of bi-exponential I_D with parameters A₁ = 1, A₂ = 15, $\tau$₁ = 40 µs, and $\tau$₂ = 5 µs using digital random signal with IRF_D = 1.45 µs. Upper left panel: excitation (blue line) and PL (red line) signal in time; upper right panel: amplitudes of Fourier components in excitation (blue line) and PL (red line); lower left panel: retrieved PL decay curve in a linear scale; lower right panel: retrieved PL decay (red line) compared to the original decay (blue line) in a semilogarithmic scale.

Download Full Size | PDF

Figure 2 also shows a simulated measurement of bi-exponential PL decay I_D (see Eq. (6)) using the aforementioned modulated I_EXC with a random distribution of duty cycle (random digital signal).

Figure 2, which shows the digital modulation, can be directly compared with Fig. 3, which presents the simulated measurement of the same bi-exponential PL decay I_D using a random analog I_EXC signal. In both figures, the parameters of the used bi-exponential I_D were A₁ = 1, A₂ = 15, $\tau$₁ = 40 µs and $\tau$₂ = 5 µs. Impulse response function (IRF) for random digital signal was IRF_D = 1.45 µs and for random analog signal it was IRF_A = 2.07 µs.

 

Fig. 3. Simulated measurement of bi-exponential I_D with parameters A₁ = 1, A₂ = 15, $\tau$₁ = 40 µs and $\tau$₂ = 5 µs using analog random signal with IRF_A = 2.07 µs. Upper left panel: excitation (blue line) and PL (red line) signal in time; upper right panel: amplitudes of Fourier components in excitation (blue line) and PL (red line); lower left panel: retrieved PL decay curve in a linear scale; lower right panel: retrieved PL decay (red line) compared to the original decay (blue line) in a semilogarithmic scale.

Download Full Size | PDF

By comparing the simulated PL decay retrieval for the digital and analog modulation, we can conclude, in agreement with our expectations, that both the approaches lead to identical PL decay. Therefore, digital modulation of excitation intensity can be used without losing the credibility of the retrieved PL decay. Although the spectrum of a digital signal features a narrower central peak in the Fourier space, it has higher amplitudes for higher frequencies. Therefore, the digital signal reaches a lower IRF width compared to the analog one.

We present the experimental confirmation of the agreement in Section 4. It is worth noting that the applicability of the analog modulation approach had been previously verified with a streak camera or TCSPC [21,23].

3.2 Direct PL lifetime map reconstruction

The original approach to PL lifetime imaging (FLIM) was based on the retrieval of a PL map for each delay after excitation, as described in Section 2.2. The real-life measurements showed that this procedure makes the post-processing routine highly time-consuming. Therefore, we propose an alternative approach that allows determining the FLIM spectrogram solely from the number of reconstructions, which equals the expected number of lifetimes in the measured sample. Namely, by determining a set of viable PL lifetimes in the sample, we can mathematically reconstruct the amplitude maps directly for the given lifetime $\tau$. This approach is very convenient for samples with a set of PL markers, mapping samples with distinct defects, or color centers emitting with well-defined lifetime.

In the cases where we know a priori the lifetimes present in a sample—for instance, a sample with a set of PL markers [28]—we can work with this information. However, in the opposite case, it is necessary to take the so-called zeroth step, i.e., illuminate the entire measured area of a sample with a homogeneous excitation spot and determine the PL decay curve I_DA0 representing the whole sample, using Eq. (4). The I_DA0 curve can be used to extract all present lifetimes via fitting. Since the entire spectrum of lifetimes is obtained from the zeroth step, it subsequently suffices to fit only amplitudes of each lifetime during the I_DA investigation (see text below)_. The literature states that it is appropriate to assume the fitting with a bi-exponential or tri-exponential decay [14]. Hence, we tested the new principle of post-processing for bi-exponential decay (see Eq. (6)).

The direct PL lifetime map reconstruction is based on the single-pixel camera concept. Therefore, it can use the same measurement routine. The measured sample was illuminated with random patterns (masks), which follow the temporal fluctuation of intensity according to I_EXC. The I_DA corresponding to each mask was reconstructed according to Eq. (4). The obtained I_DA curves were then fitted with a multi-exponential function with a fixed set of lifetimes. The fitting provided us with the amplitudes corresponding to the individual lifetimes.

If we consider the bi-exponential decay curve of the sample (see Eq. (6)), we obtain the parameters $\tau$₁ and $\tau$₂ from the fitted I_DA0. We assume the presence of $\tau$₁, $\tau$₂ in all measured I_DA curves and, therefore, we only fit parameters A₁ and A₂ of all I_DA curves. Because the sample is illuminated by the number of masks M (given by the compression ratio k), we also get M different values of A₁ and A₂, which creates vectors A₁SPC, A₂SPC of size M. Since the amplitudes A₁ and A₂ are a linear superposition of all PL decays within the illuminated area of the sample, we can employ the same retrieval algorithms as we use for the PL intensity map.

The knowledge of the used masks and A₁SPC, A₂SPC vectors can be used to reconstruct amplitude maps using Eq. (7). In this case of bi-exponential decay, we obtain two amplitude maps of lifetime H$\tau$₁ and H$\tau$₂.

In the case of mono-exponential decay, the reconstructed map H$_\tau$ represents the amplitude distribution for a given $\tau$. Nevertheless, if it is an n-exponential decay, the lifetime $\tau$ in each pixel (FLIM spectrogram) must be determined. There are multiple approaches to calculating the overall decay lifetime of a complex PL decay curve. Here we consistently used the weighted average:

The process of amplitude map retrieval is described in more detail in a scheme assuming the bi-exponential decay ($\tau$₁ = 20 ns, $\tau$₂ = 70 ns), as shown in Fig. 4. Amplitude distribution of $\tau$₁ can be observed in H$\tau$₁ map, and amplitude distribution of $\tau$₂ is shown in H$\tau$₂ map. Using Eq. (8) we calculated the total FLIM spectrogram with the mean lifetime.

 

Fig. 4. (A) Example of fitting I_DA curve amplitudes A₁ and A₂, where the distribution of $\tau$₁ and $\tau$₂ is already known from I_DA0 fitting. The amplitude fitting provides vectors A₁SPC and A₂SPC. (B) Reconstructed amplitude map H$\tau$₁. (C) Reconstructed amplitude map H$\tau$₂. (D) Calculated FLIM spectrogram based of knowledge of H$\tau$₁ and H$\tau$₂.

Download Full Size | PDF

The proposed approach significantly accelerated the post-processing procedure, and the whole FLIM spectrogram can be determined using n reconstructions for an n-exponential decay. Thus, in the case of bi-exponential decay, there are only two reconstructions of the undetermined system. Moreover, this approach offers excellent resistance to noise in the measured data, as shown in the next subsection.

Due to its principles, the presented direct lifetime retrieval is more suited for materials with several emitting species with sharp PL lifetime values, where the contribution of each species varies over the scene. In such a case, the method outperforms the standard approach, as we will show later. In the case where each pixel of the scene features a different lifetime value from a broad distribution, the method can still attain reasonable results by including more lifetimes and calculating the mean PL lifetime according to Eq. (8). Nevertheless, for a very broad lifetime distribution, the use of the standard frame-by-frame retrieval can be considered as a better option.

3.3 Comparison of noise effect on both post-processing approaches

The new approach to the FLIM spectrogram evaluation introduced in Section 3.2 significantly reduces the time required for post-processing compared to the original approach described in Section 2.2. In this section, we compare both approaches with respect to the quality of the obtained FLIM spectrogram with different levels of noise present in the measured data.

In our previous work, we showed that the quality of the retrieved I_DA and the 2D scene reconstruction is significantly more affected by the noise level present in I_PL compared to the I_EXC. Furthermore, it was shown that the quality of 2D scene reconstruction is only negligibly affected by the change in the compression ratio compared to the amount of noise in the system [25]. Therefore, for the following simulations, the compression ratio was always kept at k = 0.4, and we tested only the noise present in the PL signal I_PL, which we set to 0%, 0.5%, 1%, and 1.5%, respectively. The I_EXC signal was kept noiseless. The resulting FLIM spectrogram of lifetimes F was always compared with the ground truth U to extract the percentage error of the reconstructed image R:

The spectrograms obtained by the original approach described in Section 2.2 are labeled as FLIM_B, while the spectrograms obtained by the new, proposed approach are labeled as FLIM_A. The simulation results are summarized in Fig. 5, where the individual row represents the different noise levels in the I_PL signal, while the columns indicate gradually the H$\tau$₁, H$\tau$₂, FLIM_A and compare them to the FLIM_B. The dependence of the error R on the noise level is then provided in Fig. 6.

 

Fig. 5. Above the line is the ground truth, which can be compared to the simulated results below the line. The results of simulations of noise effect on both post-processing approaches. The new approach, represented through FLIM_A (the third column), is supplemented by reconstructions of partial amplitude maps H$\tau$₁ (the first column) and H$\tau$₂ (the second column). The results of the original approach are represented through FLIM_B (the fourth column). Each row corresponds to chosen noise level in the system (0-1.5%).

Download Full Size | PDF

 

Fig. 6. Evaluation of FLIM spectrogram reconstruction error—see Eq. (9)—via FLIM_A (direct PL lifetime map) and FLIM_B (frame-by-frame standard analysis) approach for a different amount of noise in a system (I_PL signal). Middle and right panel: Histograms of individual spectrograms for both FLIM_A and FLIM_B approaches (logarithmic y-axis).

Download Full Size | PDF

For this simulation, a scene with two areas $\tau$₁ = 20 ns and $\tau$₂ = 70 ns and corresponding amplitudes A₁ = 1 and A₂ = 1 was employed. The “sample” combines parts with a mono-exponential decay with a region featuring a bi-exponential decay.

The TVAL3 algorithm was used to reconstruct the undetermined systems in Eq. (7) [29,30]. In the case of FLIM_B, it was the intensity PL maps reconstruction, where the main parameters of the algorithm were mu (2¹¹) and beta (2⁷). In the case of FLIM_A, amplitude maps reconstruction was used, where the main parameters were set to mu (2⁹), and beta (2⁶). The mentioned parameters mu and beta were optimized for both approaches independently to attain the best reconstruction results with the lowest residues.

Reconstructed pixels that were below 10% of the PL amplitude were removed from the statistics for both FLIM_A and FLIM_B in Fig. 5 and also in Fig. 6. The fitting curves for both FLIM_A (I_DA's) and FLIM_B (3D-datacube) cases were always in the range of 0 ns to 400 ns, which is sufficient for the entered parameters $\tau$₁ and $\tau$₂.

For a rigorous discussion of results, it is important to realize that the FLIM_B results (original approach) are dominantly affected by noise in I_SPC (see Fig. 1(B)). This noise arises from inaccuracies in the reconstruction of individual PL maps m(x,y,t) when creating a 3D datacube (see Fig. 1(C)). Due to the character of the image retrieval, the noisy data entering the fit can cause a vast local error of the PL lifetime value in a few pixels. Therefore, FLIM_B spectrograms could deviate from the ground truth values locally, while the overall agreement could be maintained.

On the contrary, the FLIM_A approach could introduce an error already in the zeroth step of the process, when the whole sample is homogeneously illuminated, and the set of lifetimes contained in the sample is analyzed via I_DA0 fitting. The obtained lifetimes from the zeroth step are further considered during the remaining I_DA evaluation, where they are used to calculate the mean PL lifetime via amplitudes A₁ and A₂. Since the amplitude maps are calculated from an undetermined system with the condition of a low total image variation, we attained smooth images without local errors despite the present noise. On the other hand, the error of the initial lifetime fit propagates to the final FLIM spectrogram. Therefore, in our simulations, the reconstruction of 0% noise in the system, FLIM_B (original FLIM analysis) has a higher accuracy than FLIM_A. However, in a situation with a PL noise of 0.5% and higher, FLIM_A is more accurate than FLIM_B (see Fig. 6).

Although it may seem that an imprecise fit of PL lifetimes in the zeroth step will highly affect the FLIM_A results, our tests proved that this effect is not pronounced. We summarized in Table 1 all the lifetimes obtained after the zeroth step (I_DA0 fitting), which all suffer from the inevitable fitting error. Nevertheless, the total error of the FLIM_A spectrograms—see Fig. 6— increases with the noise in the fitted data and does not follow the lifetime errors. As an example, the worst zeroth-step lifetime fit is associated with the noise of 1% in the system (see Table 1), while the highest total reconstruction error is linked to the data with the noise of 1.5% (see Fig. 6). To outline the fitting issue, we present results obtained after a single I_DA0 trace fitting. However, the imprecise determination of lifetimes can be improved by statistical averaging over several measurements in the zeroth step.

Table 1. Summary of results of I_DA0 fitting in zeroth step of FLIM_A determination process.

View Table

Figure 6 also shows histograms of individual spectrograms for both the FLIM_A and the FLIM_B approaches. Note that the y-axis uses a logarithmic scale. The spectrograms obtained by the FLIM_B approach show an increasing deviation of lifetimes with an increasing amount of noise. The increasing noise also blurs the spectrograms obtained by FLIM_A but a higher number of pixels is accurately determined by this approach than by FLIM_B. This is a result of the zeroth step because the use of predetermined lifetimes constrains the resulting values and does not allow for significant deviations outside of the set range.

Overall, it can be stated that the direct PL lifetime map reconstruction (FLIM_A) is more stable in terms of noise. The standard frame-by-frame retrieval (FLIM_B) may be locally accurate, while some pixels may be entirely out of scale, and it is difficult to determine whether this is an effect of the noise or an actual sample property.

It is beneficial to address new methods for accurately determining the parameters of exponential curves [31], which would further improve both of the approaches considered. Nevertheless, the new approach is less time-consuming for post-processing than the standard one.

4. Experimental implementation

4.1 Optical setup

The advanced optical setup stemmed from the original proof-of-principle experiments [23]. Its scheme is shown in Fig. 7.

 

Fig. 7. The 2D-RATS method in a single-pixel camera configuration implemented into a microscope optical setup.

Download Full Size | PDF

Temporal modulation was ensured via a modulated Cobolt S06-01 MLD (405 nm) laser. The modulation signal was produced by the Digilent Cmod A7 development kit and was generated in the FPGA Xilinx Artix-7 (VIVADO software package). The bitstream is generated via a Linear Feedback Shift Register (LFSR) from flip-flops and XNOR gate feedback, configured in FPGA. The output of the LFSR meets many randomness tests [32]. Using the above-mentioned configuration, one can generate any bitstream with the desired repetition period of the random signal. Thus, it is possible to choose as long a period as necessary for one measurement and thus avoid the effect of I_EXC periodicity described in our previous work [25].

A temporally modulated laser beam is expanded using a GBE20-A Thorlabs beam expander and guided to the digital micromirror device (DMD) through mirrors M₁ and M₂ so that one of the reflected beams is reflected in the normal direction of the DMD chip surface. In order to achieve an even distribution of the DMD-generated illumination pattern, the laser beam diameter is magnified 20 times in front of the DMD, so it significantly exceeds the dimensions of the DMD chip.

The DLPLCR65EVM DMD from Texas Instruments was controlled via a DLPLCRC900EVM module. We used 35 × 35 pixel patterns both in the simulations (Section 3) and in the demonstration measurements (Section 4). However, rather than the entire DMD chip size, only a limited part of the chip in its center was used to generate patterns. This enabled us to use a low numerical aperture in the setup.

Since the DMD pixel size is on the micrometer scale (7.56 µm), diffraction on the pixels should be taken into account [33,34]. Therefore, the diffracted beam from the DMD is collected with an AR-coated fused silica lens L₁ (Ø50mm, f = 75 mm) and then filtered using a low pass filter in the configuration of L₁ and the second AR-coated fused silica lens L₂ (Ø25mm, f = 50 mm) and iris aperture. Hence, only one diffraction order passes through the low pass filter and hits the Thorlabs N-BK7 beam splitter BS (CM1-BS013 (50:50)). The reflected part of the beam is targeted on a PDA10A2 photodiode for I_EXC detection, while the transmitted part of the beam goes through a 405 nm bandpass filter F₁ (Thorlabs FBH405-10) to avoid potential parasite I_PL generation in N-BK7 glass (BS).

Subsequently, the beam is transmitted through a dichroic mirror DM (Thorlabs DMSP425, 425 nm cut-off) and imaged by a microscope objective—a 4X Olympus plan achromat objective (0.10 NA, 18.5 mm WD). Using the 4X Olympus plan achromat lens, we ideally get a single pixel size of 12.6 µm and a field of view of about 450 × 450 µm. The stated values were confirmed by means of calibration grids with a defined line spacing of 50 µm.

The illumination pattern imaged by a microscopic lens on a sample can be captured using a CMOS camera (IDS UI-3240ML-M-GL) to find the optimal focus. The microscopic lens also collects the generated I_PL signal, which is reflected and focused on a photomultiplier (Hamamatsu H10721-20) through a dichroic mirror and lens L₃ (Ø25mm, f = 12 mm). Prior to PL detection, the collected light is filtered by the Thorlabs 500 nm cut-off filter F₂ (FELH0500) to avoid a possible back reflection of I_EXC. Thus, all decay curves correspond to PL in a range above 500 nm of wavelength.

The temporal waveforms of both the I_EXC and the I_PL signals are collected via the TiePie Handyscope HS6-1000XM. The impulse response function (IRF) depends on the modulation frequency and many additional parameters and we provide the IRF width for each experiment.

In the current configuration, where we used a cw-operated laser at 125 mW, we attained an intensity of the excitation light of 0.35 mW at the sample plane. This means that the setup efficiency was about 0.3%. Note that the use of the excitation laser efficiency was not optimized, and it was highly affected by using an excessive beam diameter at the DMD, the use of a part of the DMD chip, the selection of a single diffraction order, or the beam splitter ratio—all of these parameters can be tuned to improve efficiency. Still, we reached an efficiency a hundred times higher compared to the proof-of-principle diffusor-based 2D RATS setup (whose efficiency was approximately 0.003%) [23].

4.2 Digital and analog modulation of the excitation signal

By comparing the simulated PL decay retrieval for the digital and analog modulation in Section 3.1, we concluded, in agreement with our expectations, that both the approaches lead to identical PL decay. We used the new optical setup to carry out the same comparison experimentally.

Both approaches (analog vs. digital random I_EXC) were compared in a single-point measurement (0D-RATS) configuration, where the SCHOTT OG565 absorption filter was used as a test sample. The setup described in this article was used for the digital excitation modulation, while the proof-of-principle setup was employed for the analog excitation modulation [21]. For the analog modulation, we reached an improved analog modulation speed by tightly focusing an expanded excitation beam (f = 25 mm, spot size 2.5 µm) on a fine diffusor produced with SiC1200 abrasives. Both the beam distance from the diffuser center of 115 mm and the rotation speed of 100 Hz contributed to the fast modulation. Therefore, the IRF width for the analog case was w_A = 71 ns. The digital modulation was adjusted to reach a similar IRF width of w_D = 47 ns.

To gain a rigorous comparison, the I°_D curve obtained using the random digital signal was convolved with the Gaussian function G(w) with a FWHM w equal to the root mean square difference of w_A and w_D values:

The experimental results in a semilogarithmic representation are presented in Fig. 8. They confirm that the digital and analog modulation of the signal can be equally employed in the PL decay analysis and lead to the same decay functions.

 

Fig. 8. Comparison measurement of OG565 filter using analog and digital random I_EXC with w_A = 71 ns and w_D = 47 ns.

Download Full Size | PDF

4.3 Temporal resolution of digital modulation of the excitation signal

By employing a thorough optimization of the diffuser-based intensity modulation, we reached the IRF width of 71 ns, as stated before. We now turn to show the potential of digital modulation using the Cobolt S06-01 laser. Here, we tested the variation of both the laser modulation frequency and the sampling rate—see Fig. 9. We can see that the fastest combination leads to the IRF width of 6 ns, which is an order of magnitude below the optimized diffuser-based modulation.

 

Fig. 9. IRF measurement for different modulation frequencies and different sampling frequencies.

Download Full Size | PDF

That is primarily due to the laser parameters, especially modulation. Hence, it is easy to generate a much faster random signal than using the previously described rotary diffuser-based generator [21]. Apart from this, Fig. 9 clearly shows that the IRF width is not strongly dependent on the modulation frequency. This behavior is the consequence of the rectangular character of the digital signal. From the general principles of rectangular signal generation, the signal contains rapidly rising and falling edges carrying significantly higher frequencies than the set modulation frequency of the digital signal. These frequencies would be revealed with sufficient sampling and bandwidth. Therefore, ideally, a digitizer with significantly faster sampling and a broader bandwidth should be used for such a (i.e., modulated) signal.

The employed HS6-1000XM TiePie USB-handyscope allows sampling of up to 500 MSa/s with a bandwidth of 250 MHz when scanning on two channels simultaneously. Hence the Nyquist condition for the fastest bit carrier frequency was met, but the frequencies of leading and trailing edges were truncated. As a result, the IRF width strongly depends on the sampling rate, as shown in Fig. 9.

However, since the carrier frequencies are not truncated, no aliasing occurs and, thus, the resulting I_D reconstruction is not distorted. To confirm this statement, we plot in Fig. 10 the PL decay of the LuAG:Ce luminophore for various sampling options. As in Fig. 9, each measurement was convolved with a Gaussian function to match the largest IRF width within the measurements. This allows for a direct comparison of the curves. We did not observe any notable variation in either the decay curves or the corresponding PL lifetime for the sampling rates well above the PL decay rates.

 

Fig. 10. Comparison of I_D measured on LuAG:Ce for different sampling frequencies (different IRF width). Measurements with lower IRF width were convolved with the root square deviation of IRF width related to measurements with the highest IRF width.

Download Full Size | PDF

In total, a RATS setup with digital signal modulation, which is not limited by the bandwidth of the detector and the corresponding electronics, has the ultimate resolution due to the steepness of the rectangular laser signal edge and the sampling rate.

4.4 Demonstration measurement

The 2D-RATS experiment based on the advancements presented in Section 3.2 was experimentally implemented into a microscope setup based on a single-pixel camera configuration. The technical details of the setup are described in Section 4.1. Here we provide an example of data acquired by the advanced setup and data processing.

We selected a LuAG:Ce scintillation crystal as a test sample [35], which features strong PL emission. The test sample was a thin polished monocrystal plate featuring microscopic scratches and cracks on both sides. One of the cracks was selected as the testing spot since we expected a significant increase in PL intensity at the defect. The PL intensity is higher due to high scattering, which improves PL signal outcoupling from the sample.

A 4X Olympus plan achromat objective (0.10 NA, 18.5 mm WD) was used to map the testing area of 450 × 450 µm with a pixel size of 12.6 µm (35 × 35 pixels). The compression ratio was set to k = 0.4 (490 measurements) with a single PL decay dataset measured over 10 ms. It is worth noting that the total acquisition time, which was on the order of tens of minutes, was caused dominantly by the data transfer and data handling, which can be significantly optimized. The sampling frequency was 200 MHz and the modulation frequency of the fastest bit of the random signal reached 98 MHz, which resulted in an IRF width of 9.9 ns.

In the initial analysis, we tested the measurement for a more general bi-exponential fit, even though we expected the LuAG:Ce to show a mono-exponential decay. The mapped region and the corresponding I_DA0, together with the IRF, are shown in Fig. 11. Fitting the I_DA0 revealed the PL lifetimes of $\tau$₁ = 11 ns and $\tau$₂ = 67 ns.

 

Fig. 11. Left panel: Measured area of the LuAG:Ce crystal. Right panel: Corresponding I_DA0 with fitted bi-exponential curve and revealed spectrum of the $\tau$₁ and $\tau$₂.

Download Full Size | PDF

The results of mapping H$\tau$₁, H$\tau$₂, and the FLIM spectrogram are presented in Fig. 12. The map of the first component can be assigned to random scattering points and scattered light along the crack. The map of the second component closely follows the shape of the crack, where the PL from LuAG:Ce is efficiently coupled out from the crystal. The FLIM spectrogram combines both amplitude maps. The second lifetime $\tau$₂ dominates, owing to its high amplitude. The brightness of the individual pixels in the FLIM spectrograms was scaled according to the PL intensity, i.e., the sum of the amplitudes A₁ and A₂ in a given pixel. In Fig. 12, it is also possible to observe the overlay of the measured area and the obtained FLIM spectrogram. The overlay documents the agreement between the PL mapping and the sample properties. It is worth noting that only pixels with PL intensities greater than or equal to 10% of the maximum are shown in the FLIM spectrogram.

 

Fig. 12. Upper panels: Reconstruction of H$\tau$₁ and H$\tau$₂ areas corresponding to expected lifetimes. Lower panels: FLIM spectrogram determination and its overlay with the measured scene.

Download Full Size | PDF

As we stated before, the assumption of the bi-exponential decay of LuAG:Ce was not physically correct. Moreover, $\tau$₁ is close to the IRF width, which distorts A₁SPC. Therefore, we also provide an analysis of the PL, where we assume a mono-exponential decay. Such a situation would, for instance, correspond to the mapping of a biological sample with a single PL marker. For the mono-exponential decay, we obtained only one H$\tau$ region for the lifetime of 59 ns, which corresponds well with the PL lifetime of LuAG:Ce [36]. The H$\tau$ is directly represented in the FLIM spectrogram, as shown in Fig. 13. We can see that in this case we attained a clear image of the crack with high PL intensity along with low-intensity PL regions surrounding the cracks where the PL is outcoupled from the monocrystal inefficiently.

 

Fig. 13. Upper panels: Fitted I_DA0 with mono-exponential curve and reconstructed area H$\tau$₁ for corresponding $\tau$₁. Lower panels: FLIM spectrogram determination and its overlay with the measured scene.

Download Full Size | PDF

5. Conclusion

Our paper outlines two significant advancements for the RATS method for measuring PL dynamics and FLIM. The first is the possibility of random excitation via a digitally modulated signal. Using this approach, it is possible to achieve a time resolution on the order of units of nanoseconds and to significantly simplify the optical setup. Secondly, we also present a new approach to the evaluation of the FLIM spectrogram, which significantly reduces the number of necessary reconstructions of the undetermined system and shortens the post-processing time accordingly. Moreover, the new approach to data processing reduces the required number of fitted curves in proportion to the chosen compression ratio k.

The original and the newly proposed approach to FLIM retrieval were compared in simulations regarding noise analysis. The new approach was demonstrated to reduce the possibility of locally incorrect lifetime determination. Therefore, the proposed amplitude map retrieval is more robust against increasing noise level compared to the standard analysis.

The advancements were implemented in a microscope setup based on the single-pixel camera technique. A digital micromirror device (DMD) was used to generate random patterns (masks), which also enables full illumination of the scene to determine lifetimes within the sample. In addition, the combination of a DMD and the random digital modulation of the laser increases the efficiency of the optical system a hundredfold compared to the original arrangement, which used diffusers [23].

We analyzed the advancements on synthetic data, as well as on testing measurements. Namely, we carried out the imaging of a LuAG:Ce crystal, where a crack in the crystal was mapped. The resulting FLIM spectrograms from the PL analysis were in perfect agreement with the camera images.

Overall, the newly proposed methods lead to a simplification and a higher efficiency of the measurement process. Their potential for saving resources makes them suitable for wide application and deserves further investigation.