1. Introduction

More than a hundred years ago, the diffraction limit emerged as the ultimate resolution limit for an optical microscope. It pinpoints a fundamental property of light and places a strict limit on the bandwidth of any image formed by an intensity pattern of light. Today, the field of optical nanoscopy [1] has introduced a plethora of ways [2–5] in which finer spatial information can be obtained by combining optical systems with the state transitions of conventional or specifically engineered fluorescent molecules.

Within the optical nanoscopy methods, coordinated targeted switching approaches such as STED [6] and RESOLFT [7,8] improve the spatial resolution by shrinking the effective region in the sample where the fluorescence signal is generated. The shrinking can be induced by inhibiting the emission of fluorescent photons generated by molecules located at the periphery of the focal spot (or spots). The inhibition is obtained via stimulated emission (STED) or by the long-lived ON-OFF switching states (RESOLFT) commonly found in reversibly switchable fluorescent proteins (RSFPs) [9,10] or dyes [11]. The photoswitching characteristics in RESOLFT allow to decrease the intensity requirement during imaging, thus facilitating live-cell imaging applications [12]. This new spatial resolution ability strictly depends on the complex interplay of optics and fluorophores’ photophysics introducing new challenges in modeling and predicting the performance of super-resolution imaging systems. Also, the signal generated through photoswitching brings along new information, which can be used to better estimate sample properties such as the number of labeled molecules [13] or a new source of noise [14], which can contribute in non-trivial ways to image quality. The same state transitions can contribute to the fluorescence signal generation in a non-intuitive way. In fact, each molecule can populate the ON and OFF states for a different duration [15], resulting in distinct photon emission or inhibition.

Also, cross-talk between OFF and ON switching can affect the extraction of photons carrying the finest spatial information [16]. Due to this complex behavior, the quantification of the achievable resolution of such imaging systems requires a more comprehensive description of the image formation model. Here we present a theoretical framework to derive such a model with the inputs of the imaging scheme and importantly, sample and fluorophore dependent properties such as photoswitching rates, labeling density, brightness, etc. This model can describe both positive and negative switching schemes in RESOLFT microscopy. To our knowledge, this is the first comprehensive framework describing the full image formation model including fluorophore switching behaviour, imaging parameters and sample structure type as well as predicting the effective spatial resolution. The completeness of the model allows us to quantitatively predict the signal-to-noise properties of the final image. Furthermore, we analytically derive the predicted Fourier ring correlation (FRC) [17,18], which we use as a metric for resolution estimation in our theoretical framework. This allows us to dissect the impact of different parameters on the achievable resolution of the RESOLFT imaging system and can be generalized to other forms, diffraction limited or not, of fluorescent-based point-scanning microscopy. The model not only guides a selection of optimal imaging parameters and future fluorophore engineering, but will likely also serve as a fundamental basis for the development of more sophisticated and accurate inversion algorithms.

2. Unidirectional and bidirectional switching noise

The number of fluorescent photons emitted by a conventional fluorophore is usually modeled as a Poisson distributed stochastic variable $\mathcal {P}(\mu )$ within what is commonly referred to as the linear regime, with $\mu$ scaling linearly with both the intensity and the duration of the excitation light. As the relative noise is inversely proportional to $\sqrt {\mu }$, longer exposures and/or higher intensity in the excitation are routinely used in fluorescence microscopy to increase the signal to noise ratio. However, it was brought to the attention of the community that the existence of state changes of the fluorophores invalidates this assumption under certain conditions [14]. The theory presented therein mainly treats the unidirectional state change where a fluorophore stochastically and irreversibly enters a non-fluorescent state (Fig. 1), which describes the current understanding of photobleaching. First, a fluorophore is in the fluorescent state during a period of time, which is an exponentially distributed random variable. While in the fluorescent state it will emit photons, and the number of photons is a Poisson distributed random variable. Then, it irreversibly enters a non-fluorescent state. Thus, the total emission will be a doubly stochastic process. For the scenario described above, the expected value and variance of the number of emitted photons will be [14] [19, chapter 9.2]

 

Fig. 1. Schematic illustration of fluorophores exhibiting unidirectional switching. This model is commonly used to model the process of photobleaching. The lifetime of the fluorescent state is a Poisson distributed stochastic variable.

Download Full Size | PDF

For the case of unidirectional switching, meaning that fluorophores always start in the fluorescent state and stochastically break at a rate $r_{OFF}$, the expectation and variance of

$\Phi$ during observation time $T$ can be analytically determined as

For the case of switchable fluorophores such as reversibly switchable fluorescent proteins (RSFP), the switching between states is not necessarily unidirectional [13,20]. The progression of states of RSFPs can instead be modeled as a two-state Markov process with transition rates determined by the intensity and wavelength of illuminating light, as Fig. 2 illustrates. The relationship between the rates and the intensity is often assumed linear in the low intensity regime [13,20]. The rates $r_{ON}$ and $r_{OFF}$ define the rate of switching from the OFF to ON and ON to OFF state respectively. We then calculate the probability of an RSFP being ON at time $t$ as $p_{ON}(t)$.

 

Fig. 2. Schematic illustration of fluorophores exhibiting bidirectional switching. This model is commonly used to model the process of photoswitching of the type characterising e.g. reversibly switchable fluorescent proteins.

Download Full Size | PDF

3. Image formation for point scanning microscopy

There are many different modalities of point scanning microscopy systems both diffraction limited and not. Examples include confocal, two-photon, STED and RESOLFT. Applications and implementations of these systems can be very different, but their image formation processes can all be modeled with a common framework. The basic principle of a point scanning microscope is that the illumination, usually focused light, is sequentially scanned point by point with respect to the sample to probe the fluorescence signal. For each scan step, the illumination is shifted a distance $(i, j)$ with respect to the sample to form the final image. We will first focus on a single scan step in the acquisition. Here the illumination light interacts with the sample inducing photon emission from a subset of molecules. The number of emitted photons from any point in the sample will be a discrete stochastic variable with an expected value and variance determined by the illumination at the location of the fluorophore. For a given spatial distribution of the illumination pattern, the fluorophore emission will in turn depend on its spatial coordinate. It is thus convenient to here introduce the concept of the spatial emission kernel of a point scanning system $\mathcal {P}(x, y) = \mathcal {P}_{x, y}$, where $x = y = 0$ is the center of the illumination pattern in any scan step. Note here that $\mathcal {P}_{x, y}$ denotes the same parameter as $\mathcal {P}$ in section 2 with added spatial dependence due to patterned illumination. Each $\mathcal {P}_{x, y}$ is then a stochastic variable with expected value $\mathbb {E}[\mathcal {P}_{x, y}]$ and variance $Var[\mathcal {P}_{x, y}]$. $\mathbb {E}[\mathcal {P}_{x, y}]$ is then the expected number of emitted photons from a fluorophore located at $(x, y)$ and $Var[\mathcal {P}_{x, y}]$ is the variance of the number of emitted photons from a fluorophore located at $(x, y)$. It is worth emphasizing here that apart from the illumination, the values of $\mathbb {E}[\mathcal {P}_{x, y}]$ and $Var[\mathcal {P}_{x, y}]$ heavily depend on the switching rates and emission rate of the fluorophores used, as seen from Eq. (1), Eq. (2) and Eq. (6). In section 5 we will examine the properties of $\mathcal {P}_{x, y}$ in the context of RESOLFT imaging with reversibly switchable fluorescent proteins as labels.

Having introduced the notion of $\mathcal {P}_{x, y}$ describing the expectation and variance of the emission from a single fluorophore at position $(x, y)$, we readily note that if the fluorophores in the sample are assumed to be independent of each other, then the expectation and variance of the total emission, $Em(x, y)$, from a spatial coordinate given the fluorophore density in the sample, $d(x, y)$, is

For a conventional point scanning system, the photons emitted from the illuminated area in the sample during a scan step will be collected by the microscope objective and focused again in the image plane, $I_{P_1}$, of the detection path. The expected image at this plane is described as the convolution between the expected emission with the point spread function (PSF) of the optical system, where the integrated sum of $PSF(x, y)$ equals the collection efficiency of the full system.

4. Derivation of resolution measures

The spatial resolution of an optical system can be estimated with different methods, both in the spatial and frequency domain. One commonly used method for resolution estimation is Fourier ring correlation (FRC). This resolution measure has a well-defined mathematical expression and is well suited for integration into our analysis. An FRC value is commonly taken as the sample correlation between the discrete Fourier transforms of two images, $f_n$, at a certain radius in frequency space.

We define an image $\mathcal {I}[m, n]$ as a sampled observation of the stochastic signal $\mathcal {S}(i, j)$, and postulate that $\mathcal {I}[m, n]$ can be divided into two components, a noiseless signal component and a pure noise component.

In the point scanning image formation model, the image of a sample density $d(x,y)$ is $\mathcal {I}[m,n]$. If the image $\mathcal {I}[m,n]$ is sampled sufficiently high so that aliasing is negligible, we can express

4.1 Factors affecting resolution

As seen from Eq. (35), the final image resolution is clearly dependent on not only the shape and power of the illumination, but also on the response of the fluorophores and not least the properties of the sample and labeling parameters. In section 3 and 4 we simply denoted $d(x,y)$ as fluorophore density. If discussing labeling density however, we can further model $d(x,y)$ as being the product of a sample density $\kappa (x,y)$, representing the density of target molecules in the sample, and a scalar value $\beta$ representing the labelling density, or number of fluorophores per target molecule. We can now express $d(x,y)$ as

Another affecting factor, clearly seen from Eq. (35), is the scanning step size $\Delta _i \Delta _j$. The reasoning behind it is that by sampling our image more times we will increase the total power spectrum of the signal quadratically, while the noise component will only scale linearly. However, it should be noted at this point that massively decreasing the scanning step size will slow down the imaging and deteriorate the signal through photobleaching. Finally, the switching and emission properties of the fluorophores naturally affect the resolution as they are the main factors defining the $H_E$ and $H_{Var}$ functions. We will explore these aspects in section 6.

5. Image formation for RESOLFT microscopy

$\mathcal {P}$ is introduced in section 2 as a random variable describing the number of photons emitted by a conventional fluorophore, characterized by $\mathbb {E}[\mathcal {P}]$ and $Var[\mathcal {P}]$. In section 3 we introduced the concept of spatial emission kernel of a point scanning system, where the emission of photons is dependent on the spatial position.

In point scanning and parallelized RESOLFT nanoscopy, the relevant photon emission for each scanning point takes place around the spatial region defined by the zero (or "zeros") intensity area of the OFF pattern (i.e., the doughnut, periodic line patterns). Ideally, all the RSFPs located in the periphery of zero should be switched off and not emit. However, since most of the RSFPs appear to have a residual probability of switching back ON, even under OFF-switching illumination, we will not have, at least with currently available RSFPs, a perfect confinement of the ON-state population. Using the equations derived in section 2, we can readily add the spatial dependence of the illumination intensity and firstly calculate the probability of a fluorophore at position $(x,y)$ being in the ON-state, and secondly, derive the expected emission and its variance at position $(x,y)$ during a certain observation time. Figure 3 illustrates these spatial functions during a representative RESOLFT imaging scheme.

 

Fig. 3. Figure showing a) an example sequence of different illumination patterns during a typical RESOLFT imaging scheme and b) the spatial distribution of fluorophores in the ON state, $p_{ON}(x,y)$, resulting from such a sequence at two different time points. Panel c) and d) shows the spatially dependent expectation and variance of the fluorescent photon emission after different read-out times.

Download Full Size | PDF

6. Simulations

Equation (35) allows to evaluate the effect of different parameters on the FRC value, but it is not trivial to exactly predict the relative increase or decrease in effective spatial resolution from a given change in a parameter. To get a better sense of the behavior of the system, we created a tool to numerically predict the resolution given a set of parameters of the fluorophores, the illuminations, the optical system and the sample. The fluorophore, illumination and optical parameters allow us to predict the $H_E$ and $H_{Var}$ kernels, and the additional knowledge of the sample power spectrum allows for the full estimation of the FRC curve from which we can extract the resolution measure.

Additionally, in order to corroborate that the simulations accurately predict the FRC resolution of a system exhibiting those parameters, we used a previously developed simulation tool [20] to simulate images acquired with a RESOLFT microscope, where fluorophores are simulated individually according to the same Markov model as presented in section 2. The simulation tool allows us to simulate two or more images of exactly the same structure but independent noise. We can then measure the FRC of those images and compare to the predicted values. As a first investigation, we tagged a few relevant questions and examined how the absolute resolution depends on realistic parameters. For the results presented below, we have assumed that the sample being imaged is spectrally flat (Fig. 4(a-b)), meaning that the average power of the Fourier transform of the sample density is independent of frequency. In Eq. (35) this translated to $\phi _D$ being constant over $r_k$.

 

Fig. 4. a) For the FRC resolution predictions a spectrally flat sample is assumed. b) After imaging, the power spectrum of the image reflects the shape of the transfer function. c) Graph showing the predicted resolution in blue line depending on the read-out time used. Yellow dots show the measured FRC resolution from simulated images. d) Plot shows the predicted resolution normalized for each curve between its lowest and highest value within the plotted range in order emphasize the shift in optimal read-out time. Values are plotted against the read-out time for four different fluorophore emission rates. e) Plot shows the predicted resolution depending on the power spectrum of the sample. Average sample density is kept constant. Yellow dots show the measured FRC resolution from simulated images. f) Plot shows the predicted resolution depending on read-out time for four different sets of parameters. For all four fluorophores, the product of emission rate and labeling density is kept constant. g) Lines in graph plot the predicted resolution depending on the saturation factor used for the OFF-switching light. Blue line predicts the resolution in the ideal case of zero intensity in the center of the OFF-illumination. Red line shows the imperfect case where the center of the OFF-illumination sees an illumination power 3% of the maximum illumination power. Yellow dots show the measured FRC resolution from simulated images with zero intensity in the center of the OFF-illumination. h) Plot shows the predicted resolution depending on the scanning step size used for two different sample powers.

Download Full Size | PDF

The question of optimal read-out time is inspired by the work of Cooper et al. [14]. By setting the fluorophore parameters such as brightness, ON-OFF rates and background, to mimic those experimentally measured for rsEGFP2 we evaluate the FRC for different read-out times. The finest resolution is achieved with an optimum read-out time at around 0.5 ms, Fig. 4(c). This can be understood by considering that the relative uncertainty of the detected photons has two components. One is the uncertainty of the total ON-time for each fluorophore, and one is the uncertainty of the random photon emission while the fluorophore is in the ON-state. At short read-out times, the total uncertainty will be dominated by the uncertainty from the random photon emission. For longer read-out times, this uncertainty will decrease while instead the uncertainty from the random total ON-time will increase. Somewhere in the middle, there is an optimal time where the total relative uncertainty is minimal. From Fig. 4(d), we can see that the exact value of the optimal read-out time also depends on other parameters of the system. Here we only varied the emission rate $r_{fl}$ and see that higher emission rates not only increase the maximum achievable resolution but also shifts the value of the optimal read-out time to shorter times.

An interesting sample-dependent parameter that is often not included in the discussions of the effective resolution of microscopy systems is the structure of the sample being imaged. As seen in Eq. (35), and as may be intuitive from the formulation of the image formation process, both the power spectrum and the average sample density are relevant for predicting the FRC resolution. In Fig. 4(e) we plot the predictions of the FRC resolution for different sample powers, keeping the average sample density constant. This can be seen as increasing the sparsity of the sample. Indeed, sparser samples, such as punctate structures, have a higher ratio of spectral power to average value than dense structures such as membranes. The plot clearly shows that as the spectral power increases, so does the resolution if the average value is kept constant i.e. images of sparser structures feature the finest spatial resolution.

As discussed in section 4.1, also labelling density is a relevant sample parameter. In Fig. 4(f), we show four curves resulting from predictions where the product of fluorophore emission rate and labeling density is kept constant. This means that the total emission rate from any point in the sample is kept constant. Despite this constant product, we see that increasing the labeling density still increases the spatial resolution. This suggests that at least within this range, it is valuable to increase the labeling density of the sample, even if that correlates to a proportionally similar decrease in emission rate, or brightness, of the individual fluorophores.

Another parameter that is commonly considered to be of utmost importance for final spatial resolution is the light intensity used to drive the RSFPs in the OFF state, which increases the spatial confinement of the region that generated the fluorescence signal. In the ideal scenario where the illumination at the center of the light pattern is perfectly zero, it is quite straightforward that higher OFF-switching light intensities will improve the final spatial resolution. The blue curve in Fig. 4(g) shows that our predictions agree with this assertion. In a more realistic scenario, however, the intensity in the center of the pattern might not be a perfect zero. Therefore, we added a prediction for the case where the intensity in the center is 3% of the maximum intensity of the pattern. In this case, we can see that increasing the illumination power does not strictly increase the resolution but instead there is an optimal parameter.

A final parameter that we investigated is the effect of scanning step size on image resolution. Just like many other parameters, we can tell from Eq. (35) that decreasing the scanning step size will strictly increase the predicted FRC resolution. In Fig. 4(h) we plot the absolute resolution values over a range of scanning step sizes for two different sample powers. In those curves, as expected, the resolution improves with smaller scanning step size but we can also draw the conclusion that the scanning step size is more influential for lower spectral power samples.

7. Discussion

In the previous sections, we presented a framework to describe the characteristics of the acquired data from a point scanning microscope. We describe the pixel values of the image as being the raw data values acquired from the acquisition. For most commonly used microscopes, this very well represents the workflow used. We acknowledge that there are also many ways to process the data in order to create a more useful representation of the likely underlying sample, such as the many different available deconvolution algorithms. However, the FRC values of the raw data are accurate and quantifiable measures of the information content available at different spatial frequencies. One should also note that although well-known and commonly accepted by many, the FRC resolution is only one way to define resolution. We use this resolution measure since it clearly relates to the ratio of signal power to noise power at a certain spatial frequency. Therefore, it estimates how well the acquired data reflects the underlying frequencies of the specimen.

In section 6 we discuss and show the impact of a number of parameters, both sample and image-acquisition dependent, on the achievable spatial resolution in RESOLFT microscopy. RSFPs show an optimal read-out time, which does not coincide with the complete OFF-switching of the whole RSFPs population as one could predict. It is instead found somewhere in between, due to the additional contribution of the photoswitching noise. The calculation of this value can easily be extended to different RSFPs variants by varying the switching rates $r_{ON}$ and $r_{OFF}$ in the formulation of $\mathcal {P}$, facilitating the search for the optimal imaging scheme. Also, the impact on imaging of RSFPs with different background levels can be calculated by changing the ratio between the switching rates. Therefore, we can even use this simulation tool to screen for better RSFPs by pinpointing the set of parameters that are crucial for the imaging.

Labeling density together with fluorophore brightness also affects the spatial resolution. Interestingly, a high labeling density of dim fluorophores is found to be a better option than bright fluorophores with poor labeling when searching for the best spatial resolution. Finally, we explore a parameter which is poorly quantified when discussing the spatial resolution, which is the structure of the sample, here defined by the power spectrum. Our simulation show that sparse samples give images with higher resolution than dense samples, thus confirming a clear dependence of the achievable spatial resolution to this specific sample characteristic.

In this work, we discussed only a few representative imaging and sample conditions, but our theoretical framework can be generally applicable to many other RSFPs characteristics or imaging schemes.

8. Conclusion

As advanced light microscopy techniques push the limits for extracting high resolution information about a sample by taking advantage of the properties of the fluorophores, it becomes increasingly important to also advance our understanding about the processes underlying the formation of this new imaging data. For example, how photoswitching or labeling densities impact the statistical properties of the acquired image. Therefore, modelling the final image dependence on parameters of the sample, fluorophores and imaging system, can help to further increase the achievable spatial resolution of the imaging system, as well as streamline the workflow of experiments by minimizing the time required for empirically finding the optimal imaging scheme. The work presented herein introduces such a framework for the case of point scanning microscopes, focusing specifically on systems based on the RESOLFT principle. We show how the switching response of the fluorophores impacts the optimal choice of parameters, but also stress the importance of acknowledging that many parameters outside the actual imaging system influence the final FRC resolution of the image. We believe this framework and the prediction tools developed can be of considerable benefit for the microscopy community and also represent a step forward in the basic understanding of the systems so commonly used in the life sciences. Finally, these simulation tools can serve as a guide for rational development of new probes and labeling strategies featuring a combination of characteristics that will result in the best spatial resolution.