1. Introduction

The combination of genetically encoded fluorescent calcium indicators [1,2] (GECIs) and various new forms of one-photon [3–18] and multi-photon microscopy [19–26], and especially the recent emergence of high-speed volumetric imaging technologies [21,24], have revolutionized neuroscience by enabling population-level neuronal activity recording at high speed and resolution across a wide range of model systems. For recording in weakly scattering specimens such as larval zebrafish or C. elegans, a variety of one-photon (1P) techniques have been developed, from non-neuron-resolved widefield imaging [3,4] to approaches that rely on scanning of confined excitation pattern in one or multiple dimensions to achieve neuronal resolution and optical sectioning. These include confocal microscopy [5], various forms of Light Sheet Microscopy [6–9], and approaches based on Bessel beams [14,15]. Techniques that leverage computational signal extraction such as light field microscopy (LFM) [10,11,27,12] and other computational volumetric point-spread function (PSF) encoding techniques [16,17,18] offer the advantage of relying on relatively simple optics only and achieving high volumetric frame rates without the need for scanning. Initially applied to transparent specimen only, LFM has recently been extended into the more strongly scattering mammalian brain by exploiting spatio-temporal demixing of GECI activity signals [27,12,13].

Generally, 1P-based neuronal imaging is restricted to the regime of ballistic and quasi-ballistic propagation of photons in the brain tissue, where a large or at least a discernable fraction of fluorescence emanates from the sample unscattered or only weakly scattered. The scattered fraction constitutes background and gives rise to crosstalk. In this regime, methods capable of suppressing the effects of background either through optical means (as in confocal microscopy) or computationally (as in LFM), while at the same time capturing enough signal to maintain a good signal-to-background and signal-to-noise ratio, can yield neuronal recording output of sufficiently high accuracy. After propagation over several scattering lengths, however, fluorescence loses all directional information. In this diffusive regime of scattering, 1P-based approaches fail to discern individual sources in the sample.

More complex and costly two-photon (2P) [19–24] and three-photon [25,26,21] point-scanning imaging approaches overcome this issue by a combination of longer excitation wavelengths, which are less susceptible to scattering, multi-photon excitation, which reduces the out-of-focus fluorescence background in scattering tissue, and via collection of scattered fluorescence light over a large solid angle onto a low-noise, single-pixel detector. Micro-endoscopic imaging approaches offer access to larger tissue depths by delivering light via optical fibers or GRIN lenses [28–31]. However, these approaches are typically highly invasive, suffer from off-axis optical aberrations and are limited to small numerical apertures. A range of approaches has been developed for mitigating higher-order wavefront distortions induced by strong scattering through wavefront shaping [32–40], often based on feedback from a variety of “guide-stars”, and commonly requiring iterative optimization of the wavefront correction.

Here, we explore an alternative approach to 1P-based functional imaging of neurons deep in scattering tissue. In contrast to wavefront shaping, our approach does not attempt to undo the effects of scattering but rather turns scattering from an obstacle into a tool, based on two key insights: First, we utilize the fact that fluorescence signals from GECIs are dynamic. This represents a major resource, since two sources with different temporal signals can be demixed if their spatial footprints also differ to a certain degree. Second, to obtain different spatial footprints for each individual source we leverage the phenomenon of speckle formation, i.e., the emergence of random interference patterns as light propagates through a disordered medium. Due to the different paths through the scattering tissue along which fluorescence light propagates from each source, the speckle patterns emerging from individual sources are different. Thus, light scattering in tissue, usually seen as an obstacle to high-quality imaging, can in fact enable the demixing of the temporally varying fluorescence signals from multiple neurons. The spatial contrast present in the speckle patterns becomes the key resource that allows to demix fluorescence signals emanating from different neurons deep in scattering tissue.

Speckle [41] are often seen as a phenomenon found only with highly coherent light sources, as is for example exploited in the case of laser speckle contrast imaging [42,43], or for retrieving information about a sample based on spatial and angular correlations inherent to speckle [44–47]. Fluorescence from GECIs such as GCaMP, due to its broad bandwidth, exhibits only a low degree of coherence. Nevertheless, under suitable conditions speckle can still be observed, albeit generally at reduced contrast. Since speckles are interference patterns, their contrast can be improved by decreasing the spectral detection bandwidth, which leads to an increased temporal coherence, or by decreasing the excited area, which leads to an increased spatial coherence. The potential to use contrast generated by fluorescence speckle for demixing of temporally varying signals from fluorescent sources has been demonstrated in recent work [48,49], in which artificially imprinted time-varying signals from up to 20 fluorescent beads located below a mouse skull were successfully demixed and localized. These results hint at the potential that fluorescence speckle can be exploited as a source for spatial footprint contrast and thus the demixability of neuronal functional signals in an in vivo functional imaging setting.

In the present work we demonstrate for the first time the generation and detection of speckle patterns generated by the activity of GECI-labelled neurons in vivo, and speckle-enabled spatiotemporal demixing of GECI activity. We first designed and built a hybrid imaging setup that utilized 2P excitation for controlled illumination conditions while allowing for detection of the emitted fluorescence on both, a photomultiplier tube (PMT) and a camera, the latter enabling the detection and characterization of generated speckle patterns (Section 3). We then used 1P excitation as an illumination source (Section 4) and found that under suitable illumination conditions, we were still able to demix time series activity of sources within the scattering mouse cortex in vivo, assisted by speckle-generated contrast, using an algorithm based on non-negative matrix factorization (NMF) [27,50,51].

2. Principles of speckle imaging in vivo

Speckle is a random interference pattern of light that is formed when amplitude and phase of a wavefront become strongly distorted after undergoing scattering. When recording an intensity image in the far-field of a scattering surface, or a volumetric medium illuminated by a light source, a speckle pattern can be observed. The contrast C of the speckle pattern is defined as the ratio between the standard deviation of image intensity and its mean $C = {\sigma _I}/\bar{I}$ [41]. For a coherent illumination source, as the phase front of the scattered light is increasingly randomized as a function of the number of scattering events and propagation distance, the contrast of the generated speckle approaches a maximum that is known as fully developed speckle. For fully-developed speckle, the phase is uniformly distributed over a range of 2$\pi$ and the intensity distribution satisfies Rayleigh statistics (i.e., the probability density function (PDF) of the intensity is a negative exponential [41]). In the case of partial temporal or spatial coherence, the speckle contrast is reduced [52] and the PDF of the intensity changes gradually from a negative exponential to a narrower shape (e.g., a Gaussian distribution), implying more uniform intensity.

2.1 Theoretical principles of speckle formation

Light propagation in brain tissue is dominated by Mie scattering [53] and characterized by a scattering coefficient, ${\mu _s}$, the inverse of which is the photons’ mean free path and has a typical range of ∼50–100 µm for visible light in the mouse cortex [54]. The distribution of scattering angles can be approximated by the Henyey-Greenstein function with an anisotropy parameter g of ∼0.9 [55], i.e., a strongly forward-directed scatter probability. These two parameters are often summarized into a quantity known as the transport mean free path, ${l^\mathrm{\ast }} = 1/[{{\mu_s}({1 - g} )} ]$, which intuitively corresponds to the distance after which the propagation direction of photons is fully randomized.

The speckle intensity distribution is governed by the PDF of the optical path lengths, $\rho (s )$, which can vary widely for different scattering media. In general, the speckle contrast C is given by [56]:

Here, the integral is over all interfering waves, and g is the well-known coherence function. The quantity $\rho \left( {\Delta s,\left\langle {\Delta s} \right\rangle } \right)$ in Eq. (1) is the PDF of optical path length differences and is related to the PDF of the optical path lengths as follows:

Here, $\left\langle {\Delta s} \right\rangle $ is the ensemble-averaged path length difference, the value of which depends on the wavelength, spectral bandwidth, and reduced scattering length. Note that Eq. (2) only holds if the interfering partially coherent waves can be assumed to be statistically independent. In this case, $\rho \left( {\Delta s,\left\langle {\Delta s} \right\rangle } \right)$ can be expressed as the autocorrelation function of $\rho (s )$, as in Eq. (2).

For a perfectly coherent source (coherence length ${l_c} \to \infty $ or $\left\langle {\Delta s} \right\rangle \ll {l_c}$), the coherence function approaches ${|{g({\Delta s} )} |^2} \to 1$, and the upper limit for the speckle contrast C is 1 for fully polarized light (e.g., observed through a polarizer) and $1/\sqrt 2 $ for unpolarized light (e.g., polarization scrambled by scattering, observed without polarizer). For an incoherent source ($\left\langle {\Delta s} \right\rangle \gg {l_c}$), ${|{g({\Delta s} )} |^2} \to 0$ and thus $C = 0$. Regardless of the exact form of $\rho (\textrm{s} )$, Eq. (1) implies that the speckle contrast decreases with decreasing source coherence (smaller ${l_c}$).

To obtain an accurate expression for the speckle contrast, knowledge of the PDF of optical path length, $\rho (s )$, is required. In general, for more complex scattering media such as biological tissue, $\rho (s )$ would have to be simulated numerically, for example using a Monte Carlo approach [57]. For more well-defined scattering media, analytical expressions for $\rho (s )$ have been derived from to the diffusion equation that describes light transport in scattering media [58]. For optically thick samples (i.e., medium thickness much larger than transport mean free path [59]), $\rho (s )$ decays as ${\sim} {s^{ - 1.5}}$ for small path lengths s [60] and decays exponentially for larger s. In the case of an optically thin medium, $\rho (s )$ is characterized by an exponential decay across a wide range of path lengths s.

For fluorescence emitted from a single GECI-labelled neuron in vivo, the speckle contrast is much smaller than one. This is both due to the low temporal coherence that results from the broad spectral bandwidth of fluorescence (typically, tens of nanometers), as well as the low spatial coherence that arises from the fact that a neuron contains many independent point source emitters (fluorophores). The spectral components emitted from a single point on the neuron undergo different scattering events and interfere, which results in lower speckle contrast if the full emission spectrum is included in the detection. However, the effective coherence length of the detected speckle can be increased, and thus the speckle contrast enhanced, by narrow-band spectral filtering of the emission.

As a rule of thumb, in the fully developed speckle regime, contrast degrades as ∼ $1/\sqrt N $, where N is the number of sources. When imaging speckle with a microscope, the number of sources N can be estimated as the number of resolved spots in the observed volume. The resolved spot size is usually diffraction-limited by the numerical aperture of the microscope objective. Thus, for typical microscopes, the fluorescence emitted from a single neuron already constitutes a spatially extended and spectrally broad light source.

In addition to the effects of finite temporal and spatial coherence, the dynamic processes occurring in living brain tissue lead to changes in its exact scattering transmission properties, which in turn leads to a decorrelation of each individual speckle pattern over time. We discuss and experimentally characterize dynamic decorrelation effects in Section 3.4 below.

2.2 Imaging neuronal activity via speckle demixing in vivo

The key conceptual idea behind our speckle imaging and demixing approach is that we treat scattering as a resource for encoding neuronal activity information rather than aiming to suppress or overcome it, as is done in most other approaches for imaging in scattering media. In speckle imaging and demixing, the speckle contrast becomes the key resource that limits demixability of multiple sources in the presence of noise. We therefore carefully explored the practical limits and usefulness of speckle imaging and demixing by analyzing how the following key parameters interact and identifying optimal trade-offs.

First, as discussed in the previous section, a smaller spectral detection bandwidth increases temporal coherence, and hence speckle contrast, but decreases signal strength, and hence SNR. Second, a smaller excitation area on each neuron increases spatial coherence, and hence speckle contrast, but also decreases signal strength. The size of the excitation area on each neuron may not be controllable in all optical modalities but can to some degree be controlled using molecular tools (e.g., soma-localized indicator expression [2]). Third, a larger spatial overlap of speckles from multiple neurons decreases spatial coherence, and hence speckle contrast, but at the same time the differing temporal dynamics of the underlying neurons can be used to demix their individual speckle patterns with higher fidelity. The degree to which individual neurons’ speckle patterns overlap is determined by the exact excitation modality and speckle imaging geometry.

In our speckle imaging approach, we excited neurons expressing GCaMP7f across many cerebral cortex regions such as somatomotor, somatosensory, retrosplenial and visual areas and from cortical layers one to three either using 1P or 2P light in a customized epi-fluorescence microscope and imaged speckle patterns onto a camera sensor (Fig. 1). We refer to the axial distance between the excited neurons and the plane that is imaged onto the camera sensor (via the microscope objective and subsequent optics in the camera arm) as the speckle distance, ${Z_i}$ (Fig. 1(a)). Speckle patterns formed by individual neurons overlap partially or fully on the sensor depending on neurons’ position and the imaging distance.

 

Fig. 1. Schematic illustration of speckle imaging and demixing in the mouse brain. (a) Schematic depiction of three neurons emitting fluorescence, two in the same z-plane and one (green neuron) in a z-plane further away from the brain surface. The green neuron’s plane is denoted as the ${Z_i} = 0$ plane. Two imaging planes are indicated by dashed lines at distances from the green neuron of ${Z_i} = {d_0}$ and ${Z_i} = {d_0} + d$, respectively. (b) Schematic illustration of three distinct, partially overlapping and partially developed speckle patterns generated by the neurons in (a), as may be observed when imaging the speckle field at smaller distance ${Z_i} = {d_0}$. (c) Illustration of completely overlapping, fully developed speckle generated by the same three neurons indicated in (a) when imaged at a larger distance, ${Z_i} = {d_0} + d$. Speckle are color-coded according to neuron of origin. (d) Same data as in (c) without color-coding according to neuron of origin. An inverted greyscale is used to ease comparison with (c), i.e., black indicates high brightness. Pronounced brightness is seen in locations where speckle from multiple neurons coincide and interfere constructively. (e) Illustration of three weakly correlated neuronal activity time traces. By spatio-temporal demixing of the corresponding speckle footprints, it is possible to recover the activity time traces, both for partially and fully overlapping speckle fields.

Download Full Size | PDF

The contrast of the combined patterns decreases approximately with the square root of the number of active neuron speckles that overlap on the sensor (see Section 3.3). At the same time, the contrast of the speckle pattern grows with speckle distance ${Z_i}$ up to the limit of fully developed speckles. For our imaging situation, randomization of paths and thus the transition to fully developed statistics progresses quickly (i.e., over shorter distances) while the imaged plane is still within the scattering medium (brain), and then continues to progress more slowly (i.e., over longer distances) when propagating outside of the scattering medium. As an individual speckle pattern develops, bright intensity spots known as speckle grains become more confined and sparser. This is a consequence of the fact that the intensity distribution of fully developed speckles follows a Rayleigh distribution, in which the probability for low intensity grains is exponentially higher than that for high intensity grains. This is illustrated in Fig. 1(b), where the neuron drawn in green exhibits a smaller grain size than the speckle from the other two neurons: Light emitted from the green neuron underwent a longer propagation and thus more scattering events, which resulted in a sparser intensity distribution. At the same time, the speckle pattern generated by the neuron drawn in green has a larger spatial footprint at an imaging distance ${Z_i} = {d_0}$ due to its greater distance from the imaging plane than the neurons drawn in blue and red.

For a certain larger value of ${Z_i} = {d_0} + d,$ the footprints may completely fill or become larger than the imaging field of view (FOV), as illustrated in Fig. 1(c). At such a large distance, all three individual speckle patterns may also be fully developed, so that their speckle grain sizes are comparable. Figure 1(d) schematically depicts an intensity image as it would be captured on a camera, generated by three superimposed speckle patterns originating from different neurons. Due to the complete intermingling of the three patterns, distinct temporal GECI activity signals from the three illuminated neurons (illustrated in Fig. 1(e)) are crucially required for demixing the individual speckle patterns, in conjunction with a speckle contrast that is sufficiently above the camera noise floor.

3. Speckle characterization using two-photon excitation

To characterize the achievable speckle contrast from neuron fluorescence and to find the optimal imaging parameters, we first excited single neurons using 2P scanning. This initial 2P characterization modality allows for greater specificity and flexibility in determining and optimizing the relevant parameters, such as the spatial extent of excitation, the speckle distance, the depth of the excited neurons, spectral detection bandwidth, background, contrast, dynamical decorrelation timescales, and the number of camera pixels illuminated by individual speckle grains and the overall pattern from each source.

3.1 Experimental setup

Figure 2(a) schematically depicts the hybrid 2P microscope setup used for characterization studies: Excitation pulses from an ultrafast Ti:Sapphire laser (920 nm, ∼140 fs, 80 MHz, Coherent Chameleon) were laterally scanned using a galvo-galvo scan path and reflected off a dichroic mirror (DM) towards the main microscope objective (Nikon CFI 175 LWD, 16×/0.8). The focused pulses were scanned across small fields-of-view on the order of the size of a single neuron, 20 µm × 20 µm (other scan areas are examined in section 3.2). Fluorescence generated from excited neurons was split 10:90 between the standard non-descanned PMT-based detection arm of the 2P microscope (Scientifica Slicescope) and a custom imaging detection arm at a side port of the microscope body. The image formed at the side port was relayed and demagnified using a tube lens and microscope objective in reverse orientation. An sCMOS camera (Andor Zyla 5.5) was mounted on a translation stage to allow for adjustment of the imaging plane independent from the excitation focal plane, which in turn was set by translating the main microscope objective up and down. In this way, the detection arm was capable of imaging fluorescence speckle at different axial imaging planes above an illuminated target neuron by simply moving the camera back from the focal plane of the demagnifying objective and thus selecting the desired speckle distance, ${Z_i}$. The fluorescence signal emanating from the side port was passed through one of a range of emission filters with different bandwidths (default filter: 30 nm bandwidth, 515 nm center wavelength), demagnified 3.75-fold, and finally detected using an sCMOS camera.

 

Fig. 2. Experimental setup for speckle generation by two-photon (2P) excitation. (a) 2P excitation beam path, PMT, detection path, and camera-based detection path, with demagnification stage. Speckle are imaged at different distances ${Z_i}$ from the objective front focal plane by translating the camera away from the native rear image plane, as indicated. Arrows beside objective illustrate objective translation for focus adjustment of 2P scanning excitation. BS: beamsplitter, DM: dichroic mirror, NBF: 515 nm narrowband filter, WD: objective working distance. All distances are in units of µm. (b) Top panel: Sample raw speckle footprint from a single neuron at depth 415 µm from the brain surface and imaged at ${Z_i}$ = 590 µm. Inset in bottom left corner: Standard 2P image of the single neuron being scanned, as recorded on the PMT. Bottom panel: Processed version of raw image from the top panel, bandpass-filtered to remove background noise. Scale bar: 50 µm for both top and bottom panels.

Download Full Size | PDF

We targeted neurons up to depths of 400–450 µm from the brain surface. The maximal practical neuron depth is limited by the signal-to-noise ratio (SNR) of the speckle pattern captured on the camera. When targeting a neuron at a greater depth, fewer signal photons arrive at the sensor due to reduced 2P excitation efficiency. At the same time, the scattering of 2P excitation light leads to an increased background fluorescence and thus contributes to an overall lower SNR. Speckle emerging from these neurons were imaged at a range of distances ${Z_i}$ (0–1645 µm) by translating the camera along the axial direction.

To set the imaging distance to a desired value ${Z_i}$, the camera was translated by a distance ${\sim} {M^2} \cdot {Z_i}$, where M is the combined lateral magnification of the bare microscope and the demagnification optics in the custom detection arm (Fig. 2(a), inset). We chose the overall lateral magnification M to be 3.84-fold at ${Z_i} = 0$ (bare microscope magnification 14.4-fold, reduced by a factor of 3.75 via demagnifying optics), guided by the requirement that the photon budget per pixel arriving at the sensor for the largest ${Z_i}$ value should be reasonably above the camera read noise floor. At the same time, since the axial magnification scales as $\sim {M^2}$, it was desirable to keep M low enough to allow reaching a large range of ${Z_i}$ in a reasonably compact setup. Without post-microscope demagnification and at high 2P power, the camera noise floor became limiting at ${Z_i}$ = 300 µm, where ∼10 signal photons per pixel were detected in our experiments. Adding demagnification concentrates photons onto fewer pixels, such that larger distances ${Z_i}$ can be observed at comparable SNR. With a 3.75× demagnification, the signal photon budget translates to a lower bound of ∼25 photons/px at ${Z_i}$ = 700 µm. To realize even larger distances (Fig. 3), we also employed 5× demagnification, for which a lower bound of ∼8 photons/px were to be expected for our largest values of ${Z_i}$ = 1645 µm. In frames where the targeted neuron was active, we detected as many as 50 photons/px at ${Z_i}$ = 1645 µm. Supplement 1 includes further details on the imaging optics.

 

Fig. 3. Speckle contrast versus speckle distance ${{\boldsymbol Z}_{\boldsymbol i}}$. (a) Examples of raw speckle footprints from a single neuron, at four different speckle distances ${Z_i}$ = 320, 450, 590, and 900 µm. Post-microscope demagnification: 3.75× (${Z_i}$ = 320, 450, 590), 5× (${Z_i}$ = 900 µm). Scale bar: 50 µm. Image size in camera pixels given in upper right corners. (b) Contrast-enhanced and high-pass filtered versions of the images shown in (a), for qualitative visualization of speckle. (c) Fully processed, median- and high-pass-filtered speckle images as used for quantitatively evaluating speckle contrast, with contrast values given in bottom left corners. (d) Speckle contrast as a function of speckle distance ${Z_i}$. Two different demagnification optics were used to record the whole range, as indicated in the legend. Data at each ${Z_i}$ includes at least 6 different neurons from 4 different animals. Gray-shaded area highlights the axial range around the brain surface.

Download Full Size | PDF

Figure 2(b) shows an example of a raw frame as recorded by the camera when imaging speckle at 8.6 Hz from a single neuron, 2P-excited with a frame rate of 60 Hz, at imaging distance ${Z_i} = 590$ µm and with an emission filter centered at 515 nm with 30 nm bandwidth. Each camera frame was exposed for 100 ms, thus integrating over six 2P excitation scan frames. In addition, the images shown in Fig. 2(b) are integrated over three camera frames, resulting in an effective frame rate of 2.87 Hz. This frame rate and integration time were chosen as a trade-off between the requirement to obtain a signal level high enough for evaluating the speckle contrast across a range of imaging distances (see below), and the requirements of the intended application, which is to record the dynamics of calcium indicators with minimum frame rates of ∼5–10 Hz. Speckles are readily visible in the raw single-frame image (top) and become clearer after filtering out structures larger than 17 µm (bottom).

3.2 Effect of speckle distance ${Z_i}$ and excitation area

We first examined the contrast of a single-neuron speckle pattern as a function of speckle distance ${{Z}_{i}}$ by translating the camera backwards and thus imaging the speckle field at different planes above the fluorescent neuron. As discussed before, larger ${{Z}_{i}}$ leads to more developed speckle and therefore a higher contrast, but also results in larger overall speckle footprints.

Figure 3(a) shows the raw speckle images obtained with identical scan- and imaging parameters as in Fig. 2(b), for a range of speckle distances ${Z_i}$. Figure 3(b) shows the corresponding images after basic background removal and display contrast scaling, both to enhance visual clarity. Figure 3(c) shows the footprints after post-processing with a 3 × 3 px median filter to reduce pixel-to-pixel noise, as well as a spatial high-pass filter to remove large blood vessel structures. These images were used for evaluating the speckle contrast quantitatively. We chose cut-on spatial frequencies manually for each ${Z_i}$, such that large structures like blood vessels would get removed. A typical cut-on spatial frequency corresponded to a feature size of 10–25 pixels (i.e., structures larger than this value were filtered out). This value was chosen by manual inspection such that it would correspond to the diameter of a typical small blood vessel in the frame. In all cases, the cut-on spatial frequency was chosen such that the speckle grains (typically, 4–8 px) were not affected by the filter. In contrast to larger structures, it is difficult to reject fine contaminating structures that have a size scale that overlaps with the distribution of speckle grain sizes, such as very small blood vessels. However, fine structures tend to be much more uniformly distributed in our imaging data than larger structures. So, while they may introduce a global offset in the extracted speckle contrast value, their presence does not introduce a bias in our results. Ultimately, any biological variations in the density of fine blood vessels are a source of error that is reflected in the error bars of our contrast quantifications.

We note that for the speckle footprint example for ${Z_i}$ = 900 µm shown in the right-most column of Fig. 3(a)-(c), the larger demagnification of 5× was used, to reduce number of illuminated pixels on the camera further and thus increase the SNR.

As is evident from the filtered images, a more connected speckle structure was generated at smaller distances from the neuron, implying partially developed speckle. This changed gradually with increasing distance from the neuron. In particular, sparser intensity distributions were generated for larger ${Z_i}$. Figure 3(d) shows the speckle contrast as a function of speckle distance ${Z_i}$ for eight different neurons from four different animals, located at depths between 410 and 450 µm. We observed that for low values of ${Z_i}$, the contrast at first increased monotonically. As the imaging plane approached the sample surface at ${Z_i}$ = 450 µm, the contrast reached a maximum and then decreased and stabilized within a lower range of ∼0.05 to 0.07. The reason for the sudden drop in contrast between ${Z_i}$ = 450 µm and ${Z_i}$ = 590 µm lies in additional contrast introduced by both absorption and phase shifts due to surface structures that we were unable to filter out completely. Informed by this result, in subsequent experiments we chose speckle distances from the excited neurons to be close to the surface of the brain. For values of ${Z_i}$ located far above the surface, the signal-to-noise ratio decreased markedly as the speckle patterns became spread out onto more and more pixels, thus limiting the usefulness of these large speckle distances.

We next examined how the speckle contrast is affected by the size of the excited area on a neuron (Fig. 4(a)). Since a larger excited area (and hence, effective source size) leads to decreased spatial coherence, we expected the contrast to decrease with excited area. This is indeed what we found by comparing an excited area of ${A_1}$ = 20 × 20 µm² to ${A_2}\; = \; $5 × 5 µm² (Fig. 4(b)). For this comparison, the total number of excitation laser pulses delivered to the areas ${A_1}$ and ${A_2}$ was kept constant, as was the frame rate of the camera. The pixel pitch (distance between subsequent scanned points) of excitation scanning for both cases was 0.16 µm, whereas the pixel dwell time was proportionally longer for the smaller scan area (2.1 µs versus 33.6 µs) to maintain a constant excitation frame rate. For each of the two area sizes, the contrasts generated from five individual neurons located at depths ranging from 400 to 450 µm and imaged at ${Z_i} = $ 450 µm were evaluated (Fig. 4(b)). In agreement with our qualitative expectation, we found a 24% increase of the median contrast for the smaller excitation area size. We note that while the 5 × 5 µm² excitation areas completely overlapped with neurons, the larger 20 × 20 µm² excitation areas were such that on average, ∼35% of the scan area did not coincide with the neuron and thus did not result in any fluorescence photons. Nonetheless, according to the $1/\sqrt N $ scaling rule of thumb for speckle contrast (N being the number of emitting diffraction-limited spots that contribute to the speckle), the contrast should still have increased threefold. This is clearly not the case in our experiments, which we tentatively attribute to incomplete speckle development in our measurement geometry.

 

Fig. 4. Effect of excitation area and detection bandwidth on speckle contrast. (a) Graphical representation of two excitation areas on a single neuron. Area A₁ (20 × 20 µm²) is 16 times larger than area A₂ (5 × 5 µm²) while the total number of pulses in each area was the same. (b) Speckle contrast for two different excitation areas from five different neurons. (c) Speckle contrast for two different fluorescence detection bandwidths, from five different neurons. Excitation area for (c) is ∼3 × 3 µm².

Download Full Size | PDF

Another parameter that we expected to directly affect speckle contrast is the spectral bandwidth of fluorescence as set by the emission bandpass filter: A larger bandwidth implies lower temporal coherence and hence lower speckle contrast. This can intuitively be understood by considering light of two different wavelengths being emitted from a point source: The two waves would get scattered at slightly different angles and thus accumulate different path length differences, resulting in slightly different speckle. Upon acquisition of an intensity image these speckle would be summed incoherently and average out, resulting in an overall lower speckle contrast.

As mentioned in Section 2.1, to obtain an expression relating temporal coherence to speckle contrast, knowledge of the probability density function of path length differences is required, which in a volumetrically scattering sample can be obtained either through measurement or through simulation. Assuming the volume’s path length probability density function plays a role equivalent to the surface height probability density function in a surface scatterer [41], the contrast for a reflection geometry with normal incident and detection angles and fully polarized light detection can be calculated as:

Here, the surface height fluctuations are assumed to be Gaussian with standard deviation ${\sigma _h}$. ${\lambda _c}$ is the center wavelength of the detection filter. Using this equation, the improvement in speckle contrast when comparing a 10-nm to a 30-nm detection bandwidth can be estimated to be ∼70% (assuming ${\sigma _h}/{\lambda _c} = 200$).

The effect of detection bandwidth on speckle contrast is shown in Fig. 4(c) for two different bandwidths with the same center wavelength of 515 nm. As expected, speckle contrast increases when decreasing detection bandwidth from 30 nm to 10 nm (n = 5 different neurons with excitation area of 3 × 3 µm²). Note that the observed increase of 50% in median contrast was obtained despite the smaller number of photons (and thus lower SNR) that arrived at the sensor in the lower-bandwidth case.

3.3 Contrast as a function of number of excited neurons

We proceeded by studying how the contrast of overlapping speckle patterns from individual neurons changes when increasing the number of simultaneously 2P-excited neurons: Fig. 5(a)-(d) shows speckle patterns from individually excited neurons at a depth of 450 µm and speckle distance ${Z_i} = 590$ µm. We ensured that the neurons were active at least once during the recording and selected the brightest frame for the contrast calculations (Fig. 5(j)). Examples of two, three and four overlaid speckle footprints generated by neurons being excited simultaneously are shown in Fig. 5(e)-(h). As is visually evident from the speckle footprints, the resulting speckle pattern exhibited lower contrast as the number of simultaneously excited ROIs (neurons) increased. Figure 5(i) depicts the PMT signal for the full FOV of 2P scanning, with the locations of the target neurons in the mouse brain highlighted. Figure 5(j) shows a quantitative summary of how contrast decreased as the number of ROIs (neurons) increased. Despite a noticeable amount of variability across different mice and regions of the brain, the data is explained well by the theoretical expectation of a ${\sim} 1/\sqrt N $ scaling of contrast with number of sources. The dashed line is the average of fits to the individual measurement series (each consisting of 1–4 ROIs in one mouse and one region).

 

Fig. 5. Speckle contrast as a function of simultaneously excited neurons. (a-d) Raw speckle footprints from 2P-excited single neurons imaged at ${Z_i} = 590$ µm. Scale bar: 50 µm for all panels. (e) Speckle from neurons 1 and 2 excited simultaneously. (f) Speckle from neurons 3 and 4 excited simultaneously. (g) Three and (h) four neurons excited simultaneously. (i) Lateral locations of neurons 1–4 in the 2p scanning FOV (individual excited areas: 15 × 15 µm). All neurons are at depth 450 µm from the brain surface. Excitation frame rate for individual region-of-interest (ROI) was 60 Hz and 30, 20 and 15 Hz for 2, 3 and 4 ROIs, respectively. (j) Contrast as a function of the number of ROIs (neurons) excited simultaneously. Boxplots include data from four different mice and five distinct regions of imaging. Dashed line: Fit with function $a/\sqrt N $, where N is the number of ROIs. Optimal a = 0.08. R-squared = 0.65.

Download Full Size | PDF

To summarize the results presented in this section, by increasing the number of excited neurons from one to four the contrast was reduced by up to ∼50% while the contrast from single neurons was ∼10%. These values delineate the regime within which demixing of the temporal signals from multiple neurons must be performed.

3.4 Speckle decorrelation dynamics

Imaging speckle in vivo is challenging since living brain tissue contains components, most prominently vasculature, that change dynamically, leading to a decorrelation of speckle patterns over time [47,61,62]. Quantifying speckle decoherence dynamics can provide insights into the underlying tissue properties, as is exploited in techniques such as laser speckle contrast imaging [42] and derived methods, including dynamic light scattering imaging [43]. In techniques that are based on the reversal of the effects of light scattering (which have mostly only been demonstrated in static scattering media as opposed to in vivo tissue), as well as in our speckle demixing approach, speckle decorrelation is a limiting factor that requires careful characterization.

Literature reports of characteristic decorrelation timescales vary considerably and are commonly obtained with a highly coherent excitation source, in contrast to the much less coherent fluorescence that the present work relies on. Decorrelation is commonly quantified by evaluating the well-known intensity autocorrelation function ${g^{(2 )}}(\tau )\; $ of speckle patterns and extracting a characteristic decay time. In the context of optical phase conjugation, speckle formed by passage of a laser beam through a flap of mouse skin have been shown to decorrelate on a timescale of tens of seconds if the skin flap is firmly clamped, but on a timescale of tens of milliseconds if the skin is held only loosely [61]. In the context of time-reversed ultrasound-encoded focusing through turbid media, an initial fast decorrelation (<10 ms) was observed, followed by a slower decorrelation on a timescale of several seconds [47]. The fast decorrelation was attributed to blood flow and could be rescued by blocking blood flow, whereas the slower decorrelation was due to other motion. Periodic decays and revivals of speckle correlations due to breathing were observed. Two timescales of decorrelation were also observed in the context of focusing through dynamical scattering media by wavefront optimization in acute brain slices of 300 µm thickness [62]. Here, the fast timescale was found to be ∼100 ms, followed by a slow one of ∼5 s. In the context of multispeckle diffusing wave spectroscopy, the correlation time has been measured as a function of depth by stepwise insertion of a laser-coupled glass fiber probe into the mouse brain in vivo. At these relatively great depths, characteristic decorrelation times ranging from ∼4 ms at 1.1 mm to 0.6 ms at 3.2 mm were found.

To obtain measurements of speckle decorrelation dynamics in our fluorescence speckle imaging modality in mouse cortex, we performed characterization experiments using the custom 2P- and speckle imaging microscope described above. We pointed the 2P excitation laser beam statically (i.e., without scanning) at individual neurons in the cortex of awake, head-restrained mice on a linear treadmill, at a series of depths and at varying distances from large blood vessels. The neurons were labelled with green fluorescent protein (GFP), both to obtain maximally strong fluorescence emission, and to exclude any effect in the decorrelation dynamics data that would arise from the time-varying emission characteristics of functional indicators such as GCaMP. We recorded fluorescence speckle patterns on an EMCCD camera (Andor iXon Ultra 897, EM gain 500–800), with the sensor cooled to –70°C, at frame rates of 382 or 501 Hz. Raw data was motion-corrected using the NoRMCorre package (global rigid correction, followed by patch-by-patch rigid correction [63]). For each targeted neuron, a standard 2P image was recorded to show the neuron position relative to surrounding structures.

A typical result is shown in Visualization 1: Whereas parts of the speckle pattern remained largely static over timescales exceeding one minute, fast motion of speckle grains on a timescale of few milliseconds can be observed elsewhere, in a pattern that is strongly reminiscent of a tubular flow, potentially due to blood flow in vasculature. An individual frame from the recording is shown in Fig. 6(a). The highly static nature of some of the speckle grains became apparent by taking the mean image over entire recordings, as shown in Fig. 6(b): Whereas in the “flow” areas only few and low-contrast speckle grains are discernible in the mean image, in the other areas pronounced speckle grains are visible that closely match the speckle grain positions in an individual frame from the recording.

 

Fig. 6. Speckle decorrelation dynamics. (a) Speckle pattern generated by pointing a 2P beam statically at a single neuron; single frame from a 90 s movie recorded at frame rate 501 Hz. Scale bar: 10 camera pixels (camera pixel size: 16 µm). See Visualization 1 for raw data video. (b) Mean image over entire recording. (c) Correlation between mean of initial 200 frames (24 ms) and subsequent frames as a function of time, with linear fits to sliding-window median (red lines; text insets: slope k of linear fits). Yellow trace: unfiltered frames. Blue trace: high-pass-filtered frames (cut-on spatial frequency: 10 pixels per cycle). (d) Second-order autocorrelation function g⁽²⁾ (blue trace) for one pixel from the recording. Inset: same autocorrelation function, zoomed out to longer lag range. Black trace: interpolation between local maxima of g⁽²⁾. Red trace: fit to interpolated data with sum of two exponentials. (e) Heatmap of ratio of g⁽²⁾ at lag 0 and at lag 50 ms for each pixel. (f) Ratio of pixels with pronounced fast decay to less prominent fast decay as a function of neuron depth and distance to nearest in-plane blood vessel. Disk areas indicate ratio, colors are random, for clarity.

Download Full Size | PDF

We quantified the decorrelation dynamics in two different ways: First, by plotting the correlation of each frame with the mean of the first 200 frames (corresponding to the first 24 ms) in the recording (Fig. 6(c)), for both raw and high-pass-filtered frames. This correlation decreases only slowly, from an initial value of 0.82 to 0.74 after 45 s for the raw frames, and from 0.55 to 0.41 for the high-pass-filtered frames. A linear fit to a sliding-window median-filtered version of the raw values reveals a decay rate of the correlation coefficient of –0.0017 (raw) and –0.0037 per second (high-pass filtered). The correlation occasionally collapses for periods ranging from a few frames to several seconds, but reliably returns to the overall trendline. We attribute these collapses to motion bursts of the animal, which despite head restraint can translate to temporary translations of brain tissue relative to the optical axis on a scale of <5 µm. For the high-pass-filtered frames, the correlation is on an overall lower level and decays somewhat more quickly, albeit with a rate that is on the same order of magnitude as for the unfiltered frames. In the unfiltered frames, larger structures such as blood vessels may contribute to an overall greater correlation level. However, even after removal of all structures that are larger than speckle grains by high-pass filtering (cut-on frequency: 10 pixels per cycle), the individual frames exhibit a sustained degree of correlation over the observed time span.

As a second, complimentary quantification method, the pixel-wise autocorrelation function ${g^{(2 )}}(\tau )$ (normalized to the mean over time for each pixel) is shown for an example pixel in Fig. 6(d). In agreement with findings in the literature, the function initially decays quickly, over lags in the range of 10–100 ms. This is followed by a much slower decay, with a timescale of tens of seconds. In some of our recordings, a periodic decay and recovery of ${g^{(2 )}}$ with a period of ∼40 ms is visible. We attribute this period decay to the mouse heart rate, which is in the range of 8–15 Hz. The collapses we observe are at roughly twice that frequency, which may be due to the distinct systole and diastole phases of the cardiac cycle. To fit a decay model to the data, we interpolated between the peaks associated with the cardiac cycle and fit the sum of two exponentials to the interpolated data. The two exponential decay terms had ${\raise0.7ex\hbox{$1$} \!\mathord{/ {\vphantom {1 e}}}\!\lower0.7ex\hbox{$e$}}$ decay times of 39 ± 7 ms and 126 ± 20 seconds (95% confidence interval), respectively. We observed that the initial decay of ${g^{(2 )}}(\tau )$ was more pronounced for pixels that were in the parts of the speckle field that visually appear as “flowing” areas. To quantify this, we formed the decay ratio ${g^{(2 )}}(0 )\; /\; {g^{(2 )}}({50\; \textrm{ms}} )$ for each pixel and plotted the result as an image (Fig. 6(e)): The areas visually identified as exhibiting “flowing” speckle show larger values of the ratio, thus visualizing the areas affected by flow. We next classified the pixels into “flow” and “no-flow” (i.e., relatively pronounced versus less pronounced decay of ${g^{(2 )}}$ within 50 ms) by manually identifying an area that constitutes a baseline no-flow region and thresholding the decay ratio at two standard deviations above the mean of the baseline region. We plotted the relative abundance of strong fast-decay pixels in Fig. 6(f) as a function of neuron depth and distance to the targeted neuron to the nearest major in-plane blood vessel. We observed a tendency towards a larger fraction of “flow” pixels in neurons that were closer to in-plane blood vessels, and that were less deep in the tissue. Due to the small number of neurons recorded (n = 9) and the complexity involved in identifying nearby blood vessels in 3D across the entire speckle propagation volume above the targeted neurons, no statistically significant trends can be reported here.

Overall, these decorrelation studies yielded the remarkable result that for each single-neuron speckle pattern recorded, a sizeable fraction of speckle grains were stable over minute-long timescales in the cortex of awake, head-restrained, walking mice. The initial, fast decay of speckle correlation, presumably due to blood flow, only affected parts of the speckle field, so that appreciable speckle contrast remained over minute-long timescales that enabled us to attempt demixing of neuronal activity signals by spatio-temporal demixing of speckle movies, as described in the following section.

4. Speckle demixing using one-photon excitation

Based on our 2P-excited characterizations of parameters affecting speckle contrast in the previous section, we next aimed to explore the limits of speckle-enabled demixing of larger numbers of neurons in vivo using one-photon excitation. We first describe our optical excitation strategy which was designed to minimize background fluorescence (Section 4.1). Next, we present speckle patterns generated by 1P excitation of single neurons at 300 µm depth (Section 4.2). Finally, in Section 4.3, we discuss multi-speckle imaging and demixing of temporally varying neuronal activity signals using 1P excitation.

4.1 Suppression of background fluorescence

In contrast to 2P excitation, unconstrained 1P excitation generates fluorescence background, in the presence of which the detection of speckles becomes more difficult. We therefore implemented an oblique 1P excitation strategy in our epi-fluorescence detection configuration. To do so we focused the excitation beam from a 488 nm solid state laser (Coherent Sapphire 488-30, 30 mW) into a laterally offset position within the back focal plane (BFP) of the objective (Nikon CFI75 LWD 16×/0.8) (Fig. 7(a)). Thereby we produced an oblique excitation beam [7] that exited the objective at an angle of $\theta $ = 37° (Fig. 7(b)), thus minimizing background fluorescence above and below the intersection area of the oblique beam and the imaging focal plane.

 

Fig. 7. Oblique one-photon (1P) illumination configuration and fluorescence background rejection. (a) Illustration of oblique collimated excitation beam generated by offsetting an incoming focused beam from the optical axis in the back-focal-plane (BFP) of the objective. (b) 1P beam path characterization using a 3D-suspension of 6-µm fluorescent beads. Background image: YZ-slice from 2P stack, grey scale indicates x coordinate. Overlay: Estimated position of 1P oblique beam. Beam diameter d = 80 µm. Inset: Photo of oblique excitation beam propagating through an agar phantom. (c) Camera image obtained without pinhole for ${Z_i}\; = $ 450 µm and objective focal plane at 460 µm depth. Frame is cropped to size of illuminated camera area when no pinhole is in place. Red dashed circle: Location of the 1P beam at brain surface. Blue circle: imaged area with pinhole in place for ${Z_i} = $ 400 µm. Inset: Image obtained with pinhole in place, showing reduced background.

Download Full Size | PDF

We characterized the width d (in a plane orthogonal to the objective’s optical axis, as indicated in Fig. 7(b)), and the oblique angle $\theta $ of the 1P excitation beam by imaging fluorescence from a 3D-suspension of 6-µm fluorescent beads (Fig. 7(b)). This allowed us to find the plane of intersection of the oblique beam with the imaging focal plane and co-register its position with the 2P imaging system.

To further reduce background, we placed a 500-µm-diameter pinhole in a plane conjugated to the front focal plane of the objective and centered it on the optical axis. This loosely confocal detection geometry, in which the size of pinhole was intentionally chosen to be much larger than the FWHM of the excitation Airy disk, allowed us to reject a significant fraction of fluorescence that was excited by scattered excitation light above and below the imaging front focal plane. We estimated that the pinhole would reject background photons originating from more than 40 µm above and below the focal plane in the purely ballistic case [64].

The effect of inserting the pinhole can be observed by comparing the camera images shown in Fig. 7(c) (without pinhole) and the inset in Fig. 7(c) (with pinhole): Blur due to out-of-focus background is reduced and image contrast is enhanced with the pinhole in place.

In addition to these optical background minimization strategies, we suppressed fluorescence from neuronal processes such as axons and dendrites by imaging mice expressing the soma-confined GECI SomaGCaMP7f [2] (for details on experimental subjects and animal procedures see Supplement 1 ).

4.2 1P-excited speckle from a single neuron

Having established oblique illumination and pinhole-based out-of-focus background suppression, we next aimed to observe speckle patterns from single neurons excited by our oblique 1P illumination approach. To this end, we first recorded 2P structural stacks of mouse cortex to identify relatively isolated labeled neurons that were reachable with our approach while minimizing the excitation of other neurons by the oblique beam on its way to the target neuron. Regions with only one active neuron in a 1P-illuminated volume are rare, especially in animals designed to yield a dense expression of GCaMP. Figure 8(a) shows an example of a selected target depth at 290 µm in the 2P structural stack. The sample was moved laterally such that the isolated neuron selected for 1P excitation was placed in the intersection region between the oblique excitation beam and the imaging front focal plane. As a control, we then shifted the sample laterally, so that the excitation beam missed the targeted neuron (cyan arrow in Fig. 8(a) and lateral view in Fig. 8(b)).

 

Fig. 8. One-photon speckle imaging of a single neuron. (a) Single cropped frame from 2P recording of the target plane (290 µm below brain surface), in grey color scale. The size of the cropped frame roughly coincides with the projection of the oblique region illuminated by the 1P beam on its way into the sample. White spot is the targeted neuron. Overlaid image in cyan color scale: Same as white, but sample shifted by 55 µm to move neuron out of oblique beam as a control as illustrated in (b). Cyan arrow indicates direction of shift of sample for obtaining control measurement. (b) Schematic illustration of 1P illumination path and the single neuron location for speckle excitation (white disk) and control (cyan disk) in an axial section (y-z) view. (c), (d) Standard deviation projections along the time axis of a 2-minute speckle movie recorded on the camera for unshifted (target neuron in excitation region) and shifted (control, target neuron not in excitation region) conditions, (e) Examples of typical neuron activity time series as recorded in standard two-photon imaging (orange trace) and via NMF speckle demixing algorithm (blue trace). Note: Traces represent different recordings, shown at arbitrary offset along horizontal axis. Scale bars: 50 µm

Download Full Size | PDF

Figure 8(c) shows an example of a speckle pattern generated by aiming the 1P excitation beam at a single neuron. When shifting the neuron out of the beam, we observed that the speckle pattern disappeared (Fig. 8(d)). The temporal activity, i.e., the modulation of global speckle pattern brightness due to neuronal activity, was extracted by spatio-temporal non-negative matrix factorization (NMF, see Supplement 1 ) of time series of speckle images (Fig. 8(e), blue trace). We qualitatively compared the obtained GECI activity trace to a trace that was obtained at a different time from the same neuron by subsequent standard 2P imaging (orange trace) and found good qualitative agreement.

4.3 1P-excited, speckle-enabled demixing of multiple sources

With successful generation and imaging of a single-neuron speckle pattern, we next aimed at exciting larger numbers of neurons simultaneously and recording their partially overlapping speckle patterns. To allow for effective temporal demixing, we recorded 10-minute-long movies.

The measurement setup used for 1P-exciting multiple neurons was identical to what is described in Section 4.1. Prior to 1P imaging, 2P structural stacks were recorded from the target volume to obtain a good estimate of the number of neurons that were later illuminated by the 1P beam. We used a 500-µm pinhole and 10-nm-bandwidth emission filter and acquired a total of 20 recordings (10 minutes duration each, 12 frames per second) with the objective front focal plane at depths ranging from 230 µm to 300 µm and from multiple locations in seven mice expressing SomaGCaMP7f after viral injection. This large number of measurements was necessary to investigate both, various regions of the brain, and various numbers of active neurons in the volumetric region excited by the 1P beam. Each speckle movie was cropped and rigid motion correction was performed using the NoRMCorre algorithm that is a part of the CaImAn package [63] (see Supplement 1). A three-frame averaging was applied to increase SNR before the background was estimated via rank-1 matrix factorization and subtracted. Next, spatio-temporal demixing was performed by NMF. The rank parameter of NMF was chosen in an iterative fashion by manually subtracting components that contained a meaningful spatial and temporal signal and continuing the demixing (typical NMF rank: 8–12). Finally, similar components were merged if their spatial and temporal components were highly correlated (more details on the demixing algorithm can be found in Supplement 1).

Figure 9 shows representative demixing results obtain from a 10-minute recording of 1P-excited speckle-containing fluorescence from multiple neurons: In this recording, we were able to demix nine independent speckle footprints (Fig. 9(a)) and their corresponding temporal activity traces using NMF (Fig. 9(b)). The footprints have different sizes on the camera: larger spatial footprints imply deeper source location. The objective front focal plane was positioned at 265 µm depth while the speckle distance ${Z_i}$ was set to 300 µm by translating the camera away from the rear image plane of the microscope, as described above. The excitation power was 200 µW post-objective, in an 80-µm-diameter oblique beam. See also Supplemental Fig. 1 for additional representative results from two recordings in different mice.

 

Fig. 9. Spatio-temporal demixing of 1P-excited speckle from multiple sources. (a) Nine demixed spatial components from mouse cortex, speckle imaging distance Z_i = 300 µm. Raw (top in pair) and high-pass-filtered (bottom in pair) images are shown for each component. High-pass cut-on spatial period: 29.5 µm. White number on left in each pair indicates component number. Number on right indicates maximum brightness value in raw component. Scale bar: 50 µm. Color bar shown right of component 9. (b) Temporal components corresponding to spatial components shown in (a). Gray lines: Raw time traces. Blue line: fits to raw traces constrained by a model of Ca²⁺ dynamics. (c) Enlarged composite image of high-pass-filtered spatial components no. 3 (red), 6 (green), and 7 (blue). Scale bar: 50 µm. See also Supplemental Fig. 1.

Download Full Size | PDF

We note that in the absence of a ground truth, it is difficult to ascertain whether individual spatial footprints and corresponding temporal activity traces originate from individual neurons. Successful demixing of two sources requires that, given a certain noise level, the spatial footprints recorded from each source must exhibit a minimum level of differentiability (i.e., non-overlap, difference in pattern). In a practical application it is only of secondary importance through which mechanism this difference is generated, and all potential sources of differentiability would have to be harvested by an efficient spatio-temporal demixing approach. The following sources of spatial differentiability are present in our data to varying degrees: (1) Spatial separation of the sources, which leads to differences in their emission intensity profiles at the surface due to defocus and scattering (for axial separation) or partial non-overlap of the emission patterns (for lateral separation). (2) Different absorbing features encountered on the path from the source to the camera, including in the brain (blood vessels) and at the surface. (3) Speckle, both from volumetric scattering in the brain and from scattering at the surface. Speckle patterns, due to their random nature, will often have the highest sensitivity to source location: As soon as two neurons are separated in space by more than the speckle memory effect length, their speckle patterns differ. If they are separated by less than the speckle memory effect length, then their patterns will appear similar but displaced relative to each other. Both effects are visible in Fig. 9(c), which shows a composite image of three spatial components. The speckle memory effect length is on the order of the scattering length, i.e., ∼50 µm for green light in mouse cortex. Overall, if two neurons exhibit activity patterns that are highly correlated in time and with insufficient spatial differentiability, then they will not get demixed by our spatio-temporal demixing algorithm.

Individual demixed components exhibited a localized distribution of source brightness, such as component 2 in Fig. 9, rather than a spatially extended speckle pattern. We attribute this to superficial neurons that got excited by scattered excitation light and were located close enough to the optical axis such that their fluorescence got transmitted through the pinhole. If these neurons are close to the speckle imaging focal plane, i.e., a distance ${Z_i}$ above the depth targeted by the oblique excitation beam, then they appear focused and localized on the camera. Since their footprints on the camera are strongly localized, such superficial neurons do not strongly affect the speckle contrast and demixing performance.

The model that underlies NMF is based on the assumption that the spatial components are static. In our case, however, the spatial components, i.e., the speckle footprints, decohere over time as described in Section 3.4. This model mismatch leads to a decreased correlation of the extracted NMF components with ground truth, as we show using simulated data in Supplement 1. Under assumptions that are approximately equivalent to our experimental conditions and the observed speckle decoherence rate, we found that the correlation between the temporal components as output by NMF and ground truth decreased from 0.74 ± 0.16 (without decorrelation, mean ± standard deviation) to 0.68 ± 0.14 (with realistic decorrelation). We observed that NMF performance decreased gradually with increasing decorrelation and did not break down abruptly in the parameter regime studied. In future work, it would be beneficial to extend standard NMF to mitigate this effect by including a model for the slow decorrelation of the speckle footprints. This has the potential of increasing the accuracy of the extracted neuronal dynamics. We note that our specific NMF-based pipeline (Supplement 1) includes iterative NMF runs followed by merging of strongly correlated temporal components, which may partly compensate for the effects of decorrelation.

As expected from the considerations in Section 1 and the results in Section 3, simultaneous excitation of multiple neurons that were located close enough laterally for their speckle footprints to overlap on the camera resulted in a lower speckle contrast than what we observed for single neurons. Based on an estimate of active neuron density obtained from 2P planar movies processed with the calcium imaging data analysis package CaImAn [65], we estimate that ≳40 active neurons were contained within the volume excited by the oblique 1P beam down to a depth of 400 µm (data not shown). This implies that the number of components extracted through speckle demixing is not limited by the number of active neurons, but by the obtainable speckle contrast and SNR, as well as the performance of NMF under decorrelation.

5. Discussion

In the present work we for the first time provide an experimental, in-vivo investigation of speckle generation, speckle imaging and spatio-temporal demixing enabled by speckle-encoded neuronal activity signals in the scattering mammalian brain tissue. We first studied speckle generation through 2P excitation of individual neurons and hybrid detection of fluorescence in both a standard two-photon microscope PMT detection arm and on a camera. This hybrid modality allowed us to systematically optimize speckle imaging parameters, such as the speckle distance ${Z_i}$, the excitation area, and detection bandwidth, and thus maximize speckle contrast. We then extended 2P excitation to multiple neurons and quantified the resulting contrast of the overlapping speckle. We were able to confirm the theoretical expectation that the contrast decays as ∼$1/\sqrt N $ with the number of source neurons N, thus establishing an important limit on the available contrast budget in a multi-neuron speckle imaging scenario. We then established the use of a 1P oblique-beam illumination strategy for background-reduced speckle generation, as well as the use of a quasi-confocal pinhole for background suppression. Together, these parameter optimizations and experimental innovations allowed us to demonstrate spatio-temporally demixing of speckle from multiple neurons in vivo.

We note that speckle demixing does not immediately provide high-resolution information on the locations of the sources in the brain, although the overall diameter of speckle footprints can be used to coarsely estimate the source depth. We anticipate that in future in vivo work the relative lateral positions of sources could potentially be inferred by analyzing remaining directional information in the scattered wavefront for more shallow sources, and by exploiting the speckle memory effect [49,66] for deep sources.

The limits of speckle-based demixing are dictated by the achievable speckle contrast, which in turn is determined by the degree of spatial and temporal coherence of the individual sources, and the combined contrast resulting from the superposition of the individual speckle fields and any background in a specific imaging modality. It will be of particular interest to determine whether the observed speckle contrast can be increased further, and whether it is limited by fundamental coherence properties of the sources involved, or whether other effects play a limiting role, such as dynamic decorrelation of speckle due to physiological dynamics in the brain tissue.

Our speckle-assisted demixing results were obtained in mice expressing a soma-localized GECI, which we chose to control the spatial extent of emission from each neuron, and thus improve spatial coherence and speckle contrast. For pan-neuronal indicators (i.e., expression not localized to nucleus or soma), the single-neuron speckle contrast and thus the demixability would decrease due to the emission from labelled processes, and background would increase, thus increasing requirements for background suppression. The depth reach of our 1P-excited, speckle-assisted demixing scheme is currently limited to layer 2/3 in mouse cortex. The depth is limited by the scatter-induced broadening of the excitation beam: At greater depths, the increased background due to off-target excitation becomes stronger and ultimately washes out the obtainable speckle contrast, both due to lack of coherence and due to noise on the background.

We note that our approach exhibits good scalability in the lateral dimensions: If the speckle patterns generated by laterally separated neurons do not overlap on the camera, then their respective speckle contrast values are independent and can be demixed without any loss of contrast due to interference from overlapping speckle. The degree of speckle pattern overlap is determined by the depth of a neuron and the distance at which the pattern is imaged. As a rule of thumb, the imaging distance can be chosen such that the extent of a speckle pattern is roughly equivalent to the neuron’s depth, which would roughly ensure that the lateral separation of neurons is large enough for non-overlapping footprints.

In summary, in the present work, we systematically explored the relevant parameter space for designing an in vivo speckle imaging and demixing modality and for maximizing the key resource in any such modality, the speckle contrast. We thereby identify trade-offs and measure parameters in vivo that can serve as a reference for future work on speckle-encoded demixing.