1. Introduction

Flexible optical endoscopes are an important tool for a variety of applications, from clinical procedures to biomedical investigations. A common use of such endoscopes is macroscopic imaging inside hollow organs. Another important application is micro-endoscopy, where micron-scale structures such as single neurons are imaged or optically excited at depths beyond the reach of conventional microscopes, as conventional microscopes are limited in their penetration depth due to tissue scattering and absorption [1].

In recent years, various solutions for small diameter micro-endoscopes have been developed [2,3]: Micro-endoscopes that are based on single-mode fibers are bend-insensitive, but require distal optical elements such as scanners and lenses [2], or spectral dispersers [4,5] to produce an image. Such distal elements may significantly enlarge the endoscope’s diameter, increasing tissue damage, and consequently limiting its use for deep-tissue imaging. Developing a flexible, lensless micro-endoscope with a minimal diameter is thus a sought after goal for minimally-invasive deep-tissue imaging.

Currently, the solutions for constructing lensless endoscopes are based either on imaging fiber bundles [2] or wavefront-shaping [6–10]. Imaging fiber bundles consist of thousands of single-mode cores packed together, where conventionally each core functions as a single pixel. While common and straightforward to use, conventional lensless bundle-based endoscopes suffer from limited resolution, pixelation, poor axial sectioning, and a small and fixed working distance.

While axial-sectioning can be obtained in fiber bundles by addition of confocal scanning [2] or structured illumination [11], the working distance is fixed to the fiber facet or its image, and does not allow three-dimensional (3D) imaging.

In recent years, a number of works demonstrated the use of wavefront-shaping for microendoscopy [6,7,9,10,12–15]. In wavefront shaping, a computer-controlled spatial light modulator (SLM) is used to compensate the phase randomization and mode-mixing in fiber bundles or multi-mode fibers, in a principle similar to the decades-old works in holography [16, 17]. However, since the phase distortions in long fibers are sensitive to fiber bending, even with the state-of-the-art digital wavefront control a direct feedback from the fiber distal end or precise knowledge of the fiber shape [15] are still required to determine the wavefront correction. These requirements make the application of wavefront-shaping techniques for flexible endoscopes very challenging in most practical imaging scenarios. Recently, speckle correlations in the optical transmission-matrices of fiber bundles, known as ’memory effect’ correlations, have been exploited for computationally reconstructing distal images from proximal measurements, without wavefront correction [18,19]. However, these approaches currently allow only two-dimensional imaging, and provide high-fidelity reconstruction only for simple objects.

Here, we present a lensless two-photon microendoscope based on wavefront-shaping and a fiber bundle, which does not require any a-priori knowledge of the fiber transmission-matrix or access to the distal end, even after arbitrary bending of the bundle. In our approach the wavefront correction is performed in-situ, by iterative optimization of two-photon fluorescence (2PF) signals detected at the proximal end.

Our approach is based on the fact that iterative optimization of a nonlinear signal leads to diffraction limited focusing [20], even when the signal is collected by a spatially-integrating detector having no spatial resolution. This principle has been recently exploited for focusing light through scattering samples [20] or multi-mode fibers [21], and here we show its use for endoscopic imaging.

Our imaging technique is composed of two steps: the first is focusing of the excitation beam on the target object by iterative wavefront optimization. After the wavefront-correction has been found, two-photon imaging is performed by scanning the formed focus in three-dimensions using the single wavefront correction, exploiting the ’memory-effect’ of imaging fiber bundles [8,18,19].

2. Methods

The experimental setup is illustrated in Fig. 1. A beam from a Ti:Sapphire laser oscillator, with an incorporated dispersion compensation module (Mai-Tai DeepSee eHP, Spectra-Physics) producing 85 fs pulses at 785nm wavelength, is reflected off a galvanometric mirror (GVS012, Thorlabs). The galvanometric mirror is imaged on a phase-only SLM (X13138-02, Hamamatsu Photonics) by a 4-f telescope (L₁ = 35mm, L₂ = 500mm). The SLM plane is imaged on the proximal facet of a fiber bundle (either Schott 1533385, 50 cm long with a measured core-to-core distance of 7μm in Figs. 2(a)–4(b), or Fujikura FIGH-03-215S, 0.5’ long with a measured core-to-core distance of 6 μm in Figs. 5(d)–5(i)) using an additional telescope (L₃ = 250mm, Obj₁; 20X Plan Achromat, Olympus). Two-photon fluorescent (2PF) target objects were placed simultaneously at several distances of 0.8mm-3mm from the fiber distal end. The excited 2PF signal was collected by the same bundle, propagated back to the proximal end, separated from the laser wavelength by a dichroic mirror (FF605-Di02-25×36, Semrock), and focused on a detector (either an sCMOS camera (Zyla 4.2 Plus, Andor) or a photomultiplier tube, PMT (X13138-02, Hamamtasu)), after additional filtering (FF01-510/84-25 bandpass, FF01-650/SP-25 short-pass filter, Semrock).

 

Fig. 1 Setup: 85 fs long laser pulses, provided by a Ti:Sapphire mode-locked laser with a dispersion-compensation module, are reflected off a galvanometric mirror and imaged on an SLM. The SLM is imaged on the proximal facet of a fiber-bundle. Two-photon fluorescent (2PF) targets are placed at short distances from the distal end of the fiber. The excited 2PF is collected by the same fiber and detected at the proximal end using a PMT or an sCMOS camera. The detected 2PF signal is used as a feedback for an iterative wavefront-shaping optimization process, aimed at maximizing the total detected 2PF, forming a sharp focus on the target. 3D two-photon imaging is achieved by scanning the formed focus with the SLM and galvanometric mirror. A reference camera is used only for results inspection.

Download Full Size | PDF

 

Fig. 2 Experimental focusing through a fiber-bundle with proximal-only detection. (a) The speckle pattern at the object plane (z=2.2mm) before optimization. (b) Same as (a), with a 2PF object in the field of view (marked by arrow). (c) Same as (a) after optimization of the epi-detected 2PF, showing sharp focusing. The side-lobes are a result of the lattice periodicity of the bundle cores. (d) Evolution of the proximally-detected 2PF during the optimization process. Scale Bars: 100 μm. (color-bars are normalized such that the mean intensity in (a) is 1).

Download Full Size | PDF

 

Fig. 3 Lensless two-photon imaging obtained by scanning the focal spot generated in-situ in Fig. 2(c). (a,c) Reference camera bright-field images of fluorescent objects placed simultaneously at two axial planes. (b,d) Corresponding two-photon images obtained through the bundle. (e) Two-photon images at various focal planes near the plane of (b), demonstrating axial sectioning. Scale Bars: (a,c) 100 μm, (b,d) 0.05 radians, (the farthest object has a smaller angular extent).

Download Full Size | PDF

 

Fig. 4 Focal spot characterization. (a) Axial characterization of the focus width obtained by scanning the focal spot in the z-direction on a thin 2PF object, by adding a parabolic phase to the wavefront-correction. (b) Lateral characterization using the reference camera. Measured FWHMs are 5.3 ± 0.1 and 5.5 ± 0.1μm in the x- and y-axis respectively. Scale Bar, 5 μm.

Download Full Size | PDF

 

Fig. 5 Focusing with suppressed side-lobes using bundles with disordered cores. (a,b,c) Focusing results using a fiber bundle with ordered cores (Schott 1563385), used in Figs. 2(a)–4(b). (a) Image of the intensity distribution on the fiber distal facet, showing the cores ordered arrangement. (b) Fourier transform of (a) in log scale. (c) optimized focal spot (white line: intensity cross-section along the dashed line). (d,e,f) Same as (a–c), using a bundle with disordered cores (Fujikura FIGH-03-215S), showing suppressed side-lobes. (g) Comparison of the angular scanning range (the ’memory effect’) of the two fibers. The Fujikura fiber shows a smaller memory-effect range, likely due to light propagation between the cores (see (d)), resulting in a smaller FoV. (h,i) Two-photon imaging of fluorescent beads through the disordered fiber: (h) bright-field image of the object, taken with the reference camera. (i) two-photon image through the bundle obtained with the proposed approach. Scale Bars: (a,c,d,f) 50 μm, (b,e) 100 μm⁻¹ (h) 10 μm, (i) 0.02 rad.

Download Full Size | PDF

Before each experiment the optimal dispersion compensation value was found in-situ by maximizing the total collected 2PF signal. This resulted in pulses of less than 140fs duration (FWHM) at the bundle output for a 1m long Schott bundle (see discussion). After dispersion compensation, 2PF imaging is performed in-situ in two steps: first, the SLM wavefront-correction for focusing is found by iteratively optimizing the total 2PF signal collected by the detector. This step produces a sharp focused beam on the target object [20]. After the wavefront correction is found, the focused spot is raster-scanned in 3D by adding a wavefront tilt with the SLM and/or Galvo for lateral scanning, and a parabolic phase on the SLM for axial scanning. The total spatially-integrated 2PF signal detected at the proximal end is used to generate a 3D image of the target objects, as in conventional 2PF microscopy [20].

For inspecting the focusing performance, a reference camera (Mako U-130B, Allied Vision) was used to directly image the object plane, using a microscope objective (Obj₂; 10X Plan Achromat, Olympus). Importantly, the reference camera was used only for results inspection, and is not required for successful focusing or imaging.

3. Results

3.1. In-situ focusing

As a first demonstration, 2PF particles of Coumarin 307 were placed at distances of 2mm − 3mm from the bundle’s distal facet, at two axial planes. Initially, the light at the object plane was a random speckle pattern, as is shown in Fig. 2 (a). After 2,500 iterations of an optimization algorithm maximizing the total collected 2PF signal, I_2PF, a sharp focal spot was formed on the object closest to the fiber facet, as shown in Fig. 2(c). The obtained focus size is of the speckle-grain diffraction-limited dimensions, as is analyzed in Fig. 4 below, and has a peak to background ratio (PBR) of ∼ 310, visible in Fig. 2(c). The position of this initial formed focus cannot be determined from the proximal end measurement. It is usually positioned at the location of the initial maximal 2PF, determined by the target object shape and the initial speckle pattern.

We have used an iterative partitioning algorithm [22,23] to optimize the 2PF signal. In this algorithm, the SLM was divided into 128X160 equally sized square segments. In each iteration, a phase, ϕ_n of zero to 2π is added to a random subset of these segments in N = 4 steps. The 2PF signal as a function of the added phase is fitted to a cosine: I_2PF(ϕ_n) = A + B · cos (ϕ_n − ϕ), using a fast Fourier transform, and the phase ϕ that maximizes the 2PF signal is added to the chosen segments. The obtained 2PF signal as a function of the iteration number is plotted in Fig. 2(d).

3.2. 3D two-photon imaging

Following the adaptive focusing, 3D imaging was performed by raster scanning the focus, independent of the initial focus position. Fast raster scanning, with a pixel dwell-time significantly shorter than the SLM refresh rate, was achieved by using a galvanometric mirror for scanning in one lateral dimension. Scanning in the other lateral dimension was implemented by addition of a linear phase ramps to the SLM wavefront correction. Scanning in the axial dimension was achieved by adding parabolic phase patterns to the SLM wavefront correction, effectively transforming the fiber bundle to an adaptive lens.

Figures 3(a)–3(d) shows the 2PF images obtained with the focus scanning for two axial planes, where the fluorescent objects were present. Thanks to the inherent axial sectioning of 2PF focused excitation, at each plane, only the objects residing in this plane are visible in Figs. 3(b) and 3(d), whereas the parts of the object residing at different axial planes do not contribute to a substantial background halo, as is visible in the conventional bright-field imaging performed with the reference camera in Figs. 3(a) and 3(c). Figure 3(e) shows additional two-photon images obtained at axial planes close to the plane of Figs. 3(a) and 3(b).

3.3. Characterization of the formed focus

In wavefront shaping, the focusing resolution is expected to be diffraction-limited, as dictated by the dimensions of a single speckle grain [24]. To characterize the axial resolution, the 2PF signal in the image stack presented in Fig. 3(e) was plotted as a function of the axial distance. Figure 4(a) displays the trace of the maximal 2PF intensity obtained at each depth. The axial resolution, defined by the FWHM of a fit to a Gaussian, is δz = 165 ± 10μm. The axial resolution, δz in 2PF microscopy, defined by the axial FWHM is expected to be 0.64 times the axial FWHM of the intensity of the focused beam, which is twice its Rayleigh range:

3.4. Comparison of ordered vs. disordered bundles for wavefront-shaping based endoscopy

As is visible in the focusing results of Fig. 2(c), the obtained focal spot is surrounded by six side-lobes, in a hexagonal arrangement. These side-lobes are the result of the hexagonal periodicity of the cores in the Schott leached fiber bundle used. Figure 5(a) shows an image of the fiber facet used in the above experiments, displaying the hexagonal lattice order. The side-lobes in the formed focus can be predicted from the 2D Fourier transform of the cores arrangement, as shown in Figs. 5(b) and 5(c) [25–27].

To obtain a focus with suppressed side-lobes, we tested a commercially-available fiber with a less ordered arrangement of cores (Fujikura FIGH-03-215S). Indeed, using such a disordered fiber the side-lobes are effectively suppressed, as shown in Figs. 5(d)–5(f). Figures 5(h) and 5(i) display a 2PF image obtained with our approach using the disordered fiber, showing accurate diffraction-limited 2PF image of the target objects (fluorescent beads). The absence of focus side-lobes is an important advantage for imaging since such side-lobes produce replicas of the imaged objects, and thus limit their transverse extent, i.e. the field of view (FoV). The limit set on the FoV by ordered fiber bundles is FoV ≈ zλ/d, where d is the core-to-core spacing and z is the distance to the object [25].

While we have expected the disordered Fujikura fiber bundle to provide a considerably larger FoV than the ordered Schott leached fiber bundle, we have noticed that this was not the case, as is quantified by measuring the scanned focus intensity as a function of the scan angle, plotted in Fig. 5(g). The narrower effective ’memory-effect’ angular scanning range of the Fujikura fiber is not an inherent limitation of disordered fibers. We attribute the smaller FoV to light propagation between the cores of this bundle, which can be observed in the lower cores-to-background contrast in Fig. 5(d), compared to Fig. 5(a). We attribute the increased light propagation between the cores in this bundle to be the result of the different manufacturing technologies between the two fiber types. While the Schott bundle is a leached fiber bundle with air-gaps between the cores, the Fujikura bundles uses disorder instead of air-gaps to reduce core-to-core coupling [28], which may present larger light propagation between the cores.

Another effect that this imperfect light guidance is causing is a narrower spectral speckle correlation bandwidth, which we have measured for this fiber (not shown). The narrower spectral correlation bandwidth leads to a lower speckle contrast [29], which is measured when the femtosecond pulsed illumination (of approximately 12nm spectral bandwidth) is used. This in turn, lowers the focus intensity enhancement obtained by wavefront shaping [30]. To increase the initial speckle contrast and focusing PBR using this fiber, a narrow BPF (LL01-785-25, Semrock, 3nm FWHM) was used in the illumination path in the experiments involving this fiber.

4. Discussion

We have demonstrated an in-situ wavefront-correction approach for two-photon microendoscopy. Since the wavefront correction is sensitive to the bending of the fiber, for each fiber orientation a different correction needs to be found. However, this may be done in a continuous manner during in-vivo experiments. In our experimental implementation, the limiting factor on the time required for determining the wavefront-correction was the refresh rate of the specific liquid-crystal SLM used (<6 Hz). This yielded optimization times of tens of minutes in our experiments. This can be significantly shortened by using faster SLMs, since fundamentally the optimization time is limited by the 2PF signal level. Using the PMT for detection, integration times as short as 0.1ms were sufficient at the first iterations of the optimization process, with the Coumarin samples used in our experiments. Significantly lower integration times are required in the following iterations when the signal grows by several orders of magnitude. Thus, finding the wavefront correction using SLMs with higher refresh rates, such as Digital Mirror Devices (DMDs) [31], or galvanometric mirrors based approaches [32] should decrease the optimization time by more than three orders of magnitudes. Using more advanced algorithms such as genetic algorithms [33] can also reduce the number of iterations required for optimization. These approaches are expected to yield an optimization time of the order of seconds or even less using fluorescent markers, as was recently demonstrated using galvanometric-mirrors for 2PF microscopy through scattering tissue [32]. In general, any similar approaches for wavefront correction using a nonlinear feedback that were originally developed for scattering media are directly applicable for bundle-based endoscopy, since a fiber bundle can be considered as a thin scattering layer [18].

After the initial focusing, image acquisition speed is limited by the galvanometric scanners speed. Faster scanners based on resonant galvanometric mirrors may be utilized for faster scanning, if signal levels are sufficiently high. Some increase in signal level may be possible by improving the laser coupling efficiency to the bundle by using a microlens array phase pattern [6,7]. However, the implementation of a microlens array for the bundles with thousands of cores used in our experiment is expected to be technically challenging. The measured coupling efficiencies in our experiment were 30% with the Schott bundle and 15% with Fujikura bundle.

An inherent limitation of wavefront-shaping based correction is the sensitivity to fiber bending. While our proximal-only detection approach does not require initial measurements prior to the insertion of the endoscope or knowledge of the fiber parameters or shape, any movement of the fiber after the optimization process will hamper the correction and decrease the focus intensity. For a static fiber left untouched, we have experimentally measured decorrelation times exceeding 12 hours. However, bending and shifts of a few millimeters caused complete decorrelation of the speckle patterns and focusing intensity in the Schott bundle. Using the Fujikura bundle, slight bending caused the focal spot to shift, with only some reduction in intensity without complete decorrelation, similar to previous works [6]. We attribute the differences in sensitivity to bending to the fact that the commercial fibers used in our work are made of multimode cores, in contrast to the single-mode cores used in [6]. The bending-induced changes may be overcome using continuous adaptive focusing to adaptively compensate for the fiber movement during imaging, especially in bundles made of single-mode cores [6]. Another possible approach is to deploy the endoscope inside a rigid cannula [34], which will significantly improve stability at the price of an increase in the overall endoscope diameter, and loss of mechanical flexibility.

The field of view (FoV) of lensless endoscopy using fiber bundles with ordered cores is limited by the diffraction side-lobes of the periodic arrangement of cores [18], and the fiber ’memory-effect’ angular range. The maximal angular FoV will be attained by using a disordered bundle having single-mode fibers, with no light propagation between the cores. Unlike the memory-effect in scattering media, the memory-effect range in fiber bundles is not limited by the fiber length, and for an ideal bundle the memory-effect FoV spans the entire NA of the fiber [18]. The minimal working distance of the presented approach is the minimal distance at which the focal spot can be formed and scanned. In order to obtain a focus which has contribution from all of the fiber cores, a minimal working distance of z > D/NA is required, where D is the bundle diameter [18]. The maximal imaging distance is limited only by the 2PF signal level.

We have used the internal dispersion-compensation module of our laser for compensating the fiber dispersion. While such single-parameter dispersion compensation proved effective for performing 2PF imaging using the <1m long fibers used in our experiments, it is not capable of fully compensating the higher order dispersion of such long fibers. Using a 1m long Schott fiber bundle, we have measured via 2PF autocorrelation an output pulse duration of 140 fs, which is close to the laser initial pulse duration of 85 fs. Longer fibers may require compensation of higher order dispersion, which can be achieved using adaptive pulse-shaping [35]. Independent of the output pulse duration, the presented focusing and imaging approach should prove effective whenever there is a sufficiently large detected 2PF signal. Interestingly, as shown in previous works in complex media [23], the spatial wavefront correction can also compensate for both temporal and spatial distortions in multi-mode systems.

5. Conclusion

We have demonstrated a minimally-invasive, lensless, two-photon micro-endoscope with in-situ wavefront correction. In contrast to prior works, our technique does not require any distal access or prior characterization, making it an interesting potential solution for imaging or optogenetic stimulation [36]. We used the non-linearity of the two-photon excitation process to generate a focal spot. Other nonlinear mechanisms such as three-photon fluorescence, second harmonic generation, or stimulated Raman scattering may also be used. Using an SLM with a faster refresh-rate may potentially allow to use our approach for freely behaving animal studies.