The lensless endoscope represents the ultimate limit in miniaturization of imaging tools: an image can be transmitted through a (multi-mode or multi-core) fiber by numerical or physical inversion of the fiber’s pre-measured transmission matrix. However, the transmission matrix changes completely with only minute conformational changes in the fiber, which has so far limited lensless endoscopes to fibers that must be kept static. In this paper, we report for the first time, to the best of our knowledge, a lensless endoscope that is exempt from the requirement of static fiber by designing and employing a custom-designed conformationally invariant fiber. We give experimental and theoretical validations and determine the parameter space over which the invariance is maintained.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Cellular-level microscopic imaging has long been a vital tool in biomedical research. Recent years have seen numerous efforts to miniaturize imaging instruments to enable cellular-level imaging in behaving animals. A recent example is the miniaturized head-mounted microscope for fluorescence imaging  (2 g, ). This approach has a light source, filters, imaging optics, and CMOS camera integrated into a head-mounted device. For an overview of miniaturized microscopes, see Ref. . A different approach bases the light delivery and collection on optical fiber, which brings the advantage that the light source and detectors can be remote rather than integrated in the head-mounted device. As an added benefit, optical fiber-based approaches can use pulsed laser sources and perform non-linear imaging—to date, demonstrations have been using piezo-electric actuator  (few grams, ) or micro-electromechanical system (MEMS) mirrors  (2.15 g, ) to perform point-scanning imaging. Systems comprising compact needle probes with graded index lenses () housed in hypodermic needles (22-gauge, ) have also been demonstrated . For an overview of the most “conventional” optical-fiber-based approaches, see Ref. .
A new approach to fiber-based endoscopes came about in 2011 [7–12], which did away with imaging optics between fiber and sample and consequently is often termed “lensless endoscopes.” This represents the ultimate limit in miniaturization, since the head-mounted device can be as small as an optical fiber itself. Cellular-level imaging in live mice by lensless endoscopes has recently been demonstrated [13,14], but the imaging modality was restricted to one-photon fluorescence imaging and fiber length to a few centimeters, and the animal subject was fixed.
The lensless endoscope is most commonly implemented with image acquisition by point scanning [7,9,11,13,14], although other strategies exist [10,15,16]. The generic concept of thelensless endoscope is: (i) the transmission matrix (TM) in the basis of localized modes  of the fiber is measured in a preliminary step; (ii) the columns of the TM are identified as the input fields that give rise to focused output fields; (iii) the phase masks that convert the laser beam into said input fields are calculated; (iv) said phase masks are displayed sequentially on a spatial light modulator (SLM) resulting in a two-dimensional scan of the output focus, all the while back-scattered fluorescence signal is recorded, as in a point-scanning microscope. The fiber can be either a multi-mode fiber (MMF) [10,13–15,18] or a multi-core fiber (MCF) [11,12,16]. In either case, its transmission matrix can be thought of as18]. In both the MMF and MCF cases, the great challenge remains that following each conformational change in the fiber, either the additional extrinsic contribution () or the new TM () must be experimentally quantified whether directly [19,20] or indirectly  in order for aberration-free imaging to continue. This is the main obstacle standing between us and a flexible lensless endoscope, which would open the possibility for minimally invasive imaging in behaving animals.
A. Twisted Multi-Core Fiber
In our earlier work, we have shown how lensless endoscopes based on MCF (as opposed to MMF) simplify many of the considerations pertaining to the TM . In particular, in Ref. , we showed that the extrinsic contribution to the TM of a MCF is simply a diagonal matrix with complex elements of unit norm and argument, which is linear in the transverse coordinate, i.e., a matrix with only two free parameters. In the present article, we take an entirely new approach to overcoming this challenge: we seek to design a MCF for which the extrinsic contribution is identity (), which would render real-time tracking of the TM unnecessary and finally allow lensless endoscopes to reach their full potential, as the endoscope fiber could be allowed to flex freely. As we show below, a MCF with a twisted geometry can fulfill this requirement.
We designed and fabricated [see Supplement 1 and Fig. 1(a)] two MCFs, “MCF1” and “MCF2,” that have the following parameters in common: number of cores ; core-to-core distance (pitch) ; outer diameter 450 μm; parabolic core index profile with index difference relative to the cladding ; and core diameter (single mode at 1 μm). The lateral image resolution achievable in a lensless endoscope with this fiber is equal to the mode-field diameter of a single core, around 4 μm. MCF1 is twisted with a helical period of , MCF2 with . See Fig. 1(b) for a schematic of the twisted MCFs as well as the coordinates that we will employ. A non-twisted MCF, “MCF0,” with the same parameters but was drawn as well to serve as a reference.
B. Twisted MCF: Intrinsic Properties
The theoretical properties of the twisted MCF are derived in Supplement 1, and a summary is given in the following. Consider first a straight section of twisted MCF of length . The center core is unaltered by the twist; the effective index of the fundamental mode in the center core at wavelength is . Generally, the core , radially offset from the center by , has its physical length modified by the twist as4) is approximate, but in Supplement 1, we demonstrate that group index and refractive index can be used interchangeably without invalidating our conclusions. Figure 2(a) shows the comparison between the native group delays measured in MCF1 and MCF2 by spectral interference  as a function of radial core offset and those calculated by Eq. (4). We observe excellent agreement. Figure 2(b) shows experimentally measured mode field diameters along with the mean and standard deviation. Both show that the twist induces no appreciable variation (standard deviation 0.045 μm) in the mode field diameter. The phase profile of the off-center modes, however, is altered as a consequence of the cores of the twisted MCF no longer being parallel to the MCF axis. This geometrical consideration predicts that the beam exiting core has an azimuthal component, the angle of the free-space beam with the MCF axis being [see inset in Fig. 2(d)] 2(d), we compare the experimentally measured , measured as the optimal coupling angle of the free-space beam entering core number , to those predicted by Eq. (5), and we observe excellent agreement. Since the off-center cores follow a curved path in space, they are expected to experience loss when is small. Our numerical simulations show that no curved path loss () due to this effect is expected for either MCF1 or MCF2. Curved path loss is expected only for twist period as detailed in Supplement 1. Nevertheless, as seen in Fig. 2(c), MCF2 does present a radially dependent loss, but a cut-back measurement from 900 mm to 345 mm revealed that their origin was not the fiber itself, so we are able to attribute them to coupling loss due to mode mismatch between the input Gaussian beam profiles and the MCF modes with a non-flat transverse phase; cf. Eq. (5).
C. Twisted MCF: Extrinsic Contribution
We consider now a bent section of the twisted MCF with length and bend radius of curvature ; cf. Fig. 1(b). Now, in addition to the native phase and group delays, extrinsic contributions to the phase delay and group delay appear. As we demonstrate in Supplement 1, when , , is independent of , and as a consequence, and . That is, when the fiber length equals an integer number of twist periods, the extrinsic contribution is the identity matrix, and so the MCF is conformationally invariant for all fiber shapes with constant radii of curvature. An intuitive explanation for this is to realize that when the condition is satisfied, a given core spends as much time on the exterior of the bent MCF as it does on the interior, so the longitudinal compression of the core on the interior compensates the longitudinal dilation on the exterior.
To demonstrate the conformational invariance, we cut MCF2 to length and MCF0 to and perform a series of characterization.
D. Conformationally Invariant Endoscope
As a first demonstration, we show how a focus at the distal end of the MCF is maintained independently of MCF bends. To do so, we use the experimental setup detailed in Supplement 1 to establish a focus at the output end of MCF2. In Fig. 3(a), we show an image of the focus established for MCF2 held straight (). Subsequently, and without changing the injection, we then displace the distal end to change the radius of curvature of the MCF to . The focus, as seen in Fig. 3(b), retains its position to within one spot size. As a base of comparison, Figs. 3(c) and 3(d) show that the focus of MCF0 is translated by a much larger amount for a miniscule conformational change (to ). A side-by-side comparison is given by Visualization 1 (MCF2) and Visualization 2 (MCF0), which show how the focus translates when the MCFs undergo the same conformational change. This showcases the dramatic reduction in extrinsic contribution in MCF2 to the level where MCF2 can be considered virtually conformationally invariant.
In Fig. 4, we show digital confocal two-photon images [see Section 2 in Supplement 1] of a USAF target acquired through MCF2 in [Fig. 4(a)] straight geometry () and [Fig. 4(b)] bent geometry (). A slight translation of the image can be observed, which is reminiscent of the translation of the focus from Figs. 3(a) and 3(b), and thus demonstrates virtually conformationally invariant lensless endoscopic imaging.
We note that the images presented here [Figs. 4(a) and 4(b)] are acquired using a continuous-wave (CW) laser source rather than a pulsed femtosecond source. The reason for this is the large native group delay that results from the twist [Fig. 2(a)]. This native group delay is static and could be compensated with a static pre-compensation, so, while not demonstrated here, there are no conceptual difficulties in rendering the setup compatible with ultrashort laser excitation, and to access nonlinear imaging modalities.
We note also that the imaging results presented here were done with forward-collection of signals rather than with epi-collection through the MCF. The reason for this is the low fill factor of the cores, which results in poor epi-collection efficiency. However, it can be envisioned to increase the epi-collection efficiency of the twisted MCF by adding an external low-index cladding, as in, e.g., Ref. .
In the above, we have shown that the lensless endoscope based on twisted MCF with an integer number of twist periods is conformationally invariant as long as the bend radius of curvature remains constant. In practice, there are two main ways that conditions can depart from this ideal condition: (i) mismatch between length and twist period, i.e., ; and (ii) non-constant bend radius of curvature, i.e., varies along the MCF. Concerning (i), we present in Supplement 1 a map of the extrinsic contribution to phase delay in the relevant parameters, permitting to identify the parametric sub-space where departure from conformational invariance does not exceed a certain threshold. Concerning (ii), it is not possible to offer insights from analytical considerations; instead, an integral must be numerically evaluated, and in Supplement 1, we calculate for realistic MCF shapes. We concede that conformational invariance naturally cannot be maintained over the entire parameter space; nevertheless, we maintain that the twisted MCF drastically reduces the extrinsic contribution over the majority of fiber conformations likely to be found in practical settings.
Finally, we remark that the shorter the twist period of a MCF, the more conformationally invariant it will be. In the limiting case of infinitesimal twist period, the MCF is conformationally invariant over the entire parameter space. In practice, this limit is unattainable because losses due to the curved path of the cores will set in at a certain twist period. In Supplement 1, we detail numerical calculations that show that for the MCF parameters used here, twist periods below 2 mm lead to losses above 1 dB/m.
We have demonstrated a lensless endoscope based on a novel, twisted MCF at an operating point where imaging performance is unaffected by the conformation of the fiber. Our findings relax the constraints on fiber shape in lensless endoscopes and pave the way towards imaging tools that reap simultaneously the two principal benefits of using nothing but an optical fiber as an imaging probe, namely, its small diameter and its flexibility.
The experimental setup (Fig. S1, Supplement 1) was designed to measure the MCF properties using two modalities: ultrashort (fs) pulses or CW in order to perform group delay and imaging measurements, respectively. The first modality was used to perform the phase-stepping spectral interferometry to measure group delays of laser pulses transmitted through different MCF cores (with respect to a reference core, usually the central core), as described in . The second modality is used to measure the point-spread function (PSF) and imaging performance during MCF conformational changes.
For phase-stepping spectral interferometry (Fig. S1, Supplement 1), the laser beam (Amplitude Systèmes -Pulse, central wavelength 1030 nm, pulse length 170 fs, repetition rate 50 MHz) is expanded with a telescope (lenses L1 and L2) to overfill the clear aperture of a SLM (Hamamatsu LCoS-SLM X10468-07). The SLM is used to segment and shape the wavefront into beamlets prior to their injection into the cores of the MCF proximal facet. For group delay measurements, only two cores are injected into (the central, reference, core, and core ) resulting in fringes at the MCF output far field. For PSF measurements, all cores are injected into in order to produce a focus at the MCF output. The necessary demagnification of the beamlets to fit the core diameters is achieved via a lens L3 and a microscope objective MO1 (Olympus Plan , NA 0.40). Light, transmitted through the cores, is collected at the fiber distal end using a second microscope objective MO2 (Nikon Plan, NA 0.25) and passes through a linear polarizer (LP) (Thorlabs LPNIR100) to maximize the contrast of the interference fringes . The image of the distal MCF facet or its near field is monitored with CCD1 (FLIR FL3-U3-32S2M-CS) in order to follow any changes of the transmitted power or the generated interference pattern while the fiber is bent. The MCF output far field is coupled into a MMF (core diameter of 62.5 μm, not shown in Fig. S1, Supplement 1), linked to an optical spectrum analyzer (OSA) (Yokogawa AQ-6315A), or imaged entirely onto a camera CCD2 (Thorlabs DCU223M) to record the PSF stability during MCF bending. Magnification of the objective MO2 and lenses L4, L5 is chosen so that only a restricted area of the far-field pattern is selected with the MMF probe (fulfilling therefore , where is related to the interference fringe spatial frequency, and is diameter of the MMF core collecting the light). The distal end of the MCF is kept fixed on a portable unit that can freely rotate while the MCF is bent. The fiber conformation is monitored using a portable camera (not shown). Finally, galvanometric scan mirrors conjugated to the SLM active area (telescopes L1 and L2) are used to scan the focused distal PSF across the sample for imaging. The galvanometric mirrors impose controllable phase tilts on the input wavefront that are translated to the output distal MCF facet owing to the diagonal MCF transmission matrix.
Agence Nationale de la Recherche (ANR-10-INSB-04-01, ANR-11-EQPX-0017, ANR-11-IDEX-0001-02, ANR-11-INSB-0006, ANR-11-LABX-0007, ANR-14-CE17-0004-01); European Regional Development Fund (CPER Photonics for Society P4S); Institut National de la Santé et de la Recherche Médicale (18CP128-00, PC201505); Eurostars (12583/19/Q DEEPBRAIN); National Eye Institute (R21EY029406); Ministère de l’enseignement supérieur et de la recherche; Hauts de France council.
SATT Sud-Est GDC Lensless Endoscope. CNRS-Weizmann ImagiNano European Associated Laboratory.
See Supplement 1 for supporting content.
1. Y. Ziv and K. K. Ghosh, “Miniature microscopes for large-scale imaging of neuronal activity in freely behaving rodents,” Curr. Opin. Neurobiol. 32, 141–147 (2015). [CrossRef]
2. D. Aharoni and T. M. Hoogland, “Circuit investigations with open-source miniaturized microscopes: past, present and future,” Front. Cell. Neurosci. 13, 141 (2019). [CrossRef]
3. A. Lombardini, V. Mytskaniuk, S. Sivankutty, E. R. Andresen, X. Chen, J. Wenger, M. Fabert, N. Joly, F. Louradour, A. Kudlinski, and H. Rigneault, “High-resolution multimodal flexible coherent Raman endoscope,” Light Sci. Appl. 7, 10 (2018). [CrossRef]
4. W. Zong, R. Wu, M. Li, Y. Hu, Y. Li, J. Li, H. Rong, H. Wu, Y. Xu, Y. Lu, H. Jia, M. Fan, Z. Zhou, Y. Zhang, A. Zang, L. Chen, and H. Cheng, “Fast high-resolution miniature two-photon microscopy for brain imaging in freely behaving mice,” Nat. Methods 14, 713–722 (2017). [CrossRef]
5. R. S. Pillai, D. Lorenser, and D. D. Sampson, “Deep-tissue access with confocal fluorescence microendoscopy through hypodermic needles,” Opt. Express 19, 7213–7221 (2011). [CrossRef]
6. B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. Cheung, and L. M. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005). [CrossRef]
7. A. J. Thompson, C. Paterson, M. A. A. Neil, C. Dunsby, and P. M. W. French, “Adaptive phase compensation for ultracompact laser scanning endomicroscopy,” Opt. Lett. 36, 1707–1709 (2011). [CrossRef]
8. T. Cizmar and K. Dholakia, “Shaping the light transmission through a multimode optical fibre: complex transformation analysis andapplications in biophotonics,” Opt. Express 19, 18871–18884 (2011). [CrossRef]
9. I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “Focusing and scanning light through a multimode optical fiber using digital phase conjugation,” Opt. Express 20, 10583–10590 (2012). [CrossRef]
10. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012). [CrossRef]
11. E. R. Andresen, G. Bouwmans, S. Monneret, and H. Rigneault, “Toward endoscopes with no distal optics: video-rate scanning microscopy through a fiber bundle,” Opt. Lett. 38, 609–611 (2013). [CrossRef]
12. E. R. Andresen, G. Bouwmans, S. Monneret, and H. Rigneault, “Two-photon lensless endoscope,” Opt. Express 21, 20713–20721 (2013). [CrossRef]
13. S. Ohayon, A. M. Caravaca-Aguirre, R. Piestun, and J. J. DiCarlo, “Minimally invasive multimode optical fiber microendoscope for deep brain fluorescence imaging,” Biomed. Opt. Express 9, 1492–1509 (2018). [CrossRef]
14. S. Vasquez-Lopez, R. Turcotte, V. Koren, M. Plöschner, Z. Padamsey, M. J. Booth, T. Cizmar, and N. Emptage, “Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber,” Light Sci. Appl. 7, 110 (2018). [CrossRef]
15. D. Loterie, S. Farahi, I. Papadopoulos, A. Goy, D. Psaltis, and C. Moser, “Digital confocal microscopy through a multimode fiber,” Opt. Express 23, 23845–23858 (2015). [CrossRef]
16. V. Tsvirkun, S. Sivankutty, E. R. Andresen, and H. Rigneault, “Wide-field lensless endoscopy with multi-core fiber,” Opt. Lett. 41, 4771–4774 (2016). [CrossRef]
17. S. M. Popoff, G. Lerosey, R. Carminati, M. Fink, A. C. Boccara, and S. Gigan, “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett. 104, 100601 (2010). [CrossRef]
18. M. Plöschner, T. Tyc, and T. Cizmar, “Seeing through chaos in multimode fibres,” Nat. Photonics 9, 529–535 (2015). [CrossRef]
19. S. Farahi, D. Ziegler, I. N. Papadopoulos, D. Psaltis, and C. Moser, “Dynamic bending compensation while focusing through a multimode fiber,” Opt. Express 21, 22504–22514 (2013). [CrossRef]
20. A. M. Caravaca-Aguirre, E. Niv, D. B. Conkey, and R. Piestun, “Real-time resilient focusing through a bending multimode fiber,” Opt. Express 21, 12881–12888 (2013). [CrossRef]
21. E. R. Andresen, S. Sivankutty, V. Tsvirkun, G. Bouwmans, and H. Rigneault, “Ultra-thin endoscopes based on multi-core fibers and adaptive optics: a status review and perspectives,” J. Biomed. Opt. 21, 121506 (2016). [CrossRef]
22. V. Tsvirkun, S. Sivankutty, G. Bouwmans, O. Vanvincq, E. R. Andresen, and H. Rigneault, “Bending-induced inter-core group delays in multicore fibers,” Opt. Express 25, 31863–31875 (2017). [CrossRef]
23. E. R. Andresen, S. Sivankutty, G. Bouwmans, L. Gallais, S. Monneret, and H. Rigneault, “Measurement and compensation of residual group delay in a multi-core fiber for lensless endoscopy,” J. Opt. Soc. Am. B 32, 1221–1228 (2015). [CrossRef]
24. S. Sivankutty, E. R. Andresen, G. Bouwmans, T. G. Brown, M. A. Alonso, and H. Rigneault, “Single shot polarimetry imaging of multicore fiber,” Opt. Lett. 41, 2105–2108 (2016). [CrossRef]