1. Introduction

Imaging inaccessible structures has become possible by modern endoscopy with applications in medical diagnosis and treatment as well as in manufacturing where the visualization of hidden structures is important to increase the quality of manufactured goods. Today there are two main approaches to build a miniaturized endoscope. The first is the miniaturization of optical elements (micro GRIN lenses) [1–3] and the second is a fiber-based approach [1,4–6] for flexible endoscopes. A conventional endoscope is a fiber bundle where each fiber corresponds to a single pixel of the captured image. However, the image resolution is limited by the fibers core-to-core separation. There are also approaches using a single mode fiber for excitation and a multimode fiber for light collection of reflected or fluorescent signal whereby miniaturized actuators scan a focused light spot at the distal tip [1].

Recently there has been an increasing interest in using a single multimode fiber (MMF) for both light delivery and collection [7–10]. MMFs provide lensless imaging with a sub-micrometer resolution over the diameter of the MMF and are therefore a suitable solution to further miniaturize endoscopes. However, light focused into a MMF is coupled in different modes, which propagate with different propagation constants and interact by coupling energy from one mode into another when the fiber is bent. This effect becomes more important with increasing core diameter, leading to modal scrambling in case of large core fibers. An input pattern will be then completely distorted at the output a MMF, but its original information will remain and can be recovered [11]. Therefore, the main drawback of imaging through a multimode fiber is that it is highly sensitive to bending [12]. As long as the fiber doesn’t move, the speckle pattern remains the same and the distortions can be compensated. When the fiber is bent, mode coupling is induced and the speckle pattern changes. To overcome this limitation, one approach is to use a completely rigid multimode fiber endoscope that is calibrated once for one specific spatial conformation of the fiber. However, for some applications, a flexible or semi-flexible endoscope is necessary. Choi et al. [8] imaged through a fiber bent within a limited range, but without compensating for bending which strongly limits the bending range. A dynamic compensation for bending has been proposed by Caravaca-Aguirre et al. [13] who have measured the transmission matrix in real-time to focus on a single spot but their method still requires a detector on the other side of the fiber.

In this paper, we present a method that can be used to dynamically compensate for bending while focusing light through a multimode fiber by digital phase conjugation. The key element of the presented approach is a virtual coherent point light source (a beacon) placed at the distal end of the MMF.

The virtual beacon point source is experimentally implemented by recording at the fiber tip, a permanent hologram of a spherical wave whose virtual focus point is in front of the fiber facet. This hologram is recorded in a thin polymer material. The method consists of recording, with a digital camera, the speckled wavefront generated by this virtual point source beacon at the proximal end of the MMF and extracting its phase and amplitude. We then use the extracted information in two separate ways. First, the phase conjugate is sent back through the fiber, which generates a real point source at the same location as the virtual point source. In this way, the focus at the distal end of the MMF is maintained even if the fiber is continuously bent. Second, we show how this virtual beacon source is used to sense the fiber conformation. The speckle pattern at the proximal MMF end is highly dependent on its conformation. We demonstrate that by comparing the intensity speckle pattern generated by the beacon to previously stored patterns corresponding to various conformations, it is possible to determine the fiber conformation without having access to its distal tip. Once the fiber conformation is known, previously extracted phase patterns can be used for phase conjugation to perform focusing. Also, the phase patterns corresponding to an array of spots could be stored in computer memory for each fiber conformation so that scanning based imaging could be performed. While scanning and imaging has not been performed in this study the result we show pave the way for the implementation of a high-resolution semi-flexible lensless endoscope based on digital scanning through an optical multimode fiber.

2. Experimental results

2.1 Digital phase conjugation on a virtual beacon source

In order to compensate for modal scrambling in a multimode fiber, we use the Digital Phase Conjugation (DPC) technique as described in [14]. When a focus spot (provided by a calibration beam) is coupled at the distal end of a MMF, it generates a speckle field at its proximal end. The phase-conjugate of this speckle field is calculated by digital holography [15]. The calculated phase is displayed on a Spatial Light Modulator (SLM), which assigns the information to a plane wave, that in turn forms the phase conjugate optical field. Accordingly, the phase-conjugated field propagates back from the proximal end through the fiber generating a focus spot at the distal end at the same position as the initial focus spot. Through this process, all the distortions are undone. The phase pattern displayed on the SLM depends on the excitation and on the bending configuration of the fiber, meaning that for every different excitation or bending configuration, a new phase pattern has to be calculated.

To focus through a multimode fiber using DPC, it is necessary, first, to place a point source (beacon of light) at the position where the focal spot is required. For a practical implementation of a multimode fiber based endoscope, the beacon needs to be generated in real-time and in reflection i.e. from the same side as the DPC apparatus. In the approach presented here, the virtual point source is recorded on a hologram, which is placed at the distal tip of the multi-mode fiber as described in Fig. 1. This virtual holographic beacon generates a wavefront that propagates from the distal to the proximal side of the fiber and is used to recreate, by phase conjugation, the original focus point, independently of the fiber conformation, as long as the phase pattern on the SLM is changed accordingly. To create such a beacon, a double clad fiber can be used to bring the calibration beam to the fiber tip through the single mode core. The calibration beam is diffracted by the hologram into the multimode cladding as if it would be coming from a point source [Fig. 1(a)]. Unfortunately double clad fibers are not commercially available for visible wavelengths. We propose then an alternative implementation where the double-clad fiber is replaced by a separate single-mode fiber alongside a multimode fiber [Fig. 1(b)]. This configuration is bulky and thus is not intended as a final endoscopic device, rather it is to demonstrate the principle of the virtual holographic beacon. However, the same process could be applied with the device showed in Fig. 1(a) whose size would be that of the double-clad fiber itself.

 

Fig. 1 Principle for recording a virtual beacon source on a photosensitive holographic material. The object beam is a diffraction-limited spot generated by an objective lens on the position where the virtual source is desired. The reference beam is delivered by a fiber using two different geometries. The object beam interferes with the reference beam and a hologram is recorded in the polymer film. We reconstruct the object point source (virtual holographic beacon) by illuminating the hologram with the same reference beam that acts as calibration beam for phase conjugation. In this step, the object beam is removed. The calibration beam is diffracted by the hologram as if it was coming from the initial focus spot, which is a virtual beacon, and gets coupled into the multimode fiber. (a) Geometry using a double-clad fiber where the reference beam comes from the single-mode core and the diffracted light is coupled in the multimode cladding. The polymer film is directly laminated on the fiber tip. (b) Geometry using a multimode fiber and a separate single-mode fiber to bring the reference beam. The polymer film is laminated on a glass slide, which acts as a planar waveguide for the reference beam coupled into it using a prism.

Download Full Size | PDF

A photosensitive holographic film (Bayfol® HX, Bayer) [16] is laminated on a glass slide, which acts as a planar waveguide for the reference beam that is coupled using a microprism. The object beam is a diffraction-limited spot generated by an objective lens on the position where the virtual source is desired. This object beam interferes with the reference beam guided in the planar structure and a hologram is recorded in the polymer film. After this step, the object beam is removed. The same reference form the single-mode fiber is used to read the hologram and forms the calibration beam for phase conjugation. The reference light is guided through the slab waveguide and diffracted by the hologram, which generates the virtual beacon source coupled into the MMF. The collected light propagates through the MMF and gets scrambled due to inter-modal coupling leading to a speckle pattern at the proximal end of the fiber. The fiber output is then imaged on a camera (CMOS1) as described in Fig. 2 where it interferes with a plane wave provided by an off-axis reference beam reflected by BS2. The off-axis digital holographic configuration enables to extract the phase distribution, which is then displayed as its conjugate on a Spatial Light Modulator (SLM). The detailed process is described in [14]. The phase-conjugated wavefront propagates back through the MMF, compensating for the modal scrambling and coming into a focus at its distal end. The real focus spot is at the same location as the virtual beacon source generated by the hologram recorded on the polymer film.

 

Fig. 2 (a) Experimental setup of Digital Phase Conjugation (DPC) through a multimode fiber using a virtual holographic beacon source. The initial beam is split into two arms with BS1. A first beam is expanded by the telescope formed by lenses L1-L2 and goes through the spatial filter OBJ3, pinhole and L4. This cleaned up and expanded beam is used as an off-axis reference beam for the recording of a digital interferogram on CMOS1. The second beam is coupled into a single mode fiber with OBJ4. The light at the distal side of the single mode fiber is coupled into a glass plate (planar waveguide) through a microprism [Fig. 1] where it acts as a calibration beam to read-out the hologram placed at the fiber distal tip. The light diffracted from the hologram is collected by the multi-mode fiber. The proximal facet of the fiber is imaged on the CMOS sensor through the 4f imaging system (OBJ2 and L3). The reference is combined with the image using the non-polarizing beamsplitter BS2 and thus generating a digital off-axis interferogram. By displaying the digital conjugate phase on the SLM and illuminating it with the off-axis reference beam, the phase conjugate wavefront is generated and is following redirected towards the fiber by reflecting on BS2. The quality of the generated focus is examined on CMOS2 through the imaging system formed by OBJ1 and L5. (b) and (c) show two different bending positions of the fiber where ΔR is the relative change in bending radius.

Download Full Size | PDF

A multi-mode fiber with a core thickness of 200μm, NA = 0.39, 1m length is used for the experiment. The laser source is an Oxxius SLIM laser emitting at 532nm. In Fig. 3 we compare the focus obtained using the virtual holographic beacon source generated by the hologram and the one obtained using a real beacon source. This real beacon source is placed at the same position by directly focusing light at the distal tip of the fiber using OBJ1 and a beam splitter to couple light through it from the laser (this part is not shown in Fig. 2). After phase conjugation, the distal tip of the fiber is imaged on CMOS2 to control the quality and the position of the generated focus spot. In Fig. 3(c) we show that a sharp focus spot is obtained by using a virtual holographic beacon point source as a seed for phase conjugation. To quantify the quality of the focus spot we calculate the enhancement, which is defined as the peak intensity over the mean intensity of the image recorded on CMOS2. The enhancement of the focus spot obtained by phase conjugation on a real source is 340 while the value obtained with the virtual beacon is lower and equal to 220. The difference between these two values can be explained by the quality of the virtual beacon generated by the hologram recorded on the polymer film and that might be not a perfect Gaussian beam as the one generated by OBJ1.

 

Fig. 3 Digital Phase Conjugation (DPC) using a virtual holographic beacon source. (a) Focal spot obtained after phase conjugation from a real beacon source (by directly focusing light at the distal tip of a 200μm core/ 0.39 NA fiber). The enhancement is evaluated to be approximately 340. (b) is the output of the fiber when a random phase is displayed on the SLM in the same experimental conditions. (c) Focal spot obtained after phase conjugation from a virtual holographic beacon source placed at the distal facet of the fiber. The enhancement is evaluated to be approximately 220, which is around 65% of what was achieved with a real beacon in (a). (d) is the output of the fiber when a random phase is displayed on the SLM. The units of the transverse axis are pixels.

Download Full Size | PDF

Using the same hologram placed at the distal tip of the fiber, a new interferogram can be recorded for another bending configuration and used for focusing again. For this process the calibration beam is brought by the single mode fiber so that the distal tip of the MMF is free.

2.2 Effects of fiber bending on the focal spot obtained by digital phase conjugation

In order to implement a system that compensates for fiber bending effects, we first characterize how bending degrades a focus spot. For this, we quantify the focus spot enhancement while the fiber is bent as well as the variations of the output speckle pattern. For the latter, we calculate the correlation coefficient between an initial intensity speckle pattern and all the new speckle patterns generated while bending the fiber.

We measure the effect of bending the MMF on the speckle intensity pattern for different fiber core diameters. A linear actuator is used to repeatedly bend the fiber as shown in Fig. 2. In this section, for the characterization, a real beacon generated by OBJ1 is used. In all measurements, the effect of bending is reported for a relative change of the bending radius (ΔR) from an arbitrary value R₀ of the Radius Of Curvature (ROC) at the chosen bending point. For this position, we record a reference speckle intensity pattern and an interferogram on CMOS1. We calculate the conjugate phase pattern corresponding to R₀ and display it on the SLM to generate a focal spot at the distal end of the fiber. Then the ROC of the fiber is changed while the excitation spot (from an objective lens) at the distal end of the fiber remains the same. The new intensity speckle pattern transmitted though the fiber is then correlated with the intensity speckle pattern (recorded at the proximal end) corresponding to R₀. At the same time, the reference phase pattern is displayed on the SLM and the enhancement of the focal spot size is calculated at the distal end. The experiment was conducted with three different fibers with core diameters of 50μm, 105μm and 200μm (supporting respectively around 1700, 7500 and 86000 modes) to investigate the correlation and enhancement dependence on the number of excited modes. In Figs. 4(a)-4(c), we show the measured normalized enhancement (distal end) and the correlation of the intensity speckle patterns with the reference intensity speckle pattern as a function of ΔR. We observe that the enhancement decreases with fiber bending which is a direct consequence of the fact that the correlation of the speckle patterns also decreases with fiber bending. The initial reference phase pattern does not compensate anymore the modal distortions of the bent fiber.

 

Fig. 4 (a) to (c) normalized enhancement of the focal spot obtained after phase conjugation (blue dots) and the correlation function of the intensity speckle patterns (red dots) with the reference intensity speckle pattern, as a function of the relative bending radius, for three fibers with different core diameters (50μm, 105μm and 200μm). The full widths at half maximum (FWHM) are respectively 1 mm, 2.7 mm and 10 mm. Figure (d) gives the absolute values for the three measurements to show that for a large core fiber the enhancement is much higher even if it drops faster.

Download Full Size | PDF

Figure 4 shows that, as the core diameter of the MMF increases, both enhancement and correlation decay faster as a function of ΔR. The full widths at half maximum (FWHM) are respectively 1 mm, 2.7 mm and 10 mm. Therefore, the sensitivity to bending of the enhancement of the focal spot is related to the core diameter of the fiber.

The larger the core diameter, the higher is the sensitivity [17]. Qualitatively, that can be explained by the strong dependency of the coupling coefficients to the mode propagation constants [18]. For closely spaced mode propagation constants, the coupling coefficients between the modes are high, which leads to a high bending sensitivity of these modes. For fibers supporting many modes and with a fixed numerical aperture, the difference between the mode propagation constant of the highest order mode and the lowest order mode is constant and doesn’t depend on the fiber diameter. However the number of modes strongly depends on the fiber core diameter (approximately as 4D²NA²/λ² where D is the core diameter, NA the fiber numerical aperture and λ the wavelength). For fibers supporting a large number of modes, the spacing between adjacent mode propagation constants is very small and therefore high coupling between the modes occurs when the fiber is bent. Hence large core fibers are much more sensitive to bending than small core fibers with a same NA. We also notice that the absolute enhancement value is higher for a larger core fiber [Fig. 4(d)] as expected since a larger number of modes are excited. Indeed, the enhancement increases with the number of controlled modes and thus with the number of excited modes [14].

In another experiment, a focal spot is positioned at different lateral positions on the distal fiber facet. This experiment aims to acquire a better understanding of the influence of the number of excited modes on the enhancement degradation of the focal spot obtained after phase conjugation as the ROC is changed. We have previously shown that an edge illumination excites more modes than center core illumination [14]. In Fig. 5, we present the results of enhancement as a function of ΔR for different lateral excitation positions. The fiber is excited at four different positions indicated in Fig. 5(a)-5(d). The excitation position is shown with a green dot on the grey circle representing the fiber facet. The blue dots correspond to the normalized enhancement and the red dots to the correlation function between the measured and the original speckle pattern. In Fig. 5(e) the absolute values of enhancement are shown and confirm that the enhancement of the focal spot obtained after phase conjugation for an edge excitation is higher than for a central excitation. We notice that the enhancement drops faster as a function of ΔR for a more central excitation [Fig. 5(a)]. The FWHM are respectively 1.1 mm, 1.7 mm, 1.9 mm and 2 mm. The reason is that central core excitation produces fewer high-order modes than edge excitation and thus a higher ratio of low-order modes in comparison to the total number of modes. The spacing between adjacent propagation constants for low-order modes is smaller than for high-order modes, leading to more intermodal coupling [19]. Thus, the high ratio of low-order modes (in comparison with the total number of modes) generated in case of central core excitation leads to a larger sensitivity with respect to bending than in the case of edge excitation. In deed for an edge excitation, more modes are excited in total and with a higher ratio of high-order modes leading to a slower decay of the enhancement.

 

Fig. 5 (a) to (d) normalized enhancement of the focal spot obtained after phase conjugation (blue dots) and the correlation function of the speckle pattern (red crosses) as a function of bending for different excitation positions at the distal facet of a 105μm core fiber. The excitation position is indicated with a green dot on the gray circle representing the fiber facet. The decay of the enhancement is slower when the fiber excitation is moved towards the edge. The FWHM are respectively 1.1 mm, 1.7 mm, 1.9 mm and 2 mm. For edge excitation more modes are excited and the absolute value of enhancement increases which is shown in (e).

Download Full Size | PDF

2.3 Focusing through a semi-flexible multimode fiber

As mentioned earlier, one solution to the fiber bending problem consists of using a rigid endoscope [10] for which the calibration of the fiber is done once for good prior to imaging. Indeed, to perform imaging through a multimode fiber, we first need to raster scan the focus on a sample. Then for each focus position, light scattered back by the sample is collected through the same fiber and measured by a detector. In this way, an image if formed [10]. Thus for every single position of the focused spot, a new phase pattern is displayed on the SLM, meaning that an interferogram has already been recorded for every position during the fiber calibration.

For certain applications flexibility is needed and thus we need to be able to dynamically compensate for the effects of bending. With the proposed beacon method, it is possible for the first time, to focus on a fixed single point without having access to the distal tip of the fiber distal end. Unfortunately, it does not provide the ability to perform imaging.

We suggest thus, the use of a semi-flexible multimode fiber for which the number of conformation is limited. For example, the ROC can be changed at a limited number of bending points along the fiber. It is then necessary to find a method to recover the fiber conformation.

The approach devised here is to use the speckle pattern generated by the virtual beacon source, propagating in the fiber from its distal to proximal end, to sense the fiber conformation (see section 2.1). This speckle pattern is then compared by correlation to previously recorded speckle patterns for different fiber conformations. We suggest then the following process for imaging: first, the fiber conformation is determined by correlation with the intensity speckle pattern stored in a database, second the ensemble of phase patterns used for raster scanning corresponding the fiber conformation are recalled form the database and displayed sequentially on the SLM. In the following experiment, we only stored one such phase pattern (not the ensemble for raster scanning) and demonstrate dynamic bending compensation by focusing on a single spot as described below.

The aim is to maintain a constant focus enhancement at the distal fiber end while the ROC is modified. We first calibrate the MMF by measuring and storing, in computer memory, the conjugated phase patterns needed to generate a focus. We calculate this patterns only for a fixed number N of fiber ROC at the chosen bending point. We also store for all the ROC values in the chosen bending range, the intensity speckle patterns generated by the virtual beacon. The vertical black lines in Fig. 6 indicate the values of ΔR for which an interferogram has been recorded. The ROC of the fiber is then changed randomly. For each increment of ΔR, the holographic beacon [Fig. 1] produces a speckle pattern after propagating in the bent MMF. This pattern is correlated with the speckle patterns corresponding to the prerecorded calibrated patterns. The phase pattern that returns the highest correlation coefficient is selected as the most likely candidate for the fiber configuration. The corresponding phase pattern is displayed on the SLM to create a focal spot at the distal fiber facet and the enhancement is calculated.

 

Fig. 6 Focusing through a fiber bent within a restricted radius of curvature range using the intensity speckle patterns generated by a holographic beacon source as a conformation sensor. The focus is generated by DPC. The blue dots give the enhancement of the focal spot obtained after phase conjugation. The red dots give the correlation of the intensity speckle pattern obtained at the proximal facet of the fiber, with the intensity speckle pattern of the closest prerecorded position of the fiber. (a) shows the case of under sampling (N = 3) which means that a significant drop of enhancement is observed between the precalibrated positions. For (b) we choose N = 9 and thus the enhancement between the precalibrated positions is maintained at a high value. Therefore the focus is maintained over the whole bending range.

Download Full Size | PDF

Given a fixed bending range, the sampling (i.e. number N of prerecorded speckle patterns) required to maintain a constant enhancement depends only on the decay rate of the enhancement of the focal spot obtained after phase conjugation. From the results presented in the previous section, we can infer that the number N of pre-recorded speckle patterns mainly depends on the core diameter of the fiber. The larger the core diameter of the fiber, the more pre-recorded patterns are necessary to ensure focusing over a certain bending range.

The results are presented in Fig. 6. If N = 3, in the specified bending range [Fig. 6(a)], we are in a case of under-sampling and the enhancement plotted in blue, can decrease by 50% between the pre-calibrated positions. In Fig. 6(b) the results of the same experiment, with a larger number of prerecorded positions (N = 9), are shown. The enhancement is maintained at an average of 0.8 for the full range of ROC. This demonstrates that a focal spot can be maintained when the fiber is bent. Without using the virtual beacon as a fiber conformation sensor, the enhancement would drop as shown in Figs. 4-5. Thus, with this system it is possible to retrieve the bending position of the fiber from the speckle pattern obtained from a holographic beacon source and to keep a high enhancement of the focal spot obtained after phase conjugation. The video attached in supplementary material (Media 1) shows the focus spot when bending is compensated for and when it is not.

The crucial point of this method is to determine the number of prerecorded positions necessary to always recognize the actual bending configuration of the fiber and to keep the focal spot within an acceptable enhancement. If there are not enough patterns (under sampling), focusing will not be possible at all positions. If the number of patterns is too high (over sampling) the speed of computation could become a problem and real-time implementation might not be possible anymore.

These results show that it is possible to recover the spatial configuration of a multimode fiber within a limited known range. Once the configuration of the fiber is known, any of the methods used to image through a multimode fiber can be applied. For scanning based imaging, the phase patterns for all focal spots have to be recorded for each possible configuration of the fiber.

This technique can be applied with a reasonable computational time for a limited number of bending joints of the fiber. Otherwise the memory required to store and computational time might be too large. The actual experiment is not optimized regarding time since only mechanical shutters and liquid crystals SLM are used. For two bending joint with a ΔR from infinity to 10mm for example, we would need to calibrate the fiber for 20 positions for each joint to keep the same ratio as in Fig. 6(b). This will result in a number of calibration positions of 20². We need then to learn the patterns needed for phase conjugation for 400 conformations of the fiber.

3. Discussion

We showed that by recording the phase and the speckle pattern for 9 bending configurations of the fiber, it is possible to maintain focusing over a 5 mm change of radius of curvature of the fiber. To implement such a system in real-time only a limited number of prerecorded patterns can be used due to limitations in computation speed. It is therefore essential to know how fast the enhancement drops when the fiber is bent in order to determine the optimal number of bending radii for which the pre-calibration has to be done. We showed that the enhancement as a function of radius of curvature is strongly related to the core diameter of the fiber. The larger the core, the faster the enhancement decreases. For a fixed number of prerecorded patterns (limited by the computation speed of the system), one can choose between high enhancement and a smaller bending range obtained with a large core fiber, or a smaller enhancement and a larger bending range for a fiber with a smaller core diameter.

The use of a virtual beacon to recover the spatial configuration of the fiber provides the basis for the implementation of a semi-flexible endoscope using a multimode fiber. However, the allowed bending configurations have to be fixed in advance in order to calibrate the fiber.

We showed an implementation using a single mode fiber alongside the MMF. We hence demonstrated that focusing through a bent multimode fiber could be done without having access to the distal tip of the fiber. An implementation with a double clad fiber would simplify the geometry and provide a compact system by avoiding a separate single mode fiber to bring the calibration beam to the fiber tip.

4. Conclusion

We have shown in this work that we can dynamically compensate for bending while focusing through a multimode fiber. The method consists of creating a virtual holographic beacon by using a single mode fiber illuminating a hologram of a point source at the distal tip of the multimode fiber. We demonstrated that the bending configuration of a multimode fiber can be found by comparing the speckle pattern originating from the virtual beacon source with previously recorded speckle patterns corresponding to various bending configurations of the fiber. Up to now, all methods to image through multimode fibers were restricted to rigid fibers, which was a major limitation. The proposed method lifts this limitation, which may pave the way to many new applications of high-resolution lens-less endoscopes based on digital scanning through optical multimode fibers.