A conventional lens has well-defined transfer function with which we can form an image of a target object. On the contrary, scattering media such as biological tissues, multimode optical fibers and layers of disordered nanoparticles have highly complex transfer function, which makes them impractical for the general imaging purpose. In recent studies, we presented a method of experimentally recording the transmission matrix of such media, which is a measure of the transfer function. In this review paper, we introduce two major applications of the transmission matrix: enhancing light energy delivery and imaging through scattering media. For the former, we identified the eigenchannels of the transmission matrix with large eigenvalues and then coupled light to those channels in order to enhance light energy delivery through the media. For the latter, we solved matrix inversion problem to reconstruct an object image from the distorted image by the scattering media. We showed the enlargement of the numerical aperture of imaging systems with the use of scattering media and demonstrated endoscopic imaging through a single multimode optical fiber working in both reflectance and fluorescence modes. Our approach will pave the way of using scattering media as unique optical elements for various biophotonics applications.
© 2015 Optical Society of America
The search for an order out of disorder has fascinated physicists for many decades. The interaction of waves with a disordered medium has provided a testing ground for this quest, which led to interesting observations. Examples include Anderson localization , coherent backscattering , random lasing , the effectiveness of time-reversal/phase conjugation , and the existence of perfect-transmission eigenchannels . In recent years, the seminal works of optical phase conjugation  and the single-channel optimization with the scattering media  reminded the research community of the feasibility of applying these concepts to the real practices. In conjunction with the technological advances in wavefront shaping devices and imaging techniques for the wavefront recording, a variety of studies have been followed to deal with more practical problems of interrogating targets located either at the opposite side or within scattering media . The methodologies include feedback-optimization of intensity at a target spot , the use of so-called memory effect to scan the spot [9, 10], the transmission matrix of scattering media [11–13], acoustic guide star in conjunction with optical phase conjugation [14, 15], and spatio-temporal focusing combined with nonlinear excitation .
In this review paper, we will mainly cover the study of the transmission matrix of scattering media and its use for biophotonics applications, which our group has pursued for a few years. The interaction of a scattering medium with light wave is highly complicated due to the multiple light scattering, which makes it extremely difficult to interrogate target objects within the medium. However, when it comes to finding the input-output relation, it is straightforward. If we confine our interest to the linear interaction and also consider that the coherence length of light source is much longer than the transit time broadening of the media, then the scattering medium is a simple linear system with respect to the electric field of the light wave. Then the input-output response of a scattering medium can be described by a so-called transmission matrix, which relates free modes at the input to those at the output. The experimental measurement of transmission matrix is rather time-consuming because it requires both the scanning of incident waves in such a way to cover input free modes and the recording of complex field, i.e. amplitude and phase, at various output channels. However, it is the complete way of describing the response of scattering media to the incident light waves. The first transmission matrix measurement in optics was made by Gigan’s group , where they used the self-interference of two different input modes generated by the same spatial light modulator. About the same time, we reported the measurement of the transmission matrix by the combination of the scanning the angle of incident wave and off-axis interference microscope .
The knowledge of the transmission matrix of a scattering medium provides us with two important opportunities. At first, we can find the eigenchannels of the scattering medium with which we can control the energy transmission through the medium. According to the random matrix theory (RMT) developed in 1980s, a particular incident wave with specific pattern can propagate through a disordered medium in the waveguide geometry without undergoing any energy loss . Mathematically, this special wave is the eigenvector of t+t, where t stands for a transmission matrix and + for a conjugate transpose operation, with maximum eigenvalue, and often called as an open eigenchannel. The underlying physical explanation for the unusual transmittance of the open eigenchannel is that a proper choice of eigenchannel as an incident wave can induce strong constructive interference of scattered waves at the opposite side of the medium. For the scattering medium with slab geometry, open eigenchannels do not exist due to the loss of energy leaking away from the detection area. However, the coupling of light to the eigenchannels with maximum eigenvalues guarantees the maximal energy transport through the medium.
The second opportunity is to make use of a scattering medium as an imaging optics. With the knowledge of the transmission matrix, we can reconstruct original object image from the image distorted by the scattering layer. The multiplication of the matrix inversion to the complex field map of a distorted wave results in an object wave at the upstream of the scattering layer. This is more general approach than the deconvolution process typically used in the conventional imaging system where shift invariance is valid. Therefore, there are unexpected advantages of using a matrix inversion over conventional imaging. It allows us to make use of the optical elements such as multimode optical fibers and fiber bundles as an aberration-free imaging optics. In addition, it is possible in principle to use scattering media made of sub-wavelength particles as a superlens to capture the near-field waves for high-resolution imaging.
In our previous studies, we explored the two opportunities of the transmission matrix approach mentioned above. For the rest of this paper, we will introduce some of them in detail. In section 2, we will cover the numerical/experimental studies for investigating the physical properties of eigenchannels [17–19] and experimental coupling of light to individual transmission eigenchannels for enhancing wave penetration through a scattering layer . In section 3, we will introduce the studies of imaging with scattering media. Our work on the enlargement of numerical aperture with the use of a scattering medium will be covered, with which the enhancement of spatial resolution of an imaging system was demonstrated . We then present aberration-free endoscopic imaging with a multimode optical fiber and a fiber bundle in both reflectance and fluorescence modes of detection [21, 22]. In section 4, we conclude this paper by discussing the future perspective of the transmission matrix approach.
2. Controlling light energy delivery through a scattering medium
2.1 Internal field distribution of transmission eigenchannels
Studies have long been conducted to understand the wave propagation in the disordered media. One of the striking predictions was the existence of particular incident waves, which can transmit through scattering media with near unity transmittance. In 1984, Dorokhov used RMT to calculate the statistical distribution of transmission eigenvalues and predicted the existence of this so-called open eigenchannels . In this section we introduce our numerical study in which we investigated the internal field distribution of various transmission eigenchannels and developed our understanding knowledge on the mechanism of perfect transmission . For this purpose, we have solved electromagnetic wave propagation through a disordered medium using the finite-difference time-domain (FDTD) method, numerically constructed a transmission matrix in an optical regime, and obtained both eigenvalues and their associated eigenchannels. We observed that open eigenchannels enhance the energy stored inside a disordered medium.
For this study, we numerically prepared 2D disordered media in open slab geometry (Fig. 1(a)). To generate a disorder, absorption-free square particles with a side length of 200 nm and a fixed refractive index, np = 2.0, are randomly distributed in the vacuum. For the filling factor of about 50%, the scattering mean free path was calculated to be about 370 nm. The thickness of the slab was set to be lth = 8, 12, and 16 μm. The vertical width of the disordered medium in the x direction is set as 130 μm for the computation, but the output field is recorded at the 90 μm width in the middle (red dashed-dotted line).
We prepare a set of angular plane waves constituting complete set for the width of 90 μm and compute their propagation through a disordered medium. At the steady state, we record electric fields along a line parallel to the x axis, which is located 50 nm behind the medium indicated by a red dashed-dotted line in Fig. 1(a). Recording is done at four successive times with a time interval of 1/(4ck0), one-quarter of optical oscillation. By applying a phase-shifting interferometry algorithm, we extract the amplitude and phase of the transmitted electric field, Eout(x,z = zd). To use the wave vector space for a basis of the transmission matrix, we perform numerical Fourier transform of Eout(x) and obtain the electric field in k space, Eout(kx′). This process is illustrated as an insertion of a virtual lens behind the medium, as shown in Fig. 1(a). Then, we construct a transmission matrix, t(kx′, kx), by the following relation:
We compare the numerically calculated transmission matrix with RMT in their singular values. We perform the singular value decomposition of the transmission matrix, t,
For the disordered media with lth = 8, 12, and 16 μm with the same np = 2.0, we calculate their transmission matrices. For each medium, we plot eigenvalues after arranging them in descending order [solid red curves in Fig. 1(b)]. With the increase of thickness, the area under the curve is reduced and thus the average eigenvalue decreases. Considering that an arbitrary input is a random superposition of eigenchannels, this decrease in average eigenvalue agrees well with the reduction in total transmission. In accordance with RMT, there always exists an eigenchannel whose transmission is unity, which is called an open eigenchannel, regardless of the thickness. This is because the thickness of the sample is much smaller than the width of the medium. We compared these numerical results with RMT. We constructed a transmission matrix following the RMT in which only the transmittance of any given medium determines the statistics of the eigenvalue distribution. The dashed blue curves in Fig. 1(b) show the eigenvalue distributions obtained from RMT. The distributions are in excellent agreement with our FDTD results. This confirms that the open slab geometry of the given disordered media can be modeled by the RMT based on the waveguide geometry at least for the considered media [5, 23].
The FDTD method differs from RMT in that it can provide the field distribution inside a disordered medium. This enables us to explore how eigenchannels propagate through the medium and how much energy they deposit into the medium. As a baseline, we first compute the propagation of a plane wave through a disordered medium whose thickness and index of particles are 16 μm and np = 2.0, respectively. The field amplitude throughout the medium is displayed in Fig. 2(a) when the incident angle is 11.5° with respect to the z axis. Inside the medium, there is a linear decrease in the amplitude of the field as the wave propagates through the medium, as is shown in Fig. 2(d), which plots the intensity averaged along the x axis as a function of the depth (z axis). This agrees well with previous studies [24, 25] conducted in the context of diffusion theory in which the transmission decreases linearly when the thickness of the medium is much larger than the transport mean free path.
For the same disordered medium, we obtain eigenchannels from singular value decomposition of the transmission matrix, and solve their propagation through the medium. Figure 2(b) displays the propagation of an open eigenchannel whose eigenvalue is 0.955. We find that the field strength inside the medium is enhanced such that its average intensity is higher than that of the input [blue curve in Fig. 2(d)]. This is in analogy with a Fabry-Perot cavity in which internal field strength increases at the resonance condition due to the constructive interference. Likewise, the constructive interference enhances the internal energy in open eigenchannels, which leads to strongly enhanced transmission.
In the case of a closed mode whose eigenvalue is close to zero [Fig. 2(c)], a steep decrease of intensity is observed along the z direction as soon as the wave enters the medium. This suggests strong destructive interference as the wave reaches the end of the medium. The intensity profile follows the exponentially decaying curve [red curve in Fig. 2(d)]. Overall, the location of the peak intensity inside the medium shifts from the center to the input side with the decrease of the eigenvalue. The internal energy of the single-channel optimizing mode is close to the open eigenchannel in connection with its enhanced transmission.
The efforts to directly observe open and closed eigenchannels have been under way, and a study shows their existence for the elastic waves . However, their observation in optics has so far been unsuccessful due to the difficulty in preparing a waveguide containing scattering media and also in covering all the optical modes.
2.2. Open eigenchannels in the Anderson localization regime
The interference of the multiple-scattered waves in the highly disordered media leads to interesting phenomena such as Anderson localization [1, 27–29]. Unlike in the diffusion regime where the transmission decreases linearly over the thickness, it decreases exponentially when the Anderson localization takes place. This is due to the dominance of local waves by the constructive interference over the incoherently interfering propagating waves . However, there also exist extended modes of high transmittance that accounts for the most of the average transmittance of the medium [5, 30]. Such extended modes, called necklace states or non-localized modes, were predicted in 1D random media  and were subsequently observed in the experiment . The underlying physics is that the propagation of energy is mediated by the coupling of neighboring localized modes, which results in the extended modes. But the existence of the extended modes in 2D or 3D disordered media and the mechanism of transmittance enhancement, if they exist, are yet to be explored.
Main challenge in the higher dimensional media is that the probability of such modes is too small to find them by chance. Arbitrary incident wave mostly couples to the localized modes under which rare contribution of extended modes is buried. Therefore finding the pure contribution of extended modes is of critical concern. We demonstrated that coupling light to open eigenchannels can be a systematic way of searching for the extended modes. Open eigenchannels in the strongly scattering media may lead to finding extended states of high transmittance. In section 2.1, we introduced the procedure of finding open eigenchannels for a medium in the diffusion regime . But as to be shown in the following, the propagating waves that used to be prevalent in the diffusion regime disappear in the Anderson localization regime and non-propagating waves start to dominate. Therefore, the mechanism of the constructive interference is expected to be dramatically different in forming open eigenchannels.
In this section, we show the internal field distribution of eigenchannels for the 2D disordered medium in the Anderson localization regime. We numerically prepare a highly disordered 2D medium whose localization length is shorter than the thickness of the medium. Using FDTD method, we compute wave propagation through the medium, and observe the localization in the internal energy distribution and the disappearance of propagating waves within the medium. We then construct a transmission matrix, extract eigenchannels and acquire their field distribution inside the medium. We found that the spatial distribution is highly localized in the open eigenchannels within a narrow channel that connects the shortest passage from input to the output planes of the medium. Our study strongly suggests that open eigenchannels induce coupling of non-propagating localized waves, which results in the extended modes of high transmittance.
For the two 16 μm-thick disordered media of np = 1.6 and 2.5, we obtain the spatial intensity map of the electromagnetic wave within the disordered media (Fig. 3(a) and 3(b)). The illumination source is a plane wave normally incident to the input plane. We observe that internal intensity decreases exponentially for np = 2.5 medium while it decreases linearly in np = 1.6 medium. In particular, prominent intensity-enhanced spots exist for np = 2.5 while the intensity is rather uniformly distributed over the space for np = 1.6 medium. These suggest that the np = 2.5 medium be in the strong localization regime while the np = 1.6 medium in the weak localization regime.
Since the FDTD method computes the complex amplitude of the field inside the medium, we can survey the properties of the field as well as the intensity in the Anderson localization regime. To this end, we take the 2D discrete Fourier transform with respect to the spatial coordinates for the field within the medium and obtain an angular spectrum in the spatial frequency. Figures 3(c) and (d) are the field amplitude in k-space for np = 1.6 and 2.5, respectively. The bright ring in the Fig. 3(c) indicates that propagating wave is dominant in the weakly scattering medium. The presence of the ring represents that waves propagate to all different directions. The radius of the ring is measured to be 1.2k0, which is equal to the effective wave vector inside the medium. Hence, it is bigger than k0, the wave number in the free space, because of the high index particles. In the strongly scattering medium (Fig. 3(d)), on the other hand, the ring pattern disappears. This indicates that the propagating waves are dominated by the non-propagating evanescent waves. Spatial Fourier transform of the evanescent waves typically exhibits broadened spectra due to the abrupt decay of the field amplitude. This is the analogue of the waves in the stop band of photonic crystal. This observation goes well with the theory of Anderson localization that the locally interfering waves dominate over the incoherently interfering propagating waves. But our analysis provides direct observation of the disappearance of propagating waves and the prevalence of the local evanescent waves in the Anderson localization regime. This also supports very well that the medium in the strong localization regime is on the mobility edge of photon.
For each singular value, we obtain a corresponding eigenchannel in the form of the coefficients of incident channels. We then solve the propagation of the constructed eigenchannel and record the map of the field within the medium. Figures 4(a)-(e) shows the intensity map inside the strongly scattering medium for the eigenchannels of transmittance corresponding to 80, 12, 2.3, 0.39, and 2.0 × 10-7%, respectively. We find that there exists a striking difference between the Anderson localization regime and the diffusion regime. In the diffusion regime, the field energy of the open eigenchannel is spread over the entire transverse extent . In the Anderson localization regime, however, it is highly concentrated in space such that a dense cluster of localized modes appears within the disordered medium (Fig. 4(a)). Moreover, the clustering is found to exist only for the eigenchannels of high transmittance. We repeatedly observe this appearance of the cluster for other media whose spatial organizations of constituting particles are different but with the same particle size and index, and the fill factor. As the transmittance becomes low, the internal waves gradually spread along the transverse direction (Fig. 4(b)-(e)). For the eigenchannels of sufficiently low transmission, the internal wave covers the entire transverse range. These suggest that the dense cluster be formed in the process of enhancing transmission through a disordered medium where non-propagating localized waves are dominant.
The existence of the dense cluster of internal modes in the Anderson localization regime can be explained as follows. According to Fig. 4(d), most waves exist in the form of non-propagating waves in the strongly scattering medium. In this case, the propagation of energy can mainly be mediated by the neighboring localized modes and by the coupling among them. This can be thought of as a kind of frustrated total internal reflection in which a neighboring pair of prisms induces the coupling of non-propagating evanescent waves and therefore enhances wave propagation. Studies in the 1D media also showed that neighboring of localized modes enhances the transmission [31, 32]
2.3. Enhancing wave transport through disordered media
In this section, we briefly introduce an experimental method to maximize energy transport through disordered media . Although various techniques are developed in the wavefront sensing and recording, the injection of waves into single eigenchannels was still a challenge. Several studies have been made to calculate individual eigenchannels in optics  and in microwave  from the recorded transmission matrix, but the injection of waves into single eigenchannels had not been realized before. Coupling waves into unique eigenchannels is difficult because of two stringent requirements. First, the transmission matrix of the disordered medium must be recorded in the short time before the medium is perturbed. Second, complex wavefronts should be generated accurately corresponding to the eigenchannels of highly complicated wavefronts derived from the measured matrix.
To overcome such difficulties, the experiment has been carried out using the experimental setup shown in Fig. 5. A disordered medium is placed between the input plane (IP) and output plane (OP). As a disordered medium, a 27 μm thick ZnO nanoparticle layer is used. The average transmittance of this medium is measured to be 0.79% at the collection angle corresponding to 0.32 NA. For the purpose of recording a transmission matrix, scanning mirrors (GM) are installed in the sample beam path. The angle, (θξ, θη), of the plane wave incident to the medium was scanned, and transmission image was recorded at each incident angle. Figure 6(b) shows the phase maps of the transmitted waves at representative incident angles. The complex field at the input plane was measured for the same set of incident angles (Fig. 6(a)). This was simply done by measuring the field after removing the disordered medium. From the two sets of complex field maps, a transmission matrix was constructed. The amplitude and phase of the transmission matrix are shown in Figs. 6(c) and 6(d), respectively. If there were no disordered medium, the matrix would be diagonal. However multiple scattering processes cause the incident beam to spread in a disordered medium. Therefore non-zero off-diagonal elements are generated in the matrix.
Singular value decomposition was performed for the measured transmission matrix in order to derive the transmission eigenchannels and their corresponding transmission eigenvalues. An optical wave corresponding to the eigenchannel was experimentally generated by using a spatial light modulator (SLM). The SLM used in this experiment operated in the phase-only mode. In order to generate eigenchannels having complicated distributions in both amplitude and phase, we used a modified method in which multiple SLM pixels act as a single unit at the target wave .
After that, using the wavefront shaping technique, the optical wave of the first transmission eigenchannel was implemented and sent to the medium. Then complex field map at the output plane was recorded (Fig. 7(b)). Figure 7(a) is the transmitted image when the uncontrolled wave (plane wave) was sent to the medium. The enhancement factor, which is defined as the ratio of the transmittance of the first eigenchannel to that of the uncontrolled wave, was 2.8 for this specific sample. When the same experimental procedure was repeated for various samples, the observed transmission enhancement was highly reproducible. The enhancement factor varies somewhat in different samples, typically in the range of 2 to 4.
Various eigenchannels are sequentially generated and their transmittances are observed (red circles in Fig. 8). The transmittance is monotonically decreased with the increase of the eigenchannel index, and the first eigenchannel has the maximum transmittance. The experimentally measured transmittance was compared with the eigenvalues (blue squares in Fig. 8) acquired from the recorded transmission matrix after accounting for the limited accuracy of wavefront shaping. Experimentally generated eigenchannels faithfully follow the prediction from measured transmission matrix. The residual discrepancy aside from the wavefront shaping inaccuracy might come from mechanical perturbation of the sample and imperfect repeatability of the GM scanning. This method was compared with the single-channel optimizing modes  in which intensity was optimized at a single point at the far side of the medium. Although the previous method shows increase in the transmittance, the transmittance (green line in Fig. 8) is well below that of the first eigenchannel. This is because the single-channel optimizing mode couples light into other eigenchannels as well as the eigenchannel with maximum eigenvalue .
3. Imaging through a scattering medium
3.1 Overcoming the diffraction limit with the use of scattering media
Image information transmitted through a disordered medium is scrambled because of multiple light scattering caused by the turbidity. However, if the transmission matrix of the medium is known, then the image distortion can be removed using the relationship deterministically connecting the output image and the transmission matrix. In this respect, a scattering medium can be a unique optical element, not just a hurdle for imaging, which possesses extra benefits that usual optics is devoid of. In the following section, we briefly introduce the basic concept of imaging with a scattering medium.
Let us consider that an input wave EIP(θx, θy) carrying image information in terms of angular spectrum is sent to a scattering medium from the input plane (x, y), which is indicated as an IP in Fig. 5. Then the transmitted wave EOP(ξ, η) at the output plane (ξ, η) is determined asFig. 5. A ZnO nano-particle layer with 15% total transmission is used as a disordered medium. We first take an image with no pattern written on the SLM, which make it serve as a flat mirror. We take 20,000 transmitted images at various incidence angles covering the angular range of illumination corresponding to 0.5 NA. Next, we display a target pattern (Fig. 9(a)) on the SLM and project the pattern to the IP. As expected, the object image is completely scrambled by a ZnO nano-particle layer (Fig. 9(b)). After applying the matrix inversion to the distorted object image, we could obtain the angular spectrum of the target object (Fig. 9(c)) from which we reconstruct object image (Fig. 9(d)). The reconstruction result shows an excellent structural correspondence with the original one. Next, the emblem of Korea University is prepared as an object as shown in Fig. 9(e) and the same procedures are repeated. The reconstructed image in Fig. 9(f) from the distortion shows a fairly good reliability.
Once TLI converts a turbid medium into a scattering lens, it provides a counter-intuitive benefit for optical imaging: enhancement of a spatial resolution for an imaging system. A conventional lens can capture only the light directing into the solid angle extended by the NA of the lens. Therefore, the achievable resolution of the imaging system is determined by the NA in general. If a scattering medium is placed between the object and the imaging system, the light with a propagation angle greater than NA can be redirected into the capture range of the lens by the multiple scattering. Thus if the acquired image through the scattering medium is carefully processed, the spatial resolution of the imaging system can be enhanced.
To demonstrate the benefit from TLI, a test object composed of 1-D stripes is generated on the SLM. We take the image with a high NA objective lens (1.0 NA) in a normal imaging configuration (Fig. 10(a)). With the diffraction limited resolution of 0.77 μm, every single structure up to the finest lines indicated by the red box (spatial period is 2.5 μm at OP) is clearly resolved as shown in Fig. 10(c). Next, the NA of the objective lens is reduced intentionally by decreasing the aperture size in the back focal plane of the objective lens which corresponds to 0.15 NA. Then some of fine structures indicated by red box in Fig. 10(c) disappear due to the lack of resolving power (Fig. 10(d)). Then, a ZnO nano-particle layer as a scattering lens is introduced between the low NA objective lens and the object as shown in Fig. 10(b). A transmission matrix for the scattering lens can be recorded up to 0.85 NA although the low NA of the objective lens because the turbid medium redirects a high-angle light to an adoptable low-angle input. With the help of the scattering lens, the transmission matrix is measured beyond the limit of the low NA objective lens of 0.15 NA. An image of the 1-D object is taken through the fully characterized scattering lens. Although it looks like a speckle pattern shown in Fig. 10(e), it contains information ranging from low- to high-angle. As the previous experiments, TLI retrieves the angular spectrum of the object embedded in the distorted image, and then reconstructs the object image. The result is presented in Fig. 10(f) and the finest lines are clearly as imaged by the high NA objective lens. For the TM recording, the illumination angle is steered from −53° to 53° (0.85 NA) in 5,000 steps along the direction orthogonal to the lines in the object. This means that the retrieved angular spectrum from the image taken by the scattering lens, which is obtained by the low NA objective lens (0.15 NA), has components up to 0.85 NA. As a consequence, the actual detection NA is increased by more than 5 fold and the spatial resolution is enhanced by the same factor. Note that the value of 0.85 NA is not a fundamental limit, which can be further improved by covering a wider angular range.
3.2. Wide-field endoscopic imaging through a single multimode optical fiber
Endoscopy has been playing a crucial role in many areas such as industrial, military and medical fields because it visualizes structures placed at a position where conventional imaging optics cannot reach. Nowadays, a fiber bundle has been widely used and thus became a standard for commercial endoscopes. However, the requirement for a large number of fibers for high-resolution imaging has posed constraints on the diameter of the endoscope, thereby causing considerable limitation on the accessibility of the device. A single multimode optical fiber can be a good alternative to the fiber bundle because its mode density is two or three orders of magnitude higher than the pixel density of the fiber bundle. However, the realization of a single-fiber endoscope has been elusive for several decades. Endoscopy using a single-fiber has a requirement that an object image be taken with a reflection configuration. Two folds distortion (light in and light out) should be addressed for imaging an object in a reflection mode. In this section, we review the recently developed endoscopic imaging, operating in a wide-field reflection mode, with a single multimode optical fiber. This method is called lensless microendoscopy by a single fiber (LMSF).
The setup for LMSF is schematically shown in Fig. 11 (for more details, see ). A He-Ne laser (λ = 633 nm) illuminates an input end (at IP) of a multimode fiber via two beam splitters (BS1 and BS2) and a 2-axis galvanometer scanning mirror (GM). The laser beam couples into the fiber and subsequently propagates toward the sample plane (SP) located at the exit of the fiber to illuminate a target object. Here we use a 1 meter-long multimode optical fiber with 200 μm core diameter, 15 μm clad thickness (230 μm in total diameter) and 0.48 NA. The total number of guiding modes for this fiber is on the order of 105. In our case, we measured a transmission matrix up to 0.22 NA. Therefore, the number of modes that we dealt with was about 20,000. For a test object we used either a USAF target or a biological tissue. The light reflected from the object is collected by the same optical fiber. The collected light is guided backward through the fiber to IP, and then the output reflection image is captured by the camera. The laser beam reflected by BS1 is combined with the beam from the fiber to form an interference image at the camera. Using an off-axis digital holography algorithm, both amplitude and phase of the image from the fiber are retrieved [35, 36].
Figure 12(c) shows typical reflection images of the USAF target recorded by the camera. Two distortion processes scramble the object image. First one is the distortion of illumination light on the way in (IP → SP) and the second one is the mixing of the reflected light (from the object) on the way out (SP → IP). In order to deal with both the problems, TLI and the speckle imaging method [37–39] are applied sequentially.
For applying TLI, a transmission matrix for the multimode fiber, from SP to IP, is measured with 15,000 images up to 0.22 NA while scanning the incident light with the angle . Some representative elements of measured transmission matrix are presented in Fig. 12(a). The transmitted patterns are random speckles confined in the fiber core and the average size of the speckle becomes finer as increasing the incident angle. Because light impinging at a high angle mostly couples to the high order modes of the fiber, which have more complicated spatial distribution.
Once we measure the transmission matrix of the fiber, TLI deals with the distortion SP → IP. The object image (each image in Fig. 12(c)) is reconstructed by TLI. Using Eq. (4), the distortion from SP to IP is reversed. More details about the inversion algorithm can be found in Refs [20, 21]. The recovered images at SP are shown in Fig. 12(d). The images still remain speckled because of the distortion in the illumination light due to the propagation from IP to SP.
The speckle imaging method resolves the distortion of illumination light at SP. When imaging an object, the angle of incident light, , at IP is scanned as shown in Fig. 12. The scanning of the incident angle at IP causes the realization of various speckle illuminations at SP. Different speckle field is generated as a function of the angle . According to the speckle imaging method, a clean object image can be obtained if we average a sufficient number of images recorded at different speckle illuminations . Then the complex speckle patterns are averaged out, leaving a clean object image. To implement this, object images are taken at 500 different , some of which are shown in Fig. 12(c). This is done in 1 second, the actual image acquisition time of LMSF. After applying TLI method to those images, the reconstructed results at SP (each in Fig. 12(d)) contain the object information even though they still look like speckle patterns. By adding all the intensity images in Fig. 12(d), the final clear object image shown in Fig. 12(e) is produced. Therefore the combination of speckle imaging and TLI method enables a single fiber microendoscopic imaging.
LMSF can provide a surveying operation as done by conventional diagnostic endoscopy. Although the change of a transmission matrix involved with a fiber modification prohibits LMSF from supporting a fully flexible endoscopic operation, we found that this operation is attainable up to a partially flexible degree. To demonstrate the endoscopic searching operation for LMSF, we translated the fiber end facing to the sample (Fig. 13(a)). A transmission matrix is measured only at the initial position of the fiber end and 500 distorted object images are acquired at different positions of the sample. Only the initial transmission matrix is repeatedly used for reconstructing the multiple object images. Figure 13(b) represents the composite image of the USAF target by stitching multiple images together. Structures ranging in a wide area, where the fiber end is translated about 800 μm in horizontally, about 1000 μm in vertically, are all clearly visible.
Since LMSF captures complex field maps, , of an object containing both amplitude and phase of reflected light, it can acquire 3-D information of the sample without depth scanning of the fiber end. The wave propagation can be numerically solved to make a focus at a different depth . Figure 14(c) is the result of the numerical refocusing from the image in Fig. 14(b) by 40 μm toward the fiber end. The villus indicated by A that used to be blurred in Fig. 14(b) comes to a focus and its boundary becomes sharp. On the other hand, the villus pointed by B gets blurred due to the defocusing. By comparing Figs. 14(b) and 14(c), we can clearly distinguish relative depths of multiple villi in the view field. This 3-D imaging ability of LMSF will eliminate the need for the physical depth-scanning such that it will dramatically enhance the speed of volumetric imaging.
3.3. Fluorescence endomicroscopy
A drawback of using fiber bundles for endoscopy is an image pixilation artifact. An image delivered through a fiber bundle is highly pixelated because individual fibers act as image pixels. We note that the mode mixing along each fiber in a bundle can be used to eliminate the pixilation artifact. For this purpose, we measured the transmission matrix of the fiber bundle and implemented scanning fluorescence imaging. In the previous studies, fluorescence mode of endoscopic imaging was demonstrated by generating a focused spot at the opposite end of a fiber and scanning the focus to acquire an object image. In this process, a transmission matrix was used for identifying the proper incident wave to form a focus and liquid crystal type SLM (LC-SLM) was used for wavefront control [40, 41]. A digital phase conjugation was also used to calibrate the image distortion by a multimode fiber [42, 43]. In those studies, the speed of an LC-SLM limited the image acquisition rate.
To solve this problem, a digital micromirror device (DMD) was used instead of LC-SLM for the purpose of high-speed wavefront shaping [44, 45]. It was used to generate a focused spot through a scattering medium or a multimode fiber in previous studies. In this section, we review our fluorescence endoscopic imaging method working with a fiber bundle. A transmission matrix for a fiber bundle is measured using a binary control of a DMD. From the transmission matrix, a specific binary pattern that generates a focused spot at an arbitrary position on the far side of the fiber bundle. We experimentally demonstrate near real-time fluorescence imaging through a fiber bundle, which is free from the pixelation artifact.
The setup is depicted in Fig. 15. The core component of the setup is an interferometric microscope equipped with a digital micromirror device (DMD, D4100, Texas Instruments) placed in the sample beam path. A fiber bundle (FIGH-06-300S, Fujikura), which has 6000 ± 600 individual fibers with average diameter of 3.45 μm and the total outer diameter of 300 ± 25 μm, is used as a fluorescence endoscopic device. The average core-to-core distance is about 3.8 μm. We record a complex field image of the wave at the sample plane (SP) located 350 μm away from the fiber bundle exit. The field produced at SP was the mixture of each output field from the individual fiber and then is a fully developed speckle pattern as a consequence. The DMD has a 1024 × 768 micromirror array and is positioned at the conjugate plane to the input plane (IP) of the fiber bundle via the 4-f imaging system composed of a tube lens (TL1) and an objective lens (OL1). We measure a TM for the fiber bundle along the direction of IP → SP. We make a single macropixel using 7 × 7 micromirrors and create M = 10,000 macropixels in total. Using 1/111 demagnification from the DMP plane to IP, the size of an individual macropixel at IP is 860 nm × 860 nm.
In order to maximize the signal-to-noise ratio (SNR) of the detection, the superposed response of the fiber bundle is measured using a DMD pattern composed of randomly chosen M/2 macropixels. The measured output response, O, at SP is expressed by
The binary pattern that generates a focus at SP can be predicted from the measured TM. To generate a focus at position i' at SP, the complex conjugate of a row vector ti’j is implemented as an input wave. By selectively turning on the input segments which have phase values within the range between 0 and π, the required pattern can be realized on the DMD. Figure 16(c) shows the binary pattern on the DMD for generating a focused spot at the center of SP. As shown in Fig. 16(d), a clean focused spot can be seen at the target position. The local intensity enhancement compared to that of the background is 343, which is about half the theoretical prediction . The discrepancy comes from the cross talk among neighboring segments on the DMD which effectively reduces the independent segments. The focus is robust under the displacement of the fiber tip up to 6.5 mm.
Next fluorescence imaging for test objects as well as living cells is demonstrated. We scan the focus along x-y plane at SP by displaying the binary patterns at a frame rate of 22,727 Hz. The fluorescence excited by the generated focus is collected by the same fiber bundle and detected by the photomultiplier tube (PMT). As a test objet, fluorescent beads with 2 μm diameter are used. The sample is placed at SP and a transmission image is taken for a comparison (Fig. 17(a)). As shown in Fig. 17(a), the pixilation artifact deteriorates fine details of the sample. A fluorescence endoscopic image is acquired using our method. The result is presented in Fig. 17(b). The pixelation is removed and the gap where there used to be no information is filled out. Each bead, whose diameter is smaller than the size of the individual fibers, is now clearly distinguishable. For this particular imaging, 70 × 70 points are scanned covering 58 μm × 58 μm area at SP and the image acquisition speed is 4.64 Hz. A stream of fluorescence images taken while translating the fiber shows the capability of real-time imaging of our method.
We also demonstrated fluorescence imaging of living cells. A stained human cell line (SNU-1074) with a fluorescence dye SRfluor 680 Carboxylate (Polyscience) is used as a sample. Figure 17(c) shows a usual transmission image of a cell taken with the fiber bundle. The image taken by our method is presented in Fig. 17(d). It verifies that the image is free from the pixelate artifact and has no loss of information.
Throughout this review, we introduced the physical properties of the transmission matrix of a scattering medium and its applicability to light energy delivery and optical imaging. While these previous studies showed conceptual advances, they are still at the proof-of-concept level and further efforts are necessary for these early studies to be useful in the real practices. For example, the study of enhancing light energy delivery works only for the slab geometry of a scattering medium. For the semi-infinite samples like human body, the concept of eigenchannels described in RMT becomes meaningless in the sense that any incident wave will be totally reflected. In order to deliver light energy deep inside the semi-infinite samples, time-resolved studies can potentially be a useful approach . In the case of endoscopic imaging, the data acquisition speed should be improved and it is necessary for making the transmission matrix stable throughout the measurements. The attempt to use a scattering medium as a far-field superlens should also deal with the measurement of transmission matrix covering the near-field waves. Finally, the transmission matrix approach relates input and output waves and it itself is not applicable to interrogating targets embedded within a scattering medium. In order to investigate targets inside a scattering medium, we will need to solve inverse problem rather than inversion problem. This challenging task is yet to be explored in the future.
This research was supported by the IT R&D Program (R2013080003), the Global Frontier Program (2014M3A6B3063710), IBS-R023-D1-2015-a00, the Basic Science Research Program (2013R1A1A2062560, 2013R1A1A2062808) and the Nano-Material Technology Development Program (2011-0020205) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning. It was also supported by the Korea Health Technology R&D Project (HI14C0748) funded by the Ministry of Health & Welfare, South Korea.
References and links
1. P. W. Anderson, “Absence of diffusion in certain random lattices,” Phys. Rev. 109(5), 1492–1505 (1958). [CrossRef]
2. P. E. Wolf, G. Maret, E. Akkermans, and R. Maynard, “Optical coherent backscattering by random-media - an experimental-study,” J. Phys. (Paris) 49(1), 63–75 (1988). [CrossRef]
5. O. N. Dorokhov, “On the coexistence of localized and extended electronic states in the metallic phase,” Solid State Commun. 51(6), 381–384 (1984). [CrossRef]
8. A. P. Mosk, A. Lagendijk, G. Lerosey, and M. Fink, “Controlling waves in space and time for imaging and focusing in complex media,” Nat. Photonics 6(5), 283–292 (2012). [CrossRef]
10. O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6(8), 549–553 (2012). [CrossRef]
12. Y. Choi, T. D. Yang, C. Fang-Yen, P. Kang, K. J. Lee, R. R. Dasari, M. S. Feld, and W. Choi, “Overcoming the diffraction limit using multiple light scattering in a highly disordered medium,” Phys. Rev. Lett. 107(2), 023902 (2011). [CrossRef] [PubMed]
13. S. M. Popoff, G. Lerosey, R. Carminati, M. Fink, A. C. Boccara, and S. Gigan, “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett. 104(10), 100601 (2010). [CrossRef] [PubMed]
15. B. Judkewitz, Y. M. Wang, R. Horstmeyer, A. Mathy, and C. Yang, “Speckle-scale focusing in the diffusive regime with time-reversal of variance-encoded light (TROVE),” Nat. Photonics 7(4), 300–305 (2013). [CrossRef] [PubMed]
16. O. Katz, E. Small, Y. Bromberg, and Y. Silberberg, “Focusing and compression of ultrashort pulses through scattering media,” Nat. Photonics 5(6), 372–377 (2011). [CrossRef]
17. W. Choi, A. P. Mosk, Q. H. Park, and W. Choi, “Transmission eigenchannels in a disordered medium,” Phys. Rev. B 83(13), 134207 (2011). [CrossRef]
19. M. Kim, W. Choi, C. Yoon, G. H. Kim, and W. Choi, “Relation between transmission eigenchannels and single-channel optimizing modes in a disordered medium,” Opt. Lett. 38(16), 2994–2996 (2013). [CrossRef] [PubMed]
20. M. Kim, Y. Choi, C. Yoon, W. Choi, J. Kim, Q.-H. Park, and W. Choi, “Maximal energy transport through disordered media with the implementation of transmission eigenchannels,” Nat. Photonics 6(9), 581-585 (2012). [CrossRef]
21. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). [CrossRef] [PubMed]
22. D. Kim, J. Moon, M. Kim, T. D. Yang, J. Kim, E. Chung, and W. Choi, “Toward a miniature endomicroscope: pixelation-free and diffraction-limited imaging through a fiber bundle,” Opt. Lett. 39(7), 1921–1924 (2014). [CrossRef] [PubMed]
23. C. W. J. Beenakker, “Random-matrix theory of quantum transport,” Rev. Mod. Phys. 69(3), 731–808 (1997). [CrossRef]
24. J. G. Rivas, R. Sprik, C. M. Soukoulis, K. Busch, and A. Lagendijk, “Optical transmission through strong scattering and highly polydisperse media,” Europhys. Lett. 48(1), 22–28 (1999). [CrossRef]
25. B. Payne, J. Andreasen, H. Cao, and A. Yamilov, “Relation between transmission and energy stored in random media with gain,” Phys. Rev. B 82(10), 104204 (2010). [CrossRef]
26. B. Gérardin, J. Laurent, A. Derode, C. Prada, and A. Aubry, “Full transmission and reflection of waves propagating through a maze of disorder,” Phys. Rev. Lett. 113(17), 173901 (2014). [CrossRef] [PubMed]
27. S. John, “Electromagnetic absorption in a disordered medium near a photon mobility edge,” Phys. Rev. Lett. 53(22), 2169–2172 (1984). [CrossRef]
28. D. S. Wiersma, P. Bartolini, A. Lagendijk, and R. Righini, “Localization of light in a disordered medium,” Nature 390(6661), 671–673 (1997). [CrossRef]
31. J. B. Pendry, “Quasi-extended electron-states in strongly disordered-systems,” J. Phys. C: Solid State Phys. 20(5), 733–742 (1987). [CrossRef]
39. Y. Choi, T. D. Yang, K. J. Lee, and W. Choi, “Full-field and single-shot quantitative phase microscopy using dynamic speckle illumination,” Opt. Lett. 36(13), 2465–2467 (2011). [CrossRef] [PubMed]
42. I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “High-resolution, lensless endoscope based on digital scanning through a multimode optical fiber,” Biomed. Opt. Express 4(2), 260–270 (2013). [CrossRef] [PubMed]
43. I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “Focusing and scanning light through a multimode optical fiber using digital phase conjugation,” Opt. Express 20(10), 10583–10590 (2012). [CrossRef] [PubMed]
44. D. B. Conkey, A. M. Caravaca-Aguirre, and R. Piestun, “High-speed scattering medium characterization with application to focusing light through turbid media,” Opt. Express 20(2), 1733–1740 (2012). [CrossRef] [PubMed]
46. D. J. McCabe, A. Tajalli, D. R. Austin, P. Bondareff, I. A. Walmsley, S. Gigan, and B. Chatel, “Spatio-temporal focusing of an ultrafast pulse through a multiply scattering medium,” Nat. Commun. 2, 447 (2011). [CrossRef] [PubMed]
47. Y. Choi, T. R. Hillman, W. Choi, N. Lue, R. R. Dasari, P. T. C. So, W. Choi, and Z. Yaqoob, “Measurement of the Time-Resolved Reflection Matrix for Enhancing Light Energy Delivery into a Scattering Medium,” Phys. Rev. Lett. 111(24), 243901 (2013). [CrossRef] [PubMed]