The optical memory effect is a well-known type of tilt/tilt wave correlation that is observed in coherent fields, allowing control over scattered light through thin and diffusive materials. Here we show that the optical memory effect is a special case of a more general class of combined shift/tilt correlations occurring in media of arbitrary geometry. We experimentally demonstrate the existence of these correlations, and provide an analytical framework that allows us to predict and understand this class of scattering correlations. This “generalized optical memory effect” can be utilized for maximizing the imaging field-of-view of deep tissue imaging techniques such as phase conjugation and adaptive optics.
© 2017 Optical Society of America
It is challenging to record clear images from deep within biological tissue. As an optical field passes through tissue, its spatial profile becomes randomly perturbed, resulting in a blurry image of the features that lie underneath. Luckily, even highly scattered optical fields still maintain a certain degree of correlation. Such scattering-based correlations have recently enabled new “hidden imaging” approaches [1–4], which reconstruct clear images from behind diffusive materials. These prior investigations have primarily exploited what is traditionally referred to as the optical memory effect [5,6]. This effect predicts that a scattered wavefront will tilt, but otherwise not change, when the beam incident upon a scattering material is also tilted by the same amount [see Fig. 1(a)]. These correlations have been observed through thin isotropically scattering screens  as well as thick forward-scattering tissue .
Recently, we reported a new type of “shift” memory effect, illustrated in Fig. 1(b), that occurs primarily in anisotropically scattering media . This form of correlation is especially important in biomedical imaging, as it offers the ability to physically shift (as opposed to tilt) a focal spot formed deep within scattering tissue by translating an incident optical beam. However, neither of the two reported types of correlation could fully explain the observations seen in thick biological tissue .
Here, we show how the optical “tilt” and “shift” memory effects are nontrivially intertwined. In fact, the two effects are manifestations of one and the same general source of correlation within a scattering process, which depends upon how an incident wavefront is both tilted and shifted [see Fig. 1(c)]. Our new “generalized memory effect” model offers a complete description of these combined shift/tilt correlations present within scattering media. With our model, we are able to explain the unexpectedly large optical memory effect observed within biological tissue, estimate the size of the isoplanatic patch in microscopy, and optimize the design of adaptive optics microscopes . Additionally, we develop and verify a Fokker–Planck light propagation model that predicts an optimal imaging/scanning strategy for a given forward-scattering sample based on its scattering response. While our model applies to coherent waves in general, we limit our attention to the optical regime.
We start out by presenting our model for the generalized memory effect. We describe how to predict the amount of expected correlation through a given slab of scattering material as function of both position and wave vector. Then, we discuss the direct applications of our findings to adaptive optics before presenting experimental memory effect measurements.
In our model, we consider propagation of monochromatic and coherent light from an “input” plane to a “target” plane . We limit ourselves to scalar waves. The forward-propagating field at the “input” surface, , and at the “target” plane, , are coupled through
It should be noted here that our model is completely general and does not make any assumptions about the location or orientation of the two planes, nor about the symmetry, shape, or other properties of the scattering sample. For example, for transmission through a slab of scattering material, the target plane is located at the back surface of the sample, and is the sample’s transmission matrix. Conversely, in a biomedical microscope setup, the target plane is typically located inside a scattering sample, like tissue, and is a field propagator connecting position to a position located inside the tissue.
At this point let us define exactly what type of correlations we are interested in. Suppose that an input field results in a target field . We hypothesize that when we shift/tilt the incident field with respect to the medium, the new target field will also experience a shift/tilt and also remain similar to the original target field. This similarity can be expressed as a correlation function involving the original matrix , and a new matrix associated with the shift/tilted field. We describe “tilting” the incident wave as a multiplication with a phase ramp, , resulting in a wavefront that is tilted by with respect to plane . Likewise, we describe the corresponding tilt at the target plane via a multiplication with .
We may now describe the new shifted/tilted situation by writing the corresponding shift/tilt matrix as , where is shifted by and at the input and target plane, respectively. By calculating the ensemble averaged value , we can find the corresponding shift/tilt correlation coefficient. However, we found that the problem is expressed more naturally if we first introduce the center-difference coordinates , (and the same for ). Using the same reasoning as above, we now find a symmetric expression for our generalized shift/tilt correlation function,2) reduces to the optical “tilt” memory effect , whereas for and , the correlation function corresponds to the anisotropic “shift” memory effect described in Ref. . Later we will show that the generalized correlation function is not simply a trivial combination of the separate shift and tilt memory effects.
Before proceeding to calculate in terms of the sample properties, we introduce the Wigner distribution function (WDF), which describes the optical field as distribution in a joint phase space of two Fourier-conjugate variables [11,12]. In our case, the variables of interest are space () and wave vector ():
We choose to work with the WDF because it will allow us to convert Eq. (1) into a function of both space and wave vector, which we may easily connect to our new correlation function of similar variables. In the paraxial approximation, the Wigner distribution function is effectively equivalent to the light field , which describes the amount of optical power at point that is propagating in direction , like a spatio-angular plot of light rays at various locations propagating in different directions.
To describe the scattering of incident light over space and wave vector, we introduce the “light field transmission function,” . This function maps the incident light field to the transmitted light field at the target plane:4) is the phase-space equivalent of Eq. (1). In Supplement 1A, we show that takes the form of a double-Wigner distribution of the transmission matrix, 11]. Although is a function of four variables, it only depends upon the scattering system’s two-variable transmission matrix, , and obeys the same properties as a WDF (e.g., realness).2) to arrive at our central result, 7) is the formulation of our new generalized memory effect, generalizing the well-known “tilt” memory effect into a full class of interrelated shift and tilt correlations. If were to be separable in and coordinates, Eq. (7) would reduce to a simple multiplication of the “tilt” and “shift” memory effects. However, as we will show below, is generally not separable, and the interaction between shift and tilt effects is nontrivial. At this point, we want to emphasize that Eq. (7) is still valid for any scattering medium or geometry. Also, these spatio-angular correlations are an intrinsic property of the scattering medium, and will thus be present regardless of the form of the input field.
3. APPROXIMATE SOLUTION FOR FORWARD SCATTERING
We are now left with the challenge of calculating . Although we could use the radiative transfer equation in the most general case, we prefer an approximate solution that gives a simple analytical form for under the condition that light is mainly scattered in the forward direction.
The model is based on the notion that the WDF’s joint description of light across and is very closely related to the light field description of rays . Similar to the approach presented in Ref. , light propagation in the sample is modeled as a series of scattering events, where each scattering event slightly changes the propagation direction of the rays while maintaining the position. Between the scattering events, light propagates along straight rays and maintains its directionality.
In the continuous limit of infinitely small steps between the scattering events, this picture translates to a Fokker–Planck equation with the following solution (full derivation in Supplement 1B):
Note that is only a function of two variables. The reduction of and to a single difference coordinate is possible because the Fokker–Planck equation is shift invariant. A similar simplification was used in the original derivation of the “tilt” optical memory effect . By assuming the average scattered intensity envelope only depended upon relative position, the resulting memory effect correlation reduced to a function of only one tilt variable, which is now a commonly applied simplification in many experiments [1–4]. Note that in this paraxial model the target intensity distribution is additionally offset by , which is exactly what is expected from pure ballistic propagation through a transparent medium of thickness . Moreover, the Fokker–Planck model is also invariant to a tilt in the incident wave. This symmetry allows for the reduction of coordinates and to . Of course, this approximation neglects the fact that rays at a high incident angle propagate a larger distance inside the sample.10), shift and tilt correlations along the target plane are not independent, but show a combined effect, represented by the cross term. The two delta functions in Eq. (9) are a direct result of the shift and tilt invariance of the Fokker–Planck model. As we show in Supplement 1C, in an actual experiment the delta functions will be replaced by the ambiguity function of the incident field, which is ideally a well-behaved sharp function.
4. MAXIMIZING THE ISOPLANATIC PATCH
We now examine the application of the generalized memory effect to adaptive optics (AO) systems. AO allows light to be focused inside scattering media by correcting for the induced wavefront distortions by means of a deformable mirror or spatial light modulator. Generally, AO systems are limited to a single plane of wavefront correction. However, a single correction plane cannot correct for a full scattering volume, and therefore, the focus scan range is limited to a small area termed the isoplanatic patch. A central question in AO is where to conjugate the correction plane to maximize the isoplanatic patch . In the case of conjugate AO, this correction plane is located at the sample’s top (input) surface, whereas in pupil AO, it is effectively located at an infinite distance from the sample. In Fig. 2, we diagram how conjugate AO and pupil AO are analogous to experiments that utilize the “tilt” [Fig. 2(a)] and “shift” memory effects [Fig. 2(b)], respectively.
To maximize the isoplanatic patch, one should simultaneously tilt and shift the incident field to maximize for a desired shift distance , while using as a free parameter. From Eq. (10), we find that the optimal scan range is achieved when9), we can substitute and in Eq. (11) to find the optimal tilt/shift combination at the input plane for a given amount of target shift : 2(c).
In Table 1, we compare the isoplanatism provided by the three different memory effects. Here, we define isoplanatism as the maximum allowed scan distance at the target plane that maintains correlation, which we solve for with appropriately defined input variables in Eq. (10). Interestingly, all memory effects decrease at the same rate of as the target plane is placed deeper inside the medium. However, the scan range of the tilt/tilt memory effect is always a factor of larger than the shift/shift memory effect. By exploiting the optimal tilt/shift combination, as given in Eq. (12), the scanning improvement is increased to a factor of 2.
For some applications, such as imaging a structure hidden far behind a forward-scattering material [1–3,15], one may be interested in maximizing the angular memory effect instead. In this case, the goal is to maximize for a desired tilt while using as a free parameter. For this scenario, we find a maximum scan range of for . This translates to a memory effect angle of . Of course, these results are based on the Fokker–Planck model, which is only valid in the forward-scattering regime . In this regime, however, the memory effect angle is significantly larger than the angle of that follows from the tilt/tilt memory effect  alone. This enhanced angular memory effect is in line with recent observations in forward-scattering tissue .
5. EXPERIMENTAL VALIDATION
We now measure and for a forward-scattering medium in two separate experiments. These experiments will verify the relation predicted in Eq. (7) and the Fokker–Planck model in Eq. (8). For simplicity, we consider and only along the horizontal dimension, with and as the horizontal components of the position and the wave vector at the sample back surface, respectively. We created our scattering samples using uniform diameter silica microspheres immersed in 1% agarose gel with a concentration of spheres per . The refractive index of the microspheres and the agarose gel are and , respectively. Using Mie theory, we find an anisotropy factor of , and a scattering mean free path of at a wavelength of 632.8 nm. This results in a transport mean free path of . We then cast this scattering mixture into layers of two thicknesses: and .
The experimental setup is illustrated in Fig. 3. Light from a 632.8 nm He–Ne laser is expanded and split into a reference path and the sample path. Light is focused onto the scattering sample using a microscope objective (Zeiss A-Plan ), and we image the sample back surface (target plane) with a charge-coupled device (AVT Stingray F-145) through a second identical objective and tube lens (). The phase of light transmitted through the sample is determined by means of off-axis holography. The sample is placed on a translation stage for lateral movement. Additionally, a pinhole or diffuser is placed directly in front of the first microscope objective to either create a pencil beam or a random speckle pattern as the input field.
A. Measurements of the Light Field Transmission Function
From Eq. (4), we know that a finite input beam will result in a WDF at the target plane, , that is a convolution between the desired light field transmission function and the WDF of the input field, . We are thus able to determine by using a pencil beam for sample illumination () at and , which approximately forms the input WDF . is then found by measuring the scattered light at the target plane across both space and angle. The pencil beam is formed by placing a 500 μm wide pinhole close to the back aperture of the first objective. After measuring the scattered field, we numerically calculate the WDF using Eq. (3) to find the scattered intensity as a function of both and . We average the WDF over 300 measurements, translating the sample over a distance of 10 μm in between each measurement to obtain . To facilitate comparison of our measurements with the Fokker–Planck model, which does not include ballistic light, we chose to remove the contribution of ballistic light by subtracting the average transmitted field from every measured field before calculating its WDF. Following prior work, it is possible to modify our model to also include ballistic light at the expense of some added complexity [16,17].
In Fig. 4, we compare our measured light field transmission functions, , to those computed with the Fokker–Plank model () in Eq. (8). The effect of the cross term between and , as predicted in , is clearly visible in our measurements from the fact that the distributions are sheared. This shear implies that the light at the edges of the diffuse spot (large ) continues to propagate, on average, in a radially outward manner (large ) after scattering. The measurements are less spread out than the Fokker–Planck model, which might be a result of the limited optical sensitivity at the edges of the objective lenses. Both a large field of view and a large numerical aperture of the microscope objective are required to measure the full extent of the light field transmission function.
B. Measurements of the Generalized Correlation Function
Next, we experimentally measure the generalized correlation function . We will directly compare these measurements to our correlation model for in Eq. (10). Additionally, we will verify our main result in Eq. (7) by comparing the measurements to , the 2D Fourier transform of the light field transmission function measurements. For the correlation measurements, we replace the pinhole in front of the first objective with a diffuser that forms a randomized input field (average speckle ) at the sample surface. We use a diffuser to minimize correlations within the input field, which manifest themselves as a convolution in our measurement of (see Supplement 1C). We tilt the random input field by translating the diffuser at the objective back aperture. We record a total of 625 scattered fields by illuminating the sample at 25 unique spatial locations and under 25 unique angles of incidence. From this data cube, we compute by taking the ensemble average of all the absolute values of the correlation coefficients between two fields separated by the same amount of shift and tilt . Finally, we normalize after subtracting the correlation value at the maximum shift, corresponding with the correlations in the ballistic light.
Figure 5 presents the results of our experiments. For the two different sample thicknesses of and , the measured 2D correlation functions in (a,d) are compared to , the Fourier transform of in (b,e), and the Fokker–Planck model in (c,f). The dashed line indicates the optimal tilt/scan condition as predicted in Eq. (11). We also show cross sections through (red stars), (blue stars), and (dashed black) on the right. These cross sections are given along the line (g,j), the line (h,k), and the optimal target plane scan condition (i,l). These last two plots demonstrate how jointly considered tilts and shifts can increase correlations along to increase the scan range (i.e., isoplanatic patch) at the target plane. For instance, for the sample, the standard deviation increases from 1.30 to 2.66 μm when comparing the correlation function along and , which corresponds to the predicted doubling of the scan range. Even though the Fokker–Planck model is a simplified paraxial description that neglects backscattering and interference, it provides an accurate prediction of the measured correlations. The cross sections for and are also in good agreement, except for Fig. 5(k). This data corresponds to the thickest sample, for which the limited field of view of the objective lens prevented us from fully measuring along . The truncation of , already observed in Fig. 4(c), results in a broadening of its Fourier transform .
The optical memory effect as reported in Ref.  has paved the way for several new imaging techniques that can “see through” thin scattering layers [1–4]. These techniques require the object of interest to be positioned at a distance behind the thin layer, and are thus not immediately applicable to situations in biomedical imaging where the object of interest is embedded in scattering tissue. The anisotropic memory effect  showed that translational spatial correlations also exists within the scattering sample, and they have been demonstrated to be suitable for imaging or focus scanning inside biological tissue [15,18]. Our new generalized memory effect model offers a new theoretical framework that encompasses the two known memory effects as special cases. Moreover, our model offers a means to optimize the field of view of adaptive optics and hidden imaging approaches.
Using the Fokker–Planck light propagation model, we have shown that the “tilt” memory effect is not only present behind, but also inside scattering layers, proving to be a factor more effective as a scanning technique than the “shift” memory effect alone. This finding supports the field-of-view (FOV) advantage of conjugate AO over pupil AO, discussed in Ref. , as the “tilt” and “shift” memory effects are utilized by conjugate AO and pupil AO, respectively. Our optimal joint tilt/shift scheme, which corresponds to an optimal AO conjugation tilt plane, maximizes the corrected FOV beyond what is predicted independently by only the “tilt” memory effect and only the “shift” memory effect. The scan ranges given in Table 1 are strictly the correlations that result from the transmission matrix. Correlations in the input beam may further extend the scan range of the different memory effects (see Supplement 1C). The Fokker–Planck model assumes continuous scattering throughout the sample and neglects backscattered light, an assumption that may not hold in strong isotropic scattering media. However, in this case the Fourier relation between and is still valid. The generalized correlation function is the phase-space equivalent of the intensity correlations introduced by Feng et al. . We envisage that there also exist phase-space equivalents of and correlations. These higher-order correlations have recently been exploited to focus light through scattering media with an unexpectedly high efficiency , and investigating their phase-space equivalents may prove equally useful.
Concluding, our new generalized memory effect model generalizes the known optical memory effects without making any assumptions about the geometry or scattering properties of the scattering system. The predicted Fourier relation between the light field transmission function and generalized correlation function has been experimentally verified in forward-scattering media. Furthermore, we found that the simple Fokker–Planck model for light propagation is surprisingly accurate in describing the full set of first-order spatial correlations inside forward-scattering media. Our new generalized memory effect model predicts the maximum distance that a scattered field can be scanned while remaining correlated to its unshifted form. In other words, the generalized memory effect provides the optimal scan mechanism for deep-tissue focusing techniques.
H2020 European Research Council (ERC) (ERC-2016-StG-714560, ERC-2016-StG-678919); Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO) (14879); Einstein Foundation Berlin; Deutsche Forschungsgemeinschaft (DFG) (EXC 257 NeuroCure); Human Frontier Science Program (HFSP) (RGP0027/2016); Krupp Foundation.
See Supplement 1 for supporting content.
1. J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012). [CrossRef]
2. O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012). [CrossRef]
3. O. Katz, P. Heidmann, M. Fink, and S. Gigan, “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photonics 8, 784–790 (2014). [CrossRef]
4. X. Yang, Y. Pu, and D. Psaltis, “Imaging blood cells through scattering biological tissue using speckle scanning microscopy,” Opt. Express 22, 3405–3413 (2014). [CrossRef]
5. I. Freund, M. Rosenbluh, and S. Feng, “Memory effects in propagation of optical waves through disordered media,” Phys. Rev. Lett. 61, 2328–2331 (1988). [CrossRef]
6. S. Feng, C. Kane, P. A. Lee, and A. D. Stone, “Correlations and fluctuations of coherent wave transmission through disordered media,” Phys. Rev. Lett. 61, 834–837 (1988). [CrossRef]
7. J. H. Li and A. Z. Genack, “Correlation in laser speckle,” Phys. Rev. E 49, 4530–4533 (1994). [CrossRef]
8. S. Schott, J. Bertolotti, J.-F. Léger, L. Bourdieu, and S. Gigan, “Characterization of the angular memory effect of scattered light in biological tissues,” Opt. Express 23, 13505–13516 (2015). [CrossRef]
9. B. Judkewitz, R. Horstmeyer, I. M. Vellekoop, I. N. Papadopoulos, and C. Yang, “Translation correlations in anisotropically scattering media,” Nat. Phys. 11, 684–689 (2015). [CrossRef]
10. J. Mertz, H. Paudel, and T. G. Bifano, “Field of view advantage of conjugate adaptive optics in microscopy applications,” Appl. Opt. 54, 3498–3506 (2015). [CrossRef]
11. M. J. Bastiaans, “Wigner distribution function and its application to first-order optics,” J. Opt. Soc. Am. 69, 1710–1716 (1979). [CrossRef]
12. M. Testorf, B. Hennelly, and J. Ojeda-Castaneda, Phase-Space Optics: Fundamentals and Applications (McGraw-Hill Education, 2009).
13. Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009), pp. 1–10.
14. H.-Y. Liu, E. Jonas, L. Tian, J. Zhong, B. Recht, and L. Waller, “3D imaging in volumetric scattering media using phase-space measurements,” Opt. Express 23, 14461–14471 (2015). [CrossRef]
15. J.-H. Park, W. Sun, and M. Cui, “High-resolution in vivo imaging of mouse brain through the intact skull,” Proc. Natl. Acad. Sci. USA 112, 9236–9241 (2015). [CrossRef]
16. A. Wax and J. E. Thomas, “Measurement of smoothed Wigner phase-space distributions for small-angle scattering in a turbid medium,” J. Opt. Soc. Am. A 15, 1896–1908 (1998). [CrossRef]
17. C.-C. Cheng and M. Raymer, “Propagation of transverse optical coherence in random multiple-scattering media,” Phys. Rev. A 62, 023811 (2000). [CrossRef]
18. I. N. Papadopoulos, J.-S. Jouhanneau, J. F. A. Poulet, and B. Judkewitz, “Scattering compensation by focus scanning holographic aberration probing (F-SHARP),” Nat. Photonics 11, 116–123 (2017). [CrossRef]
19. C. W. Hsu, S. F. Liew, A. Goetschy, H. Cao, and A. D. Stone, “Correlation-enhanced control of wave focusing in disordered media,” Nat. Phys. 13, 497–502 (2017). [CrossRef]