Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Long-distance pattern projection through an unfixed multimode fiber with natural evolution strategy-based wavefront shaping

Open Access Open Access

Abstract

Focusing light into an arbitrary pattern through complex media is desired in energy delivery-related scenarios and has been demonstrated feasible with the assistance of wavefront shaping. However, it still encounters challenges in terms of pattern fidelity and focusing contrast, especially in a noisy and perturbed environment. In this work, we show that the strategy relying on natural gradient ascent-based parameter optimization can help to resist noise and disturbance, enabling rapid wavefront optimization towards high-quality pattern projection through complex media. It is revealed that faster convergence and better robustness can be achieved compared with existing phase control algorithms. Meanwhile, a new fitness function based on cosine similarity is adopted for the algorithm, leading to higher focusing contrast without sacrificing similarity to the target pattern. As a result, long-distance projection of an arbitrary pattern can be accomplished with considerably enhanced performance through a 15-meter multimode fiber that is not fixed and susceptible to perturbation. With further engineering, the approach may find special interests for many biomedical applications, such as deep-tissue photon therapy and optogenetics, where free-space localized optical delivery encounters challenges.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Complex media like ground glass or multimode fiber (MMF) are opaque to light, as light experiences multiple scattering or mode dispersion during its propagation within the media [1,2]. The amplitude and phase of light are scrambled with only an optical speckle being formed behind the media. The advent of wavefront shaping (WFS) [35] has rendered the possibility to focus light through or within complex media by actively controlling the incident wavefront with a spatial light modulator (SLM), which brings great potential for deep-tissue optical focusing and imaging [2,68]. WFS usually requires feedback from the output field to guide the optimization of input phase and/or amplitude [9,10] through approaches that can be categorized into transmission matrix (TM) [5,11] and iterative optimization [3,9,1216]. Compared with typical single-spot focusing, focusing light into an arbitrary pattern is often favored in energy delivery-related biomedical applications, yet more demanding to achieve satisfactory pattern fidelity and focusing contrast. Noisy and complicated conditions like medium instability or environment perturbation may pose further challenges [17].

To overcome these obstacles, many TM [1820] and iterative [2125] methods have been explored. For the former, due to the interference effect in multi-spot focusing [18], additional optimization of the SLM hologram is still required after measuring the TM. In comparison, iterative WFS does not involve off-axis holographic setup and/or large measurements that are typically required for TM calibration, and the whole procedure is more straightforward and robust [26]. Most of recent efforts in multi-spot focusing and patterned projection through fixed complex media have utilized intelligent optimization algorithms such as genetic algorithm (GA) and its variants [2124], or improved ant colony algorithm (IACO) [27]. The feedback signal also extends from denoting localized intensity to considering both the intensity and uniformity of the target region [21,22]. However, these algorithms rely more or less on random variations of the population, which might be easily affected by external interference, and hence restrict the optimization speed and efficiency. With a noisy environment or an instable medium like MMF, which is ideal for minimally invasive deep-tissue applications [28], current iterative methods may not be able to adapt to fiber deformations in a timely manner, which has limited wide applications of the technique.

Different from the above-mentioned algorithms, natural evolution strategy (NES) builds upon parameter optimization via natural gradient ascent [29], and it has demonstrated excellent anti-interference ability, which is essential for iterative WFS [30,31] in perturbed environment. In this work, we propose to use separable NES (SNES) [29,32] to accelerate the global search for optimum wavefront and apply it for complex pattern projection through a long multimode fiber that is loosely placed on optical table. We first introduce the working principle of SNES, and then numerically compared the performance of various representative algorithms under different noise and decorrelation conditions. The simulation reveals superior performance of SNES on robustness and convergence speed over other state-of-the-art phase control algorithms. Moreover, vector cosine similarity (VecCos) is adopted as a new fitness function to replace the commonly used Pearson correlation coefficient (PCC), which leads to higher focusing contrast without sacrificing similarity to the target pattern. Experimentally, long-distance projection of arbitrary patterns through a 15-meter long unstable MMF is demonstrated using different algorithms, confirming the simulation observations. SNES optimized with VecCos, in particular, achieves encouraging performances, such as 60% higher contrast than that optimized with PCC, which is promising for WFS applications involving complex pattern focusing or projection in perturbed environment.

2. Methods

2.1 Principle of SNES

The NES family is a class of evolutionary algorithms that iteratively update a search distribution by adapting the distribution parameters along the natural gradient of expected fitness [29]. In the scheme of SNES, samples are drawn from the standard multivariate Gaussian distribution ${{\boldsymbol s}_k}\; \sim \mathrm{\;\ {\cal N}}({\mathbf 0,\mathrm{\;\ \mathbb{I}}} )$ with a diagonal covariance matrix. The distribution parameters $\theta = ({{\boldsymbol \mu },{ \;\boldsymbol\sigma }} )$ where ${\boldsymbol \mu } \in {\mathrm{\mathbb{R}}^N}{\boldsymbol \; }$ and ${\boldsymbol \sigma } \in \mathrm{\mathbb{R}}_ + ^N{\boldsymbol \; \; }$ denote the vectors of the mean and standard deviation respectively, map a population of ${N_p}$ samples into the search distribution (i.e., input wavefronts) such that ${{\boldsymbol z}_{\boldsymbol k}} = \textrm{}{\boldsymbol \sigma } \cdot {{\boldsymbol s}_{\boldsymbol k}} + {\boldsymbol \mu \; }({k = 1,\; \cdots {N_p}} )$. Suppose we have ${{\boldsymbol z}_{\boldsymbol k}}\; \sim \textrm{}\pi ({\boldsymbol z}|\theta )$, which is evaluated by a fitness function f, the expected fitness of the search distribution is

$$\begin{array}{{c}} {J(\theta )= \; {\mathrm{\mathbb{E}}_\mathrm{\theta }}[{f({\boldsymbol z} )} ]= \smallint f({\boldsymbol z} )\pi ({{\boldsymbol z}\textrm{|}\theta } )d\boldsymbol{z}} \end{array}$$

To maximize $J(\theta )$ with gradient ascent, the plain gradient regarding the parameters is derived

$$\begin{array}{{c}} {{\nabla _\theta }J(\theta )= {\nabla _\theta }\smallint f({\boldsymbol z} )\pi ({{\boldsymbol z}\textrm{|}\theta } )d\boldsymbol{z} = {\mathrm{\mathbb{E}}_\mathrm{\theta }}[{f({\boldsymbol z} ){\nabla_\theta }\log \pi ({{\boldsymbol z}\textrm{|}\theta } )} ]} \end{array}$$

According to Ref. [29], plain gradient ascent tends to be unstable with either oscillation or premature convergence. Natural gradient is thus put forward by multiplying the plain gradient with the inverse of Fisher information matrix ${\mathbf F} = \; {\mathrm{\mathbb{E}}_\mathrm{\theta }}[{{\nabla_\theta }\textrm{log}\pi ({{\boldsymbol z}\textrm{|}\theta } ){\nabla_\theta }\textrm{log}\pi {{({{\boldsymbol z}\textrm{|}\theta } )}^T}} ]$,

$$\begin{array}{{c}} {{{\tilde{\nabla }}_\theta }J(\theta )= {{\mathbf F}^{ - 1}}{\nabla _\theta }J(\theta )} \end{array}$$

Note that the natural gradient can be computed in a discrete manner by estimation from the search distribution ${{\boldsymbol z}_{\boldsymbol k}}\; ({k = 1,\; \cdots {N_p}} )$. The general formation for updating the search distribution via natural gradient ascent is thus given by

$$\begin{array}{{c}} {\theta \leftarrow \theta + \eta {{\tilde{\nabla }}_\theta }J(\theta )= \; \theta + \eta {{\mathbf F}^{ - 1}}{\nabla _\theta }J(\theta ),} \end{array}$$
where $\eta $ is the learning rate of parameter $\theta $. The above derivations show the general principle of NES that employs global search and natural gradient ascent. With regards to its variant SNES, several crucial modifications have been adopted to improve the performance and robustness [29]. These include fitness shaping that replaces the rank-based fitness with a set of ordered utility values ${w_k}({k = 1,\; \cdots {N_p}} )$ used for more robust gradient computation, the introduction of a local “natural” coordinate system that simplifies the Fisher matrix to an identity matrix and so on. In iterative WFS, the way to use SNES for wavefront optimization through updating the search parameters ${\boldsymbol \mu \; }$ and ${\boldsymbol \sigma }$ is illustrated in Fig. 1.

 figure: Fig. 1.

Fig. 1. Framework of SNES in iterative WFS optimization for pattern projection through complex media.

Download Full Size | PDF

2.2 Comparison of different optimization algorithms

To compare the performance on arbitrary pattern projection through complex media, SNES and representative optimization algorithms including particle swam optimization (PSO) [33], GA [12,21], and IACO [27], are studied through both simulation and experiment. For every algorithm, we control the population size of each generation to be 18, the size of controllable input wavefront to be $64 \times 64$, and the number of iterations to be 2000 to ensure convergence. The wavefront control for pattern projection is performed using phase-only modulation.

There are also specific parameters for different algorithms, for which we adopt typical settings to guarantee the best states of each algorithm, for the sake of a fair comparison. For example, for PSO, both the individual (${c_1}$) and group (${c_2}$) learning factors are 1.5. The inertia weight decreases linearly from the initial maximum ${w_{max}} = 0.2$ to the final minimum ${w_{min}} = 0.001$. For GA, the mutation rate decays exponentially from ${R_0} = 0.01$ to ${R_{end}} = 0.001$ with the decay factor $\lambda = 250$, and the offspring is half of the population size. Regarding IACO, it was recently proposed to achieve rapid convergence and strong anti-noise ability with new selection and pheromone updating rules [27]. We adopt the same parameters as the reported sets where the controlled ($0\sim 2\pi $) phase is sampled with 20 points, the probability threshold is $0.9992$, and the size of offspring is the same as the population. When it comes to SNES, the learning rates for search parameters ${\eta _\mu }$ and ${\eta _\sigma }$ as well as the set of utility values ${w_k}$ all follow the suggested settings [29,32]. The detailed parameters are summarized in Table 1.

Tables Icon

Table 1. Parameters of different optimization algorithms for pattern projection

To mimic the measurement noise encountered in experiment, such as shot noise, dark electricity noise of camera and read-out noise etc., we manually apply a multiplicative noise to the output light intensity ${\boldsymbol I} \in \mathrm{\mathbb{R}}_ + ^M$ during the simulation, described by

$$\begin{array}{{c}} {{{\boldsymbol I}_{noise}} = \boldsymbol{I} + \alpha \boldsymbol{I} \cdot \boldsymbol{\epsilon} .} \end{array}$$

Here, $\mathrm{\boldsymbol{\epsilon}}$ is a vector with the same size as ${\boldsymbol I}$, which is randomly drawn from a uniform distribution $U({ - 0.5,\; 0.5} ),$ and $\alpha > 0$ denotes the noise level.

In realistic conditions, the complex medium may be unstable, and its TM may fluctuate over time. According to the model in Ref. [34], a small perturbation $\xi $ drawn from a complex Gaussian distribution $\mathrm{{\cal N}}({0,\textrm{}\delta } )$ can be added to each of the TM elements ${\textrm{t}_{\textrm{mn}}}$ for simulation,

$$\begin{array}{{c}} {{\textrm{t}_{\textrm{mn}}} \to \frac{1}{{\sqrt {1 + {\delta ^2}} }}({{t_{mn}} + \xi } ).} \end{array}$$

Note that $\delta $ is relative to the standard deviation of the TM, which denotes the perturbation level. If we let ${T_p}$ be the persistence time of the complex medium when its TM remains unchanged and ${T_i}$ be the duration of one iteration, the perturbation level can be predicted by $\delta = 1/\sqrt {{T_p}/{T_i}} $ [29], where ${T_p}/{T_i}$ is denoted as relative persistence time.

2.3 Performance evaluation metrics and fitness function

For pattern projection through complex media, the correlation between the input and output intensity distributions, i.e., PCC, is an important metric to evaluate the pattern fidelity. Additionally, in the output field, the ratio between the mean light intensities in the target region and that of the non-target region, i.e., focusing contrast, constitutes the other key metric of performance.

Mathematically,

$$\begin{array}{{c}} {PCC = \; \frac{{\mathop \sum \nolimits_i^M \; ({{{\boldsymbol I}_i} - \bar{{\boldsymbol I}}} )\; ({{\boldsymbol I}_i^t - \overline {{{\boldsymbol I}^t}} } )}}{{\sqrt {\mathop \sum \nolimits_i^M {{({{{\boldsymbol I}_i} - \bar{{\boldsymbol I}}} )}^2}} \sqrt {\mathop \sum \nolimits_i^M {{({{\boldsymbol I}_i^t - \overline {{{\boldsymbol I}^t}} } )}^2}} }},} \end{array}$$
where ${{\boldsymbol I}_i}$ and ${\boldsymbol I}_i^t$ represent the light intensity of the output and the target pattern at position index i, and $\bar{{\boldsymbol I}}$ and $\overline {{{\boldsymbol I}^t}} $ are the mean of the output and target pattern intensity, respectively.
$$\begin{array}{{c}} {Contrast = \mu ({{{\boldsymbol I}_i}} ){|_{i \in \textrm{T}}}\; /\; \mu ({{{\boldsymbol I}_i}} ){|_{i \notin \textrm{T}}}\; ,} \end{array}$$
where $\mu ({\cdot} )$ represents the mean and $\textrm{T}$ represents the target region of the output field.

Also note that previous research usually adopted PCC as the fitness function to evaluate the performance [22,25,27]. Empirically, we found that using PCC as the feedback is not ideal for increasing the output intensity and hence the focusing contrast for the output pattern. In this work, we propose to replace the fitness function with VecCos, given by

$$\begin{array}{{c}} {VecCos = \; \frac{{\mathop \sum \nolimits_i^M \; ({{{\boldsymbol I}_i} \cdot {\boldsymbol I}_i^t} )\; }}{{\sqrt {\mathop \sum \nolimits_i^M {{({{{\boldsymbol I}_i}} )}^2}} \sqrt {\mathop \sum \nolimits_i^M {{({{\boldsymbol I}_i^{\boldsymbol t}} )}^2}} }}\; .} \end{array}$$

Apparently, the amount of calculation for VecCos is less than that of PCC, which matters for frequently called situations. What’s more, it is hypothesized that wavefront optimization with VecCos can significantly increase the focusing contrast without sacrificing correlation with the target pattern, which will be confirmed through experiment.

2.4 Experimental configuration

Figure 2 illustrates the setup built for pattern projection experiment. The complex medium here is a 15-meter long MMF that is not fixed onto the optical table and is sensitive to external perturbations. The light beam from a 532 nm continuous-wave laser (EXLSR-532-300-CDRH, Spectra Physics, USA) is expanded with a 40× objective lens (Obj1) and collimated by a convex lens (L1) subsequently. The expanded beam is incident on the digital mirror device (DMD, DLP7000, Texas Instruments Inc, USA) and is reflected before being relayed into a 4f system consisting of lenses L2 and L3 for phase modulation. An iris is used to select the -1-diffraction order component in the spectral plane of the 4f system. The modulated beam is then coupled into the 15-meter long MMF (0.22 NA, 105 µm core, SUH105, Xinrui, China). The output field from the MMF is imaged by a 40× objective (Obj2, Plan N 40X/0.65NA Olympus N1216000) and tube lens L4 before being recorded by a CMOS camera (BFS-U3-04S2M, FLIR, USA), with an imaging magnification measured to be ∼19.3. The CMOS camera is triggered by the DMD for synchronous acquisition of the output light intensities from the MMF when the DMD displays a new pattern. Basically, for each iteration sequence, the feedback signals from the CMOS camera are used to produce the offspring phase masks using an optimization algorithm, which are encoded via Lee hologram [10,35] and uploaded onto the DMD for phase modulation.

 figure: Fig. 2.

Fig. 2. Experimental setup for the feedback-based wavefront optimization for pattern projection through MMF. CMOS, complementary metaloxide semi-conductor camera; DMD, digital mirror device; FC, fiber collimator; L, lens; M, mirror; MMF, multimode fiber; Obj, objective; PC, personal computer.

Download Full Size | PDF

3. Results

3.1 Numerical simulation

In this section, we numerically compare the performances of various algorithms for pattern projection through a complex medium under the condition of different noise levels (as gauged by α) and relative persistence times (as gauged by ${T_p}/{T_i}$). The parameters of PSO, GA, IACO, and SNES are as specified in Table 1. The target pattern was a binary image showing string “PolyU”. Empirically, the noise level was set to 30% and 60$\%$, respectively. In addition, the relative persistence time of the medium, ${T_p}/{T_i}$, was set to be 60 and 30, respectively, which reasonably mimicked experimental observations. Each of the algorithms was repeated 30 times for averaging and a new TM was simulated each time. Note that in simulation, all algorithms adopted PCC as the fitness function to guide the wavefront optimization. Under different conditions, the resultant progression curves of PCC between the simulated outputs of various algorithms and the target pattern as the iteration progresses are presented in Fig. 3, with the error bars under 30 repeated tests added.

 figure: Fig. 3.

Fig. 3. Simulated progression of PCC as a function of iteration number for different optimization algorithms for the projection of “PolyU” pattern through a complex medium, under the condition of different noise levels and relative persistence times: (a) $\alpha = 30\%$ and ${T_p}/{T_i}\; = 60$; (b) $\alpha = 30\%$ and ${T_p}/{T_i}\; = 30$; (c) $\alpha = 60\%$ and ${T_p}/{T_i}\; = 60$; (d) $\alpha = 60\%$ and ${T_p}/{T_i}\; = 30$. The error bars represented the standard deviation of the 30 repeated tests for each curve.

Download Full Size | PDF

It is clearly shown that among the four algorithms, SNES evolves fastest and reaches the highest PCC (all larger than $0.8$) under all conditions, and it is least affected by the variations in noise and perturbation levels. For IACO, it evolves slowest initially, but exhibits strong immunity to noise and perturbation and grows gradually, achieving PCCs that are slightly smaller than those of SNES at the end. The other two algorithms, GA and PSO, however, are more susceptible to the measurement noise and furthermore the perturbation to the medium. In the situation with relatively low noise and long persistence time (Fig. 3(a)), GA increases at the initial phase and peaks after ∼500 iterations. After that, the performance declines gradually due to the perturbation applied to the medium’s TM every 60 iterations. PSO performs even worse than GA. Although growing fast at first, it soon decreases gradually, with similar behavior but a poorer metric than GA over the iterations.

When the relative persistence time ${T_p}/{T_i}\; $ decreases to 30 (Fig. 3(b)), it indicates an increase of perturbation level and hence the effect of decorrelation. Under this situation, the performances of SNES and IACO are not severely affected, but those of GA and PSO deteriorate significantly, both with the final PCC below 0.3. The worst performances occur in Fig. 3(d), with no surprise, in the condition with strong noise and short persistence time, where the final PCCs for SNES and IACO are slightly above and below 0.8, respectively, while for GA and IACO the values are around 0.2 after 2000 iterations. These results prove that SNES has superior features such as fast convergence and robust immunity to noise and perturbation. Besides, the error bars in Figs. 3(a)-(d) also reveal that SNES and IACO suffer from less variations than GA and especially IACO during the 30 repeated tests. It should also be noted that IACO yields nearly as good results as those of SNES, although it grows slower in the initial phase. In the next section, the performances of different methods will be further compared through real experiments.

3.2 Experiment

In addition to the algorithms explored in simulation, SNES optimized with VecCos, termed as SNES-VecCos, is introduced, which is expected to further enhance the performance. The 15-meter long and loosely placed MMF in our experiment could be considered as a dynamically changing medium as even weak air flow and platform vibrations can cause a drift of position or deformation to the MMF. Note that the decorrelation of the MMF is at high level and uncontrolled, which is thus unpredictable and might vary from case to case.

Long-distance pattern projection through the dynamically changing MMF was first conducted using the same “PolyU” target pattern as in simulation. During this experiment, it was observed the speckle decorrelation time (the time it took for the correlation between the captured speckle patterns with the initial one first dropped below 1/e, using the same input wavefront) was ∼35s. Although the observed decorrelation seemed to be a little bit strong initially, it would weaken and fluctuate over time, indicating the unfixed MMF tried to recover from disturbance. It took about 16 minutes in total for the five optimization methods, with 2,000 iterations for each method. The projected patterns captured by the camera after normalization and the progression curves of PCC and contrast as the functions of iteration number for all algorithms are provided in Fig. 4. Visually, the projected patterns with IACO and SNES have apparently higher quality in both similarity to the target pattern and image contrast, while PSO and GA seem to fail the task (Figs. 4(a)-(e)). The trend can be observed more clearly from the progression curves. In Fig. 4(f), the experimental PCC curves are well consistent with the simulated data (especially with Fig. 3(c)): the SNES and IACO approaches can resist the noise/disturbance and achieve high PCCs close to 0.8; for GA and PSO, the performance degrades slowly after several hundreds of iterations, resulting in a PCC value of around 0.6 and 0.3, respectively. The only significant deviation is that GA seems to outperform its simulation although not significantly, which may be because the conditions set in simulation turn to be more demanding than those encountered in experiment. Furthermore, unlike the simulated data that are averaged over 30 times, the results shown in Fig. 4 are based on a single measurement in experiment for each algorithm. It is hence natural to have certain randomness, although the results are typical.

 figure: Fig. 4.

Fig. 4. Experimental results of various optimization algorithms for long-distance projection of the “PolyU” pattern through a 15 m long unstable MMF. The normalized results of MMF outputs after wavefront optimization by all the algorithms: (a) PSO, (b) GA, (c) IACO, (d) SNES optimized with PCC, and (e) SNES optimized with VocCos. Note that PSO, GA, and IACO all use PCC as the fitness function. The evolution curves of (f) PCC and (g) contrast are also recorded for all the cases. The scale bar shown in (a)-(e) is 10 $\mu m$.

Download Full Size | PDF

Regarding the experimental contrast curves (Fig. 4(g)), they share overall similar trends with the PCC. Notably, SNES-VecCos outperforms its SNES peer (i.e., SNES-PCC) and IACO significantly, although the PCCs for these three approaches are very comparable (Fig. 4(f)). These are consistent with what can be seen directly from the resultant projected patterns (Figs. 4(a)-(e)), suggesting that optimization with VecCos can considerably increase the pattern contrast without sacrificing the fidelity with the target.

To further confirm the performance of the proposed SNES-VecCos, several more target pattens of different types were set to be projected through the long unstable MMF. The pattern includes a $6 \times 6$ focus array (Fig. 5(a)), a pentagram (Fig. 5(b)), and a simplified Bagua (a Chinese traditional motif incorporating the eight trigrams of the I Ching, arranged octagonally around a symbol denoting the balance of yin and yang, Fig. 5(c)). From Fig. 4(f), with SNES a desired pattern can be formed with only hundreds of iterations, which could be relatively fast by using a DMD with a maximum refresh rate of up to 23 kHz. Therefore, in this new group of experiments, wavefront optimization of various methods all ran for 1,000 iterations at the same time, with the normalized output patterns provided in Fig. 5. Qualitatively, the performances of different methods are in good agreement with those observed earlier: the fidelity with the target pattern achieved by IACO and SNES are comparable and considerably better than those obtained by GA and PSO. The relatively large performance difference after ∼1,000 iterations could be attributed to the strong decorrelation of MMF at the time. It is also obvious that SNES-VecCos achieves the highest contrast in all cases. Particularly for the Bagua pattern, the contrast yielded by SNES-VecCos is ∼60% higher than those achieved by SNES-PCC and IACO. Note that part of the Bagua pattern of SNES-VecCos appears slightly oversaturated, which, however, has very small impact to the contrast calculation, as the oversaturated pixels occupy around 1% of the total pixels. Also note that since the number of output modes to be controlled varies case by case, the optimal contrast of projected patterns that can be attained is naturally at different levels. Nevertheless, the high-quality results of arbitrary pattern projection through the long unstable MMF confirm the superior performance of the proposed SNES-VecCos method through experiments.

 figure: Fig. 5.

Fig. 5. Several more examples of arbitrary pattern projection through the 15-meter unstable MMF, using different optimization methods for comparison. Projection results for the target pattern of (a) $6 \times 6$ focus array, (b) pentagram, and (c) simplified Bagua are shown on the first, second, and third row, respectively, with the corresponding contrast value noted in each realization. Note that for each realization, the intensities of experimentally acquired images are individually normalized and have the same color bar for comparison of contrast. The scale bar shown in the last images of (a)-(c) is 10 $\mu m$.

Download Full Size | PDF

4. Discussion and conclusion

In this study, five iterative wavefront optimization algorithms were implemented and compared to project an arbitrary pattern through a complex medium, in a noisy and perturbed condition. As seen from the progression curves of PCC in both simulation and experiment, SNES evolves most rapidly and converges at the earliest stage, showing its higher search efficiency for optimal phase mask. These advantages are mostly related to the working principle of SNES: unlike other intelligent optimization algorithms that more or less involve random variations, SNES is built upon parameter optimization via natural gradient ascent. While algorithms like PSO, GA, and IACO update populations by probability, SNES generates new populations by updated search parameters each time. The computation of parameters’ natural gradients in SNES avoids direct involvement of the fitness values, increasing the robustness of parameter updating, especially with the presence of medium disturbance. Therefore, for pattern projection through a long unfixed multimode fiber, SNES optimized with VecCos is desired to replace PCC as the fitness function, which results in faster convergence and improvement of image contrast for up to 60% in experiment while maintaining the pattern projection fidelity.

Whilst promising, the new algorithm, i.e., SNES-VecCos, also sees limitation in time consumption. In our experiment, it took ∼16 minutes in total for five wavefront optimization procedures, with 2000 iterations for each algorithm. Although the DMD used in the study can theoretically update at 23 kHz, it refreshed at ∼300 frame/s to ensure synchronization with the relatively slower CMOS camera and allocate sufficient buffer for data transfer among different system components and processing by the computer. Specifically, for 2000 iterations, it takes ∼35 s for IACO, ∼23 s for PSO, GA and SNES-PCC, and ∼18s for SNES-VecCos by using our workstation with an Intel Xeon CPU (3.50 GHz), NVIDIA 3070 GPU (1.73 GHz clock, 8 GB GDDR6 memory), and 64 GB RAM. For pattern focusing or projection in real applications where the optical field decorrelates rapidly (e.g., living biological tissue), further engineering to the system is required, which may involve faster feedback on pattern formation, more efficient data transfer, processing, and control through a customized field-programmable gate array framework and so on.

In summary, SNES optimized with VecCos is proposed in this study to search for optimal wavefront and control light focusing into an arbitrary pattern through a complex medium in a noisy and perturbed environment. Different algorithms were compared with numerical simulations under different noise and perturbation levels, which agree well with the experimental results using a 15-meter unfixed MMF. Apart from the excellent anti-interference and convergence performance, adopting VecCos as the fitness function can further enhance the contrast of projected pattern. With the proposed scheme, an arbitrary pattern can be effectively generated at the distal end of the long and unstable MMF within 1000 iterations. With further improvement especially on wavefront optimization speed, the approach may find special interests for many biomedical applications, such as deep-tissue photon therapy and optogenetics based on MMF, where free-space localized optical delivery encounters challenges.

Funding

National Natural Science Foundation of China (81627805, 81930048); Research Grants Council, University Grants Committee (15217721, C7074-21GF, R5029-19); Innovation and Technology Commission - Hong Kong (GHP/043/19SZ, GHP/044/19GD); Guangdong Science and Technology Department (2019A1515011374, 2019BT02X105).

Acknowledgments

The work was supported by National Natural Science Foundation of China (NSFC) (81930048, 81627805), Hong Kong Research Grant Council (15217721, R5029-19, C7074-21GF), Hong Kong Innovation and Technology Commission (GHP/043/19SZ, GHP/044/19GD), and Guangdong Science and Technology Commission (2019A1515011374, 2019BT02X105). The authors would also like to thank the Photonics Research Institute of the Hong Kong Polytechnic University for facility support.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data and codes underlying the results presented in this paper are not publicly available at this time but may be obtained from the corresponding author upon reasonable request.

References

1. A. P. Mosk, A. Lagendijk, G. Lerosey, and M. Fink, “Controlling waves in space and time for imaging and focusing in complex media,” Nat. Photonics 6(5), 283–292 (2012). [CrossRef]  

2. S. Rotter and S. Gigan, “Light fields in complex media: Mesoscopic scattering meets wave control,” Rev. Mod. Phys. 89(1), 015005 (2017). [CrossRef]  

3. I. M. Vellekoop and A. Mosk, “Focusing coherent light through opaque strongly scattering media,” Opt. Lett. 32(16), 2309–2311 (2007). [CrossRef]  

4. I. M. Vellekoop and A. P. Mosk, “Universal optimal transmission of light through disordered materials,” Phys. Rev. Lett. 101(12), 120601 (2008). [CrossRef]  

5. S. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13(12), 123021 (2011). [CrossRef]  

6. Z. Yu, H. Li, and P. Lai, “Wavefront Shaping and Its Application to Enhance Photoacoustic Imaging,” Appl. Sci. 7(12), 1320 (2017). [CrossRef]  

7. J.-H. Park, Z. Yu, K. Lee, P. Lai, and Y. Park, “Perspective: Wavefront shaping techniques for controlling multiple light scattering in biological tissues: Toward in vivo applications,” APL Photonics 3, 100901 (2018). [CrossRef]  

8. S. Yoon, M. Kim, M. Jang, Y. Choi, W. Choi, S. Kang, and W. Choi, “Deep optical imaging within complex scattering media,” Nat. Rev. Phys. 2(3), 141–158 (2020). [CrossRef]  

9. D. Akbulut, T. J. Huisman, E. G. van Putten, W. L. Vos, and A. P. Mosk, “Focusing light through random photonic media by binary amplitude modulation,” Opt. Express 19(5), 4017–4029 (2011). [CrossRef]  

10. D. B. Conkey, A. M. Caravaca-Aguirre, and R. Piestun, “High-speed scattering medium characterization with application to focusing light through turbid media,” Opt. Express 20(2), 1733–1740 (2012). [CrossRef]  

11. S. M. Popoff, G. Lerosey, R. Carminati, M. Fink, A. C. Boccara, and S. Gigan, “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett. 104(10), 100601 (2010). [CrossRef]  

12. D. B. Conkey, A. N. Brown, A. M. Caravaca-Aguirre, and R. Piestun, “Genetic algorithm optimization for focusing through turbid media in noisy environments,” Opt. Express 20(5), 4840–4849 (2012). [CrossRef]  

13. H. Li, C. M. Woo, T. Zhong, Z. Yu, Y. Luo, Y. Zheng, X. Yang, H. Hui, and P. Lai, “Adaptive optical focusing through perturbed scattering media with a dynamic mutation algorithm,” Photonics Res. 9(2), 202–212 (2021). [CrossRef]  

14. Q. Zhao, C. M. Woo, H. Li, T. Zhong, Z. Yu, and P. Lai, “Parameter-free optimization algorithm for iterative wavefront shaping,” Opt. Lett. 46(12), 2880–2883 (2021). [CrossRef]  

15. C. M. Woo, H. Li, Q. Zhao, and P. Lai, “Dynamic mutation enhanced particle swarm optimization for optical wavefront shaping,” Opt. Express 29(12), 18420–18426 (2021). [CrossRef]  

16. C. M. Woo, Q. Zhao, T. Zhong, H. Li, Z. Yu, and P. Lai, “Optimal efficiency of focusing diffused light through scattering media with iterative wavefront shaping,” APL Photonics 7, 046109 (2022). [CrossRef]  

17. Y. Luo, S. Yan, H. Li, P. Lai, and Y. Zheng, “Towards smart optical focusing: deep learning-empowered dynamic wavefront shaping through nonstationary scattering media,” Photonics Res. 9(8), B262–B278 (2021). [CrossRef]  

18. T. Cizmar and K. Dholakia, “Shaping the light transmission through a multimode optical fibre: complex transformation analysis and applications in biophotonics,” Opt. Express 19(20), 18871–18884 (2011). [CrossRef]  

19. D. Loterie, S. Farahi, I. Papadopoulos, A. Goy, D. Psaltis, and C. Moser, “Digital confocal microscopy through a multimode fiber,” Opt. Express 23(18), 23845–23858 (2015). [CrossRef]  

20. M. Ploschner and T. Cizmar, “Compact multimode fiber beam-shaping system based on GPU accelerated digital holography,” Opt. Lett. 40(2), 197–200 (2015). [CrossRef]  

21. D. B. Conkey and R. Piestun, “Color image projection through a strongly scattering wall,” Opt. Express 20(25), 27312–27318 (2012). [CrossRef]  

22. L. Wan, Z. Chen, H. Huang, and J. Pu, “Focusing light into desired patterns through turbid media by feedback-based wavefront shaping,” Appl. Phys. B 122(7), 204 (2016). [CrossRef]  

23. Q. Feng, F. Yang, X. Xu, B. Zhang, Y. Ding, and Q. Liu, “Multi-objective optimization genetic algorithm for multi-point light focusing in wavefront shaping,” Opt. Express 27(25), 36459–36473 (2019). [CrossRef]  

24. L. Zhan, D. Liu, A. Yang, J. Miao, and J. Zhu, “Fast compensation for arbitrary focusing through scattering media,” Appl. Opt. 58(10), 2624–2629 (2019). [CrossRef]  

25. H. Li, X. Wu, G. Liu, X. Wang, S. Luo, Z. Chen, and J. Pu, “Generation of Focal Patterns With Uniform Intensity Distribution From Speckle by Hadamard-Genetic Algorithm,” IEEE Photonics J. 13(3), 1–8 (2021). [CrossRef]  

26. S. Cheng, H. Li, Y. Luo, Y. Zheng, and P. Lai, “Artificial intelligence-assisted light control and computational imaging through scattering media,” J. Innov. Opt. Health Sci. 12(04), 1930006 (2019). [CrossRef]  

27. Z. Yang, L. Fang, X. Zhang, and H. Zuo, “Controlling a scattered field output of light passing through turbid medium using an improved ant colony optimization algorithm,” Optics and Lasers in Engineering 144, 106646 (2021). [CrossRef]  

28. T. Zhong, Z. Qiu, Y. Wu, J. Guo, H. Li, Z. Yu, S. Cheng, Y. Zhou, J. Zhu, and J. Tian, “Optically Selective Neuron Stimulation with a Wavefront Shaping-Empowered Multimode Fiber,” Advanced Photonics Research 3, 2100231 (2022). [CrossRef]  

29. D. Wierstra, T. Schaul, T. Glasmachers, Y. Sun, J. Peters, and J. Schmidhuber, “Natural evolution strategies,” The Journal of Machine Learning Research 15, 3381–3387 (2014). [CrossRef]  

30. J. Yang, Q. He, L. Liu, Y. Qu, R. Shao, B. Song, and Y. Zhao, “Anti-scattering light focusing by fast wavefront shaping based on multi-pixel encoded digital-micromirror device,” Light: Sci. Appl. 10(1), 149 (2021). [CrossRef]  

31. Y. Zhao, Q. He, S. Li, and J. Yang, “Gradient-assisted focusing light through scattering media,” Opt. Lett. 46(7), 1518–1521 (2021). [CrossRef]  

32. T. Schaul, T. Glasmachers, and J. Schmidhuber, “High dimensions and heavy tails for natural evolution strategies,” in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (2011), 845–852.

33. L. Fang, H. Zuo, Z. Yang, X. Zhang, L. Pang, W. Li, Y. He, X. Yang, and Y. Wang, “Particle swarm optimization to focus coherent light through disordered media,” Appl. Phys. B 124(8), 155 (2018). [CrossRef]  

34. I. M. Vellekoop and A. P. Mosk, “Phase control algorithms for focusing light through turbid media,” Opt. Commun. 281(11), 3071–3080 (2008). [CrossRef]  

35. W.-H. Lee, “Binary computer-generated holograms,” Appl. Opt. 18(21), 3661–3669 (1979). [CrossRef]  

Data availability

Data and codes underlying the results presented in this paper are not publicly available at this time but may be obtained from the corresponding author upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1. Framework of SNES in iterative WFS optimization for pattern projection through complex media.
Fig. 2.
Fig. 2. Experimental setup for the feedback-based wavefront optimization for pattern projection through MMF. CMOS, complementary metaloxide semi-conductor camera; DMD, digital mirror device; FC, fiber collimator; L, lens; M, mirror; MMF, multimode fiber; Obj, objective; PC, personal computer.
Fig. 3.
Fig. 3. Simulated progression of PCC as a function of iteration number for different optimization algorithms for the projection of “PolyU” pattern through a complex medium, under the condition of different noise levels and relative persistence times: (a) $\alpha = 30\%$ and ${T_p}/{T_i}\; = 60$; (b) $\alpha = 30\%$ and ${T_p}/{T_i}\; = 30$; (c) $\alpha = 60\%$ and ${T_p}/{T_i}\; = 60$; (d) $\alpha = 60\%$ and ${T_p}/{T_i}\; = 30$. The error bars represented the standard deviation of the 30 repeated tests for each curve.
Fig. 4.
Fig. 4. Experimental results of various optimization algorithms for long-distance projection of the “PolyU” pattern through a 15 m long unstable MMF. The normalized results of MMF outputs after wavefront optimization by all the algorithms: (a) PSO, (b) GA, (c) IACO, (d) SNES optimized with PCC, and (e) SNES optimized with VocCos. Note that PSO, GA, and IACO all use PCC as the fitness function. The evolution curves of (f) PCC and (g) contrast are also recorded for all the cases. The scale bar shown in (a)-(e) is 10 $\mu m$.
Fig. 5.
Fig. 5. Several more examples of arbitrary pattern projection through the 15-meter unstable MMF, using different optimization methods for comparison. Projection results for the target pattern of (a) $6 \times 6$ focus array, (b) pentagram, and (c) simplified Bagua are shown on the first, second, and third row, respectively, with the corresponding contrast value noted in each realization. Note that for each realization, the intensities of experimentally acquired images are individually normalized and have the same color bar for comparison of contrast. The scale bar shown in the last images of (a)-(c) is 10 $\mu m$.

Tables (1)

Tables Icon

Table 1. Parameters of different optimization algorithms for pattern projection

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

J ( θ ) = E θ [ f ( z ) ] = f ( z ) π ( z | θ ) d z
θ J ( θ ) = θ f ( z ) π ( z | θ ) d z = E θ [ f ( z ) θ log π ( z | θ ) ]
~ θ J ( θ ) = F 1 θ J ( θ )
θ θ + η ~ θ J ( θ ) = θ + η F 1 θ J ( θ ) ,
I n o i s e = I + α I ϵ .
t mn 1 1 + δ 2 ( t m n + ξ ) .
P C C = i M ( I i I ¯ ) ( I i t I t ¯ ) i M ( I i I ¯ ) 2 i M ( I i t I t ¯ ) 2 ,
C o n t r a s t = μ ( I i ) | i T / μ ( I i ) | i T ,
V e c C o s = i M ( I i I i t ) i M ( I i ) 2 i M ( I i t ) 2 .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.