## Abstract

This paper analyzes the capacity of the free space coherent optical MIMO transmission system. Two scenarios are considered (i.e., with and without the adaptive optics compensation). It is generally accepted that the adaptive optics compensation can significantly improve the system performance, which is rigorously true when the MIMO algorithm is not implemented. However, it might not be the case in the coherent MIMO systems. When the turbulence strength is weak or moderate, this work demonstrates that the phase-only wave-front corrector will increase the mean eigen value of the coherent system capacity matrix **HH*** ^{H}* and make the eigen value distribution more even, i.e. it will decrease the maximal eigen value while increasing the average eigen value. Hence, the capacity of the system with adaptive optics increases when the channel information is not available, because the sub-channels are placed with equal powers. When the channel information is perfectly available and the water filling algorithm is used to optimize the power allocation, the system with adaptive optics could have a deteriorated performance as the capacity is more related to the large eigen values especially in the low signal to noise ratio (SNR) regime. When the turbulence strength is strong, it is found that adaptive optics will decrease both the mean and maximal eigen values for the capacity matrix

**HH**

*, and therefore the system capacity degrades, whether the channel information is available or not.*

^{H}© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

Space division multiplexing (SDM) has been proposed to further boost the transmission capacity of the optical communication systems [1]. For the free space optical (FSO) transmission system, the SDM is realized by modulating the signals on a set of spatial modes. The spatial modes can be the Laguerre-Gauss (LG) modes [2], the Bessel-Gauss modes [3] or other spatial modes provided that they are orthogonal to each other during the propagation in free space.

The orthogonality of those modes (e.g., the LG modes, maintains perfectly in the isotropic and homogenous media). However, they will be distorted due to the presence of atmosphere turbulence [4–11]. Atmosphere turbulence, which is induced by rain, water vapor, smog etc., causes random scattering and therefore, distorts the transmission beams/channels. Such a phenomenon will cause crosstalk among the modes and degrade the system capacity [9–11]. There have been quite a few researchers studying the methods to mitigate the influence of atmosphere turbulence. One way to compensate the atmosphere turbulence induced distortion is to use the phase-only adaptive optics [12–14]. A beacon wave which monitors the phase error in the transmission channel is sent [12–14] and the calculated phase error will be compensated by the phase-only spatial mask. The channel monitoring can be achieved either by sending a beacon wave in the counter direction [12] or using the beam in the signal direction but at a different wavelength [13,14].

For the non-coherent FSO system with an intensity channel, the capacity has been well studied [15–18]. For the coherent MIMO system with no real and non-negative constraint for the input, the system capacity is determined by the channel capacity matrix **HH*** ^{H}* [2]. Not only the diagonal elements of the channel matrix

**H**matter (which is surely increased by adaptive optics), but also the off diagonal elements, which stands for the crosstalk between the modes, play an important role. Up to date, there have been studies on the system capacity of the free space optical communication systems with crosstalk. In the free space non-MIMO SDM systems [5,12–14], the capacity/performance are discussed with [12–14] and without [5] the phase-only adaptive optics implemented. In [2], N. Zhao et. al. performed a thorough analysis on the capacity of the free space coherent MIMO systems, whose crosstalk is mainly from the finite pupil size. Further study on the free space MIMO systems with and without the non-phase-only adaptive optics (referred to as “full wave adaptive optics”) has been shown in [9,10]. However, the conclusion cannot be directly extended to the existing phase-only adaptive optics compensated systems. Moreover, the work in [9,10] focuses on the quantum-mechanical (Holevo-limit) capacity, which is different from the Shannon-limit capacity for the coherent optical communication systems. In our latest work [11], the Shannon-limit capacity upper bound for coherent MIMO system without adaptive optics under atmosphere turbulence was analyzed and the optimal mode sets were determined. However, the case in [11] does not consider the situation that the channel information is fully available or the adaptive optics is used. Therefore, although there have been pioneering researches [2,5,9,10,12–18] on the system capacity for the free space SDM systems, the free space coherent optical MIMO systems have not been investigated for the system capacity under atmosphere turbulence with the currently available phase-only adaptive optics. The comparison of the turbulence impacted coherent MIMO system capacity with and without the channel information has not been done either.

In this paper, we provide a thorough study on the system capacity for the free space coherent MIMO systems with and without the phase-only adaptive optics, and we will try to find the answers to the following questions. What is the performance difference for the coherent MIMO systems with and without the phase-only adaptive optics? What if the channel information is available or not? What if the turbulence strength varies from the weak/moderate regime to the strong regime? Is it possible that the phase-only adaptive optics might bring degradation to the system capacity in these scenarios?

## 2. Theory

#### 2.1 The LG modes propagation in free space

Without loss of generality, the LG modes are assumed in this work, because they form the complete orthonormal basis in free space and are frequently used in the free space SDM systems. They are characterized by the initial beam waist *w*_{0}, the free space wavelength λ_{0}, the azimuth order *m* and the radial order *p*. The detailed expressions for the LG modes can be found in Appendix A. The propagation of the LG modes under the turbulence can be simulated by integrating the perturbed wave equation [11,19], and the detailed discussion can be found in Appendix B. The index perturbation is usually described by a 3-dimentional random variable, whose correlation function is the Fourier transform of some spectrum function. Without loss of generality, the Kolmogorov spectrum is assumed [5,11]:

**FT**is the Fourier transform,

*k*and

_{x}k_{y}*k*the corresponding spatial frequencies,

_{z}*l*,

_{0}*L*and C

_{0}_{n}

^{2}the parameters associated with the spectrum shape and the turbulence strength [5,11].

In the simulation, the space is divided into several sections in the propagation direction z. For each section, a random phase screen is inserted which is generated by filtering a two-dimensional Gaussian white noise through a filter whose frequency domain response is the square root of **K**(*k*_{x}, *k*_{y}, 0). Such a multiple phase screen model is widely used in the simulation of light propagation in the presence of atmosphere turbulence [5–8,14].

#### 2.2 Channel model and capacity calculation

In the system, limited number of the LG modes is used for transmission. Without loss of generality, *Q* LG modes with either a different radial order p or a different azimuth order m are assumed in the coherent MIMO system, which indicates that the numbers of the transmitters and receivers are both Q. The receiving signal **r** can be calculated by the transmitting signal vector **x** through [2,11]

**H**is the channel matrix, whose diagonal elements indicate the remaining amplitude of the transmitted modes and the off diagonal elements indicate the crosstalk among the modes,

**n**the Gaussian i.i.d noise vector which is introduced at the transmitter and the receiver. Different from the free space intensity channels [15–18], the above channel model has no real and non-negative constraint on the input vector

**x**, and it is in accordance with the coherent MIMO channel model in [2]. The capacity of such a coherent MIMO system can be calculated by [2]

*λ*is the eigen value of the capacity matrix

_{q}**H**

^{H}**H**,

*P*the allocated power on the q

_{q}^{th}sub-channel which is represented by the q

^{th}eigen vector of

**H**

^{H}**H**, and σ

^{2}equals <

*n*(

_{q}*t*)

*n** (

_{q}*t*)> which stands for the noise power on each sub-channel.

Two cases are usually considered while evaluating Eq. (3). When the channel matrix **H** is not known at the transmitter side, equal power for each channel will be assumed to maximize the system capacity.

_{tot}stands for the total signal power. If the pilot signals can be sent to measure the channel matrix, the water filling algorithm can be used to optimize the power allocation on each sub-channel [20].

#### 2.3 Analysis of the mean eigen value of the capacity matrix HHH with and without adaptive optics

We denote the transfer coefficient from mode i to mode j as *H*_{ij}, where i and j are associated with the i^{th} and the j^{th} LG modes. It is worth noting that *H*_{ij} might not be the element of **H**, as **H** only includes *H*_{ij} as its element if and only if mode i and mode j belong to the Q-mode set (i.e. i< = Q and j< = Q) which are selected for transmission. In order to separate the cases with and without adaptive optics, a superscript of 'ad' or 'non' will be used.

First of all, it is easy to infer that under the weak/moderate turbulence, we have

where < > denotes expectation. This is true because adaptive optics can compensate the wave-front distortion and thus it increases the portion of the power on the transmitted LG mode at the receiver. This fact has been widely proved by the experiments and the simulations in [12–15].It is known that

**H**

^{H}**H**is determined by the summation of its diagonal elements.

Due to the energy conservation law and the fact that the LG modes form the complete orthonormal basis, we have

It is well accepted and has been proved in the experiments and the simulations that the power transferring mainly takes place between the neighboring modes [5–8], i.e. the modes with close values for the azimuth order *m* and the radial order *p*. Therefore, for the modes 'in the middle', i.e. the modes with quite a few neighboring modes contained in the selected Q-mode set, we have the following approximate equality under the weak/moderate turbulence

As is indicated by Eq. (6), the power will concentrate more on the center mode i if adaptive optics is implemented. For those modes with less neighboring modes contained in the selected Q-mode set, the summation of the square of the row elements of **H** will mainly depend on the value of $\u3008{\left|{H}_{ii}\right|}^{2}\u3009$, and we will have

A mode varies between “the mode with quite a few neighboring modes” and “the mode with less neighboring modes” should have $\u3008{\displaystyle \sum _{j=1}^{Q}{\left|{H}_{ij}{}^{ad}\right|}^{2}}\u3009$ to be either larger or close to$\u3008{\displaystyle \sum _{j=1}^{Q}{\left|{H}_{ij}{}^{non}\right|}^{2}}\u3009$. Considering this, we have the following inequality under the weak/moderate turbulence

Equation (11) indicates that under the weak/moderate turbulence, the summation of the diagonal elements or the mean eigen value of the capacity matrix will be increased by adaptive optics and therefore, a better system capacity performance is expected when the channel information is not available and equal power allocation has been assumed for each sub-channel.

## 3. Results and discussions

Numerical simulations have been performed to investigate the system performance. We adopt the same numerical simulation parameters as [5,11]. Although the reader can refer to [5,11] for details, it will be restated here for convenience. The signal wavelength is 850nm. The turbulence spectral parameters have the following values: *L*_{0} = 20m, and *l*_{0} = 5mm. The transmission length of the system is 1000m. Random phase screens are placed every 50m along the propagation distance [5,11]. Monte Carlo simulations are performed with 1000 realizations [5,11]. When we have a small realization number, the averaged results could fluctuate with larger variances, and 1000 is a number selected to balance the accuracy and computational complexity.

The area to characterize the beam propagation is a 280cmX280cm square, which has been divided into a grid of 2048X2048 elements. The initial beam waist of the LG modes *w*_{0} is 1.6cm. Totally 76 LG modes with the radial order p from 0 to 3 and the azimuth order m from −9 to + 9 are considered in the transmission and they propagate collinearly. It is worth mentioning that if a different mode set is chosen, the capacity of the system could take a different value but the behaviors will be similar for the system capacity with and without adaptive optics. According to [2], the root mean square (r. m. s) waist size of the LG mode can be calculated by ${w}_{z}\sqrt{2p+\left|l\right|+1}$, and hence, the largest r.m.s waist size of the LG mode is 4*w*_{0} and 4*w _{L}* for z = 0 and z =

*L*. Since the impact of finite pupil size has been fully characterized in [2], we assume that the lenses of the transmitter and the receiver are large enough to accept most of the energy of 76 LG modes, i.e. they should have their pupil radii much larger than the largest beam waist 4

*w*

_{0}and 4

*w*, which is achievable in our case. Such a configuration enables us to neglect the pupil size impact and focus on the atmosphere turbulence influence only.

_{L}As discussed above, two scenarios are considered in the simulation, i.e. with and without adaptive optics. When adaptive optics is not used, the beam transmits through the atmosphere turbulence and correlates with the ideal LG modes at the receiver side to obtain the corresponding transmission matrix **H**. When implementing adaptive optics, a beacon beam with the radial order of 0 and the azimuth order of 0 is sent to monitor the wave-front. The beacon beam can be located at the same wavelength or a different wavelength in comparison with the signal [12–14]. In our case, the beacon wavelength of 852nm is assumed. It has been verified that the wavelength separation between the signal and the beacon wave can be as large as 10nm without significantly affecting the capacity. Here, it is selected to be close to the 850nm signal wavelength. The beacon beam has its beam waist to be the same as that of the largest LG mode, and it is detected at the receiver side. A phase-only spatial mask array is used to compensate the phase distortion for the 76 transmitted LG modes. In the simulation, it is assumed that there are enough phase-adjustment mirrors to fully compensate the phase front distortion detected by the beacon beam. Such an assumption enables us to study the theoretical capacity limit of the free space transmission system with the adaptive optics compensation. The schematic to obtain the system capacity is shown in Fig. 1.

#### 3.1. Verification of the equations in section 2.3

As a proof of concept for section 2.3, an example of weak/moderate turbulence is presented. C_{n}^{2} is set to be 10^{−15} m^{-2/3}. Totally 1000 realizations have been simulated to obtain the average diagonal elements of the capacity matrix **H**^{H}**H**. Since in the simulation, the azimuth order *m* changes from −9 to + 9 and the radial order *p* changes from 0 to 3, the lowest order LG mode, i.e. the LG mode with m = 0 and p = 0 is expected to be with the maximal number of the neighboring modes. The corresponding average diagonal element for **H**^{ad}^{H}**H**^{ad} and **H**^{non}^{H}**H**^{non} are 0.9967 and 0.9969. They are very close to 1 and thus it verifies Eq. (9). Similar behaviors occur for those diagonal elements, whose corresponding modes are with a large number of the neighboring modes. The average differences between the corresponding diagonal elements are less than 0.004.

On the other hand, the LG mode with m = 9 and p = 3 are with the least number of the neighboring modes, and the corresponding average diagonal elements are 0.8427 and 0.6968 respectively for the cases with and without adaptive optics, and therefore, Eq. (10) is verified.

Combining Eq. (9-10), it can be further verified that Eq. (11) is true. The average values for the summation of the diagonal elements for **H**^{ad}^{H}**H**^{ad} and **H**^{non}^{H}**H**^{non} are 72.8994 and 71.3498 respectively.

#### 3.2. System capacity analysis

In the system simulation, the turbulence strength C_{n}^{2} varies. Firstly, it is assumed to be 10^{−14} m^{-2/3}, which is the moderate turbulence strength. The capacity calculation results, which have been averaged with respect to 1000 realizations, are shown in Fig. 2. Since the two curves for the results with and without adaptive optics are close, the subplots inside the figures magnify the results so that they are demonstrated more clearly. It can be seen from the sub-figures that the capacities of the systems behave differently with and without the channel information. When the channel information is not available, equal power is assumed for each sub-channel. The system with adaptive optics demonstrates an improved performance. However, when the channel information is available and the water filling algorithm has been used to optimize the power allocation, the system capacity is higher for the system without adaptive optics in the low SNR regime while the system with adaptive optics shows a superior performance in the high SNR regime. The threshold SNR for the performance transition is 5dB.

In order to explain the systems' behaviors in Fig. 2, the eigen value distribution of the capacity matrix **HH*** ^{H}* is shown in Fig. 3. The eigen values are sorted in the descending order and are averaged over 1000 realizations. In accordance with the conclusion in section 2.3 and 3.1, it can be seen that the mean eigen value is higher for the system with adaptive optics which explains the fact that the corresponding system capacity is higher when equal power is applied to each sub-channel. The adaptive optics tends to increase the smaller eigen values while decreasing the larger ones. This explains the fact that adaptive optics might deteriorate the capacity in the low SNR regime when the water filling algorithm is applied, because the algorithm tends to concentrate the power on the sub-channels with larger eigen values especially in the low SNR regime.

When C_{n}^{2} is 5 × 10^{−15} m^{-2/3}, the turbulence is relatively weaker. The corresponding capacity and eigen value analyses are shown in Fig. 4 and Fig. 5. Very similar results are demonstrated and the same conclusion is reached. It can be seen that the adaptive optics compensation improves the performance without the channel information (channels with equal power) and also with the channel information in the high SNR regime (channel power allocated with the water filling algorithm). The transition threshold SNR varies from 5dB to 0dB in comparison with the previous example. Below this SNR threshold, the system without adaptive optics behaves better if the water filling algorithm has been implemented with the available channel information. This is because the large eigen values are closer to each other for the two cases under the weaker turbulence. Therefore, the capacity degradation occurrence requires lower SNR for the case with adaptive optics and the fully acknowledged channel information.

When C_{n}^{2} increases to 10^{−13} m^{-2/3}, the turbulence becomes strong. In this case, the conclusion for the mean eigen value of the capacity matrix in section 2.3 will not be valid. The results shown in Fig. 6 demonstrate that the adaptive optics compensation will bring a deteriorated performance both for the cases with and without the channel information. The study on the eigen value distribution in Fig. 7 shows that the eigen values have been decreased by the adaptive optics compensation although the diagonal elements of the transfer matrix **H** have been increased. Henceforth, it is not recommended to use adaptive optics in the case of strong turbulence.

## 4. Summary

In summary, we have studied the free space coherent MIMO system capacity with and without the phase-only adaptive optics. It is found that in the case of weak/moderate turbulence, the system capacity improves with adaptive optics when the channel information is not available. When the full channel information can be obtained, the capacities of the systems with and without adaptive optics depend on the SNR. In the low SNR regime, the system without adaptive optics behaves better while in the high SNR regime, the system with adaptive optics outperforms the former. In the case of strong turbulence, the phase only adaptive optics deteriorates the capacity of the system both in the scenarios with and without the channel information. The results of this work can be further applied to the future design and analysis of the free space coherent optical MIMO transmission systems.

## Appendix

For the readers' convenience, some detailed information for the LG modes and its propagation under atmosphere is provided in the appendix.

#### A. The LG modes in free space

Without loss of generality, the LG modes are assumed in this work, because they form the complete orthonormal basis in free space and are frequently used in the free space SDM systems. Their distribution in free space can be represented by [5,11]

*r*,

*ϕ*,

*z*are the radius, the angle, and the propagation distance in the cylindrical coordinates respectively,

*w*

_{0}the initial beam waist, λ

_{0}the free space wavelength,

*m*and

*p*the azimuth order and the radial order of the LG polynomial

*L*.

_{p}^{m}Without atmosphere turbulence, the LG modes propagation obeys Eqs. (1-2) and they keep orthogonal to each other.

#### B. Atmosphere turbulence model

The impact of atmosphere turbulence on the propagating wave *φ* can be formulated by the following wave equation [11,19]

*k*

_{0}is 2π/λ

_{0}, Δ

*n*the zero-mean refractive index variation. It is a stationary random process and has its correlation function as

## Funding

National Natural Science Foundation of China (NSFC) (61775168); Shanghai Natural Science Foundation (16ZR1438600).

## References and links

**1. **J. Wang, J. Yang, I. M. Fazal, N. Ahmed, Y. Yan, H. Huang, Y. Ren, Y. Yue, S. Dolinar, M. Tur, and A. E. Willner, “Terabit free-space data transmission employing orbital angular momentum multiplexing,” Nat. Photonics **6**(7), 488–496 (2012). [CrossRef]

**2. **N. Zhao, X. Li, G. Li, and J. M. Kahn, “Capacity Limits of Spatially Multiplexed Free-Space Communication,” Nat. Photonics **9**(12), 822–826 (2015). [CrossRef]

**3. **K. Zhu, G. Zhou, X. Li, X. Zheng, and H. Tang, “Propagation of Bessel-Gaussian beams with optical vortices in turbulent atmosphere,” Opt. Express **16**(26), 21315–21320 (2008). [CrossRef] [PubMed]

**4. **Y. Zhang, M. Cheng, Y. Zhu, J. Gao, W. Dan, Z. Hu, and F. Zhao, “Influence of atmospheric turbulence on the transmission of orbital angular momentum for Whittaker-Gaussian laser beams,” Opt. Express **22**(18), 22101–22110 (2014). [CrossRef] [PubMed]

**5. **J. A. Anguita, M. A. Neifeld, and B. V. Vasic, “Turbulence-induced channel crosstalk in an orbital angular momentum-multiplexed free-space optical link,” Appl. Opt. **47**(13), 2414–2429 (2008). [CrossRef] [PubMed]

**6. **J. A. Anguita, M. A. Neifeld, and B. V. Vasic, “Spatial correlation and irradiance statistics in a multiple-beam terrestrial free-space optical communication link,” Appl. Opt. **46**(26), 6561–6571 (2007). [CrossRef] [PubMed]

**7. **J. Zhou, J. Zong, and D. Liu, “Coupled mode theory for orbital angular momentum modes transmission in the presence of atmosphere turbulence,” Opt. Express **23**(25), 31964–31976 (2015). [CrossRef] [PubMed]

**8. **J. Zhou, J. Zong, and D. Liu, “The Higher Order Statistics of OAM Modal Amplitudes Under Atmosphere Turbulence,” IEEE Photonics Technol. Lett. **28**(23), 2653–2656 (2016). [CrossRef]

**9. **N. Chandrasekaran and J. H. Shapiro, “Photon information efficient communication through atmospheric turbulence–Part I: Channel model and propagation statistics,” J. Lightwave Technol. **32**(6), 1075–1087 (2014). [CrossRef]

**10. **N. Chandrasekaran, J. H. Shapiro, and L. Wang, “Photon information efficient communication through atmospheric turbulence—Part II: Bounds on Ergodic classical and private capacities,” J. Lightwave Technol. **32**(6), 1088–1097 (2014). [CrossRef]

**11. **J. Zhou, J. Wu, J. Zong, and Q. Hu, “Optimal mode set selection for free space optical communications in the presence of atmosphere turbulence,” J. Lightwave Technol. **36**(11), 2222–2229 (2018). [CrossRef]

**12. **Y. Ren, G. Xie, H. Huang, N. Ahmed, Y. Yan, L. Li, C. Bao, M. P. J. Lavery, M. Tur, M. A. Neifeld, R. W. Boyd, J. H. Shapiro, and A. E. Willner, “Adaptive-optics-based simultaneous pre- and post-turbulence compensation of multiple orbital-angular-momentum beams in a bidirectional free-space optical link,” Optica **1**(6), 376–382 (2014). [CrossRef]

**13. **M. Li, M. Cvijetic, Y. Takashima, and Z. Yu, “Evaluation of channel capacities of OAM-based FSO link with real-time wavefront correction by adaptive optics,” Opt. Express **22**(25), 31337–31346 (2014). [CrossRef] [PubMed]

**14. **Y. Ren, G. Xie, H. Huang, L. Li, N. Ahmed, Y. Yan, M. P. J. Lavery, R. Bock, M. Tur, M. A. Neifeld, R. W. Boyd, J. H. Shapiro, and A. E. Willner, “Turbulence compensation of an orbital angular momentum and polarization-multiplexed link using a data-carrying beacon on a separate wavelength,” Opt. Lett. **40**(10), 2249–2252 (2015). [CrossRef] [PubMed]

**15. **A. Lapidoth, S. M. Moser, and M. A. Wigger, “On the capacity of free-space optical intensity channels,” IEEE Trans. Inf. Theory **55**(10), 4449–4461 (2009). [CrossRef]

**16. **J.-B. Wang, Q.-S. Hu, J. Wang, M. Chen, and J.-Y. Wang, “Tight bounds on channel capacity for dimmable visible light communications,” J. Lightwave Technol. **31**(23), 3771–3779 (2013). [CrossRef]

**17. **J.-Y. Wang, J. Dai, R. Guan, L. Jia, Y. Wang, and M. Chen, “Channel capacity and receiver deployment optimization for multi-input multi-output visible light communications,” Opt. Express **24**(12), 13060–13074 (2016). [CrossRef] [PubMed]

**18. **S. Moser, “Capacity results of an optical intensity channel with input-dependent Gaussian noise,” IEEE Trans. Inf. Theory **58**(1), 207–223 (2012). [CrossRef]

**19. **A. Belmonte, “Feasibility study for the simulation of beam propagation: consideration of coherent lidar performance,” Appl. Opt. **39**(30), 5426–5445 (2000). [CrossRef] [PubMed]

**20. **D. Tse and P. Viswanath, *Fundamentals of Wireless Communication*, Cambridge University Press (2005).