## Abstract

The average bit error rate (ABER) performance of an orbital angular momentum (OAM) multiplexing-based free-space optical (FSO) system with multiple-input multiple-output (MIMO) architecture has been investigated over atmospheric turbulence considering channel estimation and space-time coding. The impact of different types of space-time coding, modulation orders, turbulence strengths, receive antenna numbers on the transmission performance of this OAM-FSO system is also taken into account. On the basis of the proposed system model, the analytical expressions of the received signals carried by the *k*-th OAM mode of the *n*-th receive antenna for the vertical bell labs layered space-time (V-Blast) and space-time block codes (STBC) are derived, respectively. With the help of channel estimator carrying out with least square (LS) algorithm, the zero-forcing criterion with ordered successive interference cancellation criterion (ZF-OSIC) equalizer of V-Blast scheme and Alamouti decoder of STBC scheme are adopted to mitigate the performance degradation induced by the atmospheric turbulence. The results show that the ABERs obtained by channel estimation have excellent agreement with those of turbulence phase screen simulations. The ABERs of this OAM multiplexing-based MIMO system deteriorate with the increase of turbulence strengths. And both V-Blast and STBC schemes can significantly improve the system performance by mitigating the distortions of atmospheric turbulence as well as additive white Gaussian noise (AWGN). In addition, the ABER performances of both space-time coding schemes can be further enhanced by increasing the number of receive antennas for the diversity gain and STBC outperforms V-Blast in this system for data recovery. This work is beneficial to the OAM FSO system design.

© 2017 Optical Society of America

## 1. Introduction

In recent years, free-space optical (FSO) communication has attracted much attention because of its large bandwidth, license-free, low fabrication costs, and capability of providing an effective solution to the “last mile” problem [1–3]. In particular, multiplexing multiple independent data-carrying beams in a FSO system could substantially enable the total capacity to be multiplied by the number of beams under the same conditions [4,5]. Various methods have been proposed such as wavelength-division multiplexing (WDM), polarization-division multiplexing (PDM), space-division multiplexing (SDM) and so on [6–8]. For SDM in FSO, specifically, multiple-input multiple-output (MIMO) multiplexing is a well-established technology that involves the use of multiple transmitter and receiver aperture pairs and is known for larger capacity, better diversity gain, increased coverage area and stronger channel robustness compared to the conventional single-input single-output (SISO) system [6,7]. Another special approach is to introduce new degrees of freedom (DOF) such as a set of orthogonal modes, each of which carries an independent data stream with low inter-mode crosstalk that allows them (de)multiplexed with low inter-channel crosstalk efficiently [8,9]. In such a FSO system, multiple data-carried spatial orthogonal modes propagate through a single aperture pair coaxially and vortex light is just the beam with a set of orbital angular momentum (OAM) modes which can satisfy orthogonality among these beams [10,11]. In principle, OAM mode is a helically phased beam due to the phase variation of an OAM beam within a wavelength, which carries the OAM of $l\hslash $ per photon and could be characterized by$\mathrm{exp}(il\phi )$. Here *l* is known as the topological charge of the OAM state that can take theoretically any integer values, $\phi $ refers to the azimuthal angle around the propagation axis of the wave and $\hslash $ is the reduced Planck constant [8–13]. Very recently, a combination of OAM multiplexing and MIMO technology has been reported in [14,15]. In [14], a 16Gbit/s signal communication link was demonstrated using MIMO processing of two OAM modes on each of two transmitter/receiver antenna apertures. The results indicated that OAM multiplexing with MIMO technology is compatible and complementary with each other. Besides, OAM multiplexing combined with MIMO can enhance the capacity significantly. In [15], a two OAM multiplexing-based 2 × 2 FSO link using spatial diversity combined with MIMO equalization was studied and the result showed that the system capacity can be improved by the number of transmitted modes. Nevertheless, both theoretical and experimental investigations indicate that atmospheric turbulence will lead to the power of any single OAM mode spreading to other OAM modes in OAM-based FSO system because of the intensity fluctuation and phase twist, which will further destroy the orthogonality of OAM modes [9–17]. To overcome the impairment mentioned, lots of efforts have been devoted to the OAM-based FSO systems [18–20]. In [18], a mitigation scheme to improve the atmospheric turbulence tolerance of OAM-multiplexing FSO system was proposed with channel coding and wavefront correction. The results showed that the BER is improved greatly and the detrimental effect of atmospheric turbulence can be removed even in the relatively strong turbulence regime. In [19], an OAM-based FSO transmission system over strong atmospheric turbulence was demonstrated with low density parity check (LDPC). And inexpensive wave-front sensorless adaptive optics (AO) and OAM multiplexing with WDM were also considered to deal with phase distortion and improve the aggregate data rate. In [20], an atmospheric turbulence mitigation scheme was presented with minimum mean square error (MMSE) multiple-user detection (MUD) method, the numerical simulation results showed that this scheme can mitigate the effects of atmospheric turbulence and random noise simultaneously, thus enhancing the system performance. As is known, space-time coding (STC) is also an excellent technique which has been studied in FSO systems to obtain diversity gain over atmospheric turbulence [21–23]. In general, space-time coding in FSO system is used in two ways: transmitting multiplexing and transmit diversity. For transmitting multiplexing, Bell labs proposed a layered space-time architecture called vertical bell labs layered space-time (V-Blast) by the joint design of high data rate and capacity, and in which different symbol streams are transmitted through different transmit antennas at the same time [21,22]. Therefore, the receive antenna will detect a linear superposition of all symbol streams and recover them via proper signal processing. According to Ref [24], zero-forcing with ordered successive interference cancellation (ZF-OSIC) decoder has a good performance with low complexity in MIMO V-Blast system. For transmit diversity, a typical technology called space-time block codes (STBC) was generalized by Tarokh on the basis of Alamouti code which could achieve full diversity gain [25,26]. Contrary to V-Blast scheme, different duplicates of the same transmission signal is transmitted through different transmit antennas in STBC scheme [21,27]. In fact, both V-Blast and STBC schemes have been proposed to improve diversity gain and capacity in FSO systems these years. Besides, since the atmospheric turbulence distortion is random, the channel state information is quite important for FSO system, and it can be obtained by channel estimator and used for space time coding decoder at the receiver [24,28,29]. However, there are no published works on the mitigation of turbulence conditions for an OAM multiplexing-based MIMO FSO system using both space-time coding and channel estimation so far, to the best of our knowledge.

In this work, an atmospheric turbulence mitigation scheme considering space-time coding with channel estimation for OAM multiplexing-based FSO system is proposed. Each transmitter aperture contains two OAM modes with a modulated signal, resulting in double available channels. The analytical expressions of the received signals carried by the *k*-th OAM mode of the *n*-th receive antenna for V-Blast and STBC schemes are derived, respectively. With the help of channel estimator based on LS algorithm, the ZF-OSIC equalizer of V-Blast system and Alamouti scheme decoder of STBC are then adopted to mitigate the performance degradation induced by the atmospheric turbulence. After that, on the basis of V-Blast and STBC schemes, the ABER performance of the current system is analyzed with different modulation orders, turbulence strengths, and receive antenna numbers, respectively.

## 2. Theoretical model

In this section, a theoretical investigation of an OAM multiplexing-based MIMO FSO system is presented with two space-time coding schemes using channel estimation. System model is provided in section 2.1. Code design criteria of V-Blast and STBC schemes with OAM multiplexing are analytically derived in section 2.2, respectively. Section 2.3 shows the propagation model of OAM multiplexing-based space-time coding system with channel estimation over atmospheric turbulence.

#### 2.1 System model

The OAM multiplexing-based MIMO FSO system using space-time coding combined with channel estimation is proposed in Fig. 1. As is shown, the system is equipped with *M* transmitter and *N* receiver apertures. Each of the transmitter apertures possess *K* multiplexed OAM beams, resulting in *KM* OAM data channels in total. A modulated signal at 1550nm is generated and split into *KM* copies, and then the constant-amplitude training sequences are interpolated into each sub-copy. The spatial overlaps among OAM beams increase with the transmission distance because of the beam divergence from different apertures. The transmission process and signal processing strategies are different for V-Blast and STBC schemes. Some turbulence phase screens are placed every 50m to simulate atmospheric turbulence. At the receiver, the *K* distorted OAM beams due to the atmospheric turbulence from each aperture are demultiplexed with the help of the orthogonality of OAM modes. Channel estimator with the constant-amplitude training sequences are carried out with LS algorithm to obtain the channel state information, which is essential to predict the performance of this system. Then, different decoding strategies are employed at the receiver according to the selected space-time coding scheme to recover the received data streams.

#### 2.2 Code design criteria of OAM multiplexing-based V-Blast and STBC schemes

In this section, the code design criteria of OAM multiplexing-based V-Blast and STBC schemes are given, respectively.

### 2.2.1 Code design criterion of OAM multiplexing-based V-Blast scheme

In the OAM multiplexing-based V-Blast system, the split *KM* = 4 substreams are encoded into symbols and then fed to the corresponding OAM beams. The basic idea here is to exploit the multipath effectively, rather than mitigate it, by considering the multipath itself as a source of diversity that allows the parallel transmission of substreams from the same receiver. The V-Blast system uses multiplexing antenna arrays at both transmitter and receiver to provide high-capacity wireless communications. At the transmitter, the transmitted symbols can be represented with the help of [21] as

At a certain symbol instant, the equivalent channel matrix **H** with *KN × M* dimensional mutual correlation coefficient is expressed as follows:

**W**is the

*KN*-dimensional vector of the zero-mean additive white Gaussian noise (AWGN) with variance ${\sigma}_{n}^{2}={N}_{0}/2$. The received

*KN*-dimensional vector

**Y**contains a mixture of interfering signals of all transmit antennas, which are received synchronously and simultaneously.

In this work, a nonlinear detection technique with ZF-OSIC algorithm is proposed as a MIMO equalizer in the OAM multiplexing-based V-Blast system. And it is mainly based on subtraction of interference of the already detected elements by ZF algorithm of ${\overline{X}}_{VBlast-OAM}$, which is the strongest sub-stream in the receiver vector ${Y}_{VBlast-OAM}$ selected by OSIC algorithm. This will obtain a modified receiver vector where fewer interferers are present.

### 2.2.2 Code design criterion of OAM multiplexing-based STBC scheme

In the OAM multiplexing-based STBC system, the STBC coder is not only a coder but also a technique to achieve reliability of data transmission in FSO system. STBC with two transmit and receive antennas does not transmit the same symbol simultaneously for all transmit antennas, but with a delay in the transmissions [21]. That is, for the first time slot, the two modulated symbols denoted by ${x}_{1}$ and ${x}_{2}$ are simultaneously transmitted at antenna 1 and 2, respectively. Then, $-{x}_{2}^{*}$ and ${x}_{1}^{*}$ are transmitted at the consecutive second time slot, respectively. Here, $\ast $ represents the complex conjugate operation. Therefore, in each coding interval, the transmitted symbols can be defined on the basis of [22] as

**H**is given as

*k*-th OAM mode for the

*m*-th transmit antenna which is time invariant throughout the transmission of each row. That is, when the rows of matrix ${X}_{STBC-OAM}$ are transmitted, the channel transmission coefficients keep constant across two consecutive symbol transmission time slots. For this case, each symbol is transmitted twice to achieve data redundancy for efficient decoding at receiver. As a result, the signals after passing through the channel in two symbol periods can be written as a combination of Eq. (5) and Eq. (6)

*k*-th OAM mode of

*n*-th receive antenna. And${w}_{1}^{{l}_{1}}={w}_{1}^{{l}_{K}},{w}_{2}^{{l}_{1}}={w}_{2}^{{l}_{K}}$, owing to the fact that

*K*OAM beams are multiplexed and transmitted together. The above expression can be further written in the form of a matrix as follows:

**H**can be written as

#### 2.3 Propagation model of OAM multiplexing-based space-time coding system with channel estimation over atmospheric turbulence

In this section, the theoretical background of OAM propagation over atmospheric turbulence with channel estimation is presented first. Then, V-Blast and STBC schemes are introduced in the OAM multiplexing-based propagation model through atmospheric turbulence, respectively. Finally, the theoretical expressions of the received signals for V-Blast and STBC systems are derived, respectively.

### 2.3.1 Theoretical background of OAM propagation through atmospheric turbulence

As is known, the Laguerre-Gaussian (LG) beam with OAM is widely used in optical wireless communications. For a radial distance *r* from the propagation distance *z*, the optical field distribution is given in [30,31] as

*z*for beam waist with ${\omega}_{0}$. ${z}_{R}=\pi {\omega}_{0}^{2}/\lambda $ is the Rayleigh range, and $\lambda $ is the optical wavelength.

*l*denotes the radial and angular mode numbers, respectively.

The OAM modes defined above satisfies the orthogonality under the ideal condition [20], which could be used to separate OAM modes completely. However, the atmospheric turbulence will destroy the orthogonality between OAM modes inevitably and the information carried by the corresponding OAM spatial modes is also deformed. Actually, the atmospheric turbulence can be approximated by several turbulence phase screens following Von Karman optical spectrum. The spectrum of fluctuations in index of refraction ${\Phi}_{n}(k)$is given in [32] as

Under the Markov approximation, the phase spectrum following the index of refraction spectrum can be obtained with the help of [33] as

where $\Delta z$ denotes the propagation distance between the subsequent phase screens. By the time, on the basis of [34,35], the phase distribution of turbulent atmosphere can be evaluated as the Fourier transform of a complex random distribution according to Eq. (15) as follows:where $\mathrm{FFT}[\cdot ]$ denotes the Fourier transformation and C is an*N × N*array of complex random numbers with a mean of zero and a variance of one. This results in two independent phase screens from the real and imaginary parts of $\phi (x,y)$.

Due to the fact that all the propagation takes place in the *z*-direction using the electric field, the optical field distribution ${U}_{l}\left(r,\theta ,z\right)$ should be convert into the rectangular coordinate system and the amplitude of the beam can be represented by ${U}_{l}(x,y)$. Then, the beam is propagated using the Fourier optics to move between *x*-space and *k*-space.

Then, the amplitude of the electric field just prior to the first phase screen can be expressed as

Since the distortion is random, the channel state information can be obtained by channel estimation. The channel estimation scheme in this system can obtain optimal channel coefficient ${h}_{x}$ by minimizing the error matrix ${e}_{x}^{2}$ per symbol, here, ${e}_{x}^{2}$ can be written with the help of [24] as

*x*, and ${\mu}_{x}$ is the training sequence. The LS algorithm used for channel estimation is expressed as

### 2.3.2 Propagation through atmospheric turbulence with OAM multiplexing-based V-Blast scheme

At the receiver of the OAM multiplexing-based V-Blast system, the signal at each receive antenna is a linear superposition of *KM* distorted transmission signals, which are perturbed by the noise. Thus, at time *t*, the received signals at the *n*-th receive antenna can be expressed with the help of [2] as

*n*-th receive antenna, which is defined as

*n*-th receive antenna from the

*k*-th OAM mode of

*m*-th transmit antenna over the atmospheric turbulence can be written as

### 2.3.3 Propagation through atmospheric turbulence with OAM multiplexing-based STBC scheme

At the receiver of the OAM multiplexing-based STBC system, the signal at each antenna is a superposition of *K* deformed transmission signals that are mixed by noise. After passing through the channel, with the help of [2], the received multiplexing OAM modes at antenna *n* can be given by

*n*-th receive antenna from the

*k*-th OAM mode of

*m*-th transmit antenna in two symbol periods over the atmospheric turbulence can be written as

## 3. Simulation results

In this section, the impact of two space-time coding schemes (V-Blast and STBC) on the ABER performance of the OAM multiplexing-based FSO system is investigated in detail by numerical simulation. Here, the transmission distance *z* is equal to 1 km. The inner scale *l*_{0} and outer scale *L*_{0} are set to 0.001 and 20, respectively, and 20 screens are placed every 50m to simulate the atmospheric turbulence [15]. The optical wavelength is 1550 nm, the beam radius ${\omega}_{0}$ equals 3 cm, and the OAM modes are set as 3 and 5, respectively. In the simulation, a sequence of symbols at 1550nm is generated and modulated. Then, the constant-amplitude training sequences as pilot information are interpolated into the generated sequences [29]. After that, the sequences are split into four copies and encoded on the basis of the selected space-time coding scheme before they are fed to the corresponding OAM beams. At the receiver, with the help of channel estimation based on LS algorithm, the channel state information can be obtained to decode the received signal.

#### 3.1 OAM multiplexing propagation

Figure 2 presents the intensity and phase distributions of typical OAM modes (${l}_{1}=3,{l}_{2}=5$) with their propagation process. As can be seen, two OAM beams are generated and then spatially multiplexed by a beam splitter. The pseudo-random phase distribution is modeled by the Fourier transform method augmented with sub-harmonics, which also obeys the Von Karman optical spectrum statistics and can be characterized by its effective ${C}_{n}^{2}$ with a value of ${10}^{-14}$for moderate turbulence regime. According to ${r}_{0}={\left(0.423{k}^{2}{C}_{n}^{2}z\right)}^{-3/5}$, here, *r*_{0} equals 0.0785. The multiplexed OAM beam is distorted by the atmospheric turbulence after the propagation through turbulence phase screens, which will further destroy the orthogonality of OAM modes. Besides, it can be also found that OAM demultiplexing suffers from mode crosstalk. Therefore, the phase and amplitude of each OAM mode are destroyed by atmospheric turbulence severely.

#### 3.2 Channel estimation

Figure 3 shows the ABER performance of the OAM-based SISO FSO system with QPSK modulation at different SNRs. Under weak, moderate and strong turbulence conditions, the corresponding refractive-index structure constant ${C}_{n}^{2}$ equals${10}^{-15}{m}^{-2/3}$,${10}^{-14}{m}^{-2/3}$ and ${10}^{-13}{m}^{-2/3}$, respectively. The transmission distance *z* here is equal to 0.5 km. In this case, ${r}_{0}$ equals 0.4736, 0.1190 and 0.0299. As can be seen, the ABERs obtained by channel estimation have excellent agreement with those of turbulence phase screen simulations, which confirms the correctness of the proposed ABER model. It is also seen from the figure that the ABERs of this system increase with the increase of the refractive-index structure constant (from weak to strong turbulence). This indicates that the performance of the present system is degraded with the increase of atmospheric turbulence strengths, which has been also confirmed in OAM-based FSO links with MIMO processing technology [14,15]. For example, when the SNR equals 10dB, the ABERs are $2\times {10}^{-1}$, $5\times {10}^{-1}$ and $6\times {10}^{-1}$ under weak, moderate and strong turbulence conditions, respectively. This is because that the signals carried by vortex beam are deteriorated by the atmospheric turbulence, thus leading to the amplitude attenuation and phase destruction similar to the phenomenon observed in Fig. 2.

#### 3.3 Performance of the proposed OAM-FSO system

In Fig. 4(a), the ABER performance versus SNR of V-Blast and STBC schemes on the OAM multiplexing-based FSO system is plotted under different turbulence conditions. The atmospheric coherent diameter ${r}_{0}$ equals 0.3124, 0.0785 and 0.0197, respectively. For comparison, the simulated ABERs without space-time coding are also presented. As can be found, the ABER values increase with the increasing atmospheric turbulence strengths. Besides, both two space-time coding schemes can improve this FSO system performance significantly. For example, under moderate turbulence condition, when the SNR value is equal to 25dB, the ABERs for three cases of without space-time coding, with V-Blast only and with STBC only are approximately equal to $5\times {10}^{-1}$, $5\times {10}^{-3}$ and ${10}^{-6}$, respectively. This is because for the OAM multiplexing-based FSO system with space-time coding, the signal will be transmitted both in space and time, thus enhancing the transmit and receive diversity gains. In addition, when the space-time coding is adopted, the maximum information can be extracted as possible in the optimum way [27]. Furthermore, it can be seen from Fig. 4(a) that the ABER curve of the STBC system decreases faster than that of the V-Blast system for different turbulence strengths, which indicates that the STBC scheme outperforms V-Blast scheme in improving the ABER performance of this OAM FSO system. For example, in order to achieve the ABER of ${10}^{-3}$, the required SNRs are approximately equal to 30dB, 32dB and 40dB in V-Blast system under weak, moderate and strong turbulence conditions, respectively. Nevertheless, the required SNRs reduce to 5dB, 16dB and 25dB in STBC system under weak, moderate and strong turbulence conditions, respectively. This is because that the main difference between OAM-multiplexing-based V-Blast system and STBC system can be summarized as: V-Blast transmits more symbols while STBC transmits at most reliable symbol by means of diversity. Thus STBC could exploit data redundancy to achieve efficient decoding at the receiver. Figure 4(b) shows the constellations of QPSK signals propagation through the moderate turbulence. Figure 4(c) and 4(d) present the recovered constellations of signals at the receiver. As can be observed, the constellations become better defined by using space-time coding in this system with SNR equal to 32dB. Besides, the constellation performance of STBC scheme is remarkably superior to that of V-Blast scheme in this OAM system, which further confirms the result in Fig. 4(a).

Figure 5 illustrates the ABER performance against the SNR for this studied OAM multiplexing-based 2 × 2 FSO system in different atmospheric turbulence regimes with BPSK, QPSK and 8PSK modulations, respectively. The atmospheric coherent diameter ${r}_{0}$ equals 0.3124, 0.0785 and 0.0197, respectively. As be found from Fig. 5(a)-5(c), the ABER values of STBC scheme is lower than those of V-Blast scheme for different turbulence strengths. That is, the OAM-FSO system with STBC scheme performs better than that with V-Blast scheme in enhancing the ABER performance for different modulation orders. Besides, it can be also seen from this figure that as the modulation order increases, the ABER values increase for both schemes. For instance, under moderate turbulence condition, when the SNR is equal to 16dB, the corresponding ABERs of BPSK-based V-Blast system and STBC system are ${10}^{-2}$ and ${10}^{-5}$, respectively. Whereas the ABERs with QPSK and 8PSK modulations are approximately$2\times {10}^{-2}$, ${10}^{-4}$and${10}^{-1}$,${10}^{-3}$, respectively. This is because that for both schemes, the spectral efficiency is significantly enhanced with the increase of modulation order by double the modulation carrier phase intervals, thus leading to the decrease of the distance between two neighboring points in the signal constellation diagram and increase of the sensitivity to noise and interference. Thus, lower modulation order is preferred to offer acceptable ABER when transmitting signals at relatively low energy or transmitting small data. Nevertheless, higher modulation order can save the transmission bandwidth and enhance the spectral efficiency. Therefore, the selection of modulation order should be considered carefully for OAM-based FSO system design on the basis of the real application requirements.

Figure 6 presents the ABERs of QPSK modulation over the atmospheric turbulence for different receive antennas with V-Blast and STBC schemes. The atmospheric coherent diameter ${r}_{0}$ equals 0.3124, 0.0785 and 0.0197, respectively. The number of transmit antennas is set to be 2. As can be seen, when the number of transmit antennas (*M* = 2) is fixed, the ABER values have a significant decrease under all turbulence conditions with the increase of receive antennas for both schemes. For instance, to achieve the ABER of ${10}^{-4}$, the required SNRs in Fig. 6(a) are approximately equal to 39dB, 19dB, 12dB in V-Blast system with two, three and four receive antennas under weak turbulence condition, respectively. And the required SNR in Fig. 6(b) are approximately 8dB, 5dB, and 3.5dB in STBC system, respectively. This is due to the fact that for both systems, the receive diversity order is increased with the increase of receive antennas, which gives better system performance at the expense of a higher complexity. Besides, the ABER performance of STBC scheme is better than those of V-Blast system for different receive antennas. This is because for an independently and identically distributed turbulence channel with *M* transmit antennas and *N* receive antennas, the diversity order in V-Blast is more than *N*-*M* + 1 and less than *N*, while the diversity order in STBC can achieve a full diversity gain of *MN*. Therefore, as the receive antenna increases, the diversity gain in V-Blast is $2-2+1=1\to 4-2+1=3$, while in STBC is $2\times 2=4\to 4\times 2=8$ [15,22]. Therefore, it should be noted that the diversity gain in STBC is always higher than that in V-Blast system.

## 4. Conclusion

In this work, the ABER performance of an OAM multiplexing-based FSO system exploring space-time coding combined with channel estimation over atmospheric turbulence was investigated in detail. The structure of the proposed FSO system can be adjusted by changing the space-time coding schemes and number of receive antennas. With the help of channel estimator considering LS algorithm, the ZF-OSIC equalizer of V-Blast system and Alamouti decoder of STBC scheme are considered to mitigate the performance degradation induced by the atmospheric turbulence. Then, on the basis of the proposed system model, the analytical expressions of the receive signals from the *k*-th OAM mode of the *n*-th receive antenna for V-Blast and STBC schemes are derived, respectively. After that, the ABER performance of the current system was analyzed with different modulation orders, turbulence strengths, and the receive antenna numbers, respectively. The results showed that the ABERs by channel estimation match well with the turbulence phase screen simulation results and the ABER performances degrade with the increasing turbulence strengths. Both V-Blast and STBC schemes can improve the ABER performance obviously and the OAM multiplexing-based STBC system is superior to V-Blast system for data recovery. Moreover, the ABER performances of both space-time coding schemes will be improved by increasing the number of receive antennas, to increase the diversity gain. This work will be of good help for OAM multiplexing-based MIMO FSO system design.

## Funding

National Natural Science Foundation of China (Grant No.61474090); Fundamental Research Funds for the Central Universities (Grant No. JB160105); 111 Project of China (Grant No. B08038).

## References and links

**1. **Z. Qu and I. B. Djordjevic, “High-speed free-space optical continuous-variable quantum key distribution enabled by three-dimensional multiplexing,” Opt. Express **25**(7), 7919–7928 (2017). [CrossRef] [PubMed]

**2. **C. Kai, P. Huang, F. Shen, H. Zhou, and Z. Guo, “Orbital angular momentum shift keying based optical communication system,” IEEE Photonics J. **25**(7), 7919–7928 (2017).

**3. **S. Li and J. Wang, “Adaptive free-space optical communications through turbulence using self-healing Bessel beams,” Sci. Rep. **7**, 43233 (2017). [CrossRef] [PubMed]

**4. **L. Li, G. Xie, Y. Ren, N. Ahmed, H. Huang, Z. Zhao, P. Liao, M. P. J. Lavery, Y. Yan, C. Bao, Z. Wang, A. J. Willner, N. Ashrafi, S. Ashrafi, M. Tur, and A. E. Willner, “Orbital-angular-momentum-multiplexed free-space optical communication link using transmitter lenses,” Appl. Opt. **55**(8), 2098–2103 (2016). [CrossRef] [PubMed]

**5. **Y. Ren, Z. Wang, P. Liao, L. Li, G. Xie, H. Huang, Z. Zhao, Y. Yan, N. Ahmed, A. Willner, M. P. J. Lavery, N. Ashrafi, S. Ashrafi, R. Bock, M. Tur, I. B. Djordjevic, M. A. Neifeld, and A. E. Willner, “Experimental characterization of a 400 Gbit/s orbital angular momentum multiplexed free-space optical link over 120 m,” Opt. Lett. **41**(3), 622–625 (2016). [CrossRef] [PubMed]

**6. **H. H. Lu, C. Y. Lin, T. C. Lu, C. A. Chu, H. H. Lin, B. R. Chen, C. J. Wu, and W. S. Tsai, “150 m/280 Gbps WDM/SDM FSO link based on OEO-based BLS and afocal telescopes,” Opt. Lett. **41**(12), 2835–2838 (2016). [CrossRef] [PubMed]

**7. **M. Fiorani, M. Tornatore, J. Chen, L. Wosinska, and B. Mukherjee, “Spatial division multiplexing for high capacity optical interconnects in modular data centers,” Opt. Commun. Netw. **9**(2), A143–A153 (2017). [CrossRef]

**8. **H. Huang, Y. Cao, G. Xie, Y. Ren, Y. Yan, C. Bao, N. Ahmed, M. A. Neifeld, S. J. Dolinar, and A. E. Willner, “Crosstalk mitigation in a free-space orbital angular momentum multiplexed communication link using 4×4 MIMO equalization,” Opt. Lett. **39**(15), 4360–4363 (2014). [CrossRef] [PubMed]

**9. **C. Chen, H. Yang, S. Tong, and Y. Lou, “Changes in orbital-angular-momentum modes of a propagated vortex Gaussian beam through weak-to-strong atmospheric turbulence,” Opt. Express **24**(7), 6959–6975 (2016). [CrossRef] [PubMed]

**10. **A. E. Willner, “Recent advancements in optical orbital-angular-momentum multiplexing,” *in Proceedings of Latin America Optics and Photonics Conference* (OSA 2014), pp. 16–21. [CrossRef]

**11. **Y. Ren, G. Xie, H. Huang, N. Ahmed, Y. Yan, L. Li, C. Bao, P. J. Lavery, M. Tur, M. A. Neifeld, R. W. Boyd, J. H. Shapiro, and A. E. Willner, “Adaptive-optics-based simultaneous pre- and post-turbulence compensation of multiple orbital-angular-momentum beams in a bidirectional free-space optical link,” Optica **1**(6), 376–382 (2014). [CrossRef]

**12. **S. Zheng, X. Hui, J. Zhu, H. Chi, X. Jin, S. Yu, and X. Zhang, “Orbital angular momentum mode-demultiplexing scheme with partial angular receiving aperture,” Opt. Express **23**(9), 12251–12257 (2015). [CrossRef] [PubMed]

**13. **L. Zhu, C. Yang, D. Xie, and J. Wang, “Demonstration of km-scale orbital angular momentum multiplexing transmission using 4-level pulse-amplitude modulation signals,” Opt. Lett. **42**(4), 763–766 (2017). [CrossRef] [PubMed]

**14. **Y. Ren, L. Li, G. Xie, Y. Yan, Y. Cao, H. Huang, N. Ahmed, M. J. Lavery, Z. Zhao, C. Zhang, M. Tur, M. Padgett, G. Caire, A. F. Molisch, and A. E. Willner, “Experimental demonstration of 16Gbit/s millimeter-wave communications using MIMO processing of 2 OAM modes on each of two transmitter/receiver antenna apertures,” *in*Proceedings of Global Communications Conference (IEEE, 2014), pp. 3821–3826.

**15. **Y. Ren, Z. Wang, G. Xie, L. Li, A. J. Willner, Y. Cao, Z. Zhao, Y. Yan, N. Ahmed, N. Ashrafi, S. Ashrafi, R. Bock, M. Tur, and A. E. Willner, “Atmospheric turbulence mitigation in an OAM-based MIMO free-space optical link using spatial diversity combined with MIMO equalization,” Opt. Lett. **41**(11), 2406–2409 (2016). [CrossRef] [PubMed]

**16. **G. Zhu, Y. Chen, Y. Liu, Y. Zhang, and S. Yu, “Characterizing a 14 × 14 OAM mode transfer matrix of a ring-core fiber based on quadrature phase-shift interference,” Opt. Lett. **42**(7), 1257–1260 (2017). [CrossRef] [PubMed]

**17. **A. Jurado-Navas, A. Tatarczak, X. Lu, J. J. Olmos, J. M. Garrido-Balsells, and I. T. Monroy, “850-nm hybrid fiber/free-space optical communications using orbital angular momentum modes,” Opt. Express **23**(26), 33721–33732 (2015). [CrossRef] [PubMed]

**18. **S. Zhao, L. Wang, L. Zou, L. Gong, W. Cheng, B. Zheng, and H. Chen, “Both channel coding and wavefront correction on the turbulence mitigation of optical communications using orbital angular momentum multiplexing,” Opt. Commun. **376**, 92–98 (2016). [CrossRef]

**19. **Z. Qu and I. B. Djordjevic, “500 Gb/s free-space optical transmission over strong atmospheric turbulence channels,” Opt. Lett. **41**(14), 3285–3288 (2016). [CrossRef] [PubMed]

**20. **L. Yang, S. Zhao, and L. Zou, “Turbulence mitigation scheme using MMSE multiple-user detection for orbital angular momentum multiplexing communications,” *in Proceedings of IEEE 6th International Conference on Wireless, Mobile and Multimedia Networks (ICWMMN 2015)* (IEEE 2015), pp. 39–42.

**21. **B. Vucetic and J. Yuan, *Space-Time Coding* (Wiley, 2003).

**22. **M. M. Kamruzzaman, M. Wang, W. He, and X. Peng, “Relay assisted adaptive MIMO communication using STBC, VBLAST, MLSTBC and HYBRID-STBC-VBLAST for MAC,” *in Proceedings of IEEE 30th International Conference on Advanced Information Networking and Applications Workshops (WAINA 2016)* (IEEE 2016), pp. 349–354. [CrossRef]

**23. **V. Tarokh, H. Jafarkhani, and A. R. Calderbank, “Space-time block coding for wireless communications: performance results,” IEEE J. Sel. Areas Comm. **17**(3), 451–460 (1999). [CrossRef]

**24. **M. Jankiraman, *Space-Time Codes and MIMO Systems* (Artech House, 2004).

**25. **V. Tarokh, H. Jafarkhani, and A. R. Calderbank, “Space-time block codes from orthogonal designs,” IEEE Trans. Inf. Theory **45**(5), 1456–1467 (1999). [CrossRef]

**26. **M. K. Simon and V. A. Vilnrotter, “Alamouti-type space-time coding for free-space optical communication with direct detection,” IEEE Trans. Wirel. Commun. **4**(1), 35–39 (2005). [CrossRef]

**27. **N. Petrellis, “STBC-OFDM communication systems with sub-sampling support,” *in Proceedings of IEEE 5th International Conference on Modern Circuits and Systems Technologies (MOCAST 2016)* (IEEE 2016), pp. 1–4. [CrossRef]

**28. **S. J. Lee, “Effect of Least Square channel estimation errors on achievable rate in MIMO fading channels,” IEEE Commun. Lett. **11**(11), 862–864 (2007). [CrossRef]

**29. **J. Lin, “Least-Squares Channel Estimation for Mobile OFDM Communication on Time-Varying Frequency-Selective Fading Channels,” IEEE Trans. Vehicular Technol. **57**(6), 3538–3550 (2008). [CrossRef]

**30. **M. Cheng, L. Guo, J. Li, Q. Huang, Q. Cheng, and D. Zhang, “Propagation of an optical vortex carried by a partially coherent Laguerre-Gaussian beam in turbulent ocean,” Appl. Opt. **55**(17), 4642–4648 (2016). [CrossRef] [PubMed]

**31. **V. P. Aksenov, V. V. Kolosov, and E. C. Pogutsa, “Random wandering of laser beams with orbital angular momentum during propagation through atmospheric turbulence,” Appl. Opt. **53**(17), 3607–3614 (2014). [CrossRef] [PubMed]

**32. **S. Fu and C. Gao, “Influences of atmospheric turbulence effects on the orbital angular momentum spectra of vortex beams,” Photonics Res. **4**(5), B1–B4 (2016). [CrossRef]

**33. **S. M. Zhao, J. Leach, L. Y. Gong, J. Ding, and B. Y. Zheng, “Aberration corrections for free-space optical communications in atmosphere turbulence using orbital angular momentum states,” Opt. Express **20**(1), 452–461 (2012). [CrossRef] [PubMed]

**34. **J. D. Strasburg and W. W. Haroer, “Impact of atmospheric turbulence on beam propagation,” in Proceedings of Laser Systems Technology (SPIE, 2004), pp. 5413–5424.

**35. **J. D. Schmidt, *Numerical Simulation of Optical Wave Propagation* (2010).