## Abstract

In order to enhance the channel capacity and spectrum efficiency, the technology of space division multiplexing (SDM) has become research hotspot in optical communications. In this paper, a new encoding/decoding concept is proposed by the states of vector modes (polarized direction, rotational direction of phase and topological charge) for vortex beam propagating. To support encoded vector modes propagating in fiber, an OAM fiber with air-core structure is designed for encoding/decoding. Meanwhile, a new mode recognition method of judging states of the received vector modes is presented by the technology of digital image processing and digital signal processing (DSP). To verify the feasibility of encoding/decoding, an experimental platform to verify that encoded 16-QAM signal (0010_1100_1110_1010) is established. The vector modes of OAM beams can be propagated in OAM fiber with the length of 80 cm, and the received signal can be decoded to 16-QAM by image processing successfully. In addition, we also evaluate and analyze the influence factor (OAM fiber length and bit rate) on the transmitting performance in terms of BER, crosstalk and constellation figures.

© 2017 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

Recently, vortex beams with the characteristic of orbital angular momentum (OAM) have been concerned as research focus for the potential ability of enhancing the channel capacity. It is generally known that multiplexing techniques have been widely used to increase the channel capacity in optical and wireless communications, which mainly contain Wavelength Division Multiplexing (WDM), Time Division Multiplexing (TDM), Polarization Division Multiplexing (PDM), Frequency Division Multiplexing (FDM), etc. The vortex beams possess the helical phase wave-front in the form of$\mathrm{exp}(il\theta )$, where $l$ is the topological charge with the units of $\hslash $(simplify the Planck’s constant), $\theta $is the azimuth [1–5]. The phase of vortex beams possesses the phase singular, and the intensity distribution shows the shape of doughnut. The phases of beams with different $l$ are mutually orthogonal to each other, and the value of $l$ is unlimited in theory [6-7]. Therefore, it can provide the new domain for multiplexing, which is also called Space Division Multiplexing (SDM). The technology of SDM is potential to enhance the channel capacity and spectral efficiency [8].

Moreover, vortex beams are also potential to be used to encode and decode with OAM states, LP modes and the abundant states of vector modes, when it propagates in the OAM fiber. The first two have been reported by Zhu’s group and Du’s group [1,5]. Zhu’s group showed that the images were transmitted by encoding/decoding with superposition of spatial modes (LP01, LP11a, LP11b and LP11a + i × LP11b) in km-scale few-mode fiber [1]. Du’s group demonstrated that the information could be encoded and decoded with different $l$($l=\pm 1$,$\pm 3$, …,$\pm 15$) in high-dimensional for representing hexadecimal data in a free-space optical communication [5]. However, the last one has never been reported. With the development of OAM fiber (ring-core, hollow fiber, etc.), the performance of OAM propagation has been significantly improved, and more vector modes can be supported by OAM fiber with less modal couple and energy leakage [8–10].

In this paper, the new methods of encoding and decoding are demonstrated by the states of OAM combined with vector modes for transmitting information, which have never been reported. When the OAM beams propagate in an OAM fiber, the combined vector modes of HE and EH are excited in the OAM fiber, which are different from the scalar modes of LP. Compared to Zhu’s report and Du’s report, the same functions can be realized only by using 2 OAM states and 8 OAM states combined with vector modes. Therefore, it is of great significant to reduce cost and improve the performance on encoding/decoding efficiency by the new methods. In addition, as another important innovation, an OAM fiber with the structure of air-core is designed and fabricated, which can support up to 26 vector modes for encoding/decoding.

## 2. The proposed scenario of encoding/decoding based on vector modes

According to the optics theory, angular momentum of photon is comprised of spin angular momentum (SAM) and orbital angular momentum (OAM) [11-12]. The two kinds of angular momentum are different. The SAM represents the rotational direction of the polarization state of vortex beams, which includes left-hand or right-hand circular polarization [13-14]. The OAM contains the rotational direction (clockwise or counterclockwise) of helical phase and topological charge. The value of $l$ and the sign of $l$ in$\mathrm{exp}(il\theta )$ denote the topological charge and rotational direction of OAM phase (field), respectively [15]. When the vortex beams propagate in the fiber, the OAM modes have to be converted to common eigenmodes (vector modes) for propagating in OAM fiber. OAM modes can be defined as$OA{M}_{\pm l,m}^{\pm}$, where $m$ denotes the number of concentric rings in the intensity profile of OAM mode, $l$ is the topological charge [16–19]. The symbol of $\pm $ in the superscript and subscript of $OA{M}_{\pm l,m}^{\pm}$ depicts the direction of OAM circular polarization state and the rotational direction of OAM field (phase), respectively.

In fiber, the vector mode mainly contains TE, TM, HE and EH mode, of which HE and EH mode can be called the basis mode or eigenmodes for OAM modes [16-17]. The OAM modes can be expressed in one of the two vector modes (EH or HE) by linear combination, as shown in Eq. (1) and Eq. (2) [2]. It is not difficult to find that the direction of circular polarization is consistent with the rotational direction of phase, when the OAM mode is comprised of HE mode in Eq. (1). Otherwise, if the OAM mode is comprised of EH mode, the direction of circular polarization is contrary to the rotational direction of phase in Eq. (2).

In order to further illustrate the linear combination of vector modes in Eq. (1) and Eq. (2), the classification chart has been plotted in Fig. 1. Figures 1(a)-1(d) demonstrate the relationship of circular polarization and rotary phase, which are consistent with the consequence of Eq. (1) and Eq. (2) by different linear combination based on vector modes, respectively.

The abundant states of the polarization, topological charge and the rotational direction of phase are all potential physical quantity to be used to encode or decode information for optical fiber communication system. According to the above discussion about polarization, topological charge and phase, each OAM with different topological charge can be described as four different states, i.e., right-hand circular polarization, left-hand circular polarization, counterclockwise phase, and clockwise phase for a certain $l$($l\ne 0,1$). If $l\text{=}0$, the vector modes can be only described in the form of vector HE mode, $OA{M}_{0,m}^{\pm}=H{E}_{1,m}^{even}+jH{E}_{1,m}^{odd}$, which has only two circular polarization states. It doesn’t belong to the domain of OAM modes, but belong to fundamental vector modes of $H{E}_{1,m}$. If $l\text{=1}$, the vector modes can be described with two combined vector modes in the form of $OA{M}_{1,m}^{\text{+}}=H{E}_{2,m}^{even}\text{+}jH{E}_{2,m}^{odd}$ and $OA{M}_{-1,m}^{-}=H{E}_{2,m}^{even}-jH{E}_{2,m}^{odd}$ with the same direction of polarization and phase. Furthermore, if $l\ge 2$, there are four different combining methods of $OA{M}_{\pm l}^{\pm}$ by vector modes. For example, if $l\text{=3}$, the OAM modes can be depicted as $OA{M}_{3,m}^{\text{+}}=H{E}_{4,m}^{even}\text{+}jH{E}_{4,m}^{odd}$,$OA{M}_{3,m}^{-}=E{H}_{2,m}^{even}\text{+}jE{H}_{2,m}^{odd}$,$OA{M}_{-3,m}^{-}=H{E}_{4,m}^{even}\text{-}jH{E}_{4,m}^{odd}$, and $OA{M}_{-3,m}^{\text{+}}=E{H}_{2,m}^{even}\text{-}jE{H}_{2,m}^{odd}$ by the linear combination of vector modes based on Eq. (1) and Eq. (2).

The above described characteristics of OAM are potential to be used to encode/decode for information, thus we propose a new encoding/decoding scheme by topological charge, the direction of circular polarization, and the direction of rotary phase. The concept of encoding/decoding information by the linear combined vector modes is shown in Fig. 2. The whole process of encoding and decoding can be divided into five steps, i.e., encoding-convert, map-modulation, propagation, demodulation (de-mapping), and decoding-convert.

Firstly, the transmitted data is converted to binary sequence with the length of 4 or 6 bit as a group for expressing data of 0-15 or 0-64. The binary sequences are located in the 3rd-6th bit and 1st-6th bit, respectively. After that, the converted original sequences are encoded with the rules in the left of Fig. 2.

Secondly, the encoded sequences are mapped to the OAM mode in the form of $OA{M}_{\pm L,m}^{\pm}$ based on the combination of vector modes. It should be noted that the valve of $m$ is set as a fixed value of 1 in this paper for simplicity. According to the mapped $OA{M}_{\pm L,m}^{\pm}$beams, the relevant direction of polarization states, rotational direction of phase, and topological charge are modulated and generated by the spatial light modulator (SLM) and polarizer. The SLM loaded complex phase pattern is used to generate the rotational direction of phase and topological charge for OAM beams. The polarizer can control the polarized directions of OAM, which include left-hand and right-hand circular polarization.

Thirdly, the modulated OAM beams are coupled into the OAM fiber by objective lens and beam expander. The incident free-space OAM beams are propagated in the OAM fiber with special construction by exciting the vector modes for keeping the states of circular polarization. The relationships between converted free-space OAM beams and vector mode beams are consistent with the rules, which have been shown in Fig. 2 and discussed by the above.

Fourthly, the received OAM beams in the form of linear combination of vector modes is shined into free-space in the terminal of OAM fiber, and demodulated by the technology of image processing and digital signal processing (DSP) based on the received images by camera.

Finally, the demodulated beams are converted into electrical signal by the technology of image processing. Meanwhile, the data is de-mapped into original consequence and evaluated for the transmitting quality by offline processing.

## 3. The design of OAM fiber

In order to support the plenty of OAM modes, the excellent performance on the effective index separation between the vector modes is of great significance for encoding/decoding information based on states of vector modes, i.e., the direction of polarization states, the rotational direction of phase, and topological charge. In addition, the other parameter of crosstalk (mode couple) between the modes needs to be considered for improving the quality of propagation of encoded information in the OAM fiber. It has been popularly recognized that a high contrast in refractive indices is beneficial to obtain good modes separation, which can reduce the modes coupling and prevent vector modes degenerating into LP modes. In this paper, we use the linear combination of vector modes (EH, HE) to represent the OAM modes in the fiber. The effective index separation between EH and HE in a group must be greater than ${10}^{-4}$ for preventing the vector modes coupling, which is consistent with the rules of polarization maintaining fiber. The contrast of refractive index is determined by the fiber structure, manufacturing material, and the manufacturing procedure. Since the refractive indices of air or special fiber core approximately equal to 1, and the refractive indices of doped silicon dioxide with other chemical elements in annular region of ring-core or air-core fiber can reach more than 1.45, the ring-core fiber and air-core fiber may possess the higher contrast of refractive index.

In this paper, the OAM fiber with a step-index profile is designed to support plenty of vector modes which can be used to encode/decode by the states of vector modes. The constraint conditions of maximum and minimum refractive indices are designed for supporting sufficient vectors modes which possess large enough of effective index separation among the groups of vector modes. After that, we design the physical size of fiber profile which includes the radius of each circular region (layer). The procedure of design OAM fiber can be combined with gradually adding layers and checking the performance of the effective index separation. According to the consequence of checking, we modified the design synchronously. The designed final profile of OAM fiber with an annular shape is shown in Fig. 3. Due to the imperfections of the fabrication procedure, we have to adopt a compromise approach to solve the question of balance between the supported maximum number of vector modes and the separation of effective refractive indices in the designing procedure. According to the above assumption of the parameter,$m=1$, the doped area is designed to be very thin for supporting the intensity profile of single ring shape. Meanwhile, it can also suppress the crosstalk among the vector modes belonged to different groups and it is benefit to the development of SDM. Moreover, it also guarantees the large enough of separation of vector modes over the whole wavelengths of C-band. The designed profiles of physical structure and refractive indices of OAM fiber are shown in Figs. 3(a) and 3(b), respectively.

The whole profile of physical structure of OAM fiber can be divide into four layers. The refractive indices of layers from the innermost layer to the outermost layer are around 1.481, 1.487, 1.451 and 1.455, respectively. The OAM fiber is fabricated by the technology of modified chemical vapor deposition (MCVD) and pull with doping different chemical elements for adjusting the refractive indices in each layer, such as SiO2, P2O5, GeO2, and F in different layers. We use finite element analysis in the COMSOL software to compute the effective refractive index of fiber eigenmodes as well as the electric field and phase distribution of OAM modes. We also use instruments to evaluate and measure the refractive indices. With the help of the software of COMSOL and the equipment of refracted near filed analyzer (NR-9200HR), it demonstrates that the measured refractive index is consistent with simulated value on the whole, and the difference between the two refractive indices main be caused by the fabrication process in Fig. 3(b).

According to the profile of physical structure of OAM fiber, the effective indices and group indices for all supported vector modes are calculated. As shown in Fig. 4, it is obvious that the supported vector modes contains TE_{0,1}, HE_{1,1}, HE_{2,1}, HE_{3,1}, TM_{0,1}, EH_{1,1}, HE_{4,1}, EH_{2,1}, HE_{5,1}, EH_{3,1}, HE_{6,1}, EH_{4,1}, HE_{7,1}, EH_{5,1}, HE_{8,1}, EH_{6,1}, HE_{9,1}, and EH_{7,1} in Fig. 4(a). However, some vector modes can’t be combined to OAM modes, such as TE_{0,1}, HE_{1,1} and TM_{0,1}.The vector mode of HE_{2,1} only can be combined to two OAM modes, which contains$OA{M}_{1,1}^{\text{+}}$and$OA{M}_{-1,1}^{-}$. In order to coexist in fiber, the minimum separation of refractive index needs to be more than ${10}^{\text{-}4}$ for avoiding the mode couple and crosstalk. Figure 4(a) shows the good performance on the separation of effective indices, and the minimum separation is around $1.09\times {10}^{-4}$ between TE_{0,1} and HE_{1,1} in 1550nm. The modes of HE_{9,1} and EH_{7,1} are not considered, because they only support the low wavelength. The each of OAM modes combined by the EH and HE with the topological from 2 to 7 contains 4 combination ways. The OAM modes with the topological charge 1 only can be combined by 2 ways with EH modes or HE modes. So all vector modes except for TE_{0,1}, HE_{1,1}, TM_{0,1}, HE_{9,1} and EH_{7,1} can be combined to 26 different OAM modes with the topological charge from 1 to 7.

Figure 4(b) depicts the calculated group indices based on the prior N_{eff}, which also shows that the OAM modes group with larger topological charge possess larger group delay and usually transmit more slowly than those with lower larger topological. In addition, it also demonstrates that the same topological charge of OAM modes beams combined by HE modes with the same direction of polarization and the rotational phase propagate more slowly than those combined by EH modes with the opposite direction of polarization and the rotational phase.

## 4. Experimental setup and discussion

The method of encoding/decoding by the states of vector modes (i.e., polarized direction, rotational direction of phase and topological charge) has been discussed in section 2, which can provide additional dimension for encoding/decoding. Therefore, it can be used to enhance the efficiency of encoding/decoding with the limited optical devices. The OAM fiber with the structure of air-core has also been designed for supporting sufficient vector modes in section 3. The effective index separation, effective indices, and group index of the vector modes have been calculated and measured by software and instruments for the design OAM fiber, which can support at least 26 vector modes for encoding/decoding. Next, an experimental platform is established for verifying that the information can be encoded/decoded by the states of vector modes in transmitting terminal; the vector modes can be propagated in the designed OAM fiber with relative optical components by exciting the vector modes; according to the captured images, the received vector modes can be detected and analyzed by two cameras and the modules of offline processing in the receiving terminal. To verify the feasibility of the scheme of the whole system, the encoded 16-QAM signal (0010_1100_1110_1010) in the form of combining vector modes were propagated through OAM fiber with the length of 80 cm by exciting the vector modes. Meanwhile, the received signals were decoded and translated to 16-QAM by image processing in the receiving terminal. Figure 5 demonstrates the whole process of the experimental scheme, which can be divided into 5 parts, which contains the source of beams, encoding information, SLM, propagation, and decoding information.

Firstly, the generated Gaussian beam by laser is amplified and filtered to guarantee the enough intensity and purity by EDFA and BPF in single mode fiber (SMF). After that, the beam is split into 3 lines for different purpose, one is used to illuminate the SLM for generating the OAM beams and the other two lines are used as the referred beams to interfere the received the OAM beams for analyzing the phase, polarization and topological charge. The top branch beam shines into free-space by the collimator (Col) and be expanded to match the size of OAM fiber for enhancing the efficiency of couple. Following, the polarization of beams are adjusted into linearity by Col. and half wave plate (HWP), and the beams are irradiated on the SLM for generating OAM beams.

Secondly, the generated PRBS is modulated to the signal of 16-QAM, then the signal is converted to sequence group with a certain length of 4 bits for hexadecimal encoding. Following, the sequence group is mapped into the encoding sequence from 3rd bit to 6th bit based on the above of encoding rules in Fig. 2. It’s worth noting that the 1st and 2nd are not necessary to be used, because the sequence group’s length only is 4 bits for 16-QAM, which can be used as expanded bits to represent 32 or 64-QAM, respectively. The mapped sequences are divided into two lines to control SLM and circular polarizer. The 3rd and 4th bit in the mapped sequence are combined as the control signal to select the topological charge, and the 5th bit is used as control signal to select the rotational phase direction of OAM beams for SLM. Furthermore, the 6th bit in the mapped sequence is used as the control signals to control the polarized direction for circular polarizer. The 1st and 2nd bit are reserved for future use.

Thirdly, the SLM is used to generate alternant OAM beams based on the referred the control signal for selecting the topological charge and the rotational direction of phase. In this paper, we design 4 kinds of fork grating holograms for SLM to generate OAM beams. The topological charges contain 2, 4, 6 and 8 based on the 3rd-4th bit in the mapped sequence. The value of the 5th bit in the mapped sequence is used to represent the rotational phase (i.e., clockwise and counterclockwise). Thus, the topological charge can be depicted as$\pm 2$, $\pm 4$, $\pm 6$ and $\pm 8$. The sign of “+” and “-” represent the counterclockwise and clockwise direction of phase, respectively.

Fourthly, the OAM beams in free-space are illuminated into the circular polarizer and focusing lens (FL) for configuring circularly polarized direction, which contains left-hand or right-hand based on the control signal of polarized direction that located in the 6th bit in the mapped sequence. Following, the OAM beams are coupled into the OAM fiber, then the related vector modes are excited based on the incident OAM beams. The relationships of OAM modes and combined vector modes have been discussed in section 2. It is worth noting that the propagating length of beams modulated by the SLM in free-space and OAM fiber with the air-core structure is around 20 cm and 80 cm.

Finally, the beams with the OAM modes are shined into free-space again from the OAM fiber by the collimator and QWP for better directivity and polarization. Next, the OAM beams are split into 2 parts based on the different circular polarized direction by the polarizing beam splitter (PBS), which also can be called left-hand polarized OAM beams and right-hand polarized OAM beams from left to right in Fig. 5, respectively. After that, the two branch beams with different polarized direction are combined with the relevant referred Gaussian beams by a beam splitter (BS), then we can observe the interference figures. Therefore, we can decode the information (direction of circular polarization, topological charge, and rotational direction of phase) based on the observed interference figures by the technology of image processing and DSP.

The above has been discussed about the whole procedure of experiment in detail on source of beams, information encoding, SLM, propagation, and encoding/decoding information. Further, Pseudo Random Binary Sequences (PRBS) are employed to verify the performance of whole system. According to the original polynomial (${x}^{7}+{x}^{6}+1$) of PRBS7, the generated PRBS (0010, 1100, 1110, 1010, ……) with the initial values of 1111 can be depicted as 2, C, E, A, ……, in the form of hexadecimal. According to the relationship of map in Fig. 2, the sequence of 2, C, E and A can be described as$OA{M}_{+2,1}^{-}$,$OA{M}_{-8,1}^{-}$, $OA{M}_{-8,1}^{+}$,$OA{M}_{+6,1}^{-}$by the combining vector modes.

According to received images from the camera 1# and 2#, we can observe the interference fringes of the received OAM modes in Fig. 6. To analyze the polarized direction, rotational direction of phase and topological charge, we carefully distinguish the origin of received images from camera 1# or 2#, then we design reference lines (L1, L2) and reference circle (C) for analyzing the intensity distribution in the form of equipotent pulse sequences.

Figure 6(a1) depicts the consequence of L1_pulse = L2_pulse, which demonstrates that there is no rotational phase distribution in Fig. 6 (a3). Figure 6(a2) describes the expression of C_pulse = 0, which means that there are no interference fringes distributed on the circle C in Fig. 6(a3). Therefore, the received Fig. 6(a3) doesn’t belong to the OAM beams, which may be induced by the referred beams. Figure 6(a5) depicts that the expression of C_pulse = 2, which means that there are 2 interference fringes distributed on the circle C in Fig. 6(a4). Figure 6(a6) demonstrates that the consequence of L1_pulse < L2_pulse means that there’s counterclockwise rotational phase in Fig. 6(b4). Furthermore, the image of Fig. 6(b4) is captured by camera 2#, which indicates the image belongs to the right-hand circular polarization. So we can determine that Fig. 6(a4) represents the OAM beams with topological charge $l=2$, counterclockwise rotational phase and right-hand circular polarization, which also can be expressed as $OA{M}_{+2,1}^{-}$ (0010).

Figure 6(b1) depicts the consequence of L1_pulse = L2_pulse, which demonstrates that there is no rotational phase distribution in Fig. 6 (b3). Figure 6(b2) describes the expression of C_pulse = 0, which means that there are no interference fringes distributed on the circle C in Fig. 6(b3). Hence, the received Fig. 6(b3) doesn’t belong to the domain of OAM beams, which may be induced by the referred beams. Figure 6(a5) depicts that the expression of C_pulse = 8, which means that there are 8 interference fringes distributed on the circle C in Fig. 6(b4). Figure 6(b6) demonstrates that the consequence of L1_pulse > L2_pulse means that there’s clockwise rotational phase in Fig. 6(b4). Furthermore, the image of Fig. 6(b4) is captured by camera 2#, which indicates the image belongs to the right-hand circular polarization. So we can determine that Fig. 6(b4) represents the OAM beams with topological charge$l=8$, clockwise rotational phase and right-hand circular polarization. It also can be expressed as$OA{M}_{-8,1}^{-}$ (1100).

Figure 6(c2) depicts that the expression of C_pulse = 8, which means that there are 8 interference fringes distributed on the circle C in Fig. 6(c3). Figure 6(c1) demonstrates that the consequence of L1_pulse > L2_pulse means that there’s clockwise rotational phase in Fig. 6(c3). Furthermore, the image of Fig. 6(c3) is captured by camera 1#, which indicates the image belongs to the left-hand circular polarization. Therefore, we can determine that Fig. 6(c3) represents the OAM beams with topological charge$l=8$, clockwise rotational phase, and left-hand circular polarization, which also can be expressed as $OA{M}_{-8,1}^{+}$(1110). Figure 6(c6) depicts the consequence of L1_pulse = L2_pulse, which demonstrates that there is no rotational phase distribution in Fig. 6(c4). Figure 6(c5) describes the expression of C_pulse = 0, which means that there’s no interference fringes distributed on the circle C in Fig. 6(c4). So, the received Fig. 6(c4) doesn’t belong to the OAM beams, which also may be induced by the referred beams.

Figure 6(d1) depicts the consequence of L1_pulse = L2_pulse, which demonstrates that there is no rotational phase distribution in Fig. 6(d3). Figure 6(d2) describes the expression of C_pulse = 0, which means that there’s no interference fringes distributed on the circle C in Fig. 6(d3). So, the received Fig. 6(d3) doesn’t belong to the OAM beams, which may be induced by the referred beams. Figure 6(d5) depicts that the expression of C_pulse = 6, which means that there are 6 interference fringes distributed on the circle C in Fig. 6(d4). Figure 6(d6) demonstrates that the consequence of L1_pulse < L2_pulse means that there’s counterclockwise rotational phase in Fig. 6(b4). Furthermore, the image of Fig. 6(b4) is captured by camera 2#, which indicates the image belongs to the right-hand circular polarization. So we can determine that Fig. 6(d4) represents the OAM beams with topological charge$l=6$, counterclockwise rotational phase, and right-hand circular polarization, which also can be expressed as $OA{M}_{+6,1}^{-}$(1010). Next, we translate the decoded$OA{M}_{+2,1}^{-}$,$OA{M}_{-8,1}^{-}$, $OA{M}_{-8,1}^{+}$and$OA{M}_{+6,1}^{-}$to 0010, 1100, 1110 and 1010 by de-mapping, which are consistent with the transmitted PRBS. Other received images by camera 1# and 2# also can be analyzed based on the above criterion for decoding the information.

According to the received images by camera 1# and 2#, the criterion for recognizing the OAM states have been described in the previous paragraph. The encoded sequence, 0010, 1100, 1110 and 1010, can be decoded into$OA{M}_{+2,1}^{-}$,$OA{M}_{-8,1}^{-}$, $OA{M}_{-8,1}^{+}$, and$OA{M}_{+6,1}^{-}$ that are combined the vector modes. To evaluate the propagation performance of information encoded by the states of vector modes, the 16-QAM signal is selected as the testing signal, which is filled with PRBS. Bit Error Rate (BER) and constellation diagrams are used to exhibit the assessment results. The length of OAM fiber is a key factor for the vector modes propagation in the communication system. Figure 7(a) demonstrates the measured performance of BER for vector modes by propagating through different length OAM fiber. It also depicts that the performance of BER declines with the increase of OAM fiber length. Compared with F_L = 0.8m, the performance of BER degenerates about 1.8 dB, 3.2dB and 5.2 dB based on the threshold value of FEC (3.8e-3) for F_L = 1.0m, F_L = 1.2m and F_L = 1.4m, respectively. As shown in Figs. 7(b)-7(e), the constellation diagrams demonstrate that the convergent validity becomes more inattentive by increasing the length of fiber from 0.8m to 1.0m, 1.2m and 1.4m at the same SNR = 17.5dB. The primary causes mainly contain the mode degradation, mode couple, crosstalk and imperfect fabricating procedure of OAM fiber.

Furthermore, we also need to take into account another important factor of bit rate of 16-QAM, which also can lead to significant influence on the performance in terms of BER when the vector modes propagate along the certain length of OAM. In order to get better performance, the OAM fiber with the length of 0.8m are also selected for evaluating the influence of bit rate using BER and constellation diagrams at SNR = 17.5dB. Figure 8(a) indicates that the performance of BER deteriorates gradually with the increase of bit rate from 80 bps to 200 bps, which demonstrates that BER is sensitive to the transmitting rate. As shown in Figs. 8(b)-8(e), the constellation diagrams also reveal that the convergent validity becomes more inattentive with the increase of transmitting rate. The primary causes may be induced by the limited of switching rates of common SLM with the best rates of 100Hz, which can’t support the faster transmitting rates. In addition, the limited switch rate of polarizer is another important factor to decline the performance on BER and constellation diagrams.

In addition, the crosstalk between vector modes plays a very important role in SDM. The weaker crosstalk can effectively reduce the BER and enhance the quality of constellation. The length of OAM fiber is the key factor to evaluate the crosstalk. In order to simplify the measuring procedure, the single signal of $OA{M}_{+2,1}^{-}$(encoded sequence 0010) is emitted and coupled into OAM fiber for exciting the correlative combination of vector modes. Figure 9 demonstrates that crosstalk is induced by the leakage of intensity of $OA{M}_{+2,1}^{-}$ along the same circular polarization direction. With the increase of the length of OAM fiber (0.6 m −1.4 m), the crosstalk becomes worse and worse. The intensity of the received OAM modes declines to ~0.5, when it propagates through the OAM fiber with the length of 1.4 m.

So far, the SLMs combined with high speed optical switch have been used to realize 20 Gbps encoding/decoding in free-space [16]. By using the same technic in future, higher speed encoding/decoding with high efficiency and better performance on BER, constellation, and crosstalk can be achieved, which is combined to 16 OAM states by 4 vector modes for OAM beams.

## 5. Conclusion

In this paper, a new encoding/decoding method by states of vector modes (i.e., polarization direction, rotational direction of phase and topological charge) for OAM beams is proposed. Furthermore, we also design a kind of OAM fiber with the air-core structure for supporting enough vector modes (i.e., TE_{0,1}, HE_{1,1}, HE_{2,1}, HE_{3,1}, TM_{0,1}, EH_{1,1}, HE_{4,1}, EH_{2,1}, HE_{5,1}, EH_{3,1}, HE_{6,1}, EH_{4,1}, HE_{7,1}, EH_{5,1}, HE_{8,1}, EH_{6,1}, HE_{9,1} and EH_{7,1}). In order to identify the OAM modes, the special decoding approach to judge received states of the vector modes is proposed with the help of the technology of image processing. In order to verify the feasibility of encoding/decoding, we set up an experimental platform to demonstrate that encoded 16-QAM signal (i.e.,$OA{M}_{+2,1}^{-}$,$OA{M}_{-8,1}^{-}$, $OA{M}_{-8,1}^{+}$and$OA{M}_{+6,1}^{-}$) can be propagated through OAM fiber with the length of 80 cm. Finally, the received signal can be decoded to 16-QAM by the technology of image processing and DSP. In addition, we evaluated the influence factor (OAM fiber length and bit rate) on the transmitting performance in terms of BER, crosstalk and constellation figures. The primary causes of deteriorating the performance can be concluded to the limited switching rates of SLMs, polarizer and imperfect fabricating procedure. In addition, we also propose the improved methods by using the SLMs with high switching rates, polarizer with high switching rate and high performance OAM fiber.

## Funding

National Natural Science Foundation of China (Project No. 61420106011, 61601279, 61601277); Shanghai Science and Technology Development Funds (Project No. 17010500400, 15530500600, 16511104100,16YF1403900).

## References and links

**1. **L. Zhu, J. Liu, Q. Mo, C. Du, and J. Wang, “Encoding/decoding using superpositions of spatial modes for image transfer in km-scale few-mode fiber,” Opt. Express **24**(15), 16934–16944 (2016). [PubMed]

**2. **C. Brunet, P. Vaity, Y. Messaddeq, S. LaRochelle, and L. A. Rusch, “Design, fabrication and validation of an OAM fiber supporting 36 states,” Opt. Express **22**(21), 26117–26127 (2014). [PubMed]

**3. **I. B. Djordjevic, “Heterogeneous Transparent Optical Networking Based on Coded OAM Modulation,” IEEE Photonics J. **3**(3), 531–537 (2011).

**4. **I. B. Djordjevic, L. Tao, X. Lei, and W. Ting, “On the Multidimensional Signal Constellation Design for Few-Mode-Fiber-Based High-Speed Optical Transmission,” IEEE Photonics J. **4**(5), 1325–1332 (2012).

**5. **J. Du and J. Wang, “High-dimensional structured light coding/decoding for free-space optical communications free of obstructions,” Opt. Lett. **40**(21), 4827–4830 (2015). [PubMed]

**6. **B. Guan, C. Qin, R. P. Scott, N. K. Fontaine, T. Su, R. Proietti, and S. J. B. Yoo, “Polarization Diversified Integrated Circuits for Orbital Angular Momentum Multiplexing,” IEEE Photonics Technol. Lett. **27**(10), 1056–1059 (2015).

**7. **Q. Kang, P. Gregg, Y. Jung, E. L. Lim, S. U. Alam, S. Ramachandran, and D. J. Richardson, “Amplification of 12 OAM Modes in an air-core erbium doped fiber,” Opt. Express **23**(22), 28341–28348 (2015). [PubMed]

**8. **S. Li, Q. Mo, X. Hu, C. Du, and J. Wang, “Controllable all-fiber orbital angular momentum mode converter,” Opt. Lett. **40**(18), 4376–4379 (2015). [PubMed]

**9. **S. Li and J. Wang, “Performance evaluation of analog signal transmission in an orbital angular momentum multiplexing system,” Opt. Lett. **40**(5), 760–763 (2015). [PubMed]

**10. **C. Lin, I. B. Djordjevic, and M. Cvijetic, “Quantum Few-Mode Fiber Communications Based on the Orbital Angular Momentum,” IEEE Photonics Technol. Lett. **25**(1), 3–6 (2013).

**11. **J. Liu, S. Li, J. Du, C. Klitis, C. Du, Q. Mo, M. Sorel, S. Yu, X. Cai, and J. Wang, “Performance evaluation of analog signal transmission in an integrated optical vortex emitter to 3.6-km few-mode fiber system,” Opt. Lett. **41**(9), 1969–1972 (2016). [PubMed]

**12. **J. Liu and J. Wang, “Polarization-insensitive PAM-4-carrying free-space orbital angular momentum (OAM) communications,” Opt. Express **24**(4), 4258–4269 (2016). [PubMed]

**13. **J. Liu, L. Zhu, A. Wang, S. Li, S. Chen, C. Du, Q. Mo, and J. Wang, “All-fiber pre- and post-data exchange in km-scale fiber-based twisted lights multiplexing,” Opt. Lett. **41**(16), 3896–3899 (2016). [PubMed]

**14. **A. Wang, L. Zhu, S. Chen, C. Du, Q. Mo, and J. Wang, “Characterization of LDPC-coded orbital angular momentum modes transmission and multiplexing over a 50-km fiber,” Opt. Express **24**(11), 11716–11726 (2016). [PubMed]

**15. **A. Wang, L. Zhu, J. Liu, C. Du, Q. Mo, and J. Wang, “Demonstration of hybrid orbital angular momentum multiplexing and time-division multiplexing passive optical network,” Opt. Express **23**(23), 29457–29466 (2015). [PubMed]

**16. **A. J. Willner, Y. Ren, G. Xie, Z. Zhao, Y. Cao, L. Li, N. Ahmed, Z. Wang, Y. Yan, P. Liao, C. Liu, M. Mirhosseini, R. W. Boyd, M. Tur, and A. E. Willner, “Experimental demonstration of 20 Gbit/s data encoding and 2 ns channel hopping using orbital angular momentum modes,” Opt. Lett. **40**(24), 5810–5813 (2015). [PubMed]

**17. **X. Zeng, Y. Li, Q. Mo, W. Li, Y. Tian, Z. Liu, and J. Wu, “Experimental Investigation of LP11 Mode to OAM Conversion in Few Mode-Polarization Maintaining Fiber and the Usage for All Fiber OAM Generator,” IEEE Photonics J. **8**(4), 1–7 (2016).

**18. **H. Zhang, W. Zhang, L. Xi, X. Tang, X. Zhang, and X. Zhang, “A New Type Circular Photonic Crystal Fiber for Orbital Angular Momentum Mode Transmission,” IEEE Photonics Technol. Lett. **28**(13), 1426–1429 (2016).

**19. **J. Zhou, J. Zong, and D. Liu, “The Higher Order Statistics of OAM Modal Amplitudes Under Atmosphere Turbulence,” IEEE Photonics Technol. Lett. **28**(23), 2653–2656 (2016).