## Abstract

With the amplitude, time, wavelength/frequency, phase, and polarization/spin parameter dimensions of the light wave/photon almost fully utilized in both classical and quantum photonic information systems, orbital angular momentum (OAM) carried by optical vortex modes is regarded as a new modal parameter dimension for further boosting the capacity and performance of the systems. To exploit the OAM mode space for such systems, stringent performance requirements on a pair of OAM mode multiplexer and demultiplexer (also known as mode sorters) must be met. In this work, we implement a newly discovered optical spiral transformation to achieve a low-cross-talk, wide-optical-bandwidth, polarization-insensitive, compact, and robust OAM mode sorter that realizes the desired bidirectional conversion between seven co-axial OAM modes carried by a ring-core fiber and seven linearly displaced Gaussian-like modes in parallel single-mode fiber channels. We further apply the device to successfully demonstrate high-spectral-efficiency and high-capacity data transmission in a 50-km OAM fiber communication link for the first time, in which a multi-dimensional multiplexing scheme multiplexes eight orbital-spin vortex mode channels with each mode channel simultaneously carrying 10 wavelength-division multiplexing channels, demonstrating the promising potential of both the OAM mode sorter and the multi-dimensional multiplexed OAM fiber systems enabled by the device. Our results pave the way for future OAM-based multi-dimensional communication systems.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. INTRODUCTION

Spatial (mode) division multiplexing exploiting the spatial degree of freedom plays a significant role in current efforts toward higher information capacity in optical communication systems [1,2]. High-dimensional quantum photonic encoding has also been shown to increase the capacity of photonic quantum information systems and robustness of quantum key distribution [3,4]. Orbital angular momentum (OAM) modes [5], as the azimuthal orthogonal eigenmodes of light in the general cylindrical optics including most free-space optical systems and optical fibers [6,7] in addition to the radial eigenmodes [8], are characterized by a single quantum number—their topological charge $ \ell $ [5], an integer whose range is theoretically unbounded in free space, while bulk and fiber optics with limited aperture can be designed to support multiple OAM modes. As such, they constitute a high-dimensional state space that can be more conveniently exploited due to their unique symmetric profile and inherent relation to OAM of light [5] for various promising multi-mode applications, including high-capacity multiplexing in free-space [9,10] or fiber [11,12] communications, high-dimensional quantum entanglement [13,14], and multi-level encoding in quantum key distribution [3,4,15,16].

Mode multiplexing and demultiplexing, i.e., pre-transmission combination and post-transmission separation (also known as mode sorting in physics) of multiple eigenmodes according to the different $ \ell $ values, are essential functions in such multi-mode systems. The conversion between multiple co-axial OAM modes with different $ \ell $ values and spatially resolved Gaussian beams should ideally be implemented without energy loss and inter-modal energy coupling (i.e., cross talk). A variety of OAM (de)multiplexing schemes have been proposed and demonstrated. Fiber-based mode-selective couplers [17] and photonic lanterns [18] as all-fiber OAM multiplexers are convenient to use. While achieving low loss and high stability, their difficulties in achieving larger OAM mode numbers and low inter-modal cross talk remain key issues to be addressed. On-chip photonic integrated devices based on waveguide phase arrays [19,20] and micro-ring resonators [21,22] (including $\Omega $-shaped waveguides [23]) have been used for OAM (de)multiplexing, which, however, suffered from high insertion loss, need for on-chip phase adjustments, and narrow optical bandwidth due to the underlying phase-matching condition. They also generally emit particular kinds of vector vortex modes that may be unmatched to the modes supported by the transmission channel (e.g., OAM fibers). Other schemes based on planar integrated nano-scattering structures [24–26] also face problems in scaling up to high mode numbers. In the free-space, as the mode number increases, projective measurements such as Dammann gratings [27,28] have increasing diffraction loss, while those based on cascaded interference stages [29,30] or multi-plane light conversion [31,32] have increasing complexity.

In contrast, geometrical transformation [33,34] schemes only require a fixed number of phase masks, and therefore can be more efficient and scalable to larger mode numbers. Until recently, however, log-polar mapping [35–40] was the only such transformation known for OAM mode sorting. While its simplicity of only requiring two phase masks is attractive, it suffers from an inherent high spatial overlap (and therefore cross talk) between adjacent sorted modes. A novel “spiral transformation” recently proposed by the authors [41] has been shown to have the potential of fundamentally overcoming this important drawback while maintaining the simplicity of only requiring two phase masks. Despite this significant development, major optical problems remain to be overcome to achieve a fiber-optics compatible, high-performance, compact, and robust device. In this work, we report the realization of such a device and the demonstration of its application potentials in multi-dimensional multiplexed OAM information systems.

Our scheme integrates the two phase masks performing the geometrical transformation onto the opposite sides of a 5-mm-thick quartz plate as diffractive optical elements (DOEs). Apart from its compactness and robustness, such an integration scheme uses the micrometer-scale precision of photolithography process to eliminate degradation in OAM mode sorting performance due to misalignment, a problem rooted in the translational variant nature of the polar coordinates in which OAM is defined. As the distance between the two phase masks (the 5-mm thickness of the plate) is comparable to the phase masks’ cross-sectional area (${3}\;{\rm mm} \times {3}\;{\rm mm}$) needed to accommodate sufficient DOE pixels for diffractive resolution and efficiency, the resulting nonparaxiality corrupts the geometrical transformation that is defined in the paraxial regime [41]. If low cross talk between OAM modes are to be suppressed to levels compatible with the stringent OAM information system requirements, the nonparaxiality must be corrected. We solve this problem by analytically deriving and experimentally implementing a new phase distribution for the second mask to correct the phase-front distortions caused by the nonparaxiality, thereby achieving the high resolution between adjacent modes in a compact and robust fashion as promised by the spiral transformation.

A further problem, common to previous implementations of both the log-polar and spiral transformations, is the elongated shape of the sorted modes that prevents efficient coupling with single-mode fibers (SMFs), which stems from the dissimilar transformation experienced by the azimuthal and radial dimensions of the polar space when respectively converted to the horizontal and vertical dimensions in the Cartesian space. We solve this problem by replacing the previously used spherical Fourier-transformation lens with a specially designed elliptical lens to achieve circular Gaussian-like output mode profiles that can be efficiently coupled into SMFs. The resulting compact device successfully achieves low-cross-talk bidirectional mapping between OAM modes carried by a ring-core fiber (RCF) [11,12] and displaced Gaussian-like modes of SMFs over a wide optical bandwidth and irrespective of the state-of-polarization (SoP) or the spin state of the optical modes. The performance of the device is further proven by enabling the first demonstration of a 50-km fiber-based multi-dimensional multiplexed OAM transmission system, the longest such system reported to date.

## 2. SPIRAL TRANSFORMATION IN THE NONPARAXIAL REGIME FOR HIGH-RESOLUTION OAM MODE SORTING

The term *spiral transformation* refers to a class of geometrical transformations that map spirals in an input plane $ (x,y) $ to straight lines in an output plane $ (u,v) $, as illustrated in Fig. 1(a). A logarithmic spiral transformation, in particular, is expressed through the following relations between the input and output coordinates [41]:

The phase mask which redirects the input rays according to the spiral transformation Eq. (1), generally known as the unwrapper, is designed to be fabricated on one side of a quartz plate with the following phase distribution [41]:

The phase of the output light contains two parts. The first part is a linear phase gradient which originates from the azimuthal phase gradient of the input OAM mode mapped onto the straight lines according to the spiral transformation. This is the desirable phase that enables the sorting of different OAM modes to different positions in the focal plane of a lens. As the azimuthal phase excursion of the OAM mode is intercepted multiple times by an endless spiral, the available effective phase excursion along it is multiple times of $ 2\pi \ell $ (only limited by the width of the input OAM beam) and therefore high-resolution mode sorting can be achieved with low cross talk, which is the major advantage of the spiral transformation compared to the log-polar one—the latter only achieves phase excursion of $ 2\pi \ell $ along one single circle [35]. The second part is the phase acquired by the wave during propagation between the input and output planes inside the quartz plate, which is not useful for mode sorting and therefore should be removed with a phase-correction mask that is generally known as the *phase corrector*.

To achieve a compact and robust device, this second mask is designed to be fabricated on the other side of the same quartz plate. As the distance between the two phase masks (thickness of the plate) is comparable to their cross-sectional dimensions, the phase-front distortions caused by the nonparaxiality must be corrected by a new phase distribution for the second mask, which we analytically derive as

where $ (x,y) $ are related to $ (u,v) $ through the spiral transformation [Eq. (1)]. It is verified that this nonparaxial phase corrector has the expected superior phase correcting and thus mode sorting performance compared to the original paraxial phase corrector widely used in previous transformations [35–37,39–41], which is presented in Supplement 1 S2. However, it should be noted that the phase profile of the unwrapper in Eq. (2) is still derived in the paraxial regime in order to obtain an analytical solution, which could be further extended to the nonparaxial regime by numerical optimization in future work.## 3. COMPACT OAM MODE SORTER DEVICE AND CHARACTERIZATION

An array of 12 compact mode sorters have been fabricated based on the above theory, each consisting of an unwrapper, a phase corrector, and a positive lens also as a DOE element, all of which are integrated on a 3D-printed base, as shown in Fig. 1(b). The unwrapper and the phase corrector are engraved as DOEs with a cross-sectional area of ${3} \times {3}\;{{\rm mm}^2}$ on respective sides of a 5-mm-thick fused quartz plate by double-sided photolithography technique (see Supplement 1 S3, for fabrication details). Aided by complimentary optical alignment markers, the critical misalignment error between these two phase masks is $ {\lt}{5}\;\unicode{x00B5} {\rm m}$. For comparison, both the log-polar and spiral sorters are fabricated on the same plate simultaneously, and for both transformations, nonparaxial and paraxial phase correctors have been included.

The fabricated compact OAM mode sorters have been characterized, stage by stage, using the experimental setup in Supplement 1 S4, with results compared to the corresponding numerical simulations conducted with a standard Fourier-space method based on the spatial spectral decomposition of the propagating light waves. Typical values of the involved parameters are $ d = 4.95\; {\rm mm} $, $ {r_0} = 0.3\; {\rm mm} $, $ 2\pi \beta = 0.4\; {\rm mm} $, $ 2\pi a = \ln(1.6) $, $ {\lambda _0} = 1550 \; {\rm nm} $, and $ {n_{\rm quartz}} = 1.444 $ (see Supplement 1 S5 for selection of the transformation parameters), which are consistent in both simulation and experiment. The input OAM modes are “perfect vortex beams” [42,43] with the same intensity distribution (beam width of 400 µm) but varying $ \ell $ (see Supplement 1 S6), closely simulating OAM modes expanded from those of a ring-core fiber (see Supplement 1 S7, for the radial influence on the mode sorting effect). As seen in Fig. 2, after passing through the first stage phase mask, the log-polar transformation scheme unwraps the input OAM modes [Fig. 2(a)] to stripe-shaped modes with a length of 0.4 mm [Fig. 2(b)], which is determined by the selected scaling parameter $ \beta = {{0.4} / {2\pi }} {\rm mm} $. Instead, in the spiral transformation scheme the OAM modes are unwrapped to stripes whose length is extended by a factor that reflects the number of spiral turns within the beam width of the input OAM mode—nearly ${3 \times}$ in this case [Fig. 2(c)]. There is a satisfactory agreement between the experimental results with the numerical ones in terms of the length and shape of the transformed intensity patterns.

The unwrapped beams are subsequently phase-corrected in order to obtain the phase-tilted plane waves and eventually spatially resolved by a spatial Fourier transformation implemented with a standard spherical lens, which is presented in Figs. 2(d) and 2(e). As expected for both schemes, the input OAM modes are sorted as being horizontally displaced from the optical axis by a distance proportional to their topological charges:

where $ {\lambda _0} $ is the wavelength in free space, and $ f $ is the focal distance of the lens. At the telecommunication wavelength $ {\lambda _0} = 1550\;{\rm nm} $ and for $ f = 100 \; {\rm mm} $, the theoretical lateral spacing between adjacent OAM modes is $ {S_{\ell + 1}} - {S_\ell } = {{{\lambda _0}f} / {(2\pi \beta }}) = 387.5 \; {\unicode{x00B5}{\rm m}} $, which is consistent with the numerical and experimental results in Figs. (2d) and (2e).Moreover, the spiral transformation [Fig. 2(d)] clearly achieves better separation between different OAM modes than the log-polar case [Fig. (2e)] when both are using nonparaxial phase correction. The gain in performance can be identified through the *finesse*, defined as the ratio of the spacing between adjacent sorted modes over their average full width at half-maximum [40]. In this case, the finesse is nearly tripled, namely increasing from 1.1 of the log-polar scheme to 3.2 of the spiral scheme in simulation, and from 1.0 to 2.6 in experiment, which confirms the superior ability to resolve OAM modes of the spiral transformation with nonparaxial phase correction.

## 4. BIDIRECTIONAL OAM AND GAUSSIAN-LIKE MODE MAPPING

As seen in Figs. 2(d) and (2e), the intensity distributions of the sorted OAM modes are generally elongated, especially in the spiral transformation scheme where the enhancement of resolution entails the sharpening of the distributions in the direction of sorting (horizontal here), and the unwrapper’s multiple turns of the spiral carve the input annular intensity distribution into a narrower stripe, which further elongates the output mode in the vertical direction. Such intensity profiles are incompatible with the next stages of optical processing, such as coupling the demultiplexed modes into SMF channels of an optical communication system. In order to realize OAM (de)multiplexing between the RCF and SMF in fiber-based OAM systems, it is desirable to obtain the sorted modes in profiles as close to Gaussian as possible. To this end, we replace the spherical Fourier transforming lens of focal length $ f $ with a specially designed elliptical lens with horizontal focal length $ f $ and vertical focal length $ {f / 2} $, also implemented as a DOE in the same manner as shown in Fig. 1(e) and Supplement 1 S3. Passing through this lens, the optical wavefront is uniaxially Fourier transformed in the horizontal direction, allowing OAM mode separation as by Eq. (4), while in the vertical direction, the narrow light beam profile is simply inverted with 1:1 magnification instead of being Fourier transformed, therefore maintaining its sharpness. By appropriate selection of $ f = 15\;{\rm mm} $, the separated OAM modes are shaped into the intensity distributions presented in Fig. 3(a), where different OAM modes are transformed to equally spaced, circular Gaussian-like modes with the beam diameter around 54 µm. Each can be subsequently coupled into a SMF by a focusing lens, as shown in the experimental setup in Supplement 1 S4. A quantitative comparison between these Gaussian-like modes with ideal Gaussian modes in this case indicates up to 60%–70% similarity, which is presented in Supplement 1 S8.

After coupling into the SMF, the measured inter-modal cross talk is less than $ - {13}\;{\rm dB}$ for all the seven OAM modes and the two circular polarization states as seen in the mode transfer matrix in Fig. 3(b), with a 3-dB deterioration optical bandwidth of 15 nm around 1550 nm (see Supplement 1 S9). The total power loss between the incident OAM modes to the SMFs is less than 8 dB (including ${\sim}{3}\;{\rm dB}$ fiber-coupling loss mainly resulting from the difference between the sorted Gaussian-like beams and ideal Gaussian beams analyzed in Supplement 1 S8) and varies ${\sim}{1}\;{\rm dB}$ for the seven modes and across the above optical bandwidth, respectively. The three DOEs introduce an insertion loss of ${\sim }{4.5}\;{\rm dB}$—apart from the ${\sim} {0.7}\;{\rm dB}$ reflection loss of the four air–glass interfaces, the rest (3.8 dB) is mainly attributed to the diffraction loss of the three DOEs, which can be reduced by decreasing the pixel size of the DOEs for higher diffraction efficiency or by employing optical metasurfaces [44,45] with high transmission in future designs.

When the sorting device is used in reverse, due to the reversibility principle of light propagation, displaced Gaussian modes incident from the SMF at specific positions should also be transformed into corresponding co-axial vortex modes for multiplexing in the RCF, which is experimentally demonstrated in Fig. 3(c). Gaussian beams output from the SMF at different positions indeed transform into almost the same co-axial ring-shaped intensity profiles, with slightly spiral intensity variation resulting from the geometrical transformation. In order to verify their phase structure, the experimental setup in Supplement 1 S4 is used in reverse for multiplexing and seven spiral phase modulations are imposed in succession by the spatial light modulator (SLM) to reveal a clear inner Gaussian spot, as shown in Supplement 1 S10, which confirms satisfactory conversion to OAM modes with topological charges of $ - 3 \le \ell \le 3 $. After coupling into the SMF, the measured inter-modal cross talk is less than $ - {13.5}\;{\rm dB}$ for all the seven OAM modes and two circular polarization states as seen in the mode transfer matrix in Fig. 3(d). Therefore, it has been confirmed that the above compact OAM mode sorter device can be simultaneously used as an OAM multiplexer/demultiplexer between OAM modes for the RCF and displaced Gaussian-like modes for the SMF, which are essential in fiber-based OAM information systems [11,12,46–49].

## 5. COMPACT MODE SORTER APPLIED IN A 50-KM OAM FIBER TRANSMISSION SYSTEM

To further prove the device performance and demonstrate its potential applications in fiber-based multi-dimensional OAM information systems, the compact mode sorter is applied in a 50-km RCF-based data transmission system with simultaneous OAM mode-division multiplexing (MDM) and wavelength-division multiplexing (WDM) [46].

Figure 4(a) illustrates the experimental setup of the OAM-MDM-WDM data transmission system. WDM is realized by combining 10 optical carriers from narrow-linewidth tunable lasers with wavelengths ranging from 1549.8 nm to 1551.6 nm in a 0.2-nm/25-GHz grid, as shown in Fig. 4(b). The combined 10 WDM carriers are modulated by 16-GBaud quadrature phase shift keying (QPSK) signal from an arbitrary waveform generator through an I/Q modulator. The sample rate of the digital-to-analog converter is 64 GSa/s and the modulated binary data sequence is a pseudo-random binary sequence with a pattern length of ${{2}^{18}} {-} {1}$. Note that all the WDM carriers are modulated with the same QPSK signal due to the available hardware limitation in our lab.

The WDM signals are split into four branches, which are separately amplified by an erbium-doped fiber amplifier, delayed for data pattern decorrelation, collimated, and linearly polarized. After that, the four optical beams are grouped into two pairs with each pair reflected by a phase-only SLM to generate OAM modes of $ \ell = + 2 $ or $ + 3 $. Then the two pairs of OAM beams after each with one arm reflected by a mirror to invert the sign of the generated OAM order are multiplexed as orthogonally polarized modes by a polarization beam splitter (PBS, for beam combination). The generated four OAM modes of $ \langle + {2},{ X} \rangle $, $ \langle - {2},{Y} \rangle $, $ \langle + {3},{X} \rangle $, and $ \langle - {3},{Y} \rangle $ of two pairs are further combined by a beam splitter. Here ${ X}$ and ${Y}$ represent the horizontal and the vertical polarizations, respectively. Then the combined OAM modes are converted into circular polarizations by a quartz-wave plate and realize dual-polarization multiplexing by using a PBS (for beam splitting), an optical delay path (for signal decorrelation), and a PBS (for beam combination), so that the four modes in each group of $ \ell = {\pm 2}$ or ${\pm 3}$ with dual-polarizations are created. The generated eight OAM modes are then multiplexed and converted into circular polarizations, and finally focused to couple into a 50-km RCF. Therefore, the four OAM modes ($ \ell = - {3},{ - 2},{ + 2}$, and $ + {3}$) with each multiplexing two polarization modes form a eight-channel MDM scheme, and all these eight mode channels with each multiplexing 10 WDM channels eventually constitute a multi-dimensional multiplexing transmission system of 80 data channels over the 50-km RCF.

The 50-km RCF [46] supports transmission of single-radial-order OAM mode groups (MGs) of $ | \ell | = 0$ to 3 with attenuation of 0.31 dB/km, in which the four modes $ \langle \pm \ell , \pm {s} \rangle $ ($ + {s} = {L}$ and $ - { s} = {R}$ represent left- and right-handed circular polarizations) in the same MG are intentionally designed to be near-degenerate while different MGs are decoupled by the large inter-MG differential effective refractive indices, an architecture that is based on a scalable fiber-based OAM-MDM scheme by striking a balance between spatial mode counts/transmission distance and digital signal processing (DSP) complexity [12]. In this framework, the OAM mode sorter is used to demultiplex different MGs while ${4} \times {4}$ multi-input-multi-output (MIMO) equalization is used to deal with intra-MG coupling. When the RCF is injected with a single OAM topological charge of $| \ell | = 2$ or 3 and of any $s$, strong mode coupling within each MG of $ | \ell |$ during the propagation results in output RCF mode patterns, as shown in Fig. 4(c). After RCF transmission, all output modes from the fiber are collimated and imaged to the plane of the OAM mode sorter, after which each OAM MG ($| \ell | = 2$ or 3) is demultiplexed into a pair of Gaussian-like spots corresponding to ${\pm } \ell $, as shown in Fig. 4(c), each containing both SoPs of ${ \pm s}$. The measured mode transfer matrix of all these eight vortex modes ($ \ell = - {3},{ - 2},{ + 2}$, and $ + {3}$ and each with ${\pm s}$) in Fig. 4(c) shows the expected strong in-fiber intra-MG coupling ($| \ell | = 2$ or 3) and weak inter-MG coupling between $ |\ell | = 2$ and 3 with an averaged cross talk around $ - {10}\;{\rm dB}$ over the entire system of mode launch, 50-km RCF propagation, and mode sorter. For the receiving of each MG ($| \ell | = 2$ or 3), the pair of Gaussian-like beams corresponding to $ + \ell $ and $ - \ell $, each containing two polarization (${\pm s}$) multiplexed coherent signals, are simultaneously coupled into two SMF-pigtailed dual-polarization integrated coherent optical receivers (ICRs). Then the eight output electrical real-value waveforms (including the I/Q for each mode) from the ICRs are simultaneously recorded by an eight-channel real-time oscilloscope (Teledyne LeCroy 10-36ZI) at a sample rate of 80-GSa/s. The offline DSP includes timing phase recovery, ${4} \times {4}$ MIMO equalization based on conventional blind constant modulus algorithm, frequency offset compensation, and carrier phase estimation, after which the bit error rates (BERs) of all four OAM mode channels within one MG are finally evaluated. The BER measurement is repeated for each MG as well as each wavelength.

Figure 4(d) shows the constellations of the recovered QPSK signals after transmission at their best optical signal-to-noise ratio values, and Fig. 4(e) illustrates the measured BERs, with all 80 channels exceeding the 20% soft-decision forward-error-correction (FEC) threshold correcting BER of ${2.4} \times {{10}^{ - 2}}$ (note that the BER values of the two polarization modes with the same $ \ell $ are averaged to plot as one data point for concision), which demonstrates successful transmission of the 16-Gbaud QPSK signals over the 50-km OAM fiber link with an aggregate capacity of 2.56-Tbit/s, spectral efficiency (SE) of 10.24 bit/s/Hz, and a SE-distance product of 512 bit/s/Hz-km, and therefore confirms the high performance of the compact OAM mode sorter and its potential applications in multi-dimensional multiplexed OAM fiber systems.

## 6. CONCLUSION

In summary, a compact and robust OAM mode sorter has been demonstrated based on a novel nonparaxial spiral coordinate transformation. The implementation consists of an logarithmic spiral unwrapper and a nonparaxial phase corrector, respectively fabricated on the two sides of a 5-mm-thick quartz plate as flat DOE patterns, followed by a DOE elliptical lens specially designed for achieving circular Gaussian-like output mode profiles, all of which are assembled on a 3D-printed base as a compact device. Each sorter has a cross-sectional area of only ${3} \times {3}\;{{\rm mm}^2}$ and a total length of 20 mm (input facet to output facet). The DOE implementation makes it possible to be mass-produced with low cost, e.g., by a molding or imprinting process.

The novel OAM mode sorter achieves a significantly enhanced mode resolution (in this specific case improved by a factor of ${\sim}{3}$) compared to the previously log-polar transformation for OAM sorting. OAM multiplexing and demultiplexing between OAM modes of RCF and displaced Gaussian-like modes of SMF have been achieved using an elliptical lens for uniaxial Fourier transformation, with total power loss of $ {\lt} {8}\;{\rm dB}$, inter-mode cross talk of $ {\lt} - {13}\;{\rm dB}$ for all seven OAM modes $ - 3 \le \ell \le 3 $ (see Supplement 1 S11 for discussion of larger mode counts), 3-dB cross-talk deterioration optical bandwidth of 15 nm around 1550 nm, and polarization insensitive operation. The device therefore overcomes fundamental drawbacks of previous OAM mode sorters in their key performance for fiber-based multi-dimensional OAM information systems including mode resolution and fiber mode-matching, with its insertion loss significantly improvable by better DOE implementations or inverse design optimization [50].

The device performance is further verified in a sophisticated multi-dimensional OAM fiber communication system simultaneously combining multiplexing in the OAM mode, polarization mode, and wavelength dimensions, and coherent data modulation in the optical phase dimension. Combined with a high-performance ring-core fiber, the device takes the fiber-based OAM mode transmission distance over 50 km for the first time. The transmission distance and spectral-efficiency-distance-product are the highest so far reported over fiber-based OAM systems. Our mode sorter would therefore likely become a very useful component in practical multi-dimensional multiplexed optical communication and high-dimensional quantum information systems based on OAM modes.

## Funding

National Key R&D Program of China (2018YFB1801803, 2019YFA0706302); National Natural Science Foundation of China (U1701661, 11774437, 61975243, 61490715); Local Innovative and Research Teams Project of Guangdong Pearl River Talents Program (2017BT01X121); Science and Technology Program of Guangzhou (201804010302); European Union H2020 project ROAM.

## Acknowledgment

The authors thank Heyun Tan, Jian Jian, Chunchuan Yang, and Lin Liu for helpful discussions in optical experiment as well as technical assistance.

## Disclosures

The authors declare no conflicts of interest.

See Supplement 1 for supporting content.

## REFERENCES

**1. **G. Li, N. Bai, N. Zhao, and C. Xia, “Space-division multiplexing: the next frontier in optical communication,” Adv. Opt. Photonics **6**, 413–487 (2014). [CrossRef]

**2. **J. M. Kahn and D. A. B. Miller, “Communications expands its space,” Nat. Photonics **11**, 5–8 (2017). [CrossRef]

**3. **A. Sit, F. Bouchard, R. Fickler, J. Gagnon-Bischoff, H. Larocque, K. Heshami, D. Elser, C. Peuntinger, K. Gunthner, B. Heim, C. Marquardt, G. Leuchs, R. W. Boyd, and E. Karimi, “High-dimensional intracity quantum cryptography with structured photons,” Optica **4**, 1006–1010 (2017). [CrossRef]

**4. **A. Sit, R. Fickler, F. Alsaiari, F. Bouchard, H. Larocque, P. Gregg, L. Yan, R. W. Boyd, S. Ramachandran, and E. Karimi, “Quantum cryptography with structured photons through a vortex fiber,” Opt. Lett. **43**, 4108–4111 (2018). [CrossRef]

**5. **L. Allen, M. W. Beijersbergen, R. J. Spreeuw, and J. P. Woerdman, “Orbital angular momentum of light and the transformation of Laguerre-Gaussian laser modes,” Phys. Rev. A **45**, 8185–8189 (1992). [CrossRef]

**6. **G. Gibson, J. Courtial, M. Padgett, M. Vasnetsov, V. Pas’ko, S. Barnett, and S. Franke-Arnold, “Free-space information transfer using light beams carrying orbital angular momentum,” Opt. Express **12**, 5448–5456 (2004). [CrossRef]

**7. **A. E. Willner, H. Huang, Y. Yan, Y. Ren, H. Ahmed, G. Xie, C. Bao, L. Li, Y. Cao, Z. Zhao, J. Wang, M. P. Lavery, M. Tur, S. Ramachandran, A. F. Molisch, N. Ashrafi, and S. Ashrafi, “Optical communications using orbital angular momentum beams,” Adv. Opt. Photonics **7**, 66–106 (2015). [CrossRef]

**8. **E. Karimi, R. W. Boyd, P. de la Hoz, H. de Guise, J. Řeháček, Z. Hradil, A. Aiello, G. Leuchs, and L. L. Sánchez-Soto, “Radial quantum number of Laguerre-Gauss modes,” Phys. Rev. A **89**, 063813 (2014). [CrossRef]

**9. **J. Wang, J. Y. Yang, I. M. Fazal, N. Ahmed, Y. Yan, H. Huang, Y. Ren, Y. Yue, S. Dolinar, M. Tur, and A. E. Willner, “Terabit free-space data transmission employing orbital angular momentum multiplexing,” Nat. Photonics **6**, 488–496 (2012). [CrossRef]

**10. **Y. Yan, G. Xie, M. P. J. Lavery, H. Huang, N. Ahmed, C. Bao, Y. Ren, Y. Cao, L. Li, Z. Zhao, A. F. Molisch, M. Tur, M. J. Padgett, and A. E. Willner, “High-capacity millimetre-wave communications with orbital angular momentum multiplexing,” Nat. Commun. **5**, 4876 (2014). [CrossRef]

**11. **N. Bozinovic, Y. Yue, Y. Ren, N. Tur, P. Kristensen, H. Huang, A. E. Willner, and S. Ramachandran, “Terabit-scale orbital angular momentum mode division multiplexing in fibers,” Science **340**, 1545–1548 (2013). [CrossRef]

**12. **G. Zhu, Z. Hu, X. Wu, C. Du, W. Luo, Y. Chen, X. Cai, J. Liu, J. Zhu, and S. Yu, “Scalable mode division multiplexed transmission over a 10-km ring-core fiber using high-order orbital angular momentum modes,” Opt. Express **26**, 594–604 (2018). [CrossRef]

**13. **A. Mair, A. Vaziri, G. Weihs, and A. Zeilinger, “Entanglement of the orbital angular momentum states of photons,” Nature **412**, 313–316 (2001). [CrossRef]

**14. **M. Krenn, J. Handsteiner, M. Fink, R. Fickler, and A. Zeilinger, “Twisted photon entanglement through turbulent air across Vienna,” Proc. Natl. Acad. Sci. U. S. A. **112**, 14197–14201 (2015). [CrossRef]

**15. **M. Mirhosseini, O. S. Magaña-Loaiza, M. N. O’Sullivan, B. Rodenburg, M. Malik, M. P. J. Lavery, M. J. Padgett, D. J. Gauthier, and R. W. Boyd, “High-dimensional quantum cryptography with twisted light,” New J. Phys. **17**, 033033 (2015). [CrossRef]

**16. **M. Erhard, R. Fickler, M. Krenn, and A. Zeilinger, “Twisted photons: new quantum perspectives in high dimensions,” Light Sci. Appl. **7**, 17146 (2018). [CrossRef]

**17. **S. Pidishety, S. Pachava, P. Gregg, S. Ramachandran, G. Brambilla, and B. Srinivasan, “Orbital angular momentum beam excitation using an all-fiber weakly fused mode selective coupler,” Opt. Lett. **42**, 4347–4350 (2017). [CrossRef]

**18. **Z. S. Eznaveh, J. C. A. Zacarias, J. E. A. Lopez, Y. Jung, K. Shi, B. C. Thomsen, D. J. Richardson, S. Leon-Saval, and R. A. Correa, “Annular core photonic lantern OAM mode multiplexer,” in *Optical Fiber Communication (OFC) Conference* (2017), paper Tu3J.3.

**19. **T. Su, R. P. Scott, S. S. Djordjevic, N. K. Fontaine, D. J. Geisler, X. Cai, and S. J. B. Yoo, “Demonstration of free space coherent optical communication using integrated silicon photonic orbital angular momentum devices,” Opt. Express **20**, 9396–9402 (2012). [CrossRef]

**20. **B. Guan, R. P. Scott, C. Qin, N. K. Fontaine, T. Su, C. Ferrari, M. Cappuzzo, F. Klemens, B. Keller, M. Earnshaw, and S. J. B. Yoo, “Free-space coherent optical communication with orbital angular, momentum multiplexing/demultiplexing using a hybrid 3D photonic integrated circuit,” Opt. Express **22**, 145–156 (2013). [CrossRef]

**21. **S. Li, Z. Nong, S. Gao, M. He, L. Liu, S. Yu, and X. Cai, “Orbital angular momentum mode multiplexer based on bilayer concentric micro-ring resonator,” in *Asia Communications and Photonics Conference (ACP)* (2017), paper Su3K.2.

**22. **S. Li, Z. Nong, X. Wu, W. Yu, M. He, Y. Zhu, S. Gao, L. Liu, Z. Li, S. Yu, and X. Cai, “Demonstration of chip-to-chip communication based on ultra-compact orbital angular momentum (de)multiplexers,” in *Conference on Lasers and Electro-Optics (CLEO)* (2018), paper Stu3B.2.

**23. **N. Zhang, M. Scaffardi, M. N. Malik, V. Toccafondo, C. Klitis, G. Meloni, F. Fresi, E. Lazzeri, D. Marini, J. Zhu, X. Cai, S. Yu, L. Poti, A. Bogoni, and M. Sorel, “4 OAM x 4 WDM Optical switching based on an innovative integrated tunable OAM multiplexer,” in *Optical Fiber Communication (OFC) Conference* (2018), paper Th3H.1.

**24. **A. Liu, C.-L. Zou, X. Ren, Q. Wang, and G.-C. Guo, “On-chip generation and control of the vortex beam,” Appl. Phys. Lett. **108**, 181103 (2016). [CrossRef]

**25. **N. Zhou, S. Zheng, X. Cao, S. Gao, S. Li, M. He, X. Cai, and J. Wang, “Generating and synthesizing ultrabroadband twisted light using a compact silicon chip,” Opt. Lett. **43**, 3140–3143 (2018). [CrossRef]

**26. **Z. Xie, T. Lei, F. Li, H. Qiu, Z. Zhang, H. Wang, C. Min, L. Du, Z. Li, and X. Yuan, “Ultra-broadband on-chip twisted light emitter for optical communications,” Light Sci. Appl. **7**, 18001 (2018). [CrossRef]

**27. **N. Zhang, X. C. Yuan, and R. E. Burge, “Extending the detection range of optical vortices by Dammann vortex gratings,” Opt. Lett. **35**, 3495–3497 (2010). [CrossRef]

**28. **T. Lei, M. Zhang, Y. Li, P. Jia, G. N. Liu, X. Xu, Z. Li, C. Min, J. Lin, C. Yu, H. Niu, and X. Yuan, “Massive individual orbital angular momentum channels for multiplexing enabled by Dammann gratings,” Light Sci. Appl. **4**, e257 (2015). [CrossRef]

**29. **J. Leach, M. J. Padgett, S. M. Barnett, S. Frankearnold, and J. Courtial, “Measuring the orbital angular momentum of a single photon,” Phys. Rev. Lett. **88**, 257901 (2002). [CrossRef]

**30. **Y. Zhou, M. Mirhosseini, D. Fu, J. Zhao, S. M. H. Rafsanjani, A. E. Willner, and R. W. Boyd, “Sorting photons by radial quantum number,” Phys. Rev. Lett. **119**, 263602 (2017). [CrossRef]

**31. **G. Labroille, B. Denolle, P. Jian, P. Genevaux, N. Treps, and J.-F. Morizur, “Efficient and mode selective spatial mode multiplexer based on multi-plane light conversion,” Opt. Express **22**, 15599–15607 (2014). [CrossRef]

**32. **R. Saad, M. Meunier, G. Trunet, N. Barré, P. Jian, J.-F. Morizur, and G. Labroille, “Highly selective 7 orbital angular momentum mode multiplexer based on multi-plane light conversion,” in *European Conference on Optical Communication (ECOC)* (2017), pp. 1–3.

**33. **D. Casasent and D. Psaltis, “Position, rotation, and scale invariant optical correlation,” Appl. Opt. **15**, 1795–1799 (1976). [CrossRef]

**34. **W. J. Hossack, A. M. Darling, and A. Dahdouh, “Coordinate transformations with multiple computer-generated optical elements,” J. Mod. Opt. **34**, 1235–1250 (1987). [CrossRef]

**35. **G. C. G. Berkhout, M. P. J. Lavery, J. Courtial, M. W. Beijersbergen, and M. J. Padgett, “Efficient sorting of orbital angular momentum states of light,” Phys. Rev. Lett. **105**, 153601 (2010). [CrossRef]

**36. **M. P. J. Lavery, D. J. Robertson, G. C. G. Berkhout, G. D. Love, M. J. Padgett, and J. Courtial, “Refractive elements for the measurement of the orbital angular momentum of a single photon,” Opt. Express **20**, 2110–2115 (2012). [CrossRef]

**37. **M. Mirhosseini, M. Malik, Z. Shi, and R. W. Boyd, “Efficient separation of the orbital angular momentum eigenstates of light,” Nat. Commun. **4**, 2781 (2013). [CrossRef]

**38. **G. Ruffato, M. Massari, and F. Romanato, “Compact sorting of optical vortices by means of diffractive transformation optics,” Opt. Lett. **42**, 551–554 (2017). [CrossRef]

**39. **S. Lightman, G. Hurvitz, R. Gvishi, and A. Arie, “Miniature wide-spectrum mode sorter for vortex beams produced by 3D laser printing,” Optica **4**, 605–610 (2017). [CrossRef]

**40. **C. Wan, J. Chen, and Q. Zhan, “Compact and high-resolution optical orbital angular momentum sorter,” APL Photonics **2**, 031302 (2017). [CrossRef]

**41. **Y. Wen, I. Chremmos, Y. Chen, J. Zhu, Y. Zhang, and S. Yu, “Spiral transformation for high-resolution and efficient sorting of optical vortex modes,” Phys. Rev. Lett. **120**, 193904 (2018). [CrossRef]

**42. **A. S. Ostrovsky, C. Rickenstorff-Parrao, and V. Arrizón, “Generation of the ‘perfect’ optical vortex using a liquid-crystal spatial light modulator,” Opt. Lett. **38**, 534–536 (2013). [CrossRef]

**43. **M. Chen, M. Mazilu, Y. Arita, E. M. Wright, and K. Dholakia, “Dynamics of microparticles trapped in a perfect vortex beam,” Opt. Lett. **38**, 4919–4922 (2013). [CrossRef]

**44. **A. Arbabi, Y. Horie, M. Bagheri, and A. Faraon, “Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission,” Nat. Nanotechnol. **10**, 937–944 (2015). [CrossRef]

**45. **P. Genevet, F. Capasso, F. Aieta, M. Khorasaninejad, and R. Devlin, “Recent advances in planar optics: from plasmonic to dielectric metasurfaces,” Optica **4**, 139–152 (2017). [CrossRef]

**46. **J. Zhang, Y. Wen, H. Tan, J. Liu, L. Shen, M. Wang, J. Zhu, C. Guo, Y. Chen, Z. Li, and S. Yu, “80-channel WDM-MDM transmission over 50-km ring-core fiber using a compact OAM demux and modular 4x4 MIMO equalization,” in *Optical Fiber Communication (OFC) Conference* (2019), paper W3F.3.

**47. **C. Brunet, P. Vaity, Y. Messaddeq, S. LaRochelle, and L. A. Rusch, “Design, fabrication and validation of an OAM fiber supporting 36 states,” Opt. Express **22**, 26117–26127 (2014). [CrossRef]

**48. **F. Feng, X. Jin, D. O’Brien, F. P. Payne, and T. D. Wilkinson, “Mode-group multiplexed transmission using OAM modes over 1km ring-core fiber without MIMO processing,” in *Optical Fiber Communication (OFC) Conference* (2017), paper Th2A.43.

**49. **J. Zhang, G. Zhu, J. Liu, X. Wu, J. Zhu, C. Du, W. Luo, Y. Chen, and S. Yu, “Orbital-angular-momentum mode-group multiplexed transmission over a graded-index ring-core fiber based on receive diversity and maximal ratio combining,” Opt. Express **26**, 4243–4257 (2018). [CrossRef]

**50. **Z. Lin, Y. Wen, Y. Chen, Y. Zhang, and S. Yu, “Inverse design of orbital angular momentum mode demultiplexer by combining wavefront matching method and gradient descent algorithm,” in *Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC)* (2019), p. 1.