Abstract

We present the results of the comparative performance-versus-complexity analysis for the several types of artificial neural networks (NNs) used for nonlinear channel equalization in coherent optical communication systems. The comparison is carried out using an experimental set-up with the transmission dominated by the Kerr nonlinearity and component imperfections. For the first time, we investigate the application to the channel equalization of the convolution layer (CNN) in combination with a bidirectional long short-term memory (biLSTM) layer and the design combining CNN with a multi-layer perceptron. Their performance is compared with the one delivered by the previously proposed NN-based equalizers: one biLSTM layer, three-dense-layer perceptron, and the echo state network. Importantly, all architectures have been initially optimized by a Bayesian optimizer. First, we present the general expressions for the computational complexity associated with each NN type; these are given in terms of real multiplications per symbol. We demonstrate that in the experimental system considered, the convolutional layer coupled with the biLSTM (CNN+biLSTM) provides the largest Q-factor improvement compared to the reference linear chromatic dispersion compensation (2.9 dB improvement). Then, we examine the trade-off between the computational complexity and performance of all equalizers and demonstrate that the CNN+biLSTM is the best option when the computational complexity is not constrained, while when we restrict the complexity to some lower levels, the three-layer perceptron provides the best performance.

Recurrent neural networks achieving MLSE performance for optical channel equalization

Sai-Chandra-Kumari Kalla, Christian Gagné, Ming Zeng, and Leslie A. Rusch
Opt. Express 29(9) 13033-13047 (2021)

Equalization performance and complexity analysis of dynamic deep neural networks in long haul transmission systems

Oleg Sidelnikov, Alexey Redyuk, and Stylianos Sygletos
Opt. Express 26(25) 32765-32776 (2018)

Computational complexity comparison of feedforward/radial basis function/recurrent neural network-based equalizer for a 50-Gb/s PAM4 direct-detection optical link

Zhaopeng Xu, Chuanbowen Sun, Tonghui Ji, Jonathan H. Manton, and William Shieh
Opt. Express 27(25) 36953-36964 (2019)

Artificial neural networks for photonic applications—from algorithms to implementation: tutorial

Pedro Freire, Egor Manuylovich, Jaroslaw E. Prilepsky, and Sergei K. Turitsyn
Adv. Opt. Photon. 15(3) 739-834 (2023)

Bi-directional gated recurrent unit neural network based nonlinear equalizer for coherent optical communication system

Xinyu Liu, Yongjun Wang, Xishuo Wang, Hui Xu, Chao Li, and Xiangjun Xin
Opt. Express 29(4) 5923-5933 (2021)