Abstract

Practical implementation of digital signal processing for mitigation of transmission impairments in optical communication systems requires reduction of the complexity of the underlying algorithms. Here, we investigate the application of convolutional neural networks for compensating nonlinear signal distortions in a 3200 km fiber-optic 11x400-Gb/s WDM PDM-16QAM transmission link with a focus on the optimization of the corresponding algorithmic complexity. We propose a design that includes original initialisation of the weights of the layers by a filter predefined through the training a single-layer convolutional neural network. Furthermore, we use an enhanced activation function that takes into account nonlinear interactions between neighbouring symbols. To increase learning efficiency, we apply a layer-wise training scheme followed by joint optimization of all weights applying additional training to all of them together in the large multi-layer network. We examine application of the proposed convolutional neural network for the nonlinearity compensation using only one sample per symbol and evaluate complexity and performance of the proposed technique.

Joint intra and inter-channel nonlinearity compensation based on interpretable neural network for long-haul coherent systems

Du Tang, Zhen Wu, Zhongliang Sun, Xizi Tang, and Yaojun Qiao
Opt. Express 29(22) 36242-36256 (2021)

Convolutional long short-term memory neural network equalizer for nonlinear Fourier transform-based optical transmission systems

Oleksandr Kotlyar, Morteza Kamalian-Kopae, Maryna Pankratova, Anastasiia Vasylchenkova, Jaroslaw E. Prilepsky, and Sergei K. Turitsyn
Opt. Express 29(7) 11254-11267 (2021)

Equalization performance and complexity analysis of dynamic deep neural networks in long haul transmission systems

Oleg Sidelnikov, Alexey Redyuk, and Stylianos Sygletos
Opt. Express 26(25) 32765-32776 (2018)

Joint intra and inter-channel nonlinear compensation scheme based on improved learned digital back propagation for WDM systems

Xinyu Chi, Chenglin Bai, Fan Yang, Qi Qi, Ruohui Zhang, Hengying Xu, Lishan Yang, Wanxiang Bi, Tianchi Chen, and Shunchang Bai
Opt. Express 32(4) 5095-5116 (2024)

Neural network architectures for optical channel nonlinear compensation in digital subcarrier multiplexing systems

Ali Bakhshali, Hossein Najafi, Behnam Behinaein Hamgini, and Zhuhong Zhang
Opt. Express 31(16) 26418-26434 (2023)