## Abstract

A frequency-domain multiple-input multiple-output (FD-MIMO) equalizer employing a momentum-based gradient descent update algorithm is proposed for polarization multiplexing coherent receivers. Its performance in operation with dynamically varying optical channels is investigated and the impact of filter update delays, arising from the latency of the fast Fourier transforms (FFTs) and other digital signal processing (DSP) operations in the feedback loop, is assessed. We show that the proposed momentum-based gradient descent algorithm used to control the equalizer response has significantly greater tolerance to feedback delay than the conventional gradient descent algorithm. We considered a 92 Gbaud dual-polarization 64 QAM receiver, with DSP operating at two samples per symbol, and with the equalizer operating on blocks of 512 and 1024 samples (i.e., 512/1024-point FFT). We found that at an optical signal-to-noise power ratio (OSNR) of 35 dB, the momentum-based gradient descent algorithm can successfully track state-of-polarization (SOP) rotation at frequencies of up to 50 kHz and with filter update delays of up to 14 blocks (39 ns). In comparison, using the conventional gradient descent algorithm in an otherwise identical receiver, the equalizer performance starts to deteriorate at SOP rotation frequencies above 20 kHz.

Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

## 1. Introduction

The use of digital signal processing (DSP) in coherent receivers is a powerful technique to mitigate static and time-varying impairments from optical channels, such as chromatic dispersion (CD), polarization mode dispersion (PMD) and state-of-polarization (SOP) rotation, and a variety of linear filters and adaptive equalizers have been proposed [1–5]. For applications in short-reach links, it is desirable to minimize the complexity and power consumption. It has been shown that frequency-domain (FD) equalization, utilizing the fast Fourier transform (FFT), is more efficient than time-domain (TD) implementation [6–10]. However, in a practical real-time implementation of an adaptive equalizer with block-based processing, filter update delays will be induced due to the FFTs and other DSP operations in the feedback loop, which can degrade the equalizer’s tracking ability, particularly during rapid SOP fluctuations [11–14].

In this paper, we propose a momentum-based frequency-domain multiple-input multiple-output (FD-MIMO) equalizer for polarization-multiplexing coherent receivers, and carried out simulations of dual-polarization 92 Gbaud 64 QAM systems over 40 km and 80 km standard single-mode fiber (SSMF) links, with state-of-polarization rotation frequencies of up to 60 kHz. The FD-MIMO equalizer combines chromatic dispersion compensation (CDC), matched filtering (MF) and adaptive equalization (AQ), so that frequency to time domain conversions can be avoided to minimize the complexity. The adaptive equalizer is implemented in a blind manner, with the decision-directed least mean square (DD-LMS) algorithm being used for updating the filter coefficients [15]. The mean square error is calculated after carrier phase estimation (CPE) and then fed back to the adaptive equalizer. We show that the momentum-based gradient descent approach is significantly more robust to feedback latency than the conventional gradient update.

## 2. Operating principle of the FD-MIMO equalizer

Figure 1 shows the block processing of the proposed FD-MIMO equalizer with the overlap-save method being adopted due to its lower complexity compared to overlap-add approach [8,16]. The input time-domain signal is first partitioned into blocks of $M$ samples. In the overlap-save method, the overlap is achieved by taking $L$ samples from the last block, giving a total block length of $L+M=N_{FFT}$. After performing FD-MIMO equalization, the output blocks are combined together in the time domain with overlap samples being removed.

The schematic of proposed FD-MIMO equalizer is shown in Fig. 2. It is composed of two static equalizers and an adaptive equalizer. The static equalizers perform chromatic dispersion compensation and matched filtering. The adaptive equalizer mitigates polarization mode dispersion and tracks the SOP. The input time-domain signal, sampled at 2 samples per symbol, is transformed into the frequency domain using a pair of FFTs. As the chromatic dispersion is a static effect, it can be mitigated by multiplying the signal with the inverse transfer function of the dispersive link given by:

where $D$ denotes the dispersion parameter, $\lambda$ is the reference wavelength, $z$ represents the transmission distance, $c$ is the speed of light and $\omega$ denotes the angular frequency. It has been shown that in the frequency-domain block processing, the number of overlap samples is critical for effective chromatic dispersion compensation [16]. The minimum overlap is calculated assuming the Gaussian pulse broadening [16]:The implementation of frequency-domain adaptive equalizers is well documented [8,9]. The configuration of the adaptive equalizer in this work is similar to the frequency-domain equalizer implementation proposed in [9]. Nevertheless, it is worth noting that in our setup, the adaptive equalizer is updated using DD-LMS algorithm while constant modulus algorithm (CMA) method was employed in [9]. In addition, we do not divide the input signal into odd and even samples while in [9], the input signal was divided into odd and even samples to reduce the computational complexity of the adaptive equalizer.

In the proposed FD-MIMO equalizer, the four sets of filter tap weights are initialized in the time domain, each of length $L_{ADEQ}$. In the simulation, $L_{ADEQ}$ = 16. Together with the overlap samples used for chromatic dispersion compensation in the 40 km and 80 km links, the total overlap lengths $L$ were 250 and 482. The adaptive equalizer is operated in the constrained sense with the DD-LMS algorithm being employed to update the filter coefficients [15]. In order to make correct decisions on symbols using DD-LMS algorithm, phase noise needs to be removed first. In the simulation, QPSK pilot symbols inserted at the rate of $\frac {1}{32}$ are used for carrier phase estimation, giving an overhead of 3.1% [20]. The first five QPSK pilots with known phase and polarization state at the transmitter are used to obtain an initial estimation of the polarization rotation, and used to set the center tap of the filter coefficients, with the rest of taps being set to zero. Following this, FFTs are applied to convert the time-domain filter coefficients into the corresponding frequency response. After using pilot symbols for initialization of the adaptive filter coefficients, the adaptive equalizer is updated in a blind manner. Note that it is possible to use pilot symbols in updating the equalizer, which can potentially reduce the equalizer update instability and accelerate the convergence, and this will be investigated in the future.

The equalization process for the two polarizations is described by:

After converting the equalized signals back to the time domain and down-sampling to 1 sample per symbol, the phase noise is first estimated from pilot symbols and then linearly interpolated between the Wiener filter coefficients and pilot symbols. The Wiener filter is implemented as an FIR filter [21]. The inverse of the phase noise profile (i.e., $e^{-j\varphi }$) is then applied to the time-domain output signal. After removing the phase noise, the time-domain error vectors are calculated, considering only the payload symbols, given by:

where $\textbf {v}_{x,y}(K)$ denotes the time-domain adaptive equalizer output after down-sampling for $x$ or $y$ polarization as shown in Fig. 2, $\textbf {d}_{x,y}$ denotes the vector of the most likely symbols after hard decisions. The term $e^{j\varphi }$ in Eq. (5) is used to feed back the estimated phase noise after making hard decisions as the input signals (i.e., $\textbf {U}_x(K)$ and $\textbf {U}_y(K)$) of adaptive equalizer are still affected by the phase noise. The errors are then up-sampled and fed back to adjust the MIMO filter coefficients.The gradient is calculated in the constrained sense which ensures that the frequency-domain filter coefficients and time-domain filter coefficients are equivalent [8,9]. The time-domain gradients are given by:

Using conventional gradient descent for updating the filter coefficients, $\textbf {H}_{xx}(K+1)$ can be calculated in the frequency domain as:

where $\mu$ denotes the step size. $\textbf {G}_{xx}(K)$ is the frequency-domain representation of the time-domain gradient given by:Equations (6), (7) and (9) are known as constrained gradient estimation. Removing the gradient constraint can lower the calculation complexity, but will result in unstable equalization [8,9]. Therefore, the constrained gradient calculation has been employed in this work. The same operation can be applied to obtain $\textbf {H}_{xy}(K+1),\ \textbf {H}_{yx}(K+1),\ \textbf {H}_{yy}(K+1)$.

The implementation of the momentum-based gradient descent algorithm is shown in the red-dash box in Fig. 2. In contrast to the conventional gradient descent approach which only takes the current gradient into account, the momentum-based gradient estimation considers all the previous gradient updates, so that it can smooth out the gradient update oscillations [22]. Updating $\textbf {H}_{xx}(K+1)$ using momentum-based gradient is given by:

where $\textbf {S}_{xx}(K)$ denotes the frequency-domain momentum-based gradient. It is calculated as [22]: where $\textbf {G}_{xx}(K)$ is the frequency-domain conventional gradient defined in Eq. (9), $\beta$ is the momentum parameter from $[0,1]$, $\textbf {S}_{xx}(K-1)$ denotes the momentum-based gradient calculation from the previous block [22]. Updating filter coefficients $\textbf {H}_{xy}(K+1),\ \textbf {H}_{yx}(K+1)$, and $\textbf {H}_{yy}(K+1)$ using momentum-based gradient descent can be implemented in a similar approach.## 3. Simulation setup

The simulation was carried out using Matlab. We considered a 92 Gbaud dual-polarization 64 QAM receiver, with DSP operating at two samples per symbol, and with the equalizer operating on blocks of 512 samples or 1024 samples. The FFTs and other DSP operations in the feedback loop were assumed to require multiple clock cycles [23], giving rise to a delay in the feedback loop in the equalizer, and a consequent impact on the equalizer’s convergence rate. A long random input signal sequence of $9.8304\times 10^5$ symbols was generated, allowing the adaptive filter to converge to and then accurately track the signal polarization and compensate the link’s PMD. The system performance was evaluated on the last $55,536$ symbols of the signal once the adaptive equalizer had converged. At the transmitter, the dual-polarization 92 Gbaud 64 QAM signal was generated with the insertion of QPSK pilot symbols at the rate of $\frac {1}{32}$. The signal was up-sampled to 2 samples/symbol and spectrally shaped by an RRC filter with 0.01 roll-off factor, following which an ideal dual-polarization IQ modulator was applied. The transmitter laser was centered at 1550 nm and had a linewidth of 100 kHz. The launch power was set to -8 dBm.

The optical channel considered in this work was a single span of standard single-mode fiber (SSMF), with a length of 40 km or 80 km. A nonlinear wavelength-division multiplexing (WDM) system (7 channels) assuming -8 dBm launch power per channel and 100 GHz channel spacing was simulated. The impact of fiber nonlinearity was found to be negligible, and so, since the focus of the paper is on the equalizer’s tracking ability of SOP rotations, only the linear effects of chromatic dispersion, PMD and SOP rotations were included in the rest of the simulations described in this paper. The chromatic dispersion was assumed to be 16.8 $\mathrm{ps\cdot nm^{-1} \cdot km^{-1}}$. The effect of PMD was included by modeling the 40 km and 80 km fiber span as 40 and 80 cascaded 1 km sections respectively, each with random birefringence. For a given section $i$, it can be described using the Jones matrices $R_{i}$ and $D_{i}$ given by [24]:

At the receiver, the signal was combined with a local oscillator (LO) centered at 1550 nm in an ideal polarization-diverse 90 degree optical hybrid coherent receiver. The power of the LO was 13 dBm and its linewidth was 100 kHz. Photodetection and balanced detection were assumed to be ideal to cancel out common modes. The signal then passed through the proposed FD-MIMO equalizer to mitigate impairments from the optical channel (i.e., chromatic dispersion, PMD and SOP rotation) and the laser induced phase noise. Finally, the system performance was evaluated using generalized mutual information (GMI) once the adaptive equalizer had converged.

In the ideal case without delay in the filter update, the signal in block $K$ is equalized as shown in Eq. (4). However, in the presence of latency which is unavoidable in practical real-time implementations, the adaptive equalization for two polarizations will be:

## 4. Results and discussion

The SOP rotation frequency was varied from 0 to 60 kHz . The ASE noise was loaded, varying the received OSNR value. The performance was quantified using generalized mutual information (GMI)/$m$ where $m$ denotes the number of bits per symbol. For the 64 QAM signal, $m$ equals to 6. The (GMI)/m was obtained over ten runs of the simulation with random rotation and random phase shift (i.e., random $\theta$ and $\phi$) of the received signal at each SOP rotation frequency and OSNR value. The momentum parameter $\beta$ was set to 0.9 [22].

In the simulation of the 40 km link, the FFT size, corresponding to the block length, was 512 points. Figure 3 shows the performance of the FD-MIMO equalizer at an OSNR of 35 dB. Both conventional gradient descent and momentum-based gradient descent were implemented in the frequency domain as described in Section 2. In the absence of feedback latency (i.e., zero delay), the FD-MIMO equalizers using both the conventional gradient descent and the momentum-based gradient estimation successfully mitigate the phase noise, the static and dynamic distortions from the fiber, giving a GMI/$m$ of approximately 1 (12 bits/4D-symbol). When including a delay in the feedback used for the filter update, the tracking ability of the equalizer employing the conventional gradient calculation is impaired. Its performance starts to break down at a delay of 4 blocks and at a SOP rotation frequency of 60 kHz. In contrast, applying momentum-based gradient estimation, the tracking ability of the equalizer is significantly improved and is robust to delays up to 14 blocks (39 ns) with SOP rotation frequencies up to 50 kHz. At a SOP rotation frequency of 60 kHz, the equalizer’s tracking ability becomes impaired at feedback delays above 13 blocks.

At 50 kHz SOP rotation frequency and zero block delay, the received signal scatter plots for the $x$ polarization using conventional gradient descent and momentum-based gradient descent are compared in Fig. 4. The $y$ polarization performs similarly. Without a delay in the feedback loop, both methods achieve good channel equalization at 50 kHz SOP rotation frequency resulting a clean scatter plot. The noisy clouds around the ideal constellation points are mainly due to ASE noise. The scatter plots when employing the conventional gradient descent method with 8 blocks delay in the feedback loop, and the momentum-based gradient descent with 14 blocks delay are presented in Fig. 5. In the case of conventional gradient descent, the equalizer starts to fail to track 50 kHz SOP rotation at 8 blocks delay, resulting in a more distorted constellation (GMI/$m$ $\approx$ 0.28 as shown in Fig. 3). In contrast, employing momentum-based gradient descent, even at 14 blocks delay as shown in Fig. 5 (right), the equalizer is still capable of recovering the signal and the scatter plot is relatively clear (GMI/$m$ $\approx$ 0.98 as shown in Fig. 3).

The results with the 40 km link at an OSNR of 30 dB are shown in Fig. 6, and the maximum achievable GMI/m decreases slightly to 0.98 due to the additional ASE noise. The ASE noise also degrades the performance of the FD-MIMO equalizer. Similarly to the trend in Fig. 3, the conventional gradient estimation fails to track high SOP rotation ($\geq 30$ kHz) when there are delays of 2 blocks or more. In contrast, the performance of the momentum-based gradient descent algorithm only starts to deteriorate at a SOP rotation frequency of 60 kHz and at feedback delays of 8 blocks and above.

In Fig. 7, simulation results for the 80 km link with received OSNR of 27 dB are plotted, showing the performance for longer distance and lower OSNR. According to Eq. (2), the required overlap samples for chromatic dispersion compensation of 80 km SSMF are 466. The overlap samples used for the adaptive equalizer were kept at 16, and therefore, the total overlap length was 482. The FFT size was increased to 1024 points.

At the OSNR of 27 dB, the initial GMI/$m$ is around 0.91. The trends in the results are similar to those observed in the 40 km link as discussed above. However, due to the lower OSNR, the amount of ASE noise causes more perturbations to the equalizer’s tracking ability. The equalizer fails to track 10 kHz SOP rotation with feedback delay of 8 blocks or more when employing conventional gradient descent. In contrast, momentum-based gradient descent can still successfully track SOP rotations of up to 20 kHz with 13 blocks delay.

The evolution of SOP rotation can be analysed using Stokes parameters, and visualized on the Poincaré sphere [25]. Assuming a linearly polarized input (i.e., $s = [1,0,0]^{T}$) and 50 kHz SOP rotation frequency, the trajectory of the first $2^{16}$ time instances obtained using Eq. (13) are shown in Fig. 8 (left). To more exhaustively test the equalizer, we also simulated SOP fluctuations with time as random walks of the polarization angle and phase. In these simulations, $J_{SOP}$ in Eq. (13) was replaced by:

## 5. Conclusion

We have presented a frequency-domain multiple-input multiple-output equalizer for dual-polarization coherent receivers updated using a momentum-based gradient descent algorithm. Its performance was tested in a dynamically varying optical channel. The impact of feedback delays due to FFTs and other DSP operations in the feedback loop was also assessed. The proposed frequency-domain equalizer achieves SOP tracking with a high tolerance to feedback latency. In simulations of the dual-polarization 92 Gbaud 64 QAM signal transmitted over 40 km SSMF, with OSNR = 35 dB, and using 512-sample block processing, it is capable of tracking SOP rotation at frequencies of up to 50 kHz and compensating PMD even with delays of up to 14 blocks. In transmissions over 80 km SSMF, with OSNR = 27 dB, and using 1024-sample block processing, it is capable of tracking SOP rotation at frequencies of up to 20 kHz and compensating PMD even with delays of up to 13 blocks.

## Funding

Engineering and Physical Sciences Research Council (EP/R035342/1).

## Disclosures

The authors declare no conflicts of interest.

## References

**1. **S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express **16**(2), 804–817 (2008). [CrossRef]

**2. **E. Ip, A. P. T. Lau, D. J. Barros, and J. M. Kahn, “Coherent detection in optical fiber systems,” Opt. Express **16**(2), 753–791 (2008). [CrossRef]

**3. **M. S. Faruk and S. J. Savory, “Digital signal processing for coherent transceivers employing multilevel formats,” J. Lightwave Technol. **35**(5), 1125–1141 (2017). [CrossRef]

**4. **K. Zhong, X. Zhou, J. Huo, C. Yu, C. Lu, and A. P. T. Lau, “Digital signal processing for short-reach optical communications: A review of current technologies and future trends,” J. Lightwave Technol. **36**(2), 377–400 (2018). [CrossRef]

**5. **P. Winzer, A. Gnauck, C. Doerr, M. Magarini, and L. Buhl, “Spectrally efficient long-haul optical networking using 112-Gb/s polarization-multiplexed 16-QAM,” J. Lightwave Technol. **28**(4), 547–556 (2010). [CrossRef]

**6. **B. Spinnler, F. N. Hauske, and M. Kuschnerov, “Adaptive equalizer complexity in coherent optical receivers,” in European Conference on Optical Communication, (2008) p. We.2.E.4.

**7. **J. Leibrich and W. Rosenkranz, “Frequency domain equalization with minimum complexity in coherent optical transmission systems,” in Optical Fiber Communication Conference, (2010), p. OWV1.

**8. **J. J. Shynk, “Frequency-domain and multirate adaptive filtering,” IEEE Signal Process. Mag. **9**(1), 14–37 (1992). [CrossRef]

**9. **M. S. Faruk and K. Kikuchi, “Adaptive frequency-domain equalization in digital coherent optical receivers,” Opt. Express **19**(13), 12789–12798 (2011). [CrossRef]

**10. **C. Fougstedt, P. Johannisson, L. Svensson, and P. Larsson-Edefors, “Dynamic equalizer power dissipation optimization,” in Optical Fiber Communication Conference, (2016), p. W4A.2.

**11. **D. Lavery, B. C. Thomsen, P. Bayvel, and S. J. Savory, “Reduced complexity equalization for coherent long-reach passive optical networks [invited],” J. Opt. Commun. Netw. **7**(1), A16–A27 (2015). [CrossRef]

**12. **Q. Guo, B. Xu, and K. Qiu, “Studies on effects of feedback delay on the convergence performance of adaptive time-domain equalizers for fiber dispersive channels,” Opt. Eng. **55**(4), 046110 (2016). [CrossRef]

**13. **P. M. Krummrich, D. Ronnenberg, W. Schairer, D. Wienold, F. Jenau, and M. Herrmann, “Demanding response time requirements on coherent receivers due to fast polarization rotations caused by lightning events,” Opt. Express **24**(11), 12442–12457 (2016). [CrossRef]

**14. **T. Zeng, Z. he, L. Meng, J. Li, X. Li, and S. Yu, “The real time implementation of a simplified 2-section equalizer with supernal SOP tracking capability,” in Optical Fiber Communication Conference, (2020), p. M2J.7.

**15. **I. Fatadin, D. Ives, and S. J. Savory, “Blind equalization and carrier phase recovery in a 16-QAM optical coherent system,” J. Lightwave Technol. **27**(15), 3042–3049 (2009). [CrossRef]

**16. **T. Xu, G. Jacobsen, S. Popov, M. Forzati, J. Mårtensson, M. Mussolin, J. Li, K. Wang, Y. Zhang, and A. T. Friberg, “Frequency-domain chromatic dispersion equalization using overlap-add methods in coherent optical system,” J. Opt. Commun. **32**(2), 131–135 (2011). [CrossRef]

**17. **R. Kudo, T. Kobayashi, K. Ishihara, Y. Takatori, A. Sano, and Y. Miyamoto, “Coherent optical single carrier transmission using overlap frequency domain equalization for long-haul optical systems,” J. Lightwave Technol. **27**(16), 3721–3728 (2009). [CrossRef]

**18. **K. Ishihara, R. Kudo, T. Kobayashi, A. Sano, Y. Takatori, T. Nakagawa, and Y. Miyamoto, “Frequency-domain equalization for coherent optical transmission systems,” in Optical Fiber Communication Conference, (2011), p. OWW4.

**19. **K. Kikuchi, “Fundamentals of coherent optical fiber communications,” J. Lightwave Technol. **34**(1), 157–179 (2016). [CrossRef]

**20. **Y. Wakayama, E. Sillekens, L. Galdino, D. Lavery, R. I. Killey, and P. Bayvel, “Increasing achievable information rates with pilot-based DSP in standard intradyne detection,” in European Conference on Optical Communication, (2019), p. W.1.B.5.

**21. **E. Ip and J. M. Kahn, “Feedforward carrier recovery for coherent optical communications,” J. Lightwave Technol. **25**(9), 2675–2692 (2007). [CrossRef]

**22. **S. Ruder, “An overview of gradient descent optimization algorithms. arxiv 2016,” arXiv preprint arXiv:1609.04747 (2016).

**23. **J. S. Bruno, V. Almenar, and J. Valls, “FPGA implementation of a 10 GS/s variable-length FFT for OFDM-based optical communication systems,” Microprocess Microsy. **64**, 195–204 (2019). [CrossRef]

**24. **G. P. Agrawal, “Polarization effects,” * Nonlinear fiber optics* (Elsevier, 2013).

**25. **E. Collett, “The wave theory of light,” * Field guide to polarization* (Spie, Bellingham, WA, 2005).