## Abstract

We report a two-stage blind frequency domain equalization method for long-haul coherent polarization-multiplexed (pol-mux) systems using quadrature phase shift keying (QPSK) and 16-quadrature amplitude modulation (16-QAM). In the first stage, blind CD parameter prediction is conducted prior to a CD equalizer. This supports flexible path switching in optical networks. In the second stage, a frequency-domain multi-modulus algorithm (MMA) equalizer is used to cope with the residual fiber impairments and perform polarization de-multiplexing. Compared with the conventional constant modulus algorithm (CMA), MMA shows advantages including better steady state performance and a faster convergence rate. Furthermore, all the estimation and equalization algorithms are implemented in the frequency domain which potentially provides the least complexity for the pol-mux optical coherent systems. The proposed algorithm is experimentally demonstrated with an 800-km 10 Gbaud coherent optical pol-mux system. For QPSK signal, the proposed method achieves error-free transmission and shows superior convergence speed against CMA, and for 16-QAM signals, the proposed MMA outperforms CMA with more than 1-dB improvement in Q-value.

© 2012 OSA

## 1. Introduction

Enabled by coherent detection technique, advanced modulation formats improve spectral efficiency to support the growing demand of capacity in fiber transmission systems [1]. Moreover, polarization multiplexing can be utilized to double bandwidth efficiency. 100 Gb/s pol-mux quadrature phase shift keying (QPSK) systems are becoming widely deployed for optical transport networks. More recently, high-order quadrature amplitude modulation (QAM) formats are being extensively investigated to pursue further compression of the signal spectrum [2–4].

Coherent detection retains both the amplitude and phase information of the received signal, allowing digital signal processing (DSP) to compensate for linear optical channel impairments such as chromatic dispersion (CD) and polarization mode dispersion (PMD) [5–7]. Compared with time domain (TD) DSP approaches that use finite-impulse-response (FIR) filters, frequency domain (FD) implementations show advantages in terms of lower computational effort as they employ fast Fourier transforms (FFT) and block processing [8]. Among existing single carrier frequency domain equalization (SC-FDE) systems, the data-aided (DA) approach has been proposed to allow impairment compensation in a single step [9]. However this technique requires the transmission of a long training sequence for channel estimation, which reduces spectral efficiency. The FD non-data-aided (NDA) method [10] also achieves equalization in a single step, but uses a constant modulus algorithm (CMA), which exhibits large errors for high-order QAM systems and has only been demonstrated for short transmission distances.

The two-stage equalization concept [5, 8] is generally adopted by most of the NDA systems for long-haul transmission, where polarization independent effects, such as CD, are mitigated in the first stage, and the second-stage equalizer typically compensates for the polarization-dependent impairments and also performs polarization de-multiplexing. However, all previous work [11–13] required a priori knowledge of the CD parameter for the CD compensation, which is unsuitable for switched optical networks; moreover, the second stage equalizer designs of these works were all using the TD approach, which are not optimized to achieve low complexity for the NDA system.

This paper provides a comprehensive theoretical analysis and detailed experimental results of the two-stage FD equalization method for pol-mux NDA QPSK and 16-QAM systems proposed in [14]. In the first stage, a blind CD parameter prediction is conducted prior to the CD equalizer, which works over a wide range of dispersions. The second stage uses a FD multiple-input multiple-output (MIMO) equalizer, based on the multi-modulus algorithm (MMA), to provide better performance than CMA adaptation. In 800-km 40-Gb/s pol-mux QPSK and 80-Gb/s pol-mux 16-QAM transmission experiments, the proposed FD algorithms show much lower implementation complexity, without performance degradation, when compared with TD and CD parameter assisted systems. We show that our second-stage FD-MMA algorithm provides faster convergence and better steady-state error performance than a FD-CMA implementation.

The paper is organized as follows. Section 2 describes the system design including the system setup and DSP schedules. The experimental results are presented and discussed in Section 3. Finally, Section 4 summarizes the paper.

## 2. Two-stage NDA SC-FDE system design

#### 2.1 Receiver DSP architecture

Figure 1 shows the receiver DSP scheme. The real and imaginary parts (I and Q) of the two polarizations are sampled by a real-time oscilloscope to form two complex signal streams. After resampling to 2 samples/symbol, the signal is first transformed into the frequency domain in a block format. Then the signal is passed through the first FD equalizer for CD estimation and compensation. The equalizer tap length in this stage depends on the transmission distance and can be derived according to [5]. Due to the delay spread being dominated by CD, the CD equalizer tap length is much longer than that in the second stage, thus the output of the first equalizer is rearranged into shorter blocks before passing through the half-symbol-spaced 16-tap FD adaptive equalizer in the second stage. In the next step, the frequency offset is estimated using spectrum-based NDA method [15] which is implemented in FD and is independent of modulation formats. After frequency acquisition, phase estimation is carried out using decision-aided maximum likelihood algorithm (DAML) [16]. Finally, symbol de-mapping is performed before error counting.

#### 2.2 Blind CD compensation

The fixed coefficients of the first stage FD CD equalizer is determined by the quadratic spectral phase characteristic of CD [8]:

where: $\lambda $ is the carrier wavelength, $f$ is the baseband signal frequency, $c$ is the speed of light and $\beta $ is the accumulated CD in (ps/nm). An auto-correlation-based FD maximum search method with good performance for 2-fold over-sampling systems [17] is employed to estimate$\beta $. First a range of potential values of ${\beta}_{i}$ $(i=1,2,\mathrm{...},N)$is set with a scanning resolution of 100 ps/nm, then for each potential value ${\beta}_{i}$a corresponding CD compensation function ${H}_{CD,}{}_{i}$ is defined. The received signal spectrum blocks may now be filtered with each CD equalizer candidate, and then the related metric is calculated for each ${\beta}_{i}$ according to the clock-tone magnitude cost function proposed in [17]. Finally the estimated ${\beta}_{i}$ for successful CD compensation is indicated by the FD maximum metric feature. After the equalizer tap weights have been fixed by substituting in the estimated parameter, an overlap frequency domain equalization (OFDE) [8] is conducted with a 1/4 overlap ratio to mitigate the majority of the CD.#### 2.3 Frequency domain blind equalization

The second stage equalizer can operate on shorter blocks of data, so for numerical efficiency the output samples of stage one are rearranged into shorter blocks. The output samples of the CD equalizer are first transformed back to TD. Then, to let the frequency domain equalizer use 2-fold oversampling, for each polarization the samples are split into two tributaries: one for even samples or odd samples. Blocks of 16 samples are collected and four FFTs are used to transform each of these four tributaries to the frequency domain.

In the following analysis, $N$ equals to 8, which is half of the equalizer length, bold-face letters are used to represent vectors, ${[\u2022]}^{H}$denotes the Hermitian transpose, ${[\u2022]}^{*}$ indicates the complex conjugate operation, ${[\u2022]}^{T}$ stands for the matrix transpose, ${[\u2022]}^{e/o}$ stands for even or odd tributary of the samples, $i$ and $j$ indicate either $x$ or $y$ polarization, ${0}_{N}$and ${I}_{N}$ are defined as $N\times N$ zero and identity matrix, while $P={[{I}_{N}{0}_{N}]}^{T}$ and $Q={[{0}_{N}{I}_{N}]}^{T}$ are used to pad zeroes before and after the vector being multiplied, respectively.

With the conventional TD blind MIMO equalizer, the output can be formulated as [5]:

where ${w}_{ij}$ is a $2N\times 1$ column vector representing the TD equalizer coefficients. As shown in Fig. 2(a) , the frequency domain implementation of the adaptive equalizer is based on the even-odd sub-equalizers approach [10] and the 50% overlap-save sectioning method [18]. Firstly we define the $2N\times 1$ auxiliary coefficient vector${W}_{ij}^{e/o}=FFT[P\cdot {w}_{ij}^{e/o}]$, and ${E}_{ij}^{e/o}=FFT[P\cdot {({w}_{ij}^{e/o})}^{*}]$ as the FD tap weight vector used for equalization, where $P={[{I}_{N}{0}_{N}]}^{T}$ means adding zeroes after the vector being multiplied. The relationship between ${W}_{ij}^{e/o}$ and ${E}_{ij}^{e/o}$ can be expressed as [19]:where $K=\left(\begin{array}{cc}1& zeros(1,2N-1)\\ zeros(2N-1,1)& fliplr({I}_{2N-1})\end{array}\right)$ aims to convert between FFT of a vector and the FFT of its complex conjugate vector, and $fliplr(\u2022)$ means flipping the matrix from left to right. Then the FD equalizer output can be expressed as:*N*× 1 FD input vector, $G=[{0}_{N}{I}_{N}]$ is used to filter out the first $N$ samples of the equalized output represented by the

*IFFT*bracket, and ${x}_{out}/{y}_{out}$ is the remaining $N$ TD output samples.

The gradient estimation is shown in Fig. 2(b), where the TD error, ${h}_{x/y}$, is first calculated using various algorithms which will be discussed in details later, and then the FD error can be estimated by taking the FFT of the zero-padding TD error [18]:

where $Q={[{0}_{N}{I}_{N}]}^{T}$ is used to pad $N$ zeros before the original $N$ TD errors.Using the knowledge of the FD error, we can update the auxiliary coefficient vector to be:

Figure 3 compares the computational complexity for this FD second stage setup (with either CMA or MMA) with the conventional TD-CMA method. The computational complexity of TD-CMA is $(12N+2)$ multiplications per symbol [10], while FD-MMA requires$(12{\mathrm{log}}_{2}(2N)+12)$ multiplications per symbol including 24 $2N\text{-point}$ FFT/IFFT operations and $24N$ multiplications to output $2N$ symbols for two polarizations. From Fig. 3(b), it is clear that the implementation complexity of the FD method will be much lower than the TD technique when FFT size $(2N)$ is ≥8; also, the complexity of FD-MMA is slightly higher than FD-CMA because the MMA needs to update the real and imaginary parts of the errors separately.

Regarding the error calculation, CMA is a well known approach and is well established in commercial systems. Its error function can be formulated as [5]:

where $E[\u2022]$ is the expectation operation, ${s}_{n}$ indicates the transmitted symbol, e.g. $[1+j,1-j,-1+j,-1-j]$ for QPSK modulation. For MMA, the real and imaginary parts of the TD error can be calculated as [20]:Figure 4 shows the convergence principle of CMA and MMA for QPSK and 16-QAM modulation formats. As indicated in Figs. 4(a) and 4(b), the convergence contour of CMA can be thought as a single dimensional update following a constant modulus (a circle). While from Figs. 4(c) and 4(d) it is clear that the convergence principle of MMA can be considered as the sum of two one-dimensional error functions, therefore the convergence rate for MMA is expected to outperform CMA as it simultaneously minimizes the errors of the real and imaginary parts of the equalized signal around these two separate straight moduli (two moduli with same value when the QAM constellation is square).

## 3. Experimental results and discussion

Figure 5
shows the experimental setup for the NDA SC-FDE system. A pair of 10 Gsymbols/s arbitrary waveform generators are used to generate two independent baseband signal streams with a desired modulation format. The data modulates the output of a 100-kHz linewidth external cavity laser using two optical I/Q modulators, one for each polarization. After being polarization-multiplexed with a polarization beam combiner, the signals are transmitted through several 80-km amplified spans of standard single-mode fiber (SSMF), then through a commercial PMD emulator yielding 30-ps mean differential group delay (DGD). At the receiver end, the signal goes through an optical band-pass filter followed by a polarization beam splitter. The signal is then detected by an optical 90-degree hybrid with a local oscillator, balanced photodiodes and low-pass filters. An Agilent real-time oscilloscope is then used to sample the outputs of the receivers at 40 Gsamples/s, with 8 × 10^{5} samples for each polarization stored for off-line processing.

Figure 6 shows the CD estimation errors versus transmission distance. The largest estimation error for both the QPSK and the 16-QAM systems is 400 ps/nm, which demonstrates that the FD CD estimation algorithm is precise enough for the 16-tap second stage FD blind equalizer to absorb the residual fiber impairments.

For back-to-back (B2B) QPSK transmission with only PMD impairments, the measured Q-values versus different step sizes with 5000 iterations for FD-CMA and FD-MMA are plotted in Fig. 7(a) . The optimum step size of 0.003 for CMA is slightly larger than that of 0.001 for MMA, while MMA can achieve similar performance with a step size of 0.003. We can get a better performance using very small step sizes combined with a large number of iterations; however, this is very time-consuming and has high implementation complexity. Thus we only consider solutions that are a good compromise between performance and convergence time. Figure 7(b) shows the Q-values versus iterations for FD-CMA and FD-MMA algorithms with different step sizes after B2B and 800-km transmission. It is clear that with the same step size, MMA converges much faster than CMA, although a smaller step size (μ = 0.001) slows down the MMA’s convergence speed. However, it is still faster then CMA with larger step size. This characteristic of MMA can save a large number of symbols and the time needed for the blind adaptation. Figure 7(c) shows the measured Q values versus transmission distance for QPSK. The results for both TD-CMA and FD-CMA are almost the same, which shows that the FD implementation achieves a similar performance to the TD one. Moreover, the measured Q values using FD-MMA do not depend on prior knowledge of the CD, which confirms that the proposed method works well. The Q value is close to 17 dB after 800-km transmission, which means error free transmission can be achieved using either CMA or MMA. However MMA can speed up the blind adaptation with its faster convergence rate. Figures 7(d) and 7(e) show the constellation diagrams of the equalized signal for x and y polarization after 800-km transmission, respectively.

Figure 8(a) shows the B2B transmission performance for 16-QAM with different step sizes for FD-CMA and FD-MMA. Similar to the QPSK case, the optimum step size for MMA is also smaller than for CMA. In this case we use the optimum step size for MMA and CMA separately in order to gain better equalization performance. Figure 8(b) shows the Q-value versus number of iterations for MMA and CMA after B2B and 800-km transmission. MMA converges similarly to CMA with a smaller step size, which again illustrates the faster converge feature of MMA. This time MMA outperforms CMA with a better steady-state Q-value performance. Figure 8(c) shows the transmission performance for 16-QAM system. FD-MMA outperforms FD-CMA for all transmission distances. The Q-value for 800 km is about 9 dB, which is larger than the 20% soft-decision FEC limit of 7 dB for error free transmission. By comparing the constellation diagrams of the equalized 16-QAM x and y polarization signals after 800-km transmission with the QPSK case in Fig. 6, we see that the degradation of Q-value for the 16-QAM signal is due to much denser constellations and less optical power obtained by the higher-order modulation formats.

## 4. Conclusions

We have developed a two-stage FD blind equalization method for long-haul pol-mux coherent optical system. CD is blindly estimated using frequency domain maximum search method and then mitigated with OFDE in the first stage, offering the ability to support switched optical paths. MMA is adopted in the second stage with even-odd and 50% overlap-save sectioning frequency domain implementation. Experiments for 10 Gbaud QPSK and 16-QAM systems were conducted to demonstrate the proposed FD algorithms and showed good performance for a low complexity. Compared with conventional CMA, MMA is able to provide a faster convergence rate with negligible performance degradation for QPSK systems, and MMA gives more than 1 dB Q-value improvement for 16-QAM system with similar convergence speed.

## Acknowledgments

NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program. The research is also partially supported by the Australian Research Council Centre of Excellence for Ultrahigh Bandwidth Devices for Optical Systems.

## References and links

**1. **J. Yu and X. Zhou, “Ultra-high-capacity DWDM transmission system for 100G and beyond,” IEEE Commun. Mag. **48**(3), S56–S64 (2010). [CrossRef]

**2. **P. J. Winzer, A. H. Gnauck, C. R. Doerr, M. Magarini, and L. L. Buhl, “Spectrally efficient long-haul optical networking using 112-Gb/s polarization-multiplexed 16-QAM,” J. Lightwave Technol. **28**(4), 547–556 (2010). [CrossRef]

**3. **I. Fatadin, D. Ives, and S. J. Savory, “Blind equalization and carrier phase recovery in a 16-QAM coherent optical system,” J. Lightwave Technol. **27**(15), 3042–3049 (2009). [CrossRef]

**4. **T. Pfau, S. Hoffmann, and R. Noé, “Hardware-efficient coherent digital receiver concept with feedforward carrier recovery for M-QAM constellations,” J. Lightwave Technol. **27**(8), 3042–3049 (2009). [CrossRef]

**5. **S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express **16**(2), 804–817 (2008). [CrossRef] [PubMed]

**6. **M. Kuschnerov, M. Chouayakh, K. Piyawanno, B. Spinnler, E. de Man, P. Kainzmaier, M. S. Alfiad, A. Napoli, and B. Lankl, “Data-aided versus blind single-carrier coherent receivers,” IEEE Photon. J. **2**(3), 387–403 (2010). [CrossRef]

**7. **B. Spinnler, “Equalizer design and complexity for digital coherent receivers,” IEEE Sel. Top. in Quantum Electron. **16**(5), 1180–1192 (2010). [CrossRef]

**8. **R. Kudo, T. Kobayashi, K. Ishihara, Y. Takatori, A. Sano, and Y. Miyamoto, “Coherent optical single carrier transmission using overlap frequency domain equalization for long-haul optical systems,” J. Lightwave Technol. **27**(16), 3721–3728 (2009). [CrossRef]

**9. **A. V. Tran, C. Zhu, C. C. Do, S. Chen, T. Anderson, D. Hewitt, and E. Skafidas, “8×40-Gb/s optical coherent pol-mux single carrier system with frequency domain equalization and training sequences,” IEEE Photon. Technol. Lett. **24**(11), 885–887 (2012). [CrossRef]

**10. **M. S. Faruk and K. Kikuchi, “Adaptive frequency-domain equalization in digital coherent optical receivers,” Opt. Express **19**(13), 12789–12798 (2011). [CrossRef] [PubMed]

**11. **S. Yamanaka, T. Kobayashi, A. Sano, H. Masuda, E. Yoshida, Y. Miyamoto, T. Nakagawa, M. Nagatani, and H. Nosaka, “11 × 117 Gb/s PDM 16-QAM Transmission over 1440 km with a spectral efficiency of 6.4 b/s/Hz using high-speed DAC,” in European Conference and Exhibition on Optical Communication (ECOC), paper We.8.C.1 (2010).

**12. **A. H. Gnauck, P. J. Winzer, C. R. Doerr, and L. L. Buhl, “10 × 112-Gb/s PDM 16-QAM transmission over 630 km of fiber with 6.2-b/s/Hz spectral efficiency,” in Optical Fiber Communication Conference (OFC), paper PDPB8 (2009).

**13. **X. Zhou, J. Yu, M.-F. Huang, Y. Shao, T. Wang, P. Magill, M. Cvijetic, L. Nelson, M. Birk, G. Zhang, S. Ten, H. B. Matthew, and S. K. Mishra, “Transmission of 32-Tb/s capacity over 580 km using RZ-shaped PDM-8QAM modulation format and cascaded multimodulus blind equalization algorithm,” J. Lightwave Technol. **28**(4), 456–465 (2010). [CrossRef]

**14. **C. Zhu, A. V. Tran, S. Chen, L. B. Du, T. Anderson, A. J. Lowery, and E. Skafidas, “Dual-stage frequency domain equalization for long-haul coherent polarization-multiplexed QPSK and 16-QAM systems,” in European Conference on Optical Communication (ECOC), paper We.1.A.2 (2012).

**15. **T. Nakagawa, M. Matsui, T. Kobayashi, K. Ishihara, R. Kudo, M. Mizoguchi, and Y. Miyamoto, “Non-data-aided wide-range frequency offset estimator for QAM optical coherent receivers,” in Optical Fiber Communication Conference (OFC), paper OMJ1 (2011).

**16. **S. Zhang, P. Y. Kam, C. Yu, and J. Chen, “Decision-aided carrier phase estimation for coherent optical communications,” J. Lightwave Technol. **28**(11), 1597–1607 (2010). [CrossRef]

**17. **F. N. Hauske, Z. Zhang, C. Li, C. Xie, and Q. Xiong, “Precise, robust and least complexity CD estimation,” in Optical Fiber Communication Conference (OFC), paper JWA32 (2011).

**18. **J. J. Shynk, “Frequency-domain and multirate adaptive filtering,” IEEE Sig. Proc. Mag. **9**(1), 14–37 (1992). [CrossRef]

**19. **B. Porat, *A Course in Digital Singal Processing* (Wiley, 1997).

**20. **J. Yang, J.-J. Werner, and G. A. Dumont, ““The multimodulus blind equalization and its generalized algorithms,” IEEE J. Sel. Areas on Commun. **20**(5), 997–1015 (2002). [CrossRef]