## Abstract

Coherent detection in combination with digital signal processing has recently enabled significant progress in the capacity of optical communications systems. This improvement has enabled detection of optimum constellations for optical signals in four dimensions. In this paper, we propose and investigate an algorithm for the blind adaptive equalization of one such modulation format: polarization-switched quaternary phase shift keying (PS-QPSK). The proposed algorithm, which includes both blind initialization and adaptation of the equalizer, is found to be insensitive to the input polarization state and demonstrates highly robust convergence in the presence of PDL, DGD and polarization rotation.

© 2011 OSA

## 1. Introduction

Polarization and phase diverse coherent detection with digital signal processing has become an essential technique for mitigating fiber transmission impairments and therefore increasing capacity [1]. The basis of polarization and phase diverse coherent detection is that the in-phase and quadrature components of the two orthogonal polarizations are detected, corresponding to all four dimensions of the incoming optical field [2]. As all four dimensions of the field are detected, transmission induced distortions such as chromatic dispersion (CD) and polarization mode dispersion (PMD) may be compensated, and the effects of polarization dependent loss (PDL) mitigated [3]. Recently, much research has been performed into the compensation of self phase modulation (SPM) [4], [5], [6]. These digital techniques have demonstrated an increase in launch power of at most 2.5 dB [5] at great computational cost.

While the detection of all four dimensions of the incoming optical field has enabled mitigation of transmission impairments, it has also enabled the use of high-level modulation formats such as quaternary phase shift keying (QPSK) [7] and 16-state quadrature amplitude modulation (16QAM) [8]. These modulation formats are most commonly simultaneously transmitted on two orthogonal linear polarizations, doubling the achievable spectral efficiency. In this paper we will denote the use of polarization multiplexing with the prefix ‘DP’ for dual-polarization.

Recently, research has been performed into determining the optimum modulation format in four dimensions for the power constrained, uncoded case [9], [10], with some research being performed into using the extra capacity afforded by using an optimal 24-state constellation as coding overhead [11].

The modulation format presented in [9] and displayed in Fig. 1
– polarization-switched QPSK (PS-QPSK) – is of particular interest, as it has been shown to be the optimal 8-level modulation format in terms of *asymptotic power efficiency* for coherent optical communication. PS-QPSK modulation consists of a QPSK symbol (containing two bits of information) being transmitted on one of two orthogonal polarizations (determining the third bit of information). While the achievable spectral efficiency is reduced from 4 (b/s)/Hz to 3 (b/s)/Hz, an improvement of up to 1.76 dB in receiver sensitivity is possible. More recently, research into the performance of PS-QPSK in transmission has entered the literature. While this may be contrary to the trend for higher levels of modulation and more dense constellations, there is still a demand for highly robust transmission at the expense of spectral efficiency for ultra long-haul applications [12], [13]. Despite recent interest in PS-QPSK modulation, DSP algorithms specifically designed for it have not yet been reported. Recent work focusing on transmission performance of PS-QPSK over uncompensated links [14] used an equalizer utilizing a training sequence, in contrast to current DP-QPSK systems which employ blind algorithms to minimize overhead due to training symbols. The work presented in [15] which examined the performance of PS-QPSK over dispersion managed links, did not discuss equalization. The work presented in reference [14] demonstrates an increase in span loss tolerance of approximately 2 dB comparing PS-QPSK to DP-QPSK at 111 Gb/s over a transmission link of 20 spans consisting of 90 km standard single mode fiber (SSMF) with either no dispersion compensation, or full optical dispersion compensation per span. Similarly, the study presented in [15] indicates that PS-QPSK modulation offers a 2 dB increase in Q-factor over DP-QPSK in transmission at 112 Gb/s over a transmission link of 20 spans of 100 km SSMF with full optical dispersion compensation per span.

In this paper we will focus on the digital equalization of PS-QPSK, which is the optimal 8-state four-dimensional constellation in terms of *asymptotic power efficiency,* giving the greatest tolerance to noise of any four-dimensional modulation format at a given bit rate.

## 2. The polarization-switched CMA equalizer

A blind equalizer is widely considered a desirable algorithm as it enables the elimination of overhead due to training sequences. While the decision-directed algorithm may be used without training sequences, it is known to have poor convergence when used without some pre-conditioning of the filter coefficients. This is often achieved with the dual-polarization CMA (DP-CMA) [16] for dual polarization (DP) modulation formats. Due to the form of the PS-QPSK constellation, the DP-CMA is fundamentally unsuited for either filter pre-conditioning or equalization. This is due to the fact that the DP-QPSK constellation has a constant modulus per polarization, which may be obtained by summing the two input polarizations of a PS-QPSK signal. The use of a DP-CMA with a PS-QPSK signal will therefore result in both output polarizations converging to the sum of both input polarizations. This will result in all information which is encoded onto the polarization state being discarded and both output polarizations being identical.

We propose the polarization-switched constant modulus algorithm equalizer, or PS-CMA. This equalizer utilizes a decision on the relative power in each output polarization from the equalizer. The polarization with more power is assumed to contain the QPSK symbol while the other polarization is assumed to contain only noise. We therefore minimize two error signals with differing moduli: the polarization with the QPSK symbol is forced toward a unit modulus, while the other polarization is forced toward the origin. This error term is therefore described in the pseudo-code in Eq. (1):

The taps of the four filters were adapted using the least mean squares algorithm given by Eq. (2) [17]:

*denotes the Hermite conjugate. If the algorithm is initialized with fixed tap weights, it is possible to find an input polarization state which will cause mal-convergence to occur. However, the nature of the PS-QPSK signal enables us to initialize the filter coefficients such that the expected cross correlation of the equalizer output powers is minimized (see appendix), which in turn avoids mal-convergence.*

^{H}## 3. Performance analysis of the PS-CMA equalizer

The PS-CMA equalizer with PS-QPSK modulation has an attractive advantage over the DP-CMA equalizer in combination with DP-QPSK modulation, in addition to superior noise tolerance associated with the PS-QPSK format. As the two switched QPSK tributaries are orthogonal, the expected cross correlation of the output powers may be minimized during the filter initialization process. To test the polarization sensitivity of the equalizer, we noise loaded the signal to 5.8 dB SNR per bit (which corresponds to a BER of 10^{−3}) and applied a rotation of the form given by Eq. (4).

The angular parameters in Eq. (4) were uniformly sampled, providing 64 points over each dimension. The signal was then equalized with a 7 tap PS-CMA equalizer, initialized as described in the appendix. Error counting was performed over 2^{18} symbols for each point, with Q-factor then calculated according to [18]. For a convergence parameter μ = 10^{−3} and a convergence period of 2^{18} symbols, a uniform penalty of 0.1 dB in Q-factor was observed with no sensitivity to input polarization state. In this and all subsequent simulations, we have neglected the effects of varying carrier phase - estimation of which has already been demonstrated [15] - in order to isolate and characterize the properties of the PS-CMA equalizer.

An essential capability of any practical equalizer is the ability to operate effectively in the presence of polarization dependant loss (PDL). To characterize the performance of the equalizer in the presence of PDL we took the transmitted optical signal, applied a polarization rotation according to Eq. (4) and then applied loss to one polarization. A second independent polarization rotation was then performed to vary the input polarization orientation to the equalizer, before the signal was noise loaded to 5.8 dB SNR per bit. Again, all three angular parameters in Eq. (4) were uniformly sampled with 32 points over each dimension for each rotation. A sequence of 2^{18} symbols was simulated for each polarization state and amount of PDL. A 7 tap equalizer was used with a convergence parameter μ = 10^{−3}. We have plotted the mean Q-penalty resulting from PDL in Fig. 2(a)
.

From Fig. 2(a) we note that the tolerance of small levels of PDL is good, with 3 dB of PDL resulting in a Q-factor penalty of less than 1 dB. It is noted that for levels of PDL up to 5 dB, mal-convergence was not observed and penalty accumulates approximately linearly. This is in contrast to when DP-QPSK modulation is used with the standard DP-CMA equalizer: singular mal-convergence causes a dramatic increase in penalty with more than a few dB of PDL as discussed in [19], [20]. Improved performance is described in [19], using an equalizer which utilizes independent component analysis (ICA). The performance of DP-QPSK modulation with a modified DP-CMA with ICA is similar to the PS-CMA with PS-QPSK, resulting in a maximum OSNR penalty of 1.2 dB, compared with a mean Q penalty of 0.9 dB for 3dB of PDL.

Due to the time-varying birefringence of optical fiber, another important characteristic of a digital equalizer for coherent optical communication is the ability to track the time varying state of polarization at the input of the receiver. To measure the performance of the receiver in this respect, we rotated the transmitted signal by a Jones matrix with a time varying circular rotation, such that ϕ and ψ remain zero while θ is increased at a constant rate to produce a rotation with constant angular frequency. The signal was then noise loaded to 5.8 dB SNR per bit and equalized with a 7 tap PS-CMA equalizer prior to error counting. Error counting was performed over 2^{20} symbols for each simulated point with 2^{18} symbols used for equalizer convergence. Performance was measured by Q-factor against both the relative frequency of polarization rotation and the PS-CMA convergence parameter μ. The results of this simulation are presented in Fig. 2(b).

It is noted from Fig. 2(b) that an increased convergence parameter μ enables a faster polarization rotation to be tracked, at the expense of a reduction in receiver sensitivity. It is also noted that a polarization rotation frequency of approximately 0.1 mrad per symbol period may be tracked for a penalty in performance of approximately 0.5 dBQ.

We then performed simulations to determine the performance in the presence of first order differential group delay (DGD). The PS-QPSK signal was again noise loaded to 5.8 dB SNR per bit, and DGD applied on a variety of polarization axes such that the angular parameters defined in Eq. (4) were uniformly sampled with 32 points across each dimension. The signal was then equalized with a 7 tap PS-CMA equalizer with a convergence parameter μ = 10^{−3}. Errors were counted over 2^{20} symbols for each point with a convergence period of 2^{18} symbols. We found that DGD up to the length of the adaptive equalizer delay (1.5 symbol periods) could be compensated without additional penalty, and that the equalizer was insensitive to the polarization axis at which the DGD was applied.

## 4. Conclusions

We have proposed and analyzed a polarization-switched constant modulus algorithm (PS-CMA) which includes both the blind initialization and the adaptation of an equalizer. PS-QPSK modulation in combination with this algorithm exhibits remarkable robustness to PDL and input polarization state. This is particularly desirable as mal-convergence was not observed at any time for the PS-CMA with PS-QPSK modulation. This is in marked contrast to the standard DP-CMA with DP-QPSK modulation, which may experience severe singular mal-convergence issues with high PDL and certain input polarization states. PS-QPSK with the PS-CMA may therefore be considered advantageous in a high PDL channel where convergence is an issue for conventional systems and algorithms.

## Appendix

To ensure accurate convergence of the PS-CMA, we initialize the central taps to be of the form given by Eq. (5):

Our initial instantaneous equalizer outputs will therefore be equivalent to Eq. (6):

The optimum value of θ is that for which the expected correlation in equalizer output powers is minimized. This is equivalent to Eq. (7):

*θ*given by Eq. (8):

This one dimensional optimization is straightforward to solve analytically for the optimal value of *θ*, or using a direct search from a set of test values of *θ*.

## Acknowledgments

The work described in this paper was carried out with the support of EPSRC.

## References and links

**1. **A. Sano, H. Masuda, T. Kobayashi, M. Fujiwara, K. Horikoshi, E. Yoshida, Y. Miyamoto, M. Matsui, M. Mizoguci, H. Yamazaki, Y. Sakamaki, and H. Ishii, “69.1-Tb/s (432 x 171-Gb/s) C – and Extended L-Band Transmission over 240km Using PDM-16-QAM Modulation and Digital Coherent Detection,” Proc. OFC/NFOEC 2010, San Diego, CA, Mar. 21–25, 2009, PDPB7.

**2. **A. H. Gnauck, R. W. Tkach, A. R. Chraplyvy, and T. Li, “High-Capacity Optical Transmission Systems,” J. Lightwave Technol. **26**(9), 1032–1045 (2008). [CrossRef]

**3. **S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express **16**(2), 804–817 (2008). [CrossRef] [PubMed]

**4. **E. Ip and J. Kahn, “Compensation of Dispersion and Nonlinear Impairments Using Digital Backpropagation,” J. Lightwave Technol. **26**(20), 3416–3425 (2008). [CrossRef]

**5. **D. S. Millar, S. Makovejs, C. Behrens, S. Hellerbrand, R. I. Killey, P. Bayvel, and S. J. Savory, “Mitigation of Fiber Nonlinearity using a Digital Coherent Receiver,” IEEE J. Sel. Top. Quantum Electron. **16**(5), 1217–1226 (2010). [CrossRef]

**6. **L. Zhu, X. Li, E. Mateo, and G. Li, “Complementary FIR Filter Pair for Distributed Impairment Compensation of WDM Fiber Transmission,” IEEE Photon. Technol. Lett. **21**(5), 292–294 (2009). [CrossRef]

**7. **E. Ip and J. Kahn, “Digital Equalization of Chromatic Dispersion and Polarization Mode Dispersion,” J. Lightwave Technol. **27**, 3416–3425 (2007).

**8. **P. J. Winzer, A. H. Gnauck, C. R. Doerr, M. Magarini, and L. L. Buhl, “Spectrally Efficient Long-Haul Optical Networking using 112-Gb/s Polarization-Multiplexed 16-QAM,” J. Lightwave Technol. **28**(4), 547–556 (2010). [CrossRef]

**9. **M. Karlsson and E. Agrell, “Which is the most power-efficient modulation format in optical links?” Opt. Express **17**(13), 10814–10819 (2009). [CrossRef] [PubMed]

**10. **E. Agrell and M. Karlsson, “Power-Efficient Modulation Formats in Coherent Transmission Systems,” J. Lightwave Technol. **27**(22), 5115–5126 (2009). [CrossRef]

**11. **H. Bülow, “Polarization QAM Modulation (POL-QAM) for Coherent Detection Schemes,” Proc. OFC/NFOEC 2009, San Diego, CA, Mar. 22–26, 2009, OWG2.

**12. ***40G Submarine Applications,* Ciena Application Note, Ciena Corp, Available: www.ciena.com

**13. ***Alcatel-Lucent**1626**Light Manager*, Data Sheet release 6.1, Alcatel-Lucent, Available: www.alcatel-lucent.com

**14. **P. Poggiolini, G. Bosco, A. Carena, V. Curri, and F. Forghieri, “Performance evaluation of coherent WDM PS-QPSK (HEXA) accounting for non-linear fiber propagation effects,” Opt. Express **18**(11), 11360–11371 (2010). [CrossRef] [PubMed]

**15. **P. Serena, A. Vannucci, and A. Bononi, “The Performance of Polarization Switched-QPSK (PS-QPSK) in Dispersion Managed WDM Transmissions,” Proc. ECOC 2010, Th.10.E.2, Sept. 2010.

**16. **S. J. Savory, “Digital Coherent Optical Receivers: Algorithms and Subsystems,” IEEE J. Sel. Top. Quantum Electron. **16**(5), 1164–1179 (2010). [CrossRef]

**17. **S. Haykin, *Adaptive filter theory*, Prentice Hall, 2001.

**18. **N. S. Bergano, F. W. Kerfoot, and C. R. Davidsion, “Margin Measurement in Optical Amplifier Systems,” IEEE Photon. Technol. Lett. **5**(3), 304–306 (1993). [CrossRef]

**19. **M. Kuschnerov, F. N. Hauske, K. Piyawanno, B. Spinnler, M. S. Alfiad, A. Napoli, and B. Lankl, “DSP for Coherent Single-Carrier Receivers,” J. Lightwave Technol. **27**(16), 3614–3622 (2009). [CrossRef]

**20. **A. Vgenis, C. S. Petrou, C. B. Papadias, I. Roudas, and L. Raptis, “Nonsingular Constant Modulus Equalizer for PDM-QPSK Coherent Optical Receivers,” IEEE Photon. Technol. Lett. **22**(1), 45–47 (2010). [CrossRef]