## Abstract

Coherent-detection (CoD) permits to fully exploit the four-dimensional (4D) signal space consisting of the in-phase and quadrature components of the two fiber polarizations. A well-known and successful format exploiting such 4D space is Polarization-multiplexed QPSK (PM-QPSK). Recently, new signal constellations specifically designed and optimized in 4D space have been proposed, among which polarization-switched QPSK (PS-QPSK), consisting of a 8-point constellation at the vertices of a 4D polychoron called hexadecachoron. We call it HEXA because of its geometrical features and to avoid acronym mix-up with PM-QPSK, as well as with other similar acronyms. In this paper we investigate the performance of HEXA in direct comparison with PM-QPSK, addressing non-linear propagation over realistic links made up of 20 spans of either standard single mode fiber (SSMF) or non-zero dispersion-shifted fiber (NZDSF). We show that HEXA not only confirms its theoretical sensitivity advantage over PM-QPSK in back-to-back, but also shows a greater resilience to non-linear effects, allowing for substantially increased span loss margins. As a consequence, HEXA appears as an interesting option for dual-format transceivers capable to switch on-the-fly between PM-QPSK and HEXA when channel propagation degrades. It also appears as a possible direct competitor of PM-QPSK, especially over NZDSF fiber and uncompensated links.

©2010 Optical Society of America

## 1. Introduction

Optical system research is currently targeting 100 Gb/s per channel transmission and higher. At such speeds, all detrimental fiber propagation effects are exacerbated. To cope with this challenge, coherent detection (CoD) has been advocated. Thanks to CoD, multilevel constellations and polarization multiplexing (PM) have become practically exploitable, making it possible to increase the bit rate while keeping the baud rate limited. Also, adaptive electronic PMD compensation has become possible, thus removing one of the most serious practical hurdles towards ultra-high bit-rates.

At present, coherent polarization-multiplexed quadrature phase-shift-keying (PM-QPSK), also called polarization-division-multiplexed QPSK (PDM-QPSK) or dual-polarization QPSK (DP-QPSK), seems to be a forerunner for actual 100 Gb/s exploitation [1–4]. Other PM formats have also been proposed and experimentally demonstrated, such as PM binary phase-shift keying (PM-BPSK), PM quadrature amplitude modulation with 8 points (PM-8QAM [5]) and 16 points (PM-16QAM [6]), as well as higher-order PM-QAM constellations [7–9].

All these proposals have one feature in common: they use standard bi-dimensional (2D) formats from classical transmission theory and multiplex them onto two orthogonal polarizations. However, the signal space that is created by the four ‘quadratures’ of the dual-polarization electromagnetic field in the fiber is four-dimensional (4D). Therefore, one may wonder whether it would be possible to design a signal constellation optimized in 4D which could outperform the straightforward juxtaposition of a pair of 2D constellations onto orthogonal polarizations.

The first paper to propose a 4D constellation that cannot be viewed as the straightforward juxtaposition of two 2D constellations was [10]. There, a 24-point constellation was proposed, which carries 4.5 bits per symbol. The redundancy above 4 was cleverly used to get a sensitivity gain through suitable coding.

More recently, two further papers [11, 12] have explored 4D constellations. In them, numerous alternatives in terms of number of constellation points (from 2 to 32) were numerically explored by means of a sphere-packing optimization algorithm. The investigation showed that indeed certain 4D-optimized constellations have very good performance. One format in particular appears to have excellent sensitivity: it is an 8-point constellation, thus carrying three bits per symbol, whose points sit at the vertices of a regular 16-cell polychoron called hexadecachoron (note that convex bodies in 4D are not called ‘polygons’ as in 3D, but ‘polychora’ - singular: ‘polychoron’). The authors of [11,12] propose to call this format ‘polarization-switched-QPSK’ (PS-QPSK) because it may be viewed as a QPSK signal which, in every symbol time, could be launched onto either one of two orthogonal polarizations. Which polarization is used in a specific symbol time depends on the value of one information bit, whereas the other two bits generate the QPSK signal.

Other formats presented in [11, 12] appear to have good sensitivity performance. However, all of them are more complex than PS-QPSK and PM-QPSK and have a non-power-of-two number of constellation points. As a result, their adoption would require specific coding strategies and either more complex modulators or non-binary modulator driving signals. This paper is therefore devoted to a performance analysis, in the context of practical exploitation scenarios, of what we believe to be the most promising among unconventional 4D schemes: PS-QPSK.

Although the acronym PS-QPSK is certainly appropriate, we believe it might be confusing when used together with others such as PM-QPSK, PDM-QPSK or DP-QPSK. For the sake of clarity, in the following we will call it HEXA, based on the four initial letters of ‘hexadecachoron’, whose vertices make up its constellation.

According to [11], HEXA has an asymptotic sensitivity gain, for high optical signal-to-noise-ratios (OSNRs), of about 1.8 dB over PM-QPSK at the same bit rate and of 3 dB at the same baud rate. Also, in [11] various ways to practically implement the system are discussed and it is shown that the exact same transmitter (Tx) hardware configuration as for PM-QPSK could be used. The difference amounts to a few logic gates placed before the modulator drivers, correlating the four binary modulator driving signals. Regarding the receiver (Rx), the difference with PM-QPSK is in some minor details of the DSP algorithms. This opens up the very interesting possibility of a variable-rate transponder capable of using either PM-QPSK or HEXA signaling, depending for instance on channel impairments.

Even though HEXA appears from the above highlights as a very promising format, several circumstances might somewhat reduce its attractiveness. First of all, there is a large difference between its asymptotic (for vanishing bit error rate - BER) sensitivity gain over PM-QPSK, and its gain at BER=10^{−3}: the latter is only about 1 dB for same bit rate and 2.2 dB for same baud rate [11]. In addition, the bandwidth efficiency of HEXA is 75% that of PM-QPSK, because it carries 3 bits per symbol rather than 4. Therefore, to transmit a given bit-rate, HEXA needs a 33% greater baud-rate than PM-QPSK, requiring proportionally larger electrical and electro-optical component bandwidth and faster analog-to-digital converters (ADCs). Also, its larger optical bandwidth occupancy might make the 50-GHz grid a tight fit for 100 Gb/s HEXA transmission, whereas it has been proved to be quite adequate for 100 Gb/s PM-QPSK. As a result, whether HEXA could be considered as a direct alternative to PM-QPSK for the same bit rate needs suitable investigation.

As already mentioned, HEXA could be a dynamic, on-the-fly fall-back option for difficult channels. A possible scenario is also that of protection switching being activated, providing a degraded path. By switching format from PM-QPSK to HEXA, while keeping the baud rate constant, it might be possible to keep a channel in operation in these circumstances, obviously at the cost of a 25% drop in bit rate. In this scenario, however, a fair comparison should also look at the obvious alternative of simply slowing down PM-QPSK by 25%. This alone would grant PM-QPSK a 1.25 dB sensitivity boost, with no need to switch to a different format.

The above discussion assumes linearity. Whether HEXA is going to be of any practical interest also, and possibly mostly, depends on its non-linear resilience: tolerance to non-linear propagation impairments might completely change the outcome of any comparison carried out in linearity alone.

The non-linear regime was not addressed in [11, 12], so the objective of this paper is to carry out a detailed simulative performance evaluation of HEXA in the context of a realistic dense wavelength-division multiplexing (DWDM) long-haul system scenario, in comparison with PM-QPSK. Such comparison is here accomplished either at the same bit rate or at the same baud rate. Since the impact of non-linear effects is not independent of the baud rate, it was necessary to select specific values. We obtained them by imposing the total bit rate to be either 111 Gb/s or its 25%-reduced value 83.25 Gb/s, while using either HEXA or PM-QPSK.

The paper is structured as follows. In Sec. 2 the system simulation set-ups are described in detail. In Sec. 3 both back-to-back and long-haul DWDM non-linear results are presented. Finally, in Sec. 4, comments and conclusions follow.

## 2. Simulation Set-Ups

To perform meaningful comparative simulations, it is necessary to introduce enough realism into the Tx and Rx set-ups, so that their key features are captured. For PM-QPSK we used simulative Tx/Rx blocks based on modeling that has proved to yield reasonable matching with experimental results, as shown in [13]. For HEXA, we re-used as many simulation blocks as possible from the PM-QPSK set-up while introducing the few necessary modifications in a realistic manner. These set-ups are described in the following.

The electro-optical sections of the Tx’s of PM-QPSK and HEXA are identical and are based on two nested Mach-Zehnder modulators whose outputs are multiplexed through a polarization beam splitter (PBS). Modulation is NRZ, with a 5-pole Bessel filter band-limiting the drivers signals (*B*
_{-3dB} = 0.75·*R _{s}*), where

*R*is the symbol (or ‘baud’) rate. For HEXA, a suitable logic network correlates the drivers signals as shown in [11]. The HEXA eight issued signal points, arbitrarily normalized for convenience, are:

_{s}$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}1,& 1,& 1,& 1\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}-1,& -1,& -1,& -1\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}-1,& -1,& 1,& 1,\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}1,& 1,& -1,& -1\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}-1,& 1,& -1,& 1\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}1,& -1,& 1,& -1\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}-1,& 1,& 1,& -1\end{array}\right)$$

$$\phantom{\rule{.2em}{0ex}}\left(\begin{array}{cccc}1,& -1,& -1,& 1\end{array}\right)$$

where the subscripts *p* and *q* stand for ‘in-phase’ and ‘quadrature’, respectively, and *x* and *y* indicate two orthogonal polarizations. The overall electric field output by the modulator is, consequently:

Gray encoding is used for PM-QPSK. For HEXA, each signal point has six nearest neighbors at the same distance, so perfect Gray encoding is not possible. We assigned the bits so that, given a signal point, three neighbors have a one-bit difference and three neighbors a two-bit difference. The seventh point has a three-bit difference but its distance is √2 that of the neighbors thus its contribution to BER is negligible.

We simulated 9 WDM channels, either PM-QPSK or HEXA. Each channel was modulated using (2^{12} - 1) PRBSs. We used 3 orthogonal PRBSs for HEXA and 4 for PM-QPSK. The PRBSs were also orthogonal among all channels, i.e., a total of 27 (for HEXA) and 36 (for PM-QPSK) uncorrelated PRBSs were used for the overall WDM simulations.

The channel spacing was the standard Δ*f* = 50 GHz. Before wavelength multiplexing, each channel was shaped by a 2nd order Super-Gaussian optical filter with a -3dB bandwidth of 40 GHz. An identical filter was used at the Rx to de-multiplex the channels.

The coherent Rx for both PM-QPSK and HEXA is shown in Fig. 1 and includes a local oscillator (LO) that is mixed with the incoming signal in two 90° hybrids based on the device proposed in [14]. LO frequency alignment is assumed to be ideal. Four balanced photo-detectors are used to detect the optical signal. The four resulting electrical signals are then filtered by 5-pole low-pass Bessel filters (*B*
_{-3dB} = 0.75 · *R _{s}*). Finally, they are sampled by four ADCs and two samples per symbol are taken.

Note that in the case of HEXA operated at 111 Gb/s (37 Gbaud), taking two samples per symbol (S/s) means sampling at 74 GS/s, a rate that is above the current commercial limit of 56 GS/s. However, test and measurement equipment is already available at 80 GS/s and it is conceivable that in the future 74 GS/s will be commercially achieved on chip. Note that sampling at fewer than two samples per symbol is theoretically possible [15,16], but introducing sub-sampling in the context of this study would complicate the analysis with little added value, so we consider sub-sampling outside of the scope of this paper.

A digital signal-processing (DSP) section follows, which is composed of two stages. The first stage consists of two complex finite-impulse-response filters that recover chromatic dispersion (CD-FIR), performing the so-called ‘electrical dispersion compensation’ (EDC). A second adaptive ‘butterfly’ stage made up of four 11-tap FIR filters recovers the polarization reference frame (POL-FIR). It also aligns the signal phase and approximately compensates for the non-ideal (non-matched) filtering that takes place in the overall system. In an actual system it would take care of PMD and possible residual CD due to imperfect compensation from the first stage. In our simulations, the first stage is adjusted for full CD compensation. The second stage is initialized using a least-mean-square (LMS) algorithm based on a training-sequence. The LMS algorithm takes as reference the ideal expected constellation. Once convergence is reached, the LMS algorithm switches to a decision-directed mode, both for PM-QPSK and HEXA.

For PM-QPSK, threshold decision was performed, whereas for HEXA we tried two different decision algorithms. A first algorithm decided for the constellation point at minimum Euclidean distance from the received signal point. We also tested another, simpler, algorithm. It requires that the constellation entering the decision stage be suitably rotated, that is, the 4D axes on which the constellation is projected are different than those used in Eq. (1). This adds no complexity as it is simply done by feeding the second-stage equalizer LMS algorithm with a target constellation taking into account the desired rotation. The new target 8-points, obtained by a +45° polarization rotation plus a -45° phase rotation of the original constellation points [Eq. (1)], belong to the set:

The mathematical operation which transforms constellation [Eq. (1)] into constellation [Eq. (3)] is a multiplication by the following 4×4 orthogonal matrix:

In Eq. (3), the four components of each array, or ‘vector’, correspond to the four sampled signals at the output of the second-stage equalizer. The algorithm first decides for the vector component with greatest absolute value. Then it decides whether such component has positive or negative sign.

Theoretically, in additive Gaussian noise, these two HEXA decision algorithms should yield the same performance. This is because the latter algorithm, which takes the largest absolute value and then selects its sign, is equivalent to ‘maximum correlation detection’, which in turn is known to be equivalent to minimal distance detection for equi-energy symbols. It is equivalent to maximum correlation detection because it operates as if the received signal vector was ‘projected’ over each of the modified constellation vectors listed as rows in Eq. (3). The fact that these projections are represented in modified reference axes is immaterial, since projection (or inner product) is invariant to orthogonal or unitary transformations.

Our simulations confirmed the prediction of same performance, to within less than 0.1 dB, not only in linearity but also in WDM long-haul non-linear transmission.

The test-link we considered (Fig. 2) is composed of 20 spans, each including a fiber stretch with *L*
_{span}=90 km, a variable optical attenuator (VOA) and an erbium-doped fiber amplifier (EDFA) with a noise-figure *F*=5 dB and a gain that completely recovers the span loss *A*
_{span}. The purpose of the VOA is to make it possible to increase the total span loss, so that the test system is suitably stressed, as will be shown later. We analyzed two fiber types: standard single-mode fiber (SSMF) with loss *α* = 0.22 dB/km, dispersion *D* = 16.7 ps/nm/km, dispersion slope *S* = 0.07 ps/nm^{2}/km and non-linearity coefficient *γ* = 1.3 (W·km)^{−1}; large effective area non-zero dispersion-shifted fiber (NZDSF) with *α* = 0.22 dB/km, *D* = 3.8 ps/nm/km, *S* = 0.07 ps/nm^{2}/km and *γ*= 1.5 (W·km)^{−1}.

Several papers, both theoretical and experimental, have shown that in general PM-QPSK performs better over optically uncompensated links [13, 17–19]. In this case, full-EDC is done at the Rx by the first-stage CD-FIR equalizer. However, current direct-detection systems employ optical dispersion management (ODM), by means of in-line dispersion-compensating units (DCUs), and it is conceivable that it will not always be easy or possible to remove DCUs from actual plants. So, both the case of full-EDC and ODM are of interest for a realistic comparison of PM-QPSK and HEXA and we investigated both. For ODM, DCUs are placed at each span. We selected two values of in-line residues D_{res,IL}: -70 ps/nm when using SSMF and -50 ps/nm when using NZDSF. The the first-stage CD-FIR equalizer then takes care of the residual CD at the Rx, of 1,000 and 1,400 ps/nm, respectively.

The DCUs were assumed lossless and linear. Actual DCUs are far from either, but multiple-stage EDFAs are typically used that permit to launch a suitable signal power into the DCU so that its non-linearity is kept under control and, at the same time, its loss does not substantially degrade the overall amplification stage noise figure. In essence, actual systems are designed to behave as much as possible as if DCUs were lossless and linear.

Finally, ASE noise was realistically injected at each EDFA along the link, rather than added at the Rx as it is often done in simulations to speed them up. This way, signal-noise nonlinear interactions were properly taken into account. The results shown in the next section actually took the equivalent of several months of a single high-performance CPU time. This large computational burden was managed by using 16 processor cores running simultaneously.

## 3. Results

We first checked whether the HEXA simulation performance in back-to-back (btb) was aligned to the analytical prediction. We performed no-fiber, single channel HEXA simulations, with perfectly square Tx pulses and matched electrical filters at the Rx. The results are shown in Fig. 3, where the blue solid line corresponds to theory for HEXA [11] and the red dashed line to theory for PM-QPSK, for same bit-rate. The dots and squares are simulation results for HEXA and PM-QPSK, respectively. Figure 3 shows the btb performance of our simulations to match the HEXA and PM-QPSK predicted analytical performance very well, with a small penalty of about 0.1 dB. Such penalty is due to the fact that the second-stage (POL-FIR) equalizer LMS updating algorithm was turned on, which unavoidably slightly dithers the recovered constellation.

In Fig. 3, the optical signal-to-noise ratio (OSNR) scale is expressed in terms of the energy-per-bit E_{b} divided by the unilateral noise power spectral density per polarization, *N*
_{0}. This approach, standard for all non-optical communications systems, permits to obtain bit-rate independent results and is used in [11,12] as well. The conversion formula, allowing to rescale it to any given bit-rate *R _{b}* and any given optical noise bandwidth

*B*is:

_{N}Specifically, to derive the OSNR value for 111 Gb/s and the customary 0.1 nm noise bandwidth (12.487 GHz @1550 nm) from *E _{b}*/

*N*

_{0}, the conversion factor

*R*/2

_{b}*B*is 4.44, or 6.5 dB.

_{N}We then proceeded to evaluate the btb performance of the more realistic Tx and Rx setup, including the electrical and optical filters. Because of the use of non-ideal filters, the btb system sensitivities are slightly degraded for both PM-QPSK and PM-HEXA. The results are shown in Table 1. For all systems, however, the penalty with respect to the analytical results is quite limited: it ranges from a minimum of 0.15 dB for 83.25 Gb/s HEXA to a maximum of 0.33 dB for 111 Gb/s PM-QPSK. These results are so good because the second-stage equalizer automatically tends towards ‘best’ filtering, making up for most of the non-ideal matching of the transfer function of the cascaded Rx filters.

In our laboratory, an actual experimental set-up of 111 Gb/s PM-QPSK, similar to [13], currently reaches 1.5 dB from the theoretical limit. We have assessed that most of the extra 1.2 penalty vs. simulations is due to noise and distortion on the electrical driving signals of the Tx nested Mach-Zehnder modulators, and to a minor extent to ADC finite resolution and other residual imperfections. Specifically, ADC resolution in the experiment is between 4 and 5 bit per sample, frequency-dependent, whereas it is infinite in the simulations. Nonetheless, we refrained from trying to closely mimick experimental penalties in the simulations, because such penalties are highly device-dependent and in addition we deem them essentially irrelevant in the context of a *relative* comparison of HEXA vs. PM-QPSK, whose assumed Tx and Rx hardware is identical.

Following, we tested the performance of 20-span links, for both 83 and 111 Gb/s PM-QPSK and HEXA in a 50 GHz WDM grid, using either full-EDC or ODM, over either SSMF or NZDSF. We explored *P*
_{Tx} from -5 up to +5 dBm and, for each value of *P*
_{Tx}, we increased the VOA attenuation in order to estimate the span loss *A*
_{span} yielding BER=10^{−3}. In all simulations the BER was estimated through direct error counting over 2^{16} symbols (about 200,000 bits for HEXA and 260,000 bits for PM-QPSK).

The results of the link simulations are shown in Figs. 4–7. Blue and red curves refer to HEXA and PM-QPSK, respectively. These plots have certain special features. First of all, the OSNR (or *E _{b}*/

*N*

_{0}) is constant over the straight slanted linearity performance curves (dashed lines) and coincides with the btb values given in Table 1. All points in the plot above a straight line have a lower OSNR whereas all points below have a higher OSNR. The non-linear performance curves bend downward because they need a higher OSNR than in linearity to keep BER=10

^{−3}. The amount of increase in OSNR can be directly read as either the horizontal or vertical distance from the linearity curve, in dB.

A first prominent feature of all four figures is that both the 83 and 111 Gb/s HEXA curves reach a higher tolerated maximum span loss than either 83 or 111 Gb/s PM-QPSK. The gain is more pronounced over uncompensated links. Particularly striking are the SSMF-EDC and NZDSF-EDC cases (Figs. 4 and 6), where 83 Gb/s HEXA outperforms 111 Gb/s PM-QPSK by 3.3 dB. The two ODM cases (Figs. 5 and 7) differ from the EDC cases only in that the gap between 83 Gb/s HEXA and 111 Gb/s PM-QPSK is slightly smaller (3 vs. 3.3 dB). Note also that in the EDC cases 83 Gb/s HEXA has about 1 dB higher optimum *P*
_{TX} than 111 Gb/s PM-QPSK.

These results appear to confirm the potential attractiveness of HEXA as a fall-back option to keep a difficult channel in operation. Switching on-the-fly from 111 Gb/s PM-QPSK to 83 Gb/s HEXA (constant 27.75 Gbaud) provides a very substantial loss margin increase (3 dB or higher). Also, simply reducing the PM-QPSK bit-rate down to the same 83 Gb/s does not nearly yield the same margin increase (1 dB or less is gained, only). In addition, switching from 111 Gb/s PM-QPSK to 83 Gb/s PM-QPSK, rather than 83 Gb/s HEXA, would require the transponder board to be able to operate at two different baud rates, which would certainly pose some difficult implementation problems.

Our results also show, somewhat unexpectedly, that 111 Gb/s HEXA is quite competitive vs. 111 Gb/s PM-QPSK. In the EDC cases of Figs. 4–6, 111 Gb/s HEXA gains more than 2 dB of span loss vs. 111 Gb/s PM-QPSK. This occurs despite the fact that channel bandwidth, limited by the same 40 GHz Tx and Rx optical filters, is clearly much tighter for 111 Gb/s HEXA than for 111 Gb/s PM-QPSK.

Note also that in the NZDSF-EDC scenario, 111 Gb/s HEXA can handle up to 1 dBm of *P*
_{TX}, with a tolerated span loss of almost 26 dB, whereas 111 Gb/s PM-QPSK, at the same launch power, is heavily impacted by non-linearity and its tolerated span loss is only about 20.5 dB. Given the large amount of NZDSF installed in the field, this result is quite significant.

Finally, since the HEXA constellation is mapped onto just two antipodal points over the Poincaré sphere, we suspected cross-polarization modulation (XPolM, [20]) effects might be substantially influenced by the WDM channels relative launch polarization alignments. We therefore checked various configurations, including all channels launched along the same Poincaré sphere diameter, as well as using orthogonal Stokes space axes for launch of neighboring channels. The performance changed by less than 0.1 dB in terms of maximum tolerated span loss, under all test scenarios, suggesting launch polarization alignment is of negligible practical impact.

## 4. Conclusion

Our detailed performance comparison of HEXA (or PS-QPSK) vs. PM-QPSK (a.k.a. PDM-QPSK or DP-QPSK) in realistic 20-span link configurations, showed HEXA to achieve very substantial gains over PM-QPSK in terms of tolerated span loss (3 dB or greater), given the same baud-rate of 27.75 Gbaud. This gain went actually beyond the noise-only back-to-back sensitivity gain predicted by theory (2.2 dB), showing HEXA to have a better non-linear resilience than PM-QPSK. These results hint at the potential use of 83 GB/s HEXA as on-the-fly fall-back format for channels unable to keep operating at 111 Gb/s PM-QPSK.

Somewhat unexpectedly, the better non-linear resilience of HEXA appears to give even 111 Gb/s HEXA a substantial edge over 111 Gb/s PM-QPSK, especially over uncompensated NZDSF links. Clearly, the greater requested baud-rate (37 Gbaud) represents a technological problem, as electrical, electro-optical and digital circuitry (including ADCs) should operate at a higher rate. In addition, the standard 50 GHz grid might get too tight at 37 Gbaud, if repeated in-line filtering (WSS and ROADMs) substantially reduced the effective optical bandwidth.

## Acknowledgments

This work was supported by CISCO Systems within a SRA (Sponsored Research Agreement) contract. It was also supported by the EURO-FOS project, a Network of Excellence (NoE) funded by the European Commission through the 7th ICT-Framework Programme. The authors thank RSoft Design Group Inc. for supplying the optical transmission system simulation software OptSim.

## References and links

**1. **C. R. S. Fludger, T. Duthel, D. Van Den Borne, C. Schulien, E-D. Schmidt, T. Wuth, E. De Man, G.D. Khoe, and H. De Waardt, “10×111 Gbit/s, 50 GHz Spaced, POLMUX-RZ-DQPSK Transmission over 2375 km Employing Coherent Equalisation,” in *Proc. of OFC 2007*, Anaheim (CA), Feb. 2007, post-deadline paper PDP-22.

**2. **G. Charlet, J. Renaudier, H. Mardoyan, P. Tran, O. Bertran-Pardo, F. Verluise, M. Achouche, A. Boutin, F. Blache, J.-Y. Dupuy, and S. Bigo, “Transmission of 16.4 Tbit/s Capacity Over 2,550 km Using PDM QPSK Modulation Format and Coherent Receiver,” in *Proc. of OFC 2008*, San Diego (CA), Feb. 2008, post-deadline paper PDP-3.

**3. **K. Roberts, “Performance of Dual-Polarization QPSK for Optical Transport Systems,” J. Lightwave Technol. **27**, 3546–3559 (2009). [CrossRef]

**4. **M. Salsi, H. Mardoyan, P. Tran, C. Koebele, E. Dutisseuil, G. Charlet, and S. Bigo, “155×100Gbit/s Coherent PDM-QPSK Transmission over 7,200 km,” in *Proc. ECOC 2009*, in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, post-deadline paper PD2.5.

**5. **J. Yu, M. F. Huang, and X. Zhou, “8×114Gbit/s, 25GHz Spaced, PolMux-RZ-8QAM Straight-Line Transmission over 800km of SSMF,” in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, paper P4.02.

**6. **A.H. Gnauck and P.J. Winzer, “10 112-Gb/s PDM 16-QAM Transmission over 1022 km of SSMF with a Spectral Efficiency of 4.1 b/s/Hz and no Optical Filtering,” in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, paper 8.4.2.

**7. **Y. Mori, C. Zhang, M. Usui, K. Igarashi, K. Katoh, and K. Kikuchi, “200-km transmission of 100-Gbit/s 32-QAM Dual-Polarization Signals using a Digital Coherent Receiver,” in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, paper 8.4.6.

**8. **A. Sano, T. Kobayashi, K. Ishihara, H. Masuda, S. Yamamoto, K. Mori, E. Yamazaki, E. Yoshida, Y. Miyamoto, T. Yamada, and H. Yamazaki, “240-Gb/s Polarization-Multiplexed 64-QAM Modulation and Blind Detection Using PLC-LN Hybrid Integrated Modulator and Digital Coherent Receiver,” in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, post-deadline paper PD2.4.

**9. **M. Nakazawa, “Optical Quadrature Amplitude Modulation (QAM) with Coherent Detection up to 128 States,” in *Proc. of OFC 2009*, San Diego (CA), Mar. 2009, paper OThG1.

**10. **H. Bülow, “Polarization QAM Modulation (POLQAM) for Coherent Detection Schemes,” in *Proc. of OFC 2009*, San Diego (CA), Mar. 2009, paper OWG2.

**11. **M. Karlsson and E. Agrell, “Which Is the Most Power-Efficient Modulation Format in Optical Links?,” Opt. Express **17**, 10814–10819 (2009). [CrossRef] [PubMed]

**12. **E. Agrell and M. Karlsson, “Power-Efficient Modulation Formats in Coherent Transmission Systems,” J. Lightwave Technol. **27**, 5115–5126 (2009). [CrossRef]

**13. **G. Gavioli, E. Torrengo, G. Bosco, A. Carena, V. Curri, V. Miot, P. Poggiolini, F. Forghieri, S. J. Savory, L. Molle, and R. Freund, “NRZ-PM-QPSK 16×100 Gb/s Transmission over Installed Fiber with Different Dispersion Maps,” IEEE Photon. Technol. Lett. **22**, 371–373 (2010). [CrossRef]

**14. **D. Hoffman, H. Heidrich, G. Wenke, R. Langenhorst, and E. Dietrich, “Integrated Optics Eight-Port 90° Hybrid on LiNbO_{3},” J. Lightwave Technol. **7**, 794–798 (1989). [CrossRef]

**15. **E. Ip and J. M. Kahn, “Digital Equalization of Chromatic Dispersion and Polarization Mode Dispersion,” J. Lightwave Technol. **25**, 2033–2043 (2007). [CrossRef]

**16. **A. Carena, V. Curri, P. Poggiolini, G. Bosco, and F. Forghieri, “Impact of ADC Sampling Speed and Resolution on Uncompensated Long-Haul 111-Gb/s WDM PM-QPSK Systems,” IEEE Photon. Technol. Lett. **21**, 1514–1516 (2009). [CrossRef]

**17. **V. Curri, P. Poggiolini, A. Carena, and F. Forghieri, “Dispersion Compensation and Mitigation of Non-Linear Effects in 111 Gb/s WDM Coherent PM-QPSK Systems,” IEEE Photon. Technol. Lett. **20**, 1473–1475 (2008). [CrossRef]

**18. **D. van den Borne, V. Sleiffer, M. Alfiad, S. Jansen, and T. Wuth, “POLMUX-QPSK modulation and coherent detection: the challenge of long-haul 100G transmission,” in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, paper 3.4.1.

**19. **E. Grellier, J.-C. Antona, and S. Bigo, “Revisiting the evaluation of non-linear propagation impairments in highly dispersive systems,” in *Proc. of ECOC 2009*, Vienna (Austria), Sept. 2009, paper 10.4.2.

**20. **D. Wang and C. R. Menyuk, “Polarization evolution due to the Kerr nonlinearity and chromatic dispersion,” J. Lightwave Technol. **17**, 2520–2529 (1999). [CrossRef]