## Abstract

The drive for higher performance in optical fiber systems has renewed interest in coherent detection. We review detection methods, including noncoherent, differentially coherent, and coherent detection, as well as a hybrid method. We compare modulation methods encoding information in various degrees of freedom (DOF). Polarization-multiplexed quadrature-amplitude modulation maximizes spectral efficiency and power efficiency, by utilizing all four available DOF, the two field quadratures in the two polarizations. Dual-polarization homodyne or heterodyne downconversion are linear processes that can fully recover the received signal field in these four DOF. When downconverted signals are sampled at the Nyquist rate, compensation of transmission impairments can be performed using digital signal processing (DSP). Linear impairments, including chromatic dispersion and polarization-mode dispersion, can be compensated quasi-exactly using finite impulse response filters. Some nonlinear impairments, such as intra-channel four-wave mixing and nonlinear phase noise, can be compensated partially. Carrier phase recovery can be performed using feedforward methods, even when phase-locked loops may fail due to delay constraints. DSP-based compensation enables a receiver to adapt to time-varying impairments, and facilitates use of advanced forward-error-correction codes. We discuss both single- and multi-carrier system implementations. For a given modulation format, using coherent detection, they offer fundamentally the same spectral efficiency and power efficiency, but may differ in practice, because of different impairments and implementation details. With anticipated advances in analog-to-digital converters and integrated circuit technology, DSP-based coherent receivers at bit rates up to 100 Gbit/s should become practical within the next few years.

© 2008 Optical Society of America

## Corrections

Ezra Ip, Alan P. Lau, Daniel J. Barros, and Joseph M. Kahn, "Coherent detection in optical fiber systems: erratum," Opt. Express**16**, 21943-21943 (2008)

https://www.osapublishing.org/oe/abstract.cfm?uri=oe-16-26-21943

## 1. Introduction

An important goal of a long-haul optical fiber system is to transmit the highest data throughput over the longest distance without signal regeneration. Given constraints on the bandwidth imposed by optical amplifiers and ultimately by the fiber itself, it is important to maximize spectral efficiency, measured in bit/s/Hz. But given constraints on signal power imposed by fiber nonlinearity, it is also important to maximize power (or SNR) efficiency, i.e., to minimize the required average transmitted energy per bit (or the required signal-to-noise ratio per bit). Most current systems use binary modulation formats, such as on-off keying or differential phase-shift keying, which encode one bit per symbol. Given practical constraints on filters for dense wavelength-division multiplexing (DWDM), these are able to achieve spectral efficiencies of 0.8 bit/s/Hz per polarization. Spectral efficiency limits for various detection and modulation methods have been studied in the linear [1–3] and nonlinear regimes [4,5]. Noncoherent detection and differentially coherent detection offer good power efficiency only at low spectral efficiency, because they limit the degrees of freedom available for encoding of information [5].

The most promising detection technique for achieving high spectral efficiency while maximizing power (or SNR) efficiency, is coherent detection with polarization multiplexing, as symbol decisions are made using the in-phase (I) and quadrature (Q) signals in the two field polarizations, allowing information to be encoded in all the available degrees of freedom. When the outputs of an optoelectronic downconverter are sampled at Nyquist rate, the digitized waveform retains full information of the electric field, which enables compensation of transmission impairments by digital signal processing (DSP). A DSP-based receiver is highly advantageous because adaptive algorithms can be used to compensate time-varying transmission impairments. Advanced forward error-correction coding can also be implemented. Moreover, digitized signals can be delayed, split and amplified without degradation in signal quality. DSP-based receivers are ubiquitous in wireless and digital subscriber line (DSL) systems at lower data rates. In such systems, computationally intensive techniques have been demonstrated, such as orthogonal frequency-division multiplexing (OFDM) with multiple-input-multiple-output (MIMO) transmission in a real-time 1 Gbit/s wireless link [6]. Continued hardware improvements will enable deployment of DSP-based coherent optical systems in the next few years.

Experimental results in coherent optical communication have been promising. Kikuchi demonstrated polarization-multiplexed 4-ary quadrature-amplitude modulation (4-QAM) transmission at 40 Gbit/s with a channel bandwidth of 16 GHz (2.5 bit/s/Hz) [7]. This experiment used a high-speed sampling oscilloscope to record the output of a homodyne downconverter. DSP was performed offline because of the unavailability of sufficiently fast processing hardware. The first demonstration of real-time coherent detection occurred in 2006, when an 800 Mbit/s 4-QAM signal was coherently detected using a receiver with 5-bit analog-to-digital converters (ADC) followed by a field programmable gate array [8]. In 2007, feedforward carrier recovery was demonstrated in real time for 4-QAM at 4.4 Gbit/s [9]. Savory showed the feasibility of digitally compensating the chromatic dispersion (CD) in 6,400 km of SMF without inline dispersion compensating fiber (DCF), with only 1.2 dB OSNR penalty at 42.8 Gbit/s [10]. Coherent detection of large QAM constellations has also been demonstrated. For example, 16-ary transmission at 40 Gbit/s using an amplitude-phase-shift keying (APSK) format was shown by Sekine et al [11]. In 2007, Hongo et al demonstrated 64-QAM transmission over 150 km of dispersion-shifted fiber [12].

This paper provides an overview of detection and modulation methods, with emphasis on coherent detection and digital compensation of channel impairments. The paper is organized as follows: in Section 3, we review signal detection methods, including noncoherent, differentially coherent and coherent detection. We compare these techniques and the modulation formats they enable. In Section 4, we compare the bit-error ratio (BER) performance of different modulation formats in the presence of additive white Gaussian noise (AWGN). In Section 5, we review the major channel impairments in long-haul optical transmission. We show how these can be compensated digitally in single-carrier systems, and we compare digital compensation to traditional compensation methods. In Section 6, we review OFDM, which is a multi-carrier modulation format. Finally, in Section 7, we compare implementation complexities of single- and multi-carrier systems.

## 2. Notation

In this paper, we represent optical signal and noise electric fields as complex-valued, baseband quantities, which are denoted as *E _{subscript}*(

*t*). In Section 3, photocurrents are represented as real passband signals as

*I*(

_{subscript}*t*). After we derive the canonical model for a coherent receiver, in all subsequent sections, we write all signals and noises as complexvalued baseband quantities, with the convention that the output of the optoelectronic downconverter is denoted by

*y*, the transmitted symbol as

*x*, the output of the linear equalizer as

*x̃*, and the phase de-rotated output prior to symbol decisions as

*x̂*. Signals that occupy one polarization only are represented as scalars and are written in italics as shown; while dually polarized signals are denoted as vectors and are written in bold face, such as

**y**

*. Unless stated otherwise, we employ a linear (*

_{k}*x*,

*y*) basis for decomposing dually polarized signals. In Section 5 dealing with impairment compensation, matrix and vector quantities are denoted in bold face.

## 3. Optical detection methods

#### 3.1 Noncoherent detection

In noncoherent detection, a receiver computes decision variables based on a *measurement of signal energy*. An example of noncoherent detection is direct detection of on-off-keying (OOK) using a simple photodiode (Fig. 1(a)). To encode more than one bit per symbol, multi-level amplitude-shift keying (ASK) – also known as pulse-amplitude modulation – can be used. Another example of noncoherent detection is frequency-shift keying (FSK) with wide frequency separation between the carriers. Fig. 1(b) shows a noncoherent receiver for binary FSK.

The limitations of noncoherent detection are: (a) detection based on energy measurement allows signals to encode only one degree of freedom (DOF) per polarization per carrier, reducing spectral efficiency and power efficiency, (b) the loss of phase information during detection is an irreversible transformation that prevents full equalization of linear channel impairments like CD and PMD by linear filters. Although maximum-likelihood sequence detection (MLSD) can be used to find the best estimate of the transmitted sequence given only a sequence of received intensities, the achievable performance is suboptimal compared with optical or electrical equalization making use of the full electric field [13].

#### 3.2 Differentially coherent detection

In differentially coherent detection, a receiver computes decision variables based on *a measurement of differential phase between the symbol of interest and one or more reference symbol(s)*. In differential phase-shift keying (DPSK), the phase reference is provided by the previous symbol; in polarization-shift keying (PolSK), the phase reference is provided by the signal in the adjacent polarization. A binary DPSK receiver is shown in Fig. 2(a). Its output photocurrent is:

where *E _{s}*(

*t*) is the received signal,

*R*is the responsivity of each photodiode, and

*T*is the symbol period. This receiver can also be used to detect continuous-phase frequency-shiftkeying (CPFSK), as the delay interferometer is equivalent to a delay-and-multiply demodulator. A receiver for

_{s}*M*-ary DPSK,

*M*>2, can similarly be constructed as shown in Fig. 2(b). Its output photocurrents are:

A key motivation for employing differentially coherent detection is that binary DPSK has 2.8 dB higher sensitivity than noncoherent OOK at a BER of 10^{-9} [5]. However, the constraint that signal points can only differ in phase allows only one DOF per polarization per carrier, the same as noncoherent detection. As the photocurrents in Eq. (1) to (3) are not linear functions of the *E*-field, linear impairments, such as CD and PMD, also cannot be compensated fully in the electrical domain after photodetection.

A more advanced detector for *M*-ary DPSK is the multichip DPSK receiver, which has multiple DPSK receivers arranged in parallel, each with a different interferometer delay that is an integer multiple of *T _{s}* [14,15]. Since a multichip receiver compares the phase of the current symbol to a multiplicity of previous symbols, the extra information available to the detector enables higher sensitivity. In the limit that the number of parallel DPSK receivers is infinite, the performance of multi-chip DPSK approaches coherent PSK [15]. In practice, the number of parallel DPSK receivers required for good performance needs to be equal to the impulse duration of the channel divided by

*T*. Although multi-chip DPSK does not require a local oscillator (LO) laser, carrier synchronization and polarization control, the hardware complexity can be a significant disadvantage.

_{s}#### 3.3 Hybrid of noncoherent and differentially coherent detection

A hybrid of noncoherent and differentially coherent detection can be used to recover information from both amplitude and differential phase. One such format is polarization-shift keying (PolSK), which encodes information in the Stokes parameter. If we let
${E}_{x}\left(t\right)={a}_{x}\left(t\right){e}^{j{\varphi}_{x}\left(t\right)}$
and
${E}_{y}\left(t\right)={a}_{y}\left(t\right){e}^{j{\varphi}_{y}\left(t\right)}$
be the *E*-fields in the two polarizations, the Stokes parameters are *S*
_{1}=*a*
^{2}
* _{x}*-

*a*

^{2}

*,*

_{y}*S*

_{2}=2

*a*cos(

_{x}a_{y}*δ*) and

*S*

_{3}=2

*a*sin(

_{x}a_{y}*δ*), where

*δ*(

*t*)=

*ϕ*(

_{x}*t*)-

*ϕ*(

_{y}*t*) [16]. A PolSK receiver is shown in Fig. 3. The phase noise tolerance of PolSK is evident by examining

*S*

_{1}to

*S*

_{3}. Firstly,

*S*

_{1}is independent of phase. Secondly,

*S*

_{2}and

*S*

_{3}depend on the phase difference

*ϕ*(

_{x}*t*)-

*ϕ*(

_{y}*t*). As

*ϕ*(

_{x}*t*) and

*ϕ*(

_{y}*t*) are both corrupted by the same phase noise of the transmitter (TX) laser, their arithmetic difference

*δ*(

*t*) is free of phase noise. In practice, the phase noise immunity of PolSK is limited by the bandwidth of the photodetectors [16]. It has been shown that 8-PolSK can tolerate laser linewidths as large as Δ

*νT*≈0.01 [17], which is about 100 times greater than the phase noise tolerance of coherent 8-QAM (Section 5.3.3). This was a significant advantage in the early 1990s, when symbol rates were only in the low GHz range. In modern systems, symbol rates of tens of GHz, in conjunction with tunable laser having linewidths <100 kHz, has diminished the advantages of PolSK. Recent results have shown that feedforward carrier synchronization enables coherent detection of 16-QAM at Δ

_{b}*νT*~10

_{b}^{-5}[18], which is within the limits of current technology. As systems are increasingly driven by the need for high spectral efficiency, polarization-multiplexed QAM is likely to be more attractive because of its higher sensitivity (Section 4).

#### 3.4 Coherent detection

The most advanced detection method is coherent detection, where the receiver computes decision variables based on the *recovery of the full electric field*, which contains both amplitude and phase information. Coherent detection thus allows the greatest flexibility in modulation formats, as information can be encoded in amplitude and phase, or alternatively in both in-phase (I) and quadrature (Q) components of a carrier. Coherent detection requires the receiver to have knowledge of the carrier phase, as the received signal is demodulated by a LO that serves as an absolute phase reference. Traditionally, carrier synchronization has been performed by a phase-locked loop (PLL). Optical systems can use (i) an optical PLL (OPLL) that synchronizes the frequency and phase of the LO laser with the TX laser, or (ii) an electrical PLL where downconversion using a free-running LO laser is followed by a second-stage demodulation by an analog or digital electrical VCO whose frequency and phase are synchronized. Use of an electrical PLL can be advantageous in duplex systems, as the transceiver may use one laser as both TX and LO. PLLs are sensitive to propagation delay in the feedback path, and the delay requirement can be difficult to satisfy (Section 5.3.1). Feedforward (FF) carrier synchronization overcomes this problem. In addition, as a FF synchronizer uses both past and future symbols to estimate the carrier phase, it can achieve better performance than a PLL which, as a feedback system, can only employ past symbols. Recently, DSP has enabled polarization alignment and carrier synchronization to be performed in software.

A coherent transmission system and its canonical model are shown in Fig. 4. At the transmitter, Mach-Zehnder (MZ) modulators encode data symbols onto an optical carrier and perform pulse shaping. If polarization multiplexing is used, the TX laser output is split into two orthogonal polarization components, which are modulated separately and combined in a polarization beam splitter (PBS). We can write the transmitted signal as:

where *T _{s}* is the symbol period,

*P*is the average transmitted power,

_{t}*b*(

*t*) is the pulse shape (e.g., non-return-to-zero (NRZ) or return-to-zero (RZ)) with the normalization ∫|

*b*(

*t*)|

^{2}

*dt*=

*T*,

_{s}*ω*and

_{s}*ϕ*(

_{s}*t*) are the frequency and phase noise of the TX laser, and

**x**

_{k}=[

*x*

_{1,k},

*x*

_{2,k}]

^{T}is a 2×1 complex vector representing the

*k*-th transmitted symbol. We assume that symbols have normalized energy:

*E*[|

**x**

_{k}|

^{2}]=1. For single-polarization transmission, we can set the unused polarization component

*x*

_{2,k}to zero.

The channel consists of *N _{A}* spans of fiber, with inline amplification and DCF after each span. In the absence of nonlinear effects, we can model the channel as a 2×2 matrix:

where *h _{ij}*(

*t*) denote the response of the

*i*-th output polarization due to an impulse applied at the

*j*-th input polarization of the fiber. The choice of reference polarizations at the transmitter and receiver is arbitrary. Eq. (5) can describe CD, all orders of PMD, polarization-dependent loss (PDL), optical filtering effects and sampling time error [19]. In addition, a coherent optical system is corrupted by AWGN, which includes (i) amplified spontaneous emission (ASE) from inline amplifiers, (ii) receiver LO shot noise, and (iii) receiver thermal noise. In the canonical transmission model, we model the cumulative effect of these noises by anequivalent noise source

**n**(

*t*)=[

*n*

_{1}(

*t*),

*n*

_{2}(

*t*)

^{T}referred to the input of the receiver.

The *E*-field at the output of the fiber is **E**
_{s}(*t*)=[*E*
_{s,1}(*t*), *E*
_{s,2}(*t*)]^{T}, where:

Under the assumption of Fig. 4 where inline amplification completely compensates propagation loss, *P _{r}*=

*P*is the average received power,

_{t}*c*(

_{lm}*t*)=

*b*(

*t*)⊗

*h*(

_{lm}*t*) is a normalized pulse shape, and

*E*(

_{sp,l}*t*) is ASE noise in the

*l*-th polarization. Assuming the

*N*fiber spans are identical and all inline amplifiers have gain

_{A}*G*and spontaneous emission factor

*n*, the two-sided power spectral density (psd) of

_{sp}*E*(

_{sp,l}*t*) is

*S*(

_{Esp}*f*)=

*N*(

_{A}n_{sp}ℏω_{s}*G*-1)/

*G*W/Hz [20].

The first stage of a coherent receiver is a dual-polarization optoelectronic downconverter that recovers the baseband modulated signal. In a digital implementation, the analog outputs are lowpass filtered and sampled at 1/*T*=*M*/*KT _{s}*, where

*M*/

*K*is a rational oversampling ratio. Channel impairments can then be compensated digitally before symbol detection.

#### 3.4.1 Single-polarization downconverter

We first consider a single-polarization downconverter, where the LO laser is aligned in the *l*-th polarization. Downconversion from optical passband to electrical baseband can be achieved in two ways: in a homodyne receiver, the frequency of the LO laser is tuned to that of the TX laser so the photoreceiver output is at baseband. In a heterodyne receiver, the LO and TX lasers differ by an intermediate frequency (IF), and an electrical LO is used to downconvert the IF signal to baseband. Both implementations are shown in Fig. 5. Although we show the optical hybrids as 3-dB fiber couplers, the same networks can be implemented in free-space optics using 50/50 beamsplitters; this was the approach taken by Tsukamoto [21]. Since a beamsplitter has the same transfer function as a fiber coupler, there is no difference in their performances.

In the heterodyne downconverter of Fig. 5(a), the optical frequency bands around *ω _{LO}*+

*ω*and

_{IF}*ω*-

_{LO}*ω*both map to the same IF. In order to avoid DWDM crosstalk and to avoid excess ASE from the unwanted image band, optical filtering is required before the downconverter. The output current of the balanced photodetector in Fig. 5(a) is:

_{IF}where
${E}_{LO,l}\left(t\right)=\sqrt{{P}_{LO,l}}{e}^{j\left({\omega}_{LO}t+{\varphi}_{LO}\left(t\right)\right)}$
is the *E*-field of the LO laser, and *P _{LO,l}*,

*ω*and

_{LO}*ϕ*(

_{LO}*t*) are its power, frequency and phase noise.

*I*(

_{sh, l}*t*) is the LO shot noise. Assuming

*P*≫

_{LO}*P*,

_{s}*I*(

_{sh,l}*t*) has a two-sided psd of

*S*(

_{Ish}*f*)=

*qRP*A

_{LO}^{2}/Hz.. Substituting Eq.(6) into Eq. (7), we get:

where *ω _{IF}*=

*ω*-

_{s}*ω*is the IF,

_{LO}*ϕ*(

*t*)=

*ϕ*(

_{s}*t*)-

*ϕ*(

_{LO}*t*) is the carrier phase, and

*y*(

_{li}*t*) and

*y*(

_{lq}*t*) are the real and imaginary parts of:

The term
$2R\sqrt{{P}_{LO,r}}{E}_{sp,l}^{\prime}\left(t\right)$
in Eq. (8) is sometimes called *LO-spontaneous beat noise*, and
${E}_{sp,l}^{\prime}\left(t\right)=\mathrm{Im}\left\{{E}_{sp,l}\left(t\right){e}^{-j\left({\omega}_{LO}t+{\varphi}_{LO}\left(t\right)\right)}\right\}$
has a two-sided psd of
$\frac{1}{2}{S}_{\mathrm{Esp}}\left(f\right)$
.

It can similarly be shown that the currents at the outputs of the balanced photodetectors in the homodyne downconverter (Fig. 5(b)) are:

where *E*′* _{sp,li}* and

*E*′

*are white noises with two-sided psd $\frac{1}{2}{S}_{Esp}\left(f\right)$ ; and*

_{sp,lq}*I*and

_{sh,li}*I*are white noises with two-sided psd $\frac{1}{2}{S}_{Ish}\left(f\right)$ . Since it can be shown that thermal noise is always negligible compared to shot noise and ASE noise [22], we have neglected this term in Eq. (10) and (11). In long-haul systems, the psd of LO-spontaneous beat noise is typically much larger than that of LO shot noise; such systems are thus ASE-limited. Conversely, unamplified systems do not have ASE, and are therefore LO shot-noise-limited.

_{sh,lq}If one were to demodulate Eq. (8) by an electrical LO at *ω _{IF}*, as shown in Fig. 5(a), the resulting baseband signals

*I*(

_{het,l,i}*t*) and

*I*(

_{het,l,q}*t*) will be the same as Eqs. (10) and (11) for the homodyne downconverter in Fig. 5(b), with all noises having the same psd’s. Hence, the heterodyne and the

*two-quadrature*homodyne downconverters have the same performance [23]. A difference between heterodyne and homodyne downconversion only occurs when the transmitted signal occupies one quadrature (e.g. 2-PSK) and the system is LO shot-noise-limited, as this enables the use of a

*single-quadrature*homodyne downconverter that has the optical front-end of Fig. 5(a), but has

*ω*=

_{s}*ω*. Its output photocurrent is $2R\sqrt{{P}_{LO,l}}\left(\sqrt{{P}_{r}}{y}_{lq}\left(t\right)+{E}_{sp,l}^{\prime}\left(t\right)\right)+{I}_{sh,l}\left(t\right)$ . Compared to Eq. (11), the signal term is doubled (four times the power), while the shot noise power is only increased by two, thus yielding a sensitivity improvement of 3 dB compared to heterodyne or

_{LO}*two-quadrature*homodyne downconversion. This case is not of practical interest in this paper, however, as long haul systems are ASE-limited, not LO shot-noise-limited. Also, for good spectral and power efficiencies, modulation formats that encode information in both I and Q are preferred. Hence, there is no performance difference between a homodyne and a heterodyne downconverter provided optical filtering is used to reject image-band ASE for the heterodyne downconverter. Since the two downconverters in Fig. 5 ultimately recover the same baseband signals, we can combine Eqs. (10) and (11) and write a normalized, canonical equation for their outputs as:

where *n _{l}*(

*t*) is complex white noise with a two-sided psd of:

*γ _{s}* is the signal-to-noise ratio (SNR) per symbol. The values of

*γ*for homodyne and heterodyne downconverters in different noise regimes are shown in Table 1. For the shot-noise limited regime using a heterodyne or two-quadrature homodyne downconverter,

_{s}*γ*is simply the number of detected photons per symbol. We note that Eq. (12) is complex-valued, and its real and imaginary parts are the two baseband signals recovered in Fig. 5. For the remainder of this paper, it is understood that all complex arithmetic operations are ultimately implemented using these real and imaginary signals.

_{s}The advantages of heterodyne downconversion are that it uses only one balanced photodetector and has a simpler optical hybrid. However, the photocurrent in Eq. (8) has a bandwidth of *ω _{IF}*+

*BW*, where

*BW*is the signal bandwidth (Fig. 6(a)). To avoid signal distortion caused by overlapping side lobes,

*ω*needs to be sufficiently large. Typically,

_{IF}*ω*≈

_{IF}*BW*, thus a heterodyne downconverter requires a balanced photodetector with at least twice the bandwidth of a homodyne downconverter, whose output photocurrents in Eqs. (10) and (11) only have bandwidths of

*BW*(Fig. 6(b)). This extra bandwidth requirement is a major disadvantage. In addition, it is also difficult to obtain electrical mixers with baseband bandwidth as large as the IF. A summary of homodyne and heterodyne receivers is given in Table 2. A comparison of carrier synchronization options is given in Table 3.

#### 3.4.2 Dual-polarization downconverter

In the single-polarization downconverter, the LO needed to be in the same polarization as the received signal. One way to align the LO polarization with the received signal is with a polarization controller (PC). There are several drawbacks with this: first, the received polarization can be time-varying due to random birefringence, so polarization tracking is required. Secondly, PMD causes the received Stokes vector to be frequency-dependent. When a single-polarization receiver is used, frequency-selective fading occurs unless PMD is first compensated in the optical domain. Thirdly, a single-polarization receiver precludes the use of polarization multiplexing to double the spectral efficiency.

A dual-polarization downconverter is shown inside the receiver of Fig. 4(a). The LO laser is polarized at 45° relative to the PBS, and the received signal is separately demodulated by each LO component using two single-polarization downconverters in parallel, each of which can be heterodyne or homodyne. The four outputs are the I and Q of the two polarizations, which has the full information of **E**
* _{s}*(

*t*). CD and PMD are linear distortions that can be compensated quasi-exactly by a linear filter.

The lossless transformation from the optical to the electrical domain also allows the receiver to emulate noncoherent and differentially coherent detection by nonlinear signal processing in the electrical domain (Fig. 7). In long-haul transmission where ASE is the dominant noise source, these receivers have the same performance as those in Fig. 1–3. A summary of the detection methods discussed in this section is shown in Table 4.

## 4. Modulation formats

In this section, we compare the BER performance of different modulation methods for single-carrier transmission corrupted by AWGN. We assume that all channel impairments other than AWGN – including CD, PMD, laser phase noise and nonlinear phase noise – have been compensated using techniques discussed in Section 5. Since ASE and LO shot noise are Gaussian, the performance equations obtained are valid for both long haul and back-to-back systems, when the definition of SNR defined by Table 1 is used. Owing to fiber nonlinearity, it is desirable to use modulation formats that maximize power efficiency. Unless otherwise stated (PolSK being the only exception), the formulae provided assume transmission in one polarization, where noise in the unused polarization has been filtered. This condition is naturally satisfied when a homodyne or heterodyne downconverter is used. For noncoherent detection, differentially coherent detection and hybrid detection, the received optical signal needs to be passed through a polarization controller followed by a linear polarizer.

Since the two polarizations in fiber are orthogonal channels with statistically independent noises, there is no loss in performance by modulating and detecting them separately. The BER formulae provided are thus valid for polarization-multiplexed transmission provided there is no polarization crosstalk. Polarization multiplexing doubles the capacity while maintaining the same receiver sensitivity in SNR per bit. We write the received signal as:

where *x _{k}* is the transmitted symbol and

*n*is AWGN. For the remainder of this paper, our notation shall be as follows:

_{k}*M* is the number of signal points in the constellation.

*b*=log_{2}(*M*) is the number of bits encoded per symbol.

*T _{b}*=

*T*/

_{s}*b*is the equivalent bit period.

*γ _{s}*=

*E*[|

*x*|

_{k}^{2}]/

*E*[|

*n*|

_{k}^{2}] is the SNR per symbol in single-polarization transmission.

*γ _{s}*=

*E*[|

**x**

_{k}|

^{2}]/

*E*[|

**n**

_{k}|

^{2}] is the SNR per symbol in dual-polarization transmission (e.g. polarization-multiplexed or PolSK).

*γ _{b}*=

*γ*/

_{s}*b*is the SNR per bit

The maximum achievable spectral efficiency (bit/s/Hz) of a linear AWGN channel is governed by the Shannon capacity [1]:

If *N _{D}* identical channels are available for transmission, and we utilize them all by dividing the available power equally amongst the channels, the total capacity is

*b*=

*N*log

_{D}_{2}(1+

*γ*/

_{s}*N*), which is an increasing function of

_{D}*N*. Hence, the best transmission strategy is to use all the dimensions available. For example, suppose a target spectral efficiency of 4 bits per symbol is needed. Polarization-multiplexed 4-QAM, which uses the inphase and quadrature components of both polarizations, has better sensitivity than single-polarization 16-QAM.

_{D}*ASK with noncoherent detection*

Optical *M*-ary ASK with noncoherent detection has signal points evenly spaced in nonnegative amplitude [25]. The photocurrents for different signal levels thus form aquadratic series. It can be shown that the optimal decision thresholds are approximately the geometric means of the intensities of neighboring symbols. Assuming the use of Gray coding, it can be shown that for large *M* and *γ _{b}*, the BER is approximated by [5]:

*DPSK with differentially coherent detection*

Assuming the use of Gray coding, the BER for *M*-ary DPSK employing differentially coherent detection is [26]:

For binary DPSK, the above formula is exact, and can be simplified to [27]:

For quaternary DPSK, we have [27]:

where
$\alpha =\sqrt{2{\gamma}_{b}\left(1-\sqrt{\frac{1}{2}}\right)}$
and
$\beta =\sqrt{2{\gamma}_{b}\left(1+\sqrt{\frac{1}{2}}\right)}$
. *Q*
_{1}(*x*, *y*) and *I*
_{0}(*x*) are the Marcum *Q* function and the modified Bessel function of the zeroth order, respectively.

*Polarization-Shift Keying (PolSK)*

PolSK is the special case in this section where the transmitted signal naturally occupies both polarizations. Thus, polarization multiplexing cannot be employed to double system capacity. The BER for binary PolSK is [16]:

For higher-order PolSK, the BER is well-approximated by [28],

where

*n*, *θ*
_{0} and *θ*
_{1} are related to the number of nearest neighbors and the shape of the decision boundaries on the Poincaré sphere. Table 5 shows their values for 4-PolSK and 8-PolSK. Square 4-PolSK denotes the constellation where the signal points lie at the vertices of a square enclosed by the Poincaré square. In tetrahedral 4-PolSK and cubic 8-PolSK, the signal points lie at the vertices of a tetrahedron and a cube enclosed by the Poincaré square, respectively.

*PSK with coherent detection*

Assuming the use of Gray coding, the BER for *M*-ary PSK employing coherent detection is given approximately by [29]:

For the special cases of BPSK and QPSK, we have the exact expressions:

*QAM with coherent detection*

Assuming the use of Gray coding, the BER for a square *M*-QAM constellation with coherent detection is approximated by [29]:

The BER performance of 8-QAM with the signals points arranged as shown in Fig. 8 is [20]:

Using Eq. (16) to (27), we compute the SNR per bit required for each modulation format discussed to achieve a target BER of 10^{-3}, which is a typical threshold for receivers employing forward error-correction coding (FEC). The results are shown in Table 6. In Fig. 9, we plot spectral efficiency vs SNR per bit with polarization multiplexing to obtain fair comparison with PolSK (we also show results for 12-PolSK and 20-PolSK [28]). Since polarization-multiplexed ASK, DPSK and PSK all have two DOF (one per polarization), as is the case with PolSK, they all have similar slopes at high spectral efficiency. Because QAM uses all four available DOF for encoding information, it has better SNR efficiency than the other formats, and exhibits a steeper slope at high spectral efficiency.

## 5. Channel impairments and compensation techniques in single-carrier systems

In this section, we review the major channel impairments in fiber-optic transmission. We present the traditional methods of combating these, and show how compensation can be done electronically with coherent detection in single-carrier systems. Impairment compensation in multi-carrier systems is discussed in Section 6.

#### 5.1 Linear impairments

### 5.1.1 Chromatic dispersion

CD is caused by a combination of waveguide and material dispersion [22]. In the frequency domain, CD can be represented by a scalar multiplication:

where *L _{fiber}* is the length of the fiber,

*β*

_{2}is the dispersion parameter,

*β*

_{3}is the dispersion slope, and

*ω*is the signal carrier frequency. Uncompensated CD leads to pulse broadening, causing intersymbol interference (ISI). Long-haul systems use DCF to compensate CD optically [22]. However, inexact matching between the

_{s}*β*

_{2}and

*β*

_{3}of transmission fiber and DCF dictates the need for terminal dispersion compensation at high bit rates, typically 40 Gbit/s or higher [30]. In reconfigurable networks, data can be routed dynamically through different fibers, so the residual dispersion can be time-varying. This necessitates tunable dispersion compensators.

### 5.1.2 Polarization-mode dispersion

Polarization-mode dispersion (PMD) is caused by random birefringence in the fiber. In first-order PMD, a fiber possesses a “fast axis” along in polarization and a “slow axis” in the orthogonal polarization (Fig. 10). These states of polarization are known as the *principal states of polarization* (PSPs), and can be any vector in Stokes space in general. First-order PMD can be written as [31]:

where
$\mathbf{D}=diag(\frac{{e}^{j\omega \tau}DGD}{2},\frac{{e}^{-j\omega \tau}DGD}{2})$
is a diagonal matrix with *τ _{DGD}* being the differential group delay between the two PSPs, and

**R**

_{1}and

**R**

_{2}are unitary matrices that rotate the reference polarizations to the fiber’s PSPs, which are elliptical in general. When a signal is launched in any polarization state other than a PSP, the receiver will detect two pulses at each reference polarization. Ignoring CD and other effects, the impulse response measured by a polarization-insensitive direct-detection receiver is

*h*(

*t*)=

*a*

^{2}·

*δ*(

*t*-

*τ*/2)+(1-

_{DGD}*a*

^{2})·

*δ*(

*t*+

*τ*/2.), where

_{DGD}*a*

^{2}is the proportion of transmitted energy falling in the slow PSP. In this simple two-path model, we see that PMD can lead to frequency-selective fading [32]. In contrast to CD, which is relatively static, PMD (both the PSPs and the DGD) can fluctuate on time scales on the order of a millisecond [33]. Thus PMD compensators thus need to be rapidly adjustable. The statistical properties of PMD have been studied in [34–36], and it has been shown that

*τ*has a Maxwellian distribution, whose mean value

_{DGD}*$\overline{\tau}$*grows as the square-root of fiber length. In SMF,

_{DGD}*$\overline{\tau}$*is typically of order $\frac{0.1\mathrm{ps}}{\sqrt{\mathrm{km}}}$ . PMD is a significant impact on systems at bit rates of 40 Gbit/s and higher, because

_{DGD}*τ*can be a significant fraction of the symbol period. Uncompensated PMD can result in system outage [37].

_{DGD}One method of combating PMD is to use a tunable PC at the transmitter to ensure the input signal is launched into a PSP [38]. Receiver-based compensators for first-order PMD use a continuously tunable PC followed by a variable retarder, which inverts the DGD of the fiber [38,39]. By cascading multiple first-order PMD compensators, one can retrace the PMD vector of the transmission fiber. Such a device can compensate higher-order PMD [40]. Optical PMD compensators have been constructed using nonlinear chirped fiber Bragg gratings [41], planar lightwave circuits (PLC) [42] and polarization-maintaining fibers (PMF) twisted mechanically [40]. Compensation of *τ _{DGD}* as large as 1.7 symbols was demonstrated by Noé et al [40]. A major limitation of optical PMD compensation is that device performance depends on the degree of tunability, and increasing the number degrees of freedom can require costly hardware. However, optical PMD compensators are transparent to the data rate and modulation format of the transmitted signal, and have been successfully employed for very high-data-rate systems, where digital compensation is currently impossible.

Electronic PMD equalization has gained considerable recent interest. Buchali and Bülow studied the use of a feedforward equalizer (FFE) with decision feedback equalizer (DFE) to combat PMD systems using direct detection of OOK [39]. As with electronic CD compensation in direct detection of OOK [43], the loss of phase during detection prevents quasi-exact compensation of PMD.

### 5.1.3 Other linear impairments

In addition to CD and PMD, a fiber optic link can also have polarization-dependent loss (PDL) due to anisotropy of network components such as couplers, isolators, filters, multiplexers, and amplifiers [44]. In DWDM transmission, arrayed waveguide gratings (AWG), interleavers and reconfigurable add-drop (de)multiplexers (ROADMs) cause attenuation at the band edges of a channel. When a signal has to pass through cascaded elements, bandwidth narrowing can be problematic. This is a major challenge in 40 Gbit/s RZ-DPSK at 50 GHz channel spacing [45]. Bandwidth narrowing can be equalized by tunable optical equalizers, but such devices are costly, introduce loss, and are difficult to make adaptive.

### 5.1.4 Compensation of linear impairments and computational complexity

Since a dual-polarization downconverter linearly recovers the full electric field, CD and PMD can be compensated quasi-exactly in the electronic domain after photodetection. One approach is to use a tunable analog filter. However, as in the case of optical compensators, it is difficult to implement the desired transfer function exactly, and analog filters are also difficult to make adaptive. In addition, parasitic effects like signal reflections can lead to signal degradation.

With improvements in DSP technology, digital equalization of CD and PMD is becoming feasible. When the outputs of a dual-polarization downconverter are sampled above the Nyquist rate, the digitized signal contains a full characterization of the received *E*-field, allowing compensation of linear distortions by a linear filter. CD compensation using a digital infinite impulse response (IIR) filter was studied by [46]. Although an (FIR filter) allows fewer taps, it is more difficult to analyze, and may require greater receiver complexity because of the need to implement time-reversal filters. In this paper, we concentrate on CD and PMD compensation using a finite impulse response (FIR) filter.

Linear equalization using an FIR filter for dually polarized coherent systems was studied in [47,48]. The canonical system model is shown in Fig. 11. The analog outputs of a dual-polarization downconverter are passed through anti-aliasing filters with impulse responses *p*(*t*) and then sampled synchronously at a rate of 1/*T*=*M*/*KT _{s}*, where

*M*/

*K*is a rational oversampling ratio. We assume that the sampling clock has been synchronized using well-known techniques [49]. In theory, the use of a matched filter in conjunction with symbol rate sampling is optimal. In practice however, symbol-rate sampling is susceptible to sampling time errors. Fractionally spaced sampling has been shown to overcome this [50–52]. Let

*q*(

_{ij}*t*)=

*b*(

*t*)⊗

*h*(

_{ij}*t*)⊗

*p*(

*t*) and

*n*′

*(*

_{i}*t*)=

*p*(

*t*)⊗

*n*(

_{i}*t*). We can write the digital samples as:

Suppose a linear equalizer takes the *N*=2*L*+1 samples closest to symbol *k* to computes the minimum-mean-square error (MMSE) estimate of the *k*-th symbol **x̃**
* _{k}*. We have:

where
${\mathbf{y}}_{i,k}={[{y}_{i,\lfloor \frac{\mathrm{kM}}{K}\rfloor +L},{y}_{i,\lfloor \frac{\mathrm{kM}}{K}\rfloor +L-1},\cdots ,{y}_{i,\lfloor \frac{\mathrm{kM}}{K}\rfloor -L}]}^{T}$
, *i*=1, 2. It can be shown that the MMSE filter is a 2×2*N* matrix given by [19]:

where **A**=*E*⌊**y**
^{*}
_{k}**y**
* ^{T}_{k}*⌋ and

**α**=

*E*⌊

**y**

^{*}

_{k}**x**

*⌋. When an non-integer oversampling ratio (*

^{T}_{k}*K*>1) is used, there are

*K*solutions to (32) depending on the value of

*kM*mod

*K*. In the example of 3/2 oversampling, there are separate Wiener solutions for odd and even symbols because of the difference in the sampling times relative to the centers of the symbols.

Defining **ε**
* _{k}*=

**x**

*-*

_{k}**x̃**

*to be the error (in the absence of other channel effects), it can be shown that the the mean-square error (MSE) matrix associated with equalizer*

_{k}**W**is a quadratic surface:

and that *tr*{*E*} is minimized by choosing **W**=**W**
* _{opt}*. Thus for a static channel

**H**, Eq. (32) gives the optimum equalizer of length

*N*=2

*L*+1. It can be shown that as

*M*/

*K*→∞ and

*N*→∞, and the anti-aliasing filter does not introduce amplitude distortion, the power penalty of a compensated CD/PMD channel approaches zero with respect to a pure AWGN channel at the same SNR. In practice, since

**H**can be time-varying, an adaptive equalizer shown in Fig. 12 is desired. Owing to the quadratic nature of Eq. (33), we can use well-known algorithms such as least mean square (LMS) or recursive least squares (RLS) [53]. For LMS, we have the following update equation for the filter coefficients:

where **W**
^{(m)} is the equalizer coefficients use to compute **x̃**
* _{k}*, and

*μ*is the step size parameter that needs to satisfy 0<

*μ*<1/

*λ*

_{max}, where

*λ*

_{max}is the largest eigenvalue of

**A**=

*E*⌊

**y**

^{*}

_{k}**y**

*⌋. When a non-integer oversampling ratio is used,*

^{T}_{k}*K*filters are required for all possible values of

*kM*mod

*K*. The index

*m*indicates the

*m*-th use of that particular filter.

Ip and Kahn showed that when *p*(*t*) is a fifth-order Butterworth filter with a 3-dB bandwidth of 0.4(*M*/*K*)*R _{s}*, any arbitrary amount of CD and first-order PMD can be compensated with less than 2 dB power penalty provided the oversampling rate

*M*/

*K*≥3/2, and the filter length

*N*satisfies:

for mitigating CD and PMD, respectively [19]. It was found that for *M*/*K*≥3/2, the required value of *N* is insensitive to whether NRZ or RZ pulses are used, that system performance is insensitive to sampling time errors. Typically, the required value of *N* is dominated by CD considerations. In Table 7, we show the required value of *N* for different transmission distances, where inline DCF is used with 2% CD under-compensation. The target bit rate is 100 Gbit/s, and we consider polarization-multiplexed 4-QAM transmission. The complexity of directly implementing Eq. (32) is 4*NR _{s}* complex multiplications per second. For large

*N*, linear equalization is more efficiently implemented using an FFT-based block processing [54]. Suppose a block length of

*B*is chosen. An FFT-based implementation has a complexity of

*R*(

_{s}*N*+

*B*-1)(2log

_{2}(

*N*+

*B*-1)+4)/

*B*complex multiplications per second. For a given

*N*, there exists an optimum block length

*B*that minimizes the number of operations required. It can be shown that the asymptotic complexity grows as

*R*log

_{s}_{2}

*R*. We compare the complexity of single-carrier versus multi-carrier transmission (using OFDM) in Section 7.

_{s}#### 5.2 Nonlinear impairments

### 5.2.1 Fiber nonlinearity

The dominant nonlinear impairments in fiber arise from the Kerr nonlinearity, which causes a refractive index change proportional to signal intensity. Signal propagation in the presence of fiber attenuation, CD and Kerr nonlinearity is described by the non-linear Schrödinger equation (NLSE)

where *E*(*z*,*t*) is the electric field, *α* is the attenuation coefficient, *β*
_{2} is the dispersion parameter, *γ* is the nonlinear coefficient, and *z* and *t* are the propagation direction and time, respectively.

Nonlinear effects include deterministic and statistical components. The nonlinearity experienced by a signal due to its own intensity called self-phase modulation (SPM). In WDM systems, a signal also suffers nonlinear effects due to the fields of neighboring channels. These are cross-phase modulation (XPM) and four-wave-mixing (FWM) [22], and their impact can be reduced by allowing non-zero local dispersion. In the absence of ASE, and given knowledge of the transmitted data, all these nonlinear effects are deterministic, and it is possible, in principle, to pre-compensate them at the transmitter. At the receiver, it would be also be possible to employ joint multi-channel detection techniques, though complexity precludes their implementation at this time. Here, we consider only receiver-based compensation of SPM-induced impairments in the presence of CD, which leads to intra-channel nonlinear effects.

In long-haul systems, interaction between ASE noise and signal through the Kerr nonlinearity leads to nonlinear phase noise (NLPN). When caused by the ASE and signal in the channel of interest, this is called SPM-induced NLPN. When caused by the ASE and signal in neighboring channels, it is called XPM-induced NLPN. Here, we consider only receiver-based compensation of SPM-induced NLPN.

In following two sections, we discuss (i) SPM in the presence of CD, and (ii) SPM-induced NLPN. We show how these can be compensated by exploiting their correlation properties.

### 5.2.2 Self-phase modulation with chromatic dispersion: intra-channel nonlinearity

We consider a noiseless transmitted signal *E*(0,*t*)=∑* _{k}x_{k}b*(

*t*-

*kT*)=∑

_{s}*, where*

_{k}x_{k}b_{k}*T*is the symbol period,

_{s}*b*is the sampled pulse shape, and

_{k}*x*∊{

_{k}*e*

^{j2π/M},

*e*

^{j4π/M},…,

*e*

^{j2π}} are the transmitted symbols chosen from an

*M*-ary PSK constellation. We use first-order perturbation theory to gain insight into the effects of nonlinearity [55]. Let

*E*=

*E*

^{(lin)}+Δ

*E*=∑

*(*

_{k}x_{k}*b*

_{k}^{(lin)}+Δ

*b*), where

_{k}*E*

^{(lin)}is the linear solution to the NLSE(obtained by setting the right hand side of (37) to zero) and Δ

*E*is the perturbation due to Kerr nonlinearity. The NLSE can be re-written as:

For typical terrestrial links, the accumulated dispersion is such that the only the terms on the right hand side of (38) that will induce a sizeable effect on the pulse at symbol *k* are those with indices satisfying *k*=*l*+*m*-*p*. Without loss of generality, we focus on symbol *k*=0. The NLSE can then be simplified to:

The term *l*=*m*=0 is a deterministic distortion of a pulse by itself, and is referred to as *intra-channel self-phase modulation* (ISPM). The terms where *l*=0,*m*≠*l*(and *m*=0,*l*≠*m*) are distortions to a pulse by neighboring pulses, and are known as *intra-channel cross-phase modulation* (IXPM), as they are analogous to signal distortion by neighboring channels in XPM. Finally, the remaining terms *l*, *m*≠0 are called *intra-channel four-wave mixing* (IFWM), because they are analogous to the interacting frequencies in FWM. We emphasize that the “intra-channel” effects all originate from SPM.

It is well-known that in OOK with direct detection, IXPM causes timing jitter to the pulses, while IFWM causes amplitude jitter (or “ghost pulses”) in the zero bits [56,57]. One can minimize these effects by careful dispersion map design [58,59], phase alternations [60] and coding [61]. When coherent detection is used in conjunction with constant-intensity modulation formats (e.g., PSK or DPSK), IXPM is deterministic, since |*x _{l}*|

^{2}is constant. Hence the randomness in

*b*

^{(lin)}

_{0}is only due to IFWM. Let

*C*′

*(*

_{l,m}*t*) be the solution to

which represents signal propagation through one span of fiber. The IFWM-induced phase noise at symbol 0 is given by:

where Δ*b*
^{(S+X)} is the deterministic perturbation to *b*
^{(lin)}
_{0} due to ISPM and IXPM. IFWM phase noise was first studied by Wei and Liu [62], who showed that *ϕ _{IFWM}* are correlated between symbols. In long-haul transmission over multiple identical spans of fiber,

*ϕ*add coherently for each span. Lau and Kahn have shown the autocorrelation function of IFWM phase noise:

_{IFWM}*R*(

*k*,

*t*)=

*E*[

*ϕ*(

_{IFWM}*t*)

*ϕ*(

_{IFWM}*t*-

*kT*)] is given by [63]:

$$R\left(k\right)=\frac{1}{2}\sum _{m}\mathrm{Re}\left\{{C}_{m,k-m}{C}_{-m,m-k}^{*}\right\}-\frac{1}{2}\sum _{m}\mathrm{Re}\left\{{C}_{m,k}{C}_{m,-k}\right\}$$

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}+\frac{1}{2}\sum _{m}\mathrm{Re}\left\{{C}_{m,k}{C}_{m,-k}^{*}\right\}-\frac{1}{2}\sum _{m}\mathrm{Re}\left\{{C}_{m,k-m}{C}_{-m,m-k}\right\},k>0$$

for BPSK system, and

for *M*-ary PSK with *M*>2. Fig. 13 shows *R*(*k*, *t*) for single-polarization RZ-QPSK at 80 Gbit/s, where we used Gaussian pulses with 33% duty cycle. We compare Eq. (43) with Monte Carlo simulations at different sampling times *t*. The simulations used 5000-trial propagations through a typical terrestrial system with a random 32-pulse sequence. We observe good agreement between theory and simulation results. It is thus possible to reduce effects of IFWM by implementing a linear noise predictor in DSP using knowledge of *R*(*k*). Assuming the received symbols are sampled at the optimal instants, we have *θ _{k}*=arg{

*x*}+

_{k}*ϕ*(

_{IFWM}*kT*). A 1.8-dB improvement in performance was obtained using a linear predictor for IFWM when IFWM is the dominant system impairment.

### 5.2.3 Nonlinear phase noise

SPM-induced NLPN is often called the Gordon-Mollenauer (G-M) effect [64]. Fig. 14(b) illustrates the G-M effect for QPSK. The received constellation is spiral-shaped, as signal points with larger amplitude experiences larger phase shifts.

Let the ASE noise of the *i*-th amplifier be *n _{i}*~

*N*(0,

*σ*

^{2}

*I*). We assume the

*n*are i.i.d. with

_{i}*σ*

^{2}∝

*G*-1, where

*G*is the amplifier gain. In the absence of CD and multi-channel nonlinear effects, the NLPN of a system with N uniformly spaced identical amplifiers is:

It can be shown that the variance of Eq. (44) is:

where *L _{eff}* is the effective nonlinear length of each span [65]. Ho studied the probability density function (pdf) of NLPN for distributed amplification and obtained analytical formulae for the BER of PSK and DPSK systems [66]. Although

*ϕ*is not Gaussian,

_{NL}*σ*

^{2}

*is a good measure of the impact of NLPN on system performance. Lau and Kahn studied joint minimization of the NLPN and linear phase noise variances by optimizing the gains and spacings of amplifiers in long-haul transmission [67,68]. In addition, as*

_{NL}*ϕ*is correlated with instantaneous received power

_{NL}*P*, one can perform receiver-based compensation of NLPN by applying a phase rotation proportional to

_{rec}*P*. Ho and Kahn [69], Ly-Gagnon and Kikuchi [70] and Liu et al [71] have shown that the optimal phase rotation

_{rec}*θ*, which minimizes

_{opt}*σ*

^{2}

_{NL}=

*E*⌊(

*ϕ*-

_{NL}*θ*)

_{opt}^{2}⌋, is give by:

Compared to the uncompensated case, *φ*
^{2}
* _{NL}* is reduced by a factor of four (6 dB). This phase rotation can be performed by a phase modulator, or digitally by DSP. Various experiments have confirmed the performance improvement of this technique [72–74]. Ho [75] also studied mid-span phase rotation proportional to instantaneous signal power, and showed that a reduction of

*σ*

^{2}

*by a factor of nine (9.5 dB) can be obtained when the compensation is performed at 2/3 the length of the transmission link. Recently, Lau and Kahn showed that the ML detection boundaries for*

_{NL}*M*-ary PSK in the presence of NLPN are of the form ${\theta}_{\mathrm{ML}}={\mathrm{aP}}_{\mathrm{rec}}+b\sqrt{{P}_{\mathrm{rec}}}+c$ [76]. Hence, ML detection can be implemented by rotating the received signal by

*θ*and then using straight decision boundaries (Fig. 14(c)). The phase rotation techniques discussed can also be applied to 16-QAM where it has been shown that ML detection is well-approximated by straight-line decision boundaries when phase pre-compensation and/or post-compensation is implemented at the transmitter and/or receiver[76].

_{ML}When CD is also considered, NLPN becomes more complicated. For signal propagation over a single span of fiber with perfect dispersion compensation and at high OSNR, the nonlinear phase noise is given by

where *h*
_{-z} (*t*) is the impulse response due to the CD of the fiber from *z* to *L*. The variance of Eq. (47) was studied by Green [77], Kumar [78] and Ho [79], and it was shown that the temporal profile of
${\sigma}_{{\varphi}_{\mathrm{NL}}}^{2}\left(t\right)$
is asymmetric with respect to *t*=0 due to the dispersive effect of ASE noise. Kumar et al [80] and Boivin et al [81] proposed the use of optical phase conjugation to mitigate
${\sigma}_{{\varphi}_{\mathrm{NL}}}^{2}\left(t\right)$
, while Serena et al [82] presented a method of characterizing BER analytically in the presence of NLPN based on a parametric gain approach. Further statistical properties of NLPN in the presence of CD, including its pdf and psd, have yet to be investigated.

### 5.2.4 Comparison of IFWM phase noise and nonlinear phase noise

The relative impact of IFWM phase noise and NLPN depend on the system parameters. Eqs. (41) and (44) show that the statistical components of IFWM and NLPN varies as *ϕ _{IFWM}* ∝ |

*E*|

^{2}and

*ϕ*∝|

_{NL}*E*·

*n*|. Hence, the ratio $\frac{{\sigma}_{{\varphi}_{\mathrm{IFWM}}}^{2}}{{\sigma}_{{\varphi}_{\mathrm{NL}}}^{2}}\propto \frac{P}{{\sigma}^{2}}$ is a function of OSNR. For a system with

*N*amplifiers, ${\sigma}_{{\varphi}_{\mathrm{NL}}}^{2}\propto {N}^{3}$ [64], while ${\sigma}_{{\varphi}_{\mathrm{IFWM}}}^{2}\propto {N}^{2}$ , as

*ϕ*adds coherently between spans. Ho and Wang [83] and Zhang et al [84] investigated the relative impact of

_{IFWM}*ϕ*and

_{IFWM}*ϕ*for DPSK, and showed that

_{NL}*ϕ*adds coherently between spans. Ho and Wang [83] and Zhang et al [84] investigated the relative impact of

_{IFWM}*ϕ*and

_{IFWM}*ϕ*for DPSK, and showed that

_{NL}*ϕ*increases with the amount of local CD as

_{IFWM}*ϕ*requires strong pulse overlap to occur, while

_{IFWM}*ϕ*increases with decreasing CD. Zhang et al studied the effects of dispersion pre-compensation and having residual dispersion per span for 40 Gbit/s RZ-DPSK and showed that when

_{NL}*ϕ*dominates, the optimal dispersion pre-compensation is similar to that for RZ-OOK [85]. Finally, Zhu et al [86] studied the BER performance of DPSK in the presence of NLPN, IFWM phase noise and linear phase noise through semi-analytical characterizations of the joint phase noise variance.

_{IFWM}#### 5.3 Laser phase noise

Laser phase noise is caused by spontaneous emission [87], and is modeled as a Wiener process [88]:

where *ϕ*(*t*) is the instantaneous phase, *δω*(*t*) is frequency noise with zero mean and autocorrelation *R _{δωδω}*(

*τ*)=2

*π*Δ

*νδ*(

*τ*). It can be shown that the laser output ${E}_{0}\left(t\right)=A{e}^{j\left({\omega}_{c}t+\varphi \left(t\right)\right)}$ has a Lorentzian spectrum with a 3-dB linewidth Δ

*ν*. Schawlow and Townes showed that laser linewidth is inversely proportional to output power [89], so it is desirable to operate the TX and LO lasers at maximum power, attenuating their outputs as required.

Phase noise is an important impairment in coherent systems as it impacts carrier synchronization. In noncoherent detection, the carrier phase is unimportant because the receiver only measures energy. In DPSK, information is encoded by phase changes, and Δ*ν* only needs to be small enough such that the phase fluctuation over a symbol period is small.

In Eq. (12), we showed that the baseband signal *y*(*t*) is modulated by *e*
^{jϕ(t)}. In the absence of other impairments, this manifests as a rotation of the received constellation. Carrier synchronization is required to ensure *ϕ*(*t*) is small so the transmitted symbols can be detected with low power penalty. Since phase noise is a Wiener process with temporal correlation, it can be mitigated by signal processing. In the next two subsections, we consider carrier synchronization in a single polarization using a PLL and FF carrier synchronizer.

### 5.3.1 Phase-locked loop

The traditional method of carrier synchronization is to use a PLL. A system diagram of a PLL is shown in Fig. 15(a). The phase estimator removes the data modulation so that *ϕ*(*t*) can be measured. This can be done by a number of methods, including raising the signal to the *M*-th power in *M*-ary PSK [23]. The phase estimator output is an error signal that is then passed through a loop filter, producing a control signal for the LO frequency. In an OPLL, the control signal drives the LO laser, whereas in an electrical PLL, the control signal drives the electrical VCO (Fig. 5(a)). Both types of PLL can be analyzed in the same manner, and the design of PLLs has been extensively studied in [29, 90].

The performance of a PLL is usually analyzed with the linear model shown in Fig. 15(b). We assume the LO has no phase noise, while the TX laser has phase noise equal to the sum of the linewidths of the two lasers. The signal to be tracked is *ϕ _{s}*(

*t*), which evolves as Eq. (48). Owing to AWGN, the phase estimator measures

*ψ*(

*t*)=

*ϕ*(

*t*)+

*n*′(

*t*) (Fig. 15(c)), and produces a voltage

*K*(

_{c}ψ*t*) for the loop filter

*F*′(

*s*). We assume the control port of the LO has a slope

*K*Hz/V. Delay in the loop is modeled as ${e}^{-s{\tau}_{d}}$ , where

_{ν}*τ*includes signal propagation plus the rise times of intermediate components. Delay degrades system performance. An electrical PLL is usually superior to an OPLL since according to Fig. 5(a), the optical path has to pass through the optical hybrid and balanced photodetectors. However, an LO laser is likely to have a larger tuning range than an electrical LO. If the frequency drift of the lasers is significant, an OPLL may be preferable.

_{d}The performance of the PLL is determined by the loop filter. For a given *F*′(*s*), we have:

where *F*(*s*)=*K _{c}K_{ν}F*′(

*s*). Ignoring delay, the denominator of Eq. (49) gives the loop order. For a first-order loop,

*F*′(

*s*)=

*K*. The design parameter is the loop bandwidth

_{f}*ω*=

_{n}*K*. For a second-order loop,

_{c}K_{f}K_{ν}*F*(

*s*)=2

*ζω*+

_{n}*ω*

_{n}^{2}/

*s*, where

*ζ*and

*ω*are the damping factor and natural frequency, respectively. The performance 4-QAM employing a second-order PLL was studied in [91], while the performance of 8- and 16-QAM employing a second-order PLL was studied in [20]. A damping factor of 1/√2 is typically chosen as a compromise between a rapid response and low steady-state variance. For both first-and second-order PLLs, there is an optimal

_{n}*ω*that minimizes the phase-error variance:

_{n}where

The first term in Eq. (50) arises from phase noise and is proportional to *σ*
^{2}
* _{p}*=2

*π*Δ

*νT*, which is the ratio between the laser linewidth and signal bandwidth. The second term arises from AWGN, and is inversely proportional to the received OSNR. Assuming the use of a decision-directed PLL [23],

_{s}*η*=

_{c}*E*[|

*x*|

^{2}]

*E*[1/|

*x*|

^{2}] is a penalty factor associated with the transmitted constellation [20]. A larger

*ω*allows the LO to adapt more quickly to phase fluctuations in the TX laser, but the loop becomes more susceptible to noise. These two conflicting requirements give rise to an optimum

_{n}*ω*, which needs to evaluated numerically. A typical plot of

_{n}*σ*versus

_{ϕ}*ω*is shown in Fig. 16 for a second-order PLL. We have assumed single-polarization 16-QAM at 100 Gbit/s. The laser beat linewidth and received OSNR are 100 kHz and 11.5 dB, respectively, which is 1 dB above sensitivity for BER=10

_{n}^{-3}(Table 6). We observe that delay increases

*σ*. Above

_{ϕ}*τ*=125

_{d}*T*, there is no

_{b}*ω*that can give 1-dB power penalty due to phase error (

_{n}*σ*=2.7° [20]). At 100 Gbit/s, this maximum delay corresponds to 1.25 ns, which corresponds to a transmission line only ~25 cm long. Even with careful circuit design, this is probably not feasible. Hence for high-data-rate systems, FF carrier recovery techniques are likely to be required.

_{ϕ}### 5.3.2 Feedforward carrier recovery

The PLL is a feedback system as the control phase at time *t* can only depend on past input phases up to *t*-*τ _{d}*. However, the laser phase noise process described by (48) has a symmetric autocorrelation function

*E*[

*ϕ*(

*t*)

*ϕ*(

*t*-

*τ*)|=2

*π*Δ

*ν*|

*τ*|, so its value at time

*t*has the same correlation with phases before and after

*t*. The PLL is suboptimal as it does not exploit possible knowledge of {

*ϕ*(

*t*-

*τ*):

*τ*<

*τ*}.

_{d}FF carrier synchronization for an intradyne receiver using analog electronics was described in [24], while digital FF carrier synchronization was studied in [21, 92–94]. In this section, we focus on digital FF carrier synchronization. Consider the system model shown in Fig. 17(a), where we assume all other channel impairments have been compensated by the digital coherent receiver, whose outputs are
${y}_{k}={x}_{k}{e}^{j{\varphi}_{k}}+{n}_{k}$
, where *x _{k}* is the transmitted symbol, and

*ϕ*and

_{k}*n*are the carrier phase and AWGN, respectively (Fig. 17(b)). Instead of using a PLL to ensure that

_{k}*ϕ*is small, a FF phase estimator directly estimates the carrier phase and then de-rotates the received signal by this estimate so symbol decisions can be made at low BER.

_{k}The FF phase estimator has a *soft phase estimator* that first computes a symbol-by-symbol estimate *ψ _{k}* of

*ϕ*, followed by a MMSE filter

_{k}*W*(

_{p}*z*). A number of algorithms exist for finding

*ψ*. In the case of

_{k}*M*-ary PSK, raising

*y*to the

_{k}*M*-th power which removes the data modulation. For more general signaling formats, decision-directed (DD) phase estimation can be used [18]. The symbol-by-symbol estimate is corrupted by AWGN (Fig. 17(b)) so that

*ψ*=

_{k}*ϕ*+

_{k}*n*, where

^{′}_{k}*n*

^{′}

*is the projection of*

_{k}*n*onto a vector orthogonal to ${x}_{k}{e}^{j{\varphi}_{k}}$ . Since

_{k}*ϕ*is correlated by the Wiener process, we use a linear filter to compute an MMSE estimate

_{k}*$\widehat{\varphi}$*

_{k-Δ}. Using the analytical model shown in Fig. 17(c), whose input is the discrete frequency noise process

*ν*with zero mean and variance

_{k}*σ*

^{2}

*=2*

_{p}*π*Δ

*νT*, it can be shown that the MMSE filter for Δ=0 has coefficients:

_{s}where *r*=*σ*
^{2}
* _{p}*/

*σ*

^{2}

_{n′}>0 is the ratio between the magnitudes of frequency noise and AWGN, and $\alpha =\left(1+\frac{r}{2}\right)-\sqrt{{\left(1+\frac{r}{2}\right)}^{2}-1}$ . The MMSE filter is non-causal, as it has two exponentially decaying tails toward the past and future. In practice, one can truncate Eq. (53) to

*L*significant coefficients and implement it as an FIR filter with delay $\Delta =\lfloor \frac{{L}_{p}-1}{2}\rfloor $ . For any causal filter

_{p}*W*(

_{p}*z*) with

*L*coefficients and delay Δ, it can be shown that the error

_{p}*ϕ*-

_{k}*$\widehat{\varphi}$*is Gaussian distributed with zero mean and a variance of:

_{k}As with the PLL, the variance has two terms that arise from phase noise and AWGN.

### 5.3.3 Power Penalty from Phase Error

Regardless of whether a PLL or a FF carrier synchronizer is used, the coherent receiver makes symbol decisions on
${y}_{k}={x}_{k}{e}^{j{\epsilon}_{k}}+{n}_{k}$
. For a PLL, *ε _{k}*=

*ϕ*has a Tikhonov distribution [95], while for a FF carrier synchronizer,

_{k}*ε*=

_{k}*ϕ*-

_{k}*$\widehat{\varphi}$*has Gaussian distribution. In the limit of high SNR, the Tikhonov is well-approximated by the Gaussian distribution. The method for computing BER for a given phase-error distribution can be found in [95]. With the phase error variances for the PLL and the FF carrier synchronizer given by Eqs. (50) and (54), the power penalty can be determined. In Table 8, we compare the linewidth requirements for receivers that use a PLL and a FF carrier synchronizer, assuming a 1-dB power penalty at a target BER of 10

_{k}^{-3}[18]. We observe that FF carrier recovery can tolerate 50% to 100% wider laser linewidth than a PLL, and is also insensitive to propagation delay.

### 5.3.4 Carrier synchronization in polarization-multiplexed systems

When polarization multiplexing is employed, the receiver has two independent signals from which to estimate carrier phase. Consider the system models shown in Fig. 18. In the PLL-based receiver, the baseband signals for each polarization are passed through phase error estimators that compute independent estimates {*ψ _{i}*(

*t*),

*i*=1, 2} of

*ϕ*(

*t*). In the digital FF carrier synchronizer, the soft-decision phase estimators give independent estimates {

*ψ*=1, 2} of

_{i,k}, i*ϕ*. Assuming both phase estimates are equally reliable, we take their average. Since AWGN is the only impairment preventing errorless measurement of carrier phase, and since noises in the two polarizations are independent,

_{k}*ψ*(

*t*) and

*ψ*have half as much AWGN as

_{k}*ψ*(

_{i}*t*) and

*ψ*. For a given noise psd, the AWGN contribution to Eqs. (50) and (54) is halved. However, to obtain the same symbol-error rate for each polarization, the SNR per polarization must be preserved. This requires the transmit energy per symbol (now a 2×1 complex vector) be doubled. Hence, the dependence of phase error variance on

_{i,k}*SNR per symbol*is preserved, and Eqs. (50) and (54) hold for polarization-multiplexed transmission.

#### 5.4 Integration of receiver functions

A functional receiver needs to integrate carrier synchronization, linear equalization, nonlinear impairment compensation and symbol synchronization. If we consider NLPN to be the dominant nonlinear effect, and we assume IFWM is negligible, a hypothetical receiver structure is shown in Fig. 19. The sampled outputs of a dual-polarization downconverter are first passed through an NLPN compensator, which performs phase rotation proportional to the received amplitude. Linear equalization follows, yielding an output at one sample per symbol for which FF carrier synchronization is performed, followed by symbol decisions. The ordering of linear equalization before carrier synchronization is possible because from Table 8, we expect the laser linewidth Δ*ν* to satisfy ~Δ*νT _{s}*<10

^{-4}. In contrast, the memory length of the linear equalizer is only 13 symbols for the worst case considered in Table 7. Since the carrier phase does not change significantly over the equalizer’s memory, performance should not be compromised. The filter

*W*(

_{p}*z*) in Fig. 19 exploits temporal correlation of laser phase noise and IFWM to mitigate both impairments. To make the system adaptive, we employ decision feedback to update equalizer coefficients, and the error signal is also passed through a timing error detector (TED) and loop filter to yield a control signal for the sampling clock. The design of the TED and loop filter for symbol synchronization can be found in [49].

A receiver structure with NLPN compensation first, followed by linear equalization and carrier synchronization has been employed in [72] and [73], and has been shown to reduce nonlinear phase variance for QPSK under certain transmission conditions. A receiver structure that can mitigate nonlinearities (including IFWM) for more general system designs is an ongoing research topic.

## 6. Multi-carrier modulation: Orthogonal Frequency-Division Multiplexing

#### 6.1 Introduction

Orthogonal frequency-division multiplexing (OFDM) is a multi-carrier technique that has been deployed in DSL and wireless systems [96, 97], and is receiving increasing interest in optical fiber communications [98–101]. A multi-carrier system sends information over *N _{c}* frequency-division-multiplexed (FDM) channels, each modulated by a different carrier (Fig.20(a)). For a given modulation format, with coherent detection, single-carrier and multi-carrier systems offer fundamentally the same spectral efficiency and power efficiency, but these may differ in practice, because of different impairments and implementation details. In OFDM, the modulation is implemented digitally by the inverse Fast Fourier Transform (IFFT) (Fig. 20(b)). We can write a multi-carrier signal as:

where *x _{n,m}* denotes the

*m*-th symbol transmitted on sub-carrier

*n*,

*b*(

*t*)=

*rect*(

*t*/

*T*) is the pulse shape,

_{ofdm}*T*=(

_{ofdm}*N*+

_{c}*N*)

_{pre}*T*is the OFDM symbol period, and

_{c}*f*=1/(

_{d}*N*)is the frequency separation between sub-carriers. We define

_{c}T_{c}*T*as the chip period, and

_{c}*N*and

_{c}*N*are integers. Suppose we sample Eq. (55) every chip period. We have:

_{pre}We note that the block of *N _{c}* samples
$\left\{{x}_{\mathrm{ofdm}}\left(k{T}_{c}\right):{k}_{1}\left({N}_{c}+{N}_{\mathrm{pre}}\right)-\frac{{N}_{c}}{2}\le k\le {k}_{1}\left({N}_{c}+{N}_{\mathrm{pre}}\right)+\frac{{N}_{c}}{2}-1\right\}$
is the IFFT of
$\left\{{x}_{n,{k}_{1}}\left(k{T}_{c}\right):-\frac{{N}_{c}}{2}\le n\le \frac{{N}_{c}}{2}-1\right\}$
multiplied by a scaling factor. The remaining

*N*terms $\left\{{x}_{\mathrm{ofdm}}\left(k{T}_{c}\right):{k}_{1}\left({N}_{c}+{N}_{\mathrm{pre}}\right)-\frac{{N}_{c}+{N}_{\mathrm{pre}}}{2}\le k\le {k}_{1}\left({N}_{c}+{N}_{\mathrm{pre}}\right)+\frac{{N}_{c}+{N}_{\mathrm{pre}}}{2}-1\right\}$ represent a periodic extension of the IFFT (Fig. 21), and are known as the

_{pre}*cyclic prefix*. The integers

*N*and

_{c}*N*are thus the number of sub-carriers and the prefix length, respectively. We can generate Eq. (55) using the structure shown in Fig. 20(b). The inputs to the transmitter are the symbols modulating each sub-carrier, applied at a rate of 1/

_{pre}*T*. The IFFT and cyclic prefix computes the time-domain samples, and is followed by a parallel-to-serial (P/S) converter, digital-to-analog converter and Mach-Zehnder (MZ) modulator. The OFDM receiver performs the reverse operations. It can be shown that the receivers in Fig. 20(a) and (b) are equivalent when

_{ofdm}*p*(

*t*)=

*rect*(

*t*/

*N*).

_{c}T_{c}Assuming the transmitted symbols are independent: *E*⌊*x*
_{l,k}
*x*
^{*}
_{n,m}⌋=0 for *l*≠*n* or *k*≠*m*, an OFDM signal has a spectrum of:

where *P _{n}*=

*E*[|

*x*

_{n,m}|

^{2}] is the average energy per symbol on sub-carrier

*n*. A plot of Eq. (57) is shown in Fig. 22, where we assumed equal energy for all the sub-channels. The modulated sub-carriers are overlapping in frequency. The receiver is able to separate them by the Fast Fourier Transform (FFT) because 1/

*f*is an integer multiple of the chip period

_{d}*T*, i.e., the sub-carriers are “orthogonal” over the duration of an OFDM symbol.

_{c}In OFDM, oversampling can be implemented by not modulating the edge sub-carriers in Fig. 20. Supposing only *N _{u}* of the

*N*sub-carriers are used, the effective oversampling rate is

_{c}*κ*

_{2}=

*N*/

_{c}*N*. Compared to single-carrier transmission, oversampling can be easier to implement in OFDM, as it is possible to employ arbitrary ratios where a corresponding multi-rate solution for single-carrier modulation is prohibitively complex. Another advantage of OFDM is that the signal spectrum (Fig. 22) falls off more rapidly at the band edges than for a single-carrier signal. This is because the spectrum at each sub-carrier is sinc

_{u}^{2}(

*ωT*/2

_{ofdm}*π*), as opposed to sinc

^{2}(

*ωT*/2

_{s}*π*) for a single carrier. The (

*N*+

_{c}*N*)-times steeper roll-off enables tighter optical filtering, and a lower over sampling rate.

_{pre}#### 6.2 Compensation of linear impairments and computational complexity

The cyclic prefix serves a special purpose in OFDM. Suppose we make *N _{pre}* large enough that

*N*is longer than the channel impulse response duration. The received OFDM symbol (after prefix removal) is a

_{pre}T_{c}*circular convolution*between the transmitted OFDM symbol and the sampled impulse response of the channel. In the frequency domain, this corresponds to flat fading over each sub-channel, allowing a single-tap equalizer (complex multiplication) to compensate any amplitude and phase distortion at each sub-channel.

If the prefix length is insufficiently long, the circular convolution assumption no longer holds, and ISI and inter-carrier interference result [102]. It can be shown that the variance of ISI+ICI distortion at sub-carrier *k* is approximately [103]:

where
${H}_{+}[s;N;k]=\sum _{n=0}^{N-1}h\left[s+n\right]{e}^{-\frac{j2\mathrm{\pi n}k}{N}}$
and
${H}_{-}[s;N;k]=\sum _{n=0}^{N-1}h\left[-s-n\right]{e}^{-\frac{j2\pi \left(N-1-n\right)k}{N}}$
are *N*-point FFTs of the tails of the channel impulse response *h*[*n*], which cause leakage of energy between OFDM symbols. *σ*
^{2}
* _{x}*=

*E*[|

*x*|

^{2}] is the mean energy per sample in the time domain. Eq. (58) is exact when the transmitted signal samples

*x*[

*n*] are i.i.d., which requires no oversampling, no cyclic prefix and equal power on all sub-channels. In practice, Eq. (58) is a good approximation when ${\sum}_{\mid n\mid >\frac{{N}_{\phantom{\rule{.2em}{0ex}}\mathrm{pre}}}{2}+{N}_{c}}{\mid h\left[n\right]\mid}^{2}$ is negligible and the oversampling rate is low.

Dual-polarization OFDM is implemented by operating two OFDM transmitters and receivers in parallel, as shown in Fig. 23 [104]. The single-tap equalizers become 2×2 matrices. In order to compensate PMD alone, the cyclic prefix must be longer than the PMD-induced delay spread (minus one chip). However, this is usually much shorter than the cyclic prefix required to compensate CD.

The number of sub-carriers required is determined by the channel’s impulse response in conjunction with the target excess bandwidth factor (*κ*
_{1}) and oversampling ratio (*κ*
_{2}). Suppose a bit rate of *R _{b}* is required and all sub-carriers (with polarization multiplexing considered) encode

*b*bits per symbol. The effective symbol rate is

*R*=

_{s}*R*/

_{b}*b*, which gives an initial estimate of the chip rate as 1/

*T*

_{c}^{(1)}=

*R*(1+

_{s}*κ*

^{(0)}

_{1})

*κ*

^{(0)}

_{2}, for which we can compute the impulse response ${\mathbf{h}}^{\left(i\right)}\left[n\right]\stackrel{\Delta}{=}\mathbf{h}\left({\mathrm{nT}}_{c}^{\left(i\right)}\right)$ . Suppose we require a fraction

*κ*

_{3}of the impulse energy be contained within the prefix:

Eq. (59) determines *N*
^{(i)}
_{pre}. To ensure the FFT size is an integer power of two, we set:

where ⌈·⌉ denotes the nearest integer greater than or equal to. Eqs. (59) and (60) define new values of *κ*
^{(i)}
_{1}=*N*
^{(i)}
* _{pre}*/

*N*

^{(i)}

_{c}and

*κ*

^{(i)}

_{2}=

*N*

^{(i)}

_{c}/

*N*

^{(i)}

_{u}from which we can compute the new chip rate 1/

*T*

_{c}^{(i+1)}=

*R*(1+

_{s}*κ*

_{1}

^{(i)})

*κ*

_{2}

^{(i)}. We iterate this process until stable parameter values are found. Table 9 shows the number of sub-carriers required to compensate CD for various propagation distances, assuming 100 Gbit/s polarization-multiplexed 4-QAM transmission through SMF with 2% CD under-compensation. As in the single-carrier case described in Section 5.1, CD dominates PMD in dictating system complexity. The complexity of OFDM is

*R*

_{s}κ_{2}log

_{2}

*N*complex multiplications per second at the transmitter and

_{c}*R*(

_{s}*κ*

_{2}log

_{2}

*N*+4) complex multiplications at the receiver. It can be shown that the asymptotic complexity of OFDM grows as

_{c}*R*log

_{s}_{2}

*R*, and is the same as single-carrier transmission. If adaptive equalization is required however, OFDM has a simpler update algorithm for the equalizer coefficients, as the filter taps and transmitted symbols are both in the frequency domain. In an FFT-based equalizer for single-carrier modulation, the filter taps are in the frequency domain, while the symbols are in the time domain. The update equation thus requires computing an FFT. In order to keep the overhead manageable, we expect the coefficient can only be updated infrequently for single-carrier.

_{s}#### 6.3 Dynamic power allocation

For a frequency-selective channel, the optimum signal spectrum is given by water filling [105]. OFDM allows greater control of the signal spectrum as the power of each sub-carrier can be varied. In the limit of large *N _{u}*, the OFDM spectrum can approach continuous water filling. Sub-carriers transmitted at higher power can use denser constellations, such that similar BERs are obtained in all sub-carriers. OFDM with dynamic power allocation has been successfully employed in wireless systems to combat signal multi-path [106–108].

Frequency-selective channels are less fundamental to optical fiber than to wireless systems, because a fiber’s main linear impairments, CD and PMD, are unitary transformations in a dual-polarization coherent receiver. In particular, PMD merely shifts energy from one polarization to another. Nevertheless, the ability to optimize power allocation may be useful when transmitting through cascaded optical elements, such as (de)interleavers, (de)multiplexers and ROADMs. In reconfigurable networks, OFDM would enable optimal power allocation across the sub-carriers based on instantaneous channel conditions.

#### 6.4 Laser Phase Noise and Nonlinear Effects

Laser phase noise destroys the orthogonality of the sub-carriers and causes inter-channel interference (ICI), which has noise-like characteristics. Wu and Bar-Ness showed that the variance of ICI grows linearly with *N _{c}* [109]. Thus, the parameter that governs performance is the linewidth-to-sub-carrier-spacing ratio (Δ

*νT*/

_{c}*N*). Phase noise considerations favor a smaller number of sub-carriers. At low phase noise, Armada and Calvo proposed using un-modulated pilot carriers to estimate the common phase error (CPE) for all the sub-channels [110]. At higher phase noise, the carrier phase can no longer be assumed as constant over an OFDM symbol. Wu and Bar-Ness found that using an MMSE equalizer that took into account both ICI and AWGN improves system performance [111]. However, this technique does not estimate or compensate for the phase noise process, unlike FF carrier synchronization for single-carrier modulation.

_{c}Carrier synchronization in OFDM may require an iterative algorithm. Suppose at the beginning of an OFDM symbol, an initial estimate of phase *$\tilde{\varphi}$ _{k}* is known from the previous symbol. We can first de-rotate the entire OFDM symbol (

*N*chips) by exp(-

_{c}*j$\tilde{\varphi}$*), and then perform the FFT, equalization and symbol detection. We can multiply the symbol decisions by the channel’s frequency response and then take the IFFT to compute what the time samples should have been without phase noise. This allows the receiver to compute estimates

_{k}*ψ*of the carrier phase for each chip period. MMSE filtering can then be employed to find more reliable phase estimates

_{k,m}*$\tilde{\varphi}$*from

_{k,m}*ψ*. We can de-rotate the OFDM symbol again, and a second iteration follows. This process can be repeated until convergence is achieved. The performance of such an algorithm has not been characterized.

_{k,m}OFDM is potentially susceptible to fiber nonlinearity due to its high peak-to-average power ratio (PAPR) [112]. Electrical pre- and post-compensation of nonlinearity for long-haul OFDM transmission has been studied in [113,114], while the effects of FWM have been investigated by [115].

## 7. Implementation complexity: Comparison of single- and multi-carrier approaches

Digital coherent detection is, at present, limited mainly by available hardware. Optical systems transmit at symbol rates of tens of GHz, which requires very fast ADCs and powerful DSPs that can achieve the implementation complexity required. Parallelization and pipelining can be aided by block-based DSP algorithms. FFT-based implementations of both the single-carrier and OFDM systems would facilitate this. Block-based digital FF carrier recovery has been studied in [116].

An important design parameter is the number of bits required for the ADCs, and this is determined by the tolerable clipping probability and quantization noise variance [27]. Suppose we require quantization noise to be at most a fraction *k*
_{1} of the AWGN variance per quadrature (*σ*
^{2}
* _{n}*) at the baseband outputs of Fig. 5. We need Δ

^{2}/12<

*k*

_{1}

*σ*

^{2}

*, where Δ is the quantization step size. To determine the clipping probability, a pdf of the received signal is required. Suppose we assume single-carrier transmission in the regime of high uncompensated CD, or assume OFDM transmission. Then we can model the received signal as Gaussian-distributed with mean power*

_{n}*P*per quadrature. The SNR per symbol is then

_{d}*γ*=

_{s}*P*/

_{d}*σ*

^{2}

*. Setting the clipping probability to be*

_{n}*α*, the ADC needs to swing over $\pm {k}_{2}\sqrt{{P}_{d}}$ , where

*erfc*(

*k*

_{2}/√2)=

*α*. The number of quantization levels required is then:

Table 10 shows the required ADC resolution for 4-, 8- and 16-QAM assuming the receiver operates at 2 dB above sensitivity limit for a target BER of 10^{-3}. We set the clipping probability to be 10^{-4}, and the quantization noise variance to be 0.1 times the AWGN variance(*k*
_{1}=0.1, *k*
_{2}=3.89).

Finally, we consider computational complexity. Using the results of Tables 7 and 9, we compute the number of operations required for linear equalization of in single-carrier transmission and in OFDM, assuming polarization-multiplexed 4-QAM (Table 11). For single-carrier transmission, we calculated the complexity of direct implementation and FFT-based implementation using an optimized block size *B* (subject to *N*+*B*-1 being an integer power of two). We observe negligible difference between single-carrier transmission and OFDM. In particular, for distances greater than 1,000 km, single-carrier equalization is more efficiently implemented by the FFT.

## 8. Conclusions

We have reviewed the principles of coherent detection in optical communications, and described digital techniques for compensating channel impairments.

Two types of channel impairments were identified: linear and nonlinear.

Linear channel impairments include CD, PMD and optical filtering by network components. When the outputs of a dual-polarization downconverter are sampled above the Nyquist rate, CD, PMD and sampling time errors are unitary transformations that can be compensated by linear equalizers with negligible power penalty.

Nonlinear impairment compensation relies on exploiting known correlation structures for nonlinear phase processes. To combat NLPN, the receiver exploits the correlation between received amplitude and nonlinear phase shift. And to combat IFWM, the receiver uses the correlation of IFWM phase shift over neighboring symbols. In similar fashion, to combat laser phase noise, a FF carrier synchronizer relies on the temporal correlation of a Wiener process. In all three cases, the receiver performs ML phase correction.

Digital signal processing enables multi-carrier transmission using OFDM. The ability to dynamically manipulate the signal spectrum, the ease of implementing oversampling and the ease of adaptive equalization are key advantages of OFDM.

## Acknowledgments

Previously unpublished work was sponsored by Naval Research Laboratory award N00173-06-1-G035 and by a Stanford Graduate Fellowship. Previously published work was sponsored by Army Research Laboratory contract W911-QX-O6-C-0101 via a subcontract from CeLight, Inc. and by National Science Foundation Grant ECS-0335013. The views expressed in this document are those of the authors, and should not be interpreted as necessarily representing those of the research sponsors.

## References and links

**1. **C. E. Shannon, “A mathematical theory of communication,” Bell. Syst. Tech. J. **27**, 379–423 (1948).

**2. **J. M. Geist, “Capacity and cutoff rate for dense *M*-ary PSK constellations,” in *MILCOM 1990*, (Monterey, CA, USA, 1990), pp. 768–770.

**3. **K.-P. Ho, “Exact evaluation of the capacity for intensity-modulated direct-detection channels with optical amplifier noises,” IEEE Photon. Technol. Lett. **17**, 858–860 (2005). [CrossRef]

**4. **P.P. Mitra and J.B. Stark, “Nonlinear limits to the information capacity of optical fiber communications,” Nature **411**, 1027–1030 (2001). [CrossRef] [PubMed]

**5. **J. M. Kahn and K.-P. Ho, “Spectral efficiency limits and modulation/detectiontechniques for DWDMSystems,” J. Sel. Top. Quantum Electron. **10**, 259–271 (2004). [CrossRef]

**6. **V. Jungnickel, A. Forck, T. Haustein, S. Schiffermuller, C. Helmolt, F. Luhn, M. Pollock, C. Juchems, M. Lampe, S. Eichnger, W. Zirwas, and E. Schulz, “1 Gbit/s MIMO-OFDM transmission experiments,” in *Proceedings of IEEE Conference on Vehicular Technol.* (Institute of Electrical and Electronics Engineers, Dallas, 2005), pp. 861–866.

**7. **K. Kikuchi, “Coherent detection of phase-shift keying signals using digital carrier-phase estimation,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, Anaheim, 2006), Paper OTuI4.

**8. **T. Pfau, S. Hoffmann, R. Peveling, S. Bhandard, S. Ibrahim, O. Adamczyk, M. Porrmann, R. Noé, and Y. Achiam, “First real-time data recovery for synchronous QPSK transmission with standard DFB lasers,” IEEE Photon. Technol. Lett. **18**, 1907–1909 (2006). [CrossRef]

**9. **A. Leven, N. Kaneda, U.-V. Koc, and Y.-K. Chen, “Coherent receivers for practical optical communication systems,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, Anaheim, 2007), Paper OThK4.

**10. **S. J. Savory, G. Gavioli, R. I. Killey, and P. Bayvel, “Transmission of 42.8 Gbit/s polarization multiplexed NRZ-QPSK over 6400 km of standard fiber with no optical dispersion compensation,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and ElectronicsEngineers, Anaheim, 2007), Paper OTuA1.

**11. **K. Sekine, N. Kikuchi, S. Sasaki, S. Hayase, C. Hasegawa, and T. Sugawara, “40 Gbit/s, 16-ary (4 bit/symbol) optical modulation/demodulation scheme,” Electron. Lett. **41**, 430–432, (2005). [CrossRef]

**12. **J. Hongo, K. Kasai, M. Yoshida, and M. Nakazawa, “1-Gsymbol/s 64-QAM coherent optical transmission over 150 km,” IEEE Photon. Technol. Lett. **19**, 638–640 (2007). [CrossRef]

**13. **T. Foggi, E. Forestieri, G. Colavolpe, and G. Prati, “Maximum-likelihood sequence detection with closed-form metrics in OOK optical systems impaired by GVD and PMD,” J. Lightwave Technol. **24**, 3073–3087 (2006). [CrossRef]

**14. **M. Nazarathy and E. Simony, “Multichip differential phase encoded optical transmission,” IEEE Photon. Technol. Lett. **17**, 1133–1135 (2005). [CrossRef]

**15. **D. Divsalar and M. Simon, “Multiple-symbol differential detection of MPSK,” IEEE Trans. Commun. **38**, 300–308 (1990). [CrossRef]

**16. **S. Benedetto and P. Poggiolini, “Theory of polarization shift keying modulation,” IEEE Trans. Commun. **40**, 708–721 (1992). [CrossRef]

**17. **S. Betti, F. Curti, G. de Marchis, and E. Iannone, “Multilevel coherent optical system based on Stokes parameters odulation,” J. Lightwave Technol. **8**, 1127–1136 (1990). [CrossRef]

**18. **E. Ip and J. M. Kahn, “Feedforward carrier recovery for coherent optical communications,” J. Lightwave Technol. , **25**, 2675–2692 (2007). [CrossRef]

**19. **E. Ip and J.M. Kahn, “Digital equalization of chromatic dispersion and polarization mode dispersion,” J. Lightwave Technol. **25**, 2033–2043 (2007). [CrossRef]

**20. **E. Ip and J.M. Kahn, “Carrier synchronization for 3- and 4-bit-per-symbol optical transmission,” J. Lightwave Technol. **23**, 4110–4124 (2005). [CrossRef]

**21. **S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent demodulation of optical multilevel phase-shift-keying signals using homodyne detection and digital signal processing,” IEEE Photon. Technol. Lett. **18**, 1131–1133 (2006). [CrossRef]

**22. **G. P. Agrawal, *Fiber-Optic Communiation Systems*, 3rd ed. (Wiley, New York, 2002). [CrossRef]

**23. **J. R. Barry and J.M. Kahn, “Carrier synchronization for homodyne and heterodyne detection of optical quadriphase-shift keying,” J. Lightwave Technol. **10**, 1939–1951 (1992). [CrossRef]

**24. **R. Noé, “Phase noise-tolerant synchronous QPSK/BPSK baseband-type intradyne receiver concept with feedforward carrier recovery,” J. Lightwave Technol. **23**, 802–808 (2005). [CrossRef]

**25. **J. Rebola and A. Cartaxo, “Optimization of level spacing in quaternary optical communication systems,” Proc. SPIE **4087**, 49–59 (2000). [CrossRef]

**26. **J. J. Bussgang and M. Leiter, “Error rate approximations for differential phase-shift keying.” IEEE Trans. Commun. Systems **12**, 18–27 (1964). [CrossRef]

**27. **J. G. Proakis, “Probabilities of error for adaptive reception of M-phase signals,” IEEE Trans. Commun. Tech. **16**, 71–81 (1968). [CrossRef]

**28. **S. Benedetto and P. Poggiolini, “Multilevel polarization shift keying: optimum receiver structure and performance evaluation,” IEEE Trans. Commun. **42**, 1174–1186 (1994). [CrossRef]

**29. **J. G. Proakis, *Digital Communications*, 4th ed. (McGraw-Hill, New York, 2001).

**30. **M. Suzuki and N. Edagawa, “Dispersion-managed high-capacity ultra-long-haul transmission,” J. Lightwave Technol. **21**, 916–929 (2003). [CrossRef]

**31. **E. Forestieri and G. Prati, “Exact analytical evaluation of second-order PMD impact on the outage probability for a compensated system,” J. Lightwave Technol. **22**, 988–996 (2004). [CrossRef]

**32. **C. D. Poole, R. W. Tkach, A. R. Chraplyvy, and D. A Fishman, “Fading in lightwave systems due to polarization-mode dispersion,” IEEE Photon. Technol. Lett. **3**, 68–70 (1991). [CrossRef]

**33. **H. Bülow, W. Baumert, H. Schmuck, F. Mohr, T. Schulz, F. Küppers, and W. Weiershausen, “Measurement of the maximum speed of PMD fluctuation in installed field fiber,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, San Diego, 1999), Paper OWE4.

**34. **C. D. Poole, “Statistical treatment of polarization dispersion in single-mode fiber,” Opt. Lett. **13**, 687–689 (1988). [CrossRef] [PubMed]

**35. **N. Gisin, J.-P. Von der Weid, and J.-P. Pellaux, “Polarization mode dispersion of short and long single-mode fibers,” J. Lightwave Technol. **9**, 821–827 (1991). [CrossRef]

**36. **G. J. Foschini and C. D. Poole, “Statistical theory of polarization dispersion in single-mode fibers,” J. Lightwave Technol. **9**, 1439–1456 (1991). [CrossRef]

**37. **H. Bülow, “System outage probability due to first- and second-order PMD,” IEEE Photon. Technol. Lett. **10**, 696–698 (1998). [CrossRef]

**38. **H. Sunnerud, C. Xie, M. Karlsson, R. Samuelsson, and P. Andrekson, “A comparison between different PMD compensation techniques,” J. Lightwave Technol. **20**, 368–378 (2002). [CrossRef]

**39. **F. Buchali and H. Bülow, “Adaptive PMD compensation by electrical and optical techniques,” J. Lightwave Technol. **22**, 1116–1126 (2004). [CrossRef]

**40. **R. Noé, D. Sandel, M. Yoshida-Dierolf, S. Hinz, V. Mirvoda, A. Schöpflin, C. Flingener, E. Gottwald, C. Scheerer, G. Fischer, T. Weyrauch, and W. Haase, “Polarization mode dispersion compensation at 10, 20, and 40 Gb/s with various optical equalizer,” J. Lightwave Technol. **17**, 1602–1616 (1999). [CrossRef]

**41. **S. Lee, R. Khosravani, J. Peng, V. Grubsky, D. S. Starodubov, A. E. Willner, and J. Feinberg, “Adjustable compensation of polarization mode dispersion using a high-birefringence nonlinearly chirped fiber Bragg grating,” IEEE Photon. Technol. Lett. **11**, 1277–1279 (1999). [CrossRef]

**42. **T. Saida, K. Takiguchi, S. Kuwahara, Y. Kisaka, Y. Miyamoto, Y. Hashizume, T. Shibata, and K. Okamoto, “Planar lightwave circuit polarization-mode dispersion compensator,” IEEE Photon. Technol. Lett. **14**, 507–509 (2002). [CrossRef]

**43. **J. Wang and J. M. Kahn, “Performance of electrical equalizers in optical amplified OOK and DPSK systems,” IEEE Photon. Technol. Lett. **16**, 1397–1399 (2004). [CrossRef]

**44. **C. Vinegoni, M. Karlsson, M. Petersson, and H. Sunnerud, “The statistics of polarization-dependent loss in a recirculating loop,” J. Lightwave Technol. **22**, 968–976 (2004). [CrossRef]

**45. **A. H. Gnauck, P. J. Winzer, and S. Chandrasekhar, “Hybrid 10/40-G transmission on a 50-GHz Grid through 2800 km of SSMF and seven optical add-drops,” IEEE Photon. Technol. Lett. **17**, 2203–2205 (2005). [CrossRef]

**46. **G. Goldfarb and G. Li, “Chromatic dispersion compensation using digital IIR filtering with coherent detection,” IEEE Photon. Technol. Lett. **19**, 969–971 (2007). [CrossRef]

**47. **N. Amitay and J. Salz, “Linear Equalization Theory in Digital Data Transmission over Dually Polarized Fading Radio Channels,” Bell. Syst. Tech. J. **63**, 2215–2259 (1984).

**48. **J. Salz, “Digital transmission over cross-coupled linear channels,” AT&T Tech. J. **64**, 1147–1159 (1985).

**49. **H. Meyr, M. Moeneclaey, and S. Fechtel, *Digital Communication Receivers*. (John Wiley, New York, 1997).

**50. **R.D. Gitlin and S. B. Weinstein, “Fractionally spaced equalization: an improved digital transversal equalizer,” Bell. Syst. Tech. J. **60**, 275–296 (1981).

**51. **G. Ungerboeck, “Fractional tap-spacing equalizer and consequences for clock recovery in data modems,” IEEE Trans. Commun. **24**, 856–864 (1976). [CrossRef]

**52. **S. Qureshi, “Adaptive equalization,” Proceedings of the IEEE **73**, 1349–1387 (1985). [CrossRef]

**53. **B. Widrow and S. D. Stearns, *Adaptive Signal Processing*, (Prentice Hall, Englewood Cliffs, NJ, 1985).

**54. **A. Oppenheim and R. Schafer, *Discrete-Time Signal Processing*, (Prentice Hall, Englewood Cliffs, NJ, 1989).

**55. **A. Mecozzi, C. B. Clausen, and M. Shtaif, “Analysis of intra-channel nonlinear effects in highly dispersed optical pulse transmission,” IEEE Photon. Technol. Lett. **12**, 392–394 (2000). [CrossRef]

**56. **I. Shake, H. Takara, K. Mori, S. Kawanishi, and Y. Yamabayashi, “Influence of inter-bit four-wave mixing in optical TDM transmission,” Electron. Lett. **34**, 1600–1601 (1998). [CrossRef]

**57. **R.-J. Essiambre, B. Mikkelsen, and G. Raybon, “Intra-channel cross-phase modulation and four-wave mixing in high-speed TDM systems,” Electron. Lett. **35**, 1576–1578 (1999). [CrossRef]

**58. **A. Mecozzi, C. B. Clausen, M. Shtaif, S.-G. Park, and A. H. Gnauck, “Cancellation of timing and amplitude jitter in symmetric links using highly dispersed pulses,” IEEE Photon. Technol. Lett. **13**, 445–447 (2001). [CrossRef]

**59. **A. Striegler and B. Schmauss, “Compensation of intrachannel effects in symmetric dispersion-managed transmission systems,” J. Lightwave Technol. **22**, 1877–1882 (2004). [CrossRef]

**60. **N. Alic and Y. Fainman, “Data-dependent phase coding for suppression of ghost pulses in optical fibers,” IEEE Photon. Technol. Lett. **16**, 1212–1214 (2004). [CrossRef]

**61. **I.B. Djordjevic and B. Vasic, “Constrained coding techniques for suppression of intrachannel nonlinear effects in high-speed optical transmission,” J. Lightwave Technol. **24**, 411–419 (2006). [CrossRef]

**62. **X. Wei and X. Liu, “Analysis of intrachannel four-wave mixing in differential phase-shift keying transmission with large dispersion,” Opt. Lett. **28**, 2300–2302 (2003). [CrossRef] [PubMed]

**63. **A. P. T. Lau, S. Rabbani, and J. M. Kahn are preparing a manuscript to be called “On the statistics of intra-channel four-wave-mixing induced phase noise in phase modulated systems.”

**64. **J. P. Gordon and L. F. Mollenauer, “Phase noise in photonic communications systems using linear amplifiers,” Opt. Lett. **15**, 1351–1353 (1990). [CrossRef] [PubMed]

**65. **K.-P. Ho, *Phase-Modulated Optical Communication Systems*, (Springer, New York, 2005).

**66. **K.-P. Ho, “Statistical properties of nonlinear phase noise,” in *Advances in Optics and Laser Research*3, (Nova Science Publishers, New York, 2003).

**67. **A. P. T. Lau and J. M. Kahn, “Design of inline amplifiers gain and spacing to minimize phase noise in optical transmission systems,” J. Lightwave Technol. **24**, 1334–1341 (2006). [CrossRef]

**68. **A.P.T. Lau and J.M. Kahn, “Power profile optimization in phase-modulated systems in presence of nonlinear phase noise,” IEEE Photon. Technol. Lett. **18**, 2514–2516 (2006). [CrossRef]

**69. **K.-P. Ho and J. M. Kahn, “Detection technique to mitigate Kerr effect phase noise,” J. Lightwave Technol. **22**, 779–783 (2004). [CrossRef]

**70. **D.-S. Ly-Gagnon and K. Kikuchi, “Cancellation of nonlinear phase noise in DPSK transmission,” 2004 Optoelectronics and Communications Conference and International Conference on Optical Internet (OECC/COIN2004), paper 14C3-3 (2004).

**71. **X. Liu, X. Wei, R. E. Slusher, and C. J. McKinstrie, “Improving transmission performance in differential phase-shift-keyed systems by use of lumped nonlinear phase-shift compensation,” Opt. Lett. **27**, 1616–1618 (2002). [CrossRef]

**72. **K. Kikuchi, M. Fukase, and S.-Y. Kim, “Electronic post-compensation for nonlinear phase noise in a 1000-km 20-Gbit/s optical QPSK transmission system using the homodyne receiver with digital signal processing,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, Anaheim, 2007), Paper OTuA2.

**73. **G. Charlet, N. Maaref, J. Renaudier, H. Mardoyan, P. Tran, and S. Bigo, “Transmission of 40 Gb/s QPSK with coherent detection over ultra-long distance improved by nonlinearity mitigation,” in *Proceedings ECOC 2006*, Cannes, France, 2006, Postdeadline paper Th4.3.4.

**74. **G. Zhu, L. Mollenauer, and C. Xu, “Experimental demonstration of post-nonlinearity compensation in a multispan DPSK transmission,” IEEE Photon. Technol. Lett. **18**, 1007–1009 (2006). [CrossRef]

**75. **K.P. Ho, “Mid-span compensation of nonlinear phase noise,” Opt. Comm. **245**, 391–398 (2005). [CrossRef]

**76. **A. P. T. Lau and J. M. Kahn, “Signal design and detection in presence of nonlinear phase noise,” J. Lightwave Technol. **25**, 3008–3016 (2007). [CrossRef]

**77. **A.G. Green, P.P. Mitra, and L.G. L. Wegener, “Effect of chromatic dispersion on nonlinear phase noise,” Opt. Lett. **28**, 2455–2457 (2003). [CrossRef] [PubMed]

**78. **S. Kumar, “Effect of dispersion on nonlinear phase noise in optical transmission systems,” Opt. Lett. **30**, 3278–3280 (2005). [CrossRef]

**79. **K.-P. Ho and H.C. Wang, “On the effect of dispersion on nonlinear phase noise,” Opt. Lett. **31**, 2109–2111 (2006). [CrossRef] [PubMed]

**80. **S. Kumar and L. Liu, “Reduction of nonlinear phase noise using optical phase conjugation in quasi-linear optical transmission systems,” Opt. Express **15**, 2166–2177 (2007). [CrossRef] [PubMed]

**81. **D. Boivin, G.-K. Chang, J. R. Barry, and M. Hanna, “Reduction of Gordon-Mollenauer phase noise in dispersion-managed systems using in-line spectral inversion,” J. Opt. Soc. Am. A. B **23**, 2019–2023 (2006). [CrossRef]

**82. **P. Serena, A. Orlandini, and A. Bononi, “Parametric-Gain approach to the analysis of single-channel DPSK/DQPSK systems with nonlinear phase noise,” J. Lightwave Technol. **24**, 2026–2037 (2006). [CrossRef]

**83. **K.P. Ho and H.C. Wang, “Comparison of nonlinear phase noise and intrachannel four-wave mixing for RZ-DPSK signals in dispersive transmission systems,” IEEE Photon. Technol. Lett. **17**, 1426–1428 (2005). [CrossRef]

**84. **F. Zhang, C. A. Bunge, and K. Petermann, “Analysis of nonlinear phase noise in single-channel return-to-zero differential phase-shift keying transmission systems,” Opt. Lett. **31**, 1038–1040 (2006). [CrossRef] [PubMed]

**85. **F. Zhang, C. A. Bunge, K. Petermann, and A. Richter, “Optimum dispersion mapping of single-channel 40 Gbit/s return-to-zero differential phase-shift keying transmission systems,” Optics Express **14**, 6613–6618 (2006). [CrossRef] [PubMed]

**86. **X. Zhu, S. Kumar, and X. Li, “Analysis and comparison of impairments in differential phase-shift keying and on-off keying transmission systems based on the error probability,” Appl. Opt. **45**, 6812–6822 (2006). [CrossRef] [PubMed]

**87. **C. Henry, “Theory of the phase noise and power spectrum of a single mode injection laser,” J. Quantum Electron. **19**, 1391–1397 (1983). [CrossRef]

**88. **M. Tur, B. Moslehi, and J. W. Goodman, “Theory of laser phase noise in recirculating fiber-optic delay lines,” J. Lightwave Technol. **3**, 20–31 (1985). [CrossRef]

**89. **A.L. Schawlow and C. H. Townes, “Infrared and optical masers,” Phys. Rev. **112**, 1940–1949, (1958). [CrossRef]

**90. **F. M. Gardner, *Phaselock Techniques*, 3rd ed. (John Wiley, Hoboken, NJ, 2005). [CrossRef]

**91. **M. A. Grant, W. C. Michie, and M. J. Fletcher, “The performance of optical phase-locked loops in the presence of nonnegligible loop propagation delay,” J. Lightwave Technol. **5**, 592–597 (1987). [CrossRef]

**92. **K. Kikuchi, “Phase-diversity homodyne detection of multilevel optical modulation with digital carrier phase estimation,” J. Sel. Top. Quantum Electron. **12**, 563–570 (2006). [CrossRef]

**93. **M. G. Taylor, “Accurate digital phase estimation process for coherent detection using a parallel digital processor,” in *Proceedings ECOC 2005*, Glasgow, UK, 2005, Paper Tu4.2.6.

**94. **D.-S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent detection of optical quadrature phase-shift keying signals with coherent phase estimation,” J. Lightwave Technol. **24**, 12–21, (2006). [CrossRef]

**95. **F. J. Foschini, R. D. Gitlin, and S. B. Weinstein, “On the selction of a two-dimensional signal constellation in the presence of phase jitter and Gaussian noise,” Bell. Syst. Tech. J. **52**, 927–965 (1973).

**96. **A. Bahai, B. Saltzberg, and M. Ergen, *Multi-carrier Digital Communications: Theory and Applications of OFDM*, 2nd Ed. (Springer, New York, 2004).

**97. **R. Prasad, “OFDM for wireless communications systems,” (Artech House Publishers, Boston, 2004).

**98. **W. Shieh, X. Yi, and Y. Tang, “Experimental demonstration of transmission of coherent optical OFDM Systems,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, Anaheim, 2007), Paper OMP2.

**99. **W. Shieh and C. Athaudage, “Coherent optical orthogonal frequency division multiplexing,” Electron. Lett. **42**, 587–589 (2006). [CrossRef]

**100. **N. Cvijetic, L. Xu, and T. Wang, “Adaptive PMD compensation using OFDM in long-haul 10 Gb/s DWDM systems,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, Anaheim, 2007), Paper OTuA5.

**101. **A. Lowery and J. Armstrong, “Orthogonal-frequency-division multiplexing for optical dispersion compensation,” in *Proceedings of IEEE Conference on Optical Fiber Communications*, (Institute of Electrical and Electronics Engineers, Anaheim, 2007), Paper OTuA4.

**102. **W. Henkel, G. Taubock, P. Odling, P. O. Borjesson, and N. Petersson, “The cyclic prefix of OFDM/DMT-an analysis,” *IEEE International Seminar on Broadband Communications*, (Institute of Electrical and Electronic Engineers, Zurich, 2002).

**103. **D. J.F. Barros and J. M. Kahn are preparing a manuscript to be called “Optimized dispersion compensation using orthogonal frequency-division multiplexing.”

**104. **W. Shieh, X. Yi, Y. Ma, and Y. Tang, “Theoretical and experimental study on PMD-supported transmission using polarization diversity in coherent optical OFDM systems,” Optics Express **15**, 9936–9947 (2007). [CrossRef] [PubMed]

**105. **C. Cover and J. Thomas, *Elements of Information Theory*. (John Wiley, New York, 1991). [CrossRef]

**106. **C. Y. Wong, R. S. Cheng, K. B. Letaief, and R. D. Murch, “Multiuser OFDM with adaptive subcarrier, bit, and power allocation,” J. Sel. Top. Commun. **17**, 1747–1758 (1999). [CrossRef]

**107. **B. S. Krongold, K. Ramchandran, and D. L. Jones, “Computationally efficient optimal power allocation algorithms for multicarrier communication systems,” IEEE Trans. Commun. **48**, 23–27 (2000). [CrossRef]

**108. **J. Jang, K. B. Lee, and Y.-H. Lee, “Transmit power and bit allocations for OFDM systems in a fading channel,” in *Proceedings of IEEE GLOBECOM*, (Institute of Electrical and Electronics Engineers, San Francisco, 2003), pp. 858–862.

**109. **S. Wu and Y. Bar-Ness, “OFDM systems in the presence of phase noise: consequences and solutions,” IEEE Trans. Commun. **52**, 1988–1996 (2004). [CrossRef]

**110. **A. G. Armada and M. Calvo, “Phase noise and sub-carrier spacing effects on the performance of an OFDM communication system,” IEEE Commun. Lett. **2**, 11–13 (1998). [CrossRef]

**111. **S. Wu and Y. Bar-Ness, “A phase noise suppression algorithm for OFDM based WLANs,” IEEE Commun. Lett. **6**, 535–537 (2002). [CrossRef]

**112. **H. Ochiai and H. Imai, “On the distribution of the peak-to-average power ratio in OFDM signals,” IEEE Trans. Commun. **49**, 282–289 (2001). [CrossRef]

**113. **A. J. Lowery, “Fiber nonlinearity mitigation in optical links that use OFDM for dispersion compensation,” IEEE Photon Technol. Lett. **19**, 1556–1558 (2007). [CrossRef]

**114. **A. J. Lowery, “Fiber nonlinearity pre- and post-compensation for long-haul optical links using OFDM,” Optics Express **15**, 12965–12970 (2007). [CrossRef] [PubMed]

**115. **A. J. Lowery, S. Wang, and M. Premaratne, “Calculation of power limit due to fiber nonlinearity in optical OFDM systems,” Optics Express **15**, 13282–13287 (2007). [CrossRef] [PubMed]

**116. **D.-S. Ly-Gagnon, “Information recovery using coherent detection and digital signal pocessing for phase-shift-keying modulation formats in optical communication systems,” M.S. Thesis, University of Tokyo (2004).