## Abstract

The probability density function and impact of equalization-enhanced phase noise (EEPN) is analytically investigated and simulated for 100 Gb/s coherent systems using electronic dispersion compensation. EEPN impairment induces both phase noise and amplitude noise with the former twice as much as the latter. The effects of transmitter phase noise on EEPN are negligible for links with residual dispersion in excess of 700 ps/nm. Optimal linear equalizer in the presence of EEPN is derived but show only marginal performance improvement, indicating that EEPN is difficult to mitigate using simple DSP techniques. In addition, the effects of EEPN on carrier recovery techniques and corresponding cycle slip probabilities are studied.

© 2010 Optical Society of America

## 1.Introduction

Coherent detection is currently under extensive investigation for long-haul fiber-optic communication systems [1–4]. Together with the advances in high-speed digital signal processing (DSP) techniques, coherent detection is believed to be one of the next major enabling technologies to realize high spectral efficiency transmission at 100Gb/s per channel and beyond. The flexibility and scalability of DSP in coherent systems are studied for compensation of transmission impairments such as chromatic dispersion (CD) [2–5], polarization-mode dispersion (PMD) [6–8], Kerr nonlinearity [5,9–13] and carrier phase recovery [14–16]. For DSP based CD compensation, Savory et al. [17] demonstrated 10 GSym/s transmissions over 6400 km of standard single-mode fiber without inline optical dispersion compensation. More recently, Charlet et al. [18] showed the ability to compensate for 7040 km of uncompensated CD using a 1500 tap equalizer for 100Gb/s transmissions using polarization-multiplexed (PM) - Quadrature Phase-Shift Keying (QPSK) modulation with a symbol rate of 28 GSym/s. However, in spite of the tremendous success and potential promise of DSP in coherent systems, DSP based receivers suffer from their own issues. For instance, Shieh and Ho [19] provided initial assessments on the system impact of equalization-enhanced phase noise (EEPN) that results from the interaction of local oscillator (LO) phase noise with electronic CD compensation. The EEPN is found to scale with accumulated CD, linewidth as well as symbol rates. Consequently, the linewidth requirements for LO become more stringent for systems using electronic CD compensation. Xie [20,21] also provided simulation results evaluating the impact of EEPN. However, there has been no in-depth study undertaken on the nature of equalization-enhanced impairments and their effects on carrier recovery techniques. In addition, previous work on the optimal filter design to equalize CD does not consider the impact of laser phase noise. It would be of interest to know (i) what the optimal filter design is in the presence of EEPN, and (ii) what the potential improvements will be by incorporating such a new optimal filter design.

In this paper, we characterize the probability density function (pdf) and impact of EEPN for 100 Gb/s systems and beyond analytically as well as simulations. EEPN induced optical signal-to-noise ratio (OSNR) penalties obtained from simulations are shown to be non-negligible for long-haul terrestrial links especially for high spectral efficiency formats. EEPN induced phase noise is found to be twice as large as EEPN induced amplitude noise, and the effects of transmitter phase noise on EEPN is negligible for a link with 700 ps/nm residual dispersion or above. The optimal linear minimum mean-squared error (LMMSE) equalizer for the link CD is derived in the presence of LO phase noise but find the system performance is only improved marginally. The effects of EEPN on carrier phase recovery techniques such as the Viterbi-Viterbi algorithm are studied and cycle slip probabilities in presence of EEPN are characterized.

## 2.Equalization-Enhanced Phase Noise

#### A. Theoretical model and system impact

Consider a coherent communication system consisting of multiple spans of propagation without dispersion compensating fibers as shown in Figure 1. In this setup,*x _{n}* denotes independent identically distributed (i.i.d.) information symbols with a symbol rate

*R*of 27.59 GSym/s and normalized average power

*E*[∣

*x*∣

_{n}^{2}]=1. With polarization-multiplexing (PM) and a 7 % Forward Error Correction (FEC) overhead, such symbol rate corresponds to 100 Gb/s and 200 Gb/s per channel transmission using Quadrature Phase-Shift Keying (QPSK) and 16-Quadrature Amplitude Modulation (QAM) modulation formats respectively. Fiber nonlinearity and polarization effects are neglected for simplicity.

The information symbols are first passed through a pulse-shaping filter *b*(*t*) to form the transmitted signal

where *T*
_{0} is the symbol period. Optical amplifiers located along the link introduce amplified spontaneous emission (ASE) noise collectively modeled as *n*(*t*) = *n _{re}*(

*t*) +

*jn*(

_{im}*t*). The noise in each quadrature

*n*(

_{re}*t*) and

*n*(

_{im}*t*) are independent additive white Gaussian noise (AWGN) process with power spectral density

*N*

_{0}/2. In the absence of phase noises from lasers, the transmitted signal is mixed with a local oscillator at the receiver and the output is given by

where *q*(*t*) = *p*(*t*)* *g*(*t*) = *p*(*t*)* *h*(*t*)**b*(*t*) and *ν*(*t*) = *p*(*t*)**n*(*t*).In this formulation, ‘*’ denotes convolution, and *h*(*t*) and *p*(*t*) respectively denote the impulse response characterizing a channel with chromatic dispersion and electrical low pass filter after photo-detection respectively. The received signal is then sampled to form *y _{m}* =

*y*(

*mT*) where

*T*is the sampling period and

*S*=

*T*/

_{0}*T*is the over-sampling rate. To compensate for CD and demodulate the

*k*transmitted symbol

^{th}*x*, a vector of received samples

_{k}with length *N*=*2L*+*1* is formed and passed through a equalizer, or, a linear finite impulse response (FIR) filter, **w**. In absence of laser phase noise, the estimate of the transmitted symbol *x̂ _{k}* is simply given by

According to the minimum mean-squared error (MMSE) criterion, the optimal linear FIR filter lin **w**
_{lin} is derived as

where **A** and *α* are given by [2]

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.2em}{0ex}}+{N}_{0}\underset{-\infty}{\overset{\infty}{\int}}{p}^{*}\left(t-lT\right)p\left(t-mT\right)\mathrm{dt}$$

and

Denoting the estimation error as *ε* = *x̂ _{n}* −

*x*and the phase error as

_{n}*θ*= arg(

*x̂*) −arg(

_{n}*x*), the corresponding mean-squared error becomes

_{n}and the phase noise variance *σ*
^{2}
_{θ} is given by

for large optical signal-to-noise ratio (OSNR).

With laser phase noise *ϕ _{r}*(

*t*), the spectrum of a laser ${e}^{j{\varphi}_{r}\left(t\right)}$ can be modeled as a Lorentzian lineshape with 3-dB linewidth Δ

*υ*in which

Equivalently, the phase noise *ϕ _{r}*(

*t*) can be modeled as a Wiener process in which

*ϕ*(

_{r}*t*

_{2})−

*ϕ*(

_{r}*t*

_{1}) is Gaussian distributed with zero mean and variance

*σ*

^{2}

_{p}= 2

*π*Δ

*υ*∣

*t*

_{1}−

*t*

_{2}∣. In presence of receiver (Rx) or LO phase noise and uncompensated CD, the received signal becomes

Due to the fact that the multiplication and convolution operations are not commutable, introduction of Rx phase noise prior to electronic CD compensation prevents complete equalization of CD and results in additional impairments. Referred to as equalization-enhanced phase noise (EEPN) [20], this effect is otherwise not present if optical dispersion compensation techniques are used. To highlight the system impact of EEPN alone and provide a performance upper bound for all carrier recovery techniques in presence of EEPN, we will assume perfect knowledge of the laser phase noise
${e}^{j{\varphi}_{t}\left(k{T}_{0}\right)}$
and
${e}^{j{\varphi}_{r}\left(k{T}_{0}\right)}$
at the carrier recovery (CR) unit so that the estimate of the transmitted symbol is given by
${\hat{x}}_{k}={\mathbf{w}}^{T}\mathbf{y}\xb7\left({e}^{-j\left({\varphi}_{t}\left(k{T}_{0}\right)+{\varphi}_{r}\left(k{T}_{0}\right)\right)}\right)$
in this section. The effect of EEPN on practical carrier recovery techniques will be studied in later sections. From Appendix A, the pdf of EEPN for a given transmitted symbol *x _{k}* is given by

where **X** = […*x*
_{k−2},*x*
_{k−1},*x*
_{k+1},*x*
_{k+2}…], *C* is the set {1, −1} for BPSK and {1, −1, *j*, −*j*} for QPSK formats etc. Figure 2 (a)–(c) illustrate the pdf of EEPN for various laser linewidths and transmission distance for QPSK systems with electronic dispersion compensation. The EEPN induces more phase noise on *x̂ _{k}* than amplitude noise, resulting in the pdf being elliptical in nature. In presence of ASE noise, the pdf is more circular-like as shown in Figure 2 (d), indicating the relative significance of complex circularly symmetric ASE noise compared with EEPN.

To evaluate the impact of EEPN on 100-Gb/s systems, Figure 3 compares the overall phase noise variance obtained from simulations for a 1200-km non return-to-zero (NRZ)-QPSK system using optical or electronic CD compensation in presence of Rx phase noise. The case without Rx phase noise is also shown as reference. The linewidth of the Rx laser is 3 MHz and the Tx laser is assumed to be ideal with a zero linewidth. The transmission link consists of 15 spans of 80 km standard single-mode fiber (SMF) with dispersion coefficient *D* =17 ps/(nm-km) and fiber attenuation coefficient *α _{fiber}* = 0.25 dB/km. The amplifiers are assumed to be Erbium-doped fiber amplifiers (EDFA) with a noise figure of 4.5 dB. The gain of each EDFA exactly compensates for the loss of the fiber in each span. For the rest of the paper, such configurations will be assumed for each span of propagation unless otherwise stated. The carrier recovery is performed after CD compensation [17]. With CD fully compensated optically, Rx phase noise only induces a very small OSNR penalty and system performance is predominantly ASE noise limited. However, with CD fully compensated electronically, EEPN introduces significant OSNR penalties. To investigate the effect of EEPN for typical long-haul systems, Figure 4 shows the phase noise variance as a function of OSNR for a 3200 km link with an accumulated dispersion of 54400 ps/nm for various LO linewidths. The performance for systems using optical dispersion compensation with various LO linewidths is almost identical to that without Rx phase noise and is therefore not shown. Assuming Gray Coding, the FEC threshold corresponding to a bit error ratio (BER) of 2 × 10

^{−3}is also shown for reference. From the figure, it can be seen that EEPN induces a non-negligible OSNR penalty for typical DFB lasers with linewidths of several MHz. In addition, for any LO linewidth, the overall phase noise variance converges to a value corresponding to the system impact due to EEPN alone. The system tolerance to EEPN decreases dramatically for higher-order modulation formats such as 16-QAM or 64-QAM, as depicted in Figure 5 for a transmission length of 1600 km. In this case, even lasers with sub-MHz linewidth such as external-cavity lasers will induce a non-negligible OSNR penalty due to EEPN.

#### B. EEPN Induced Phase and Amplitude Noise

The interaction between Rx phase noise and electronic CD compensation produces additional phase distortions as well as amplitude distortions. To look further into the nature of equalization-enhanced impairments in general, we studied the variances of the amplitude noise Var(∣*x̂ _{k}*∣) as well as phase noise

*σ*

^{2}

_{θ}induced by EEPN for a QPSK system and simulation results are shown in Figure 6. From the figure, it can be seen that the interaction between Rx phase noise and electronic CD compensation induces more phase noise than amplitude noise even at high OSNR, in contrast to systems corrupted by AWGN alone. For various values of OSNR and uncompensated CD, the EEPN induced phase noise variance is approximately twice the amplitude noise variance, indicating that EEPN induced phase noise rather than amplitude noise is a more dominant transmission impairment for systems using electronic CD compensation.

#### C. EEPN from transmitter laser

In addition to the phase noise variance resulted from the interactions between Rx phase noise and electronic dispersion compensation, the effect of Tx laser phase noise on EEPN is investigated. Figure 7 shows the phase noise variance using **w**
_{lin} for CD compensation for various laser linewidths and propagation distance for a QPSK system. The dispersion coefficient is 17 ps/nm-km. ASE noise from optical amplifiers is not included to highlight the contribution of the overall phase noise by the Tx and Rx lasers. Two scenarios are studied: ideal Tx laser with Rx phase noise only and systems with both Rx and Tx phase noises. From Figure 6, it can be seen that for 10 km of propagation, the presence of Tx phase noise doubles the overall phase noise variance, indicating that Rx and Tx phase noise contributes equally to overall signal degradation after sampling and electronic CD compensation. On the other hand, for more than 40 km of propagation, Tx phase noise does not increase the overall phase noise variance by much. Therefore, even for a propagation distance of 40 km with an accumulated dispersion of about 700 ps/nm, EEPN from the interaction between Rx phase noise and electronic dispersion compensation already dominates system performance. In a WDM system with periodic optical dispersion compensation, residual dispersion in this range can be present for some channels due to dispersion map design for mitigation of fiber nonlinearity induced impairments or dispersion slope mismatch between the propagating fibers and dispersion compensating fibers. Consequently, the Rx laser linewidth requirement will typically be more stringent than Tx laser for WDM links even with periodic dispersion compensation.

## 3. Optimal Equalization in Presence of EEPN

In presence of EEPN, the linear MMSE equalizer **w**
_{lin} previously studied for CD compensation is no longer optimal as the derivation of **w**
_{lin} does not take into account the statistics of the Rx phase noise. In this case, the received signal is given by

Taking into account the statistics of
${e}^{j{\varphi}_{r}\left(t\right)}$
, the optimal linear MMSE estimate can now be expressed as *x̂ _{n}* =

*α*

^{T}

_{PN}

**A**

^{−1}

_{PN}

**y**=

**w**

^{T}

_{PN}

**y**where

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.2em}{0ex}}=\{\iint \sum _{n}{g}^{*}({\tau}_{1}-n{T}_{0})g({\tau}_{2}-n{T}_{0}){p}^{*}\left((Sk+L-l)T-{\tau}_{1}\right)p\left((Sk+L-m)T-{\tau}_{2}\right).$$

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}E\left[{e}^{j\left({\varphi}_{r}\left({\tau}_{2}\right)-{\varphi}_{r}\left({\tau}_{1}\right)\right)}\right]d{\tau}_{1}d{\tau}_{2}\}+{N}_{0}\underset{-\infty}{\overset{\infty}{\int}}{p}^{*}(t-lT)p(t-mT)\mathrm{dt}$$

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.2em}{0ex}}=\{\iint \sum _{n}{g}^{*}({\tau}_{1}-n{T}_{0})g({\tau}_{2}-n{T}_{0}){p}^{*}\left((Sk+L-l)T-{\tau}_{1}\right)p\left((Sk+L-m)T-{\tau}_{2}\right).$$

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.2em}{0ex}}{e}^{-\pi \Delta \upsilon \mid {\tau}_{2}-{\tau}_{1}\mid}d{\tau}_{1}d{\tau}_{2}\}+{N}_{0}\underset{-\infty}{\overset{\infty}{\int}}{p}^{*}(t-lT)p(t-mT)\mathrm{dt}$$

and

Since *p*(*t*) is a low-pass filter that filters out-of-band noise and laser linewidth is at most in the order of MHz for practical optical communication systems, it can be shown (from the Appendix B) that

and thus we can approximate the received signal as

In this case,

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}+{N}_{0}\underset{-\infty}{\overset{\infty}{\int}}{p}^{*}(t-lT)p(t-mT)\mathrm{dt}$$

and

Figure 8 shows the magnitude of the optimal filter tap coefficients **w**
_{PN} with a LO linewidth of 3 MHz and **w**
_{lin} with zero LO linewidth, both for a QPSK system with a transmission distance of 1200 km. One can see that with the received vector **y** corrupted by EEPN, the exponential decay terms in *α*
_{PN}
**, A**
^{−1}
_{PN} that capture the LO phase noise statistics result in samples of **y** further away from the center having bigger ‘weights’. However, as the samples further away from the center are more corrupted by LO phase noise, they became less relevant for the detection of the bit of interest and therefore system performance degrades with EEPN. Note also that in presence of EEPN, the samples with the largest weight are shifted from 98 to 95 samples from the center of the filter.

If we use the filter **w**
_{lin} to compensate CD in presence of EEPN, the mean-squared error will be given by

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.2em}{0ex}}=1-2{\alpha}_{\mathrm{PN}}^{H}{\mathbf{A}}^{\mathbf{-}\mathbf{1}}\alpha +{\alpha}^{H}{\mathbf{A}}^{\mathbf{-}\mathbf{1}}{\mathbf{A}}_{\mathrm{PN}}{\mathbf{A}}^{\mathbf{-}\mathbf{1}}\alpha .$$

With the optimal FIR coefficients **w**
_{PN} for CD compensation in presence of EEPN, the corresponding mean-squared error will be given by

Figure 9 shows the mean-squared error vs. OSNR in presence of EEPN for a QPSK system using different FIR filters. The transmission distance is 1200-km link with a LO linewidth of 3 MHz. The performance without Rx phase noise is also shown as reference. It can be seen that in presence of EEPN, using **w**
_{PN} does not improve the performance by much. The performance with **w**
_{PN,opt} corresponds to the case when the approximation in (16) is not used and *E*[*x _{k}*

**y**

^{*}] and

*E*[

**y**

^{*}

**y**

^{T}] are numerically obtained from simulations to form

**w**

_{PN,opt}= (

*E*[

**y**

^{*}

**y**

^{T}])

^{−1}

*E*[

*x*

_{k}**y**

^{*}]. From Figure 9, we can see that the mean-squared error using

**w**

_{PN,opt}is very close to that using

**w**

_{PN}and therefore we can safely use the approximations in (17) for further analysis. Unfortunately, the optimal FIR filter

**w**

_{PN}only provides marginal performance improvements for systems with EEPN. Consequently, EEPN is difficult to mitigate by using simple DSP algorithms.

## 4. EEPN Impact on Carrier Phase Recovery

Carrier phase recovery using phase-locked loop or other feedback-based techniques used to be common for traditional copper-wire and wireless communication systems. However, with the advent of DSP and the extremely low tolerance of propagation delay in feedback-based systems with high symbol rate, carrier phase recovery using DSP-based feed-forward techniques are currently receiving a lot of attention in coherent systems and therefore the effect of EEPN on such carrier phase recovery techniques becomes an important issue. Feedforward carrier recovery techniques can be implemented either with decision-directed (DD) or non-decision-aided (NDA) approaches. In this paper, we will focus on the effect of EEPN on the Viterbi-Viterbi algorithm for QPSK systems. Figure 10 shows the carrier phase *ϕ*(*t*) = *ϕ _{t}*(

*t*) +

*ϕ*(

_{r}*t*) together with the tracked phase for systems with optical or electrical CD compensation for a 1200-km link. To achieve phase tracking, the Viterbi-Viterbi algorithm is performed on the CD-equalized samples and the phases of the resulting output are passed through a Wiener filter described in Eq. (26) of [14] whose output is the MMSE estimate of the carrier phase. The tracked phase without EEPN is also shown as reference. As shown in Figure 9, it can be seen that in presence of EEPN, the tracked phase will have larger errors compared with systems using optical CD compensation. As the EEPN results in larger errors in carrier phase estimates, it will cause more cycle slips as it is more likely to have neighboring phases differ by 2

*π*/4 =

*π*/2, which is depicted in Figure 11.

Figure 12 shows the probability of cycle slips as a function of OSNR for a 1200-km link obtained from simulations. For low OSNR, cycle slips are mainly caused by ASE noise. However, when the OSNR is high, cycle slips are more contributed by EEPN. With the optimal MMSE equalizer **w**
_{PN} in presence of EEPN, the cycle slip probability is indeed lower compared with the case using **w**
_{lin}. However, the difference maybe too small for practical interest.

## 5. Conclusions

The pdf of EEPN are analytically derived and simulations are conducted to investigate the effect of EEPN on coherent communication systems operating at 100Gb/s and beyond. The effect of EEPN induced phase noise is approximately twice as large as EEPN induced amplitude noise. The contributions to the overall EEPN by transmitter lasers is shown to be negligible for a link with residual dispersion of 700 ps/nm or above. Optimized linear MMSE equalizers in presence of EEPN are derived but only marginal performance improvements are obtained. In addition, the effects of EEPN on cycle slip probabilities of QPSK systems using the Viterbi-Viterbi algorithm for carrier phase recovery are studied. Further investigations on the effect of EEPN on other carrier recovery techniques and experimental characterizations of EEPN in coherent systems will be topics for future research.

## APPENDIX A: Pdf of EEPN

In the absence of ASE noise and Tx laser phase noise, (11) becomes $y\left(t\right)=\left(\left[\sum _{n}{x}_{n}g(t-n{T}_{0})\right]{e}^{j{\varphi}_{r}\left(t\right)}\right)*p\left(t\right)\approx \left(\sum _{n}{x}_{n}q(t-n{T}_{0})\right){e}^{j{\varphi}_{r}\left(t\right)}$

where the validity of the approximation is explained in Appendix B. The received samples are then

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}=\sum _{n}{x}_{n}q(mT-n{T}_{0})+j{\varphi}_{r}\left(mT\right)\sum _{n}{x}_{n}q(mT-n{T}_{0})$$

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}={Q}_{m}+{Q}_{m}j{\varphi}_{r}\left(mT\right).$$

Let Δ_{m−m′} = *ϕ _{r}*(

*mT*) −

*ϕ*(

_{r}*m′T*) and

**w**

_{lin}= [

*w*

_{L}*w*

_{L−1}…

*w*

_{−L+1}

*w*

_{−L}]

^{T}. In this case, the estimation error for the

*k*

^{th}transmitted symbol is given by

$$\phantom{\rule{.2em}{0ex}}={\left[j{Q}_{Sk+L}{\Delta}_{L}\phantom{\rule{.2em}{0ex}}J{Q}_{Sk+L-1}{\Delta}_{L-1}\dots \phantom{\rule{.2em}{0ex}}J{Q}_{Sk-L+1}{\Delta}_{-L+1}\phantom{\rule{.2em}{0ex}}J{Q}_{Sk-L}{\Delta}_{-L}\right]}^{T}{\mathbf{w}}_{\mathrm{lin}}$$

$$\phantom{\rule{.2em}{0ex}}=j{w}_{L}{Q}_{Sk+L}{\Delta}_{L}+j{w}_{L-1}{Q}_{Sk+L-1}{\Delta}_{L-1}+\dots {\mathrm{jw}}_{-L+1}{Q}_{Sk-L+1}{\Delta}_{-L+1}+{\mathrm{jw}}_{-L}{Q}_{Sk-L}{\mathbf{\Delta}}_{-L}$$

$$\phantom{\rule{.2em}{0ex}}=\sum _{i=-L}^{-1}{r}_{i}{\Delta}_{i}+0+\sum _{i=1}^{L}{r}_{i}{\Delta}_{i}$$

$$\phantom{\rule{.2em}{0ex}}=\sum _{i=-L}^{-1}\left[\left({\Delta}_{i}-{\Delta}_{i+1}\right)\sum _{m=-L}^{i}{r}_{m}\right]+\sum _{i=1}^{L}\left[\left({\Delta}_{i}-{\Delta}_{i-1}\right)\sum _{m=i}^{L}{r}_{m}\right]$$

where *r _{i}*=

*jw*, and Δ

_{i}Q_{Sk+i}_{i}− Δ

_{i±1}are independent Gaussian random variables with zero mean and variance 2

*π*Δ

*υT*. For a given

*x*,

_{k}*r*,

_{−L}*r*

_{−L+1}, ⋯,

*r*

_{L−1},

*r*are functions of the neighboring symbols

_{L}**X**= […

*x*

_{k−2},

*x*

_{k−1},

*x*

_{k+1},

*x*

_{k+2}…]. Therefore, for a given neighboring symbol sequence

**X**,

*ε*will be a complex Gaussian random variable with zero mean and covariance matrix

2*π*Δ*υ*
*T*Λ(**X**) where

Therefore, for a given *x _{k}*, the pdf of EEPN is given by

$f(\epsilon \mid {x}_{k})=\frac{1}{{\mid C\mid}^{L}}\xb7\sum _{\mathbf{x}\in {C}^{L}}\frac{1}{4{\pi}^{2}\Delta \upsilon T\mid \Lambda \left(\mathbf{X}\right)\mid}\mathrm{exp}(-\frac{1}{4\pi \mathbf{\Delta}\upsilon T}\left[\mathrm{Re}\left\{\epsilon \right\}\phantom{\rule{.2em}{0ex}}\mathrm{Im}\left\{\epsilon \right\}\right]\Lambda {\left(\mathbf{X}\right)}^{-1}{\left[\mathrm{Re}\left\{\epsilon \right\}\phantom{\rule{.2em}{0ex}}\mathrm{Im}\left\{\epsilon \right\}\right]}^{T})$
where *C* is the set {1, −1} for BPSK and {1, −1, *j*, −*j*} for QPSK formats etc. and ∣ *C* ∣ denotes the cardinality of *C*. Note that in presence of ASE noise with variance *σ*
^{2} per quadrature, Λ(**X**) can be replaced by Λ(**X**) +*σ*
^{2}
**I**.

## APPENDIX B: Proof of (g(t)·ejϕr(t))*p(t)≈q(t)ejϕr(t)

Denote
$u\left(t\right)=\left(g\left(t\right)\xb7{e}^{j{\varphi}_{r}\left(t\right)}\right)*p\left(t\right)$
, the Fourier Transform of *u*(*t*) is given by *U*(*f*) = (*G*(*f*)*Φ(*f*))·*P*(*f*)

where Φ(*f*) is the Fourier Transform of
${e}^{j{\varphi}_{r}\left(t\right)}$
. Now,
$U\left(f\right)=P\left(f\right)\underset{-\infty}{\overset{\infty}{\int}}\Phi \left({f}_{1}\right)G(f-{f}_{1}){\mathrm{df}}_{1}\approx P\left(f\right)\underset{\frac{-\Delta \upsilon}{2}}{\overset{\frac{\Delta \upsilon}{2}}{\int}}\Phi \left({f}_{1}\right)G(f-{f}_{1}){\mathrm{df}}_{1}$
as the bandwidth of
${e}^{j{\varphi}_{r}\left(t\right)}$
is approximately Δ*υ*. Since the bandwidth of the low-pass filter *P*(*f*) is in the order of 30 GHz for a symbol rate of 27.59 GSym/s and Δ*υ* is at most several MHz in practice,

In this case,

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}\approx \underset{\frac{-\Delta \upsilon}{2}}{\overset{\frac{\Delta \upsilon}{2}}{\int}}\Phi \left({f}_{1}\right)P(f-{f}_{1})G(f-{f}_{1}){\mathrm{df}}_{1}$$

$$\phantom{\rule{.9em}{0ex}}\phantom{\rule{.9em}{0ex}}=\underset{\frac{-\Delta \upsilon}{2}}{\overset{\frac{\Delta \upsilon}{2}}{\int}}\Phi \left({f}_{1}\right)Q(f-{f}_{1}){\mathrm{df}}_{1}=Q\left(f\right)*\Phi \left(f\right).$$

Therefore, by taking the inverse Fourier Transform, we obtain
$u\left(t\right)\approx q\left(t\right)\xb7{e}^{j{\varphi}_{r}\left(t\right)}$
. Note that this is an alternative way to illustrate the origins of EEPN. Consider replacing *P*(*f*) with
${H}^{-1}\left(f\right)={e}^{j4{\pi}^{2}{f}^{2}{\beta}_{2}\frac{L}{2}}$
that characterize compensation of accumulated CD in a transmission link. For any given Δ*υ*, *H*
^{−1}(*f*) ≠ *H*
^{−1}(*f* − *f*1) for ∣*f*
_{1}∣ ≤ Δ*υ*/2 whenever *β*
_{2}
*L* is large enough and hence (B.2) will not follow and multiplication and convolution will not commute. Therefore, the effect of EEPN will always scale up with accumulated CD independent of laser linewidth.

## Acknowledgements

The authors would like to acknowledge the support of the Hong Kong Government General Research Fund (GRF) under project number 522009.

## References and links

**1. **E. Ip, A. P. T. Lau, D. J. F. Barros, and J. M. Kahn, “Coherent detection in optical fiber systems,” Opt. Express **16**(1), 753–791 (2008). [CrossRef]

**2. **E. Ip and J. M. Kahn, “Digital equalization of Chromatic Dispersion and Polarization Mode Dispersion,” IEEE/OSA J. Lightwave Technol. **25**(8), 2033–2043 (2007). [CrossRef]

**3. **M. G. Taylor, “Coherent detection method using DSP for demodulation of signal and subsequent equalization of propagation impairments,” IEEE Photon.Technol. Lett. **16**(2), 674–676 (2004). [CrossRef]

**4. **G. Goldfarb and G. Li, “Chromatic dispersion compensation using digital IIR filtering with coherent detection,” IEEE Photon. Technol. Lett. **19**(13), 969–971 (2007). [CrossRef]

**5. **K. Roberts, C. Li, L. Strawczyski, M. O. Sullivan, and I. Hardcastle, “Electronic precompensation of optical nonlinearity,” IEEE Photon. Technol. Lett.18(13), 403–405 (2006). [CrossRef]

**6. **F. Buchali and H. Bulow, “Adaptive PMD Compensation by Elecgtrical and Optical Techniques,” IEEE/OSA J. Lightwave Technol. **22**(4), 1116–1126 (2004). [CrossRef]

**7. **D. Schlump, B. Wedding, and H. Bulow, “Electronic equalization of PMD and chromatic dispersion induced distortion after 100 km standard fibre at 10Gb/s,” *in Proceedings ECOC*, Madrid, Spain, 1998, pp. 535–536.

**8. **J. M. Gené, P. J. Winzer, S. Chandrasekhar, and H. Kogelnik, “Simultaneous compensation of Polarization Mode Dispersion and chromatic dispersion using electronic signal processing,” IEEE/OSA J. Lightwave Technol. **26**(7), 1735–1741 (2007). [CrossRef]

**9. **H. Bulow, F. Buchali, and A. Klekamp, “Electronic dispersion compensation,” IEEE/OSA J. Lightwave Technol. **26**(1), 158–167 (2008). [CrossRef]

**10. **E. Ip and J. M. Kahn, “Compensation of Dispersion and Nonlinear Impairments Using Digital Backpropagation,” IEEE/OSA J. Lightwave Technol. **26**(20), 3416–3425 (2008). [CrossRef]

**11. **X. Li, X. Chen, G. Goldfarb, E. Mateo, I. Kim, F. Yaman, and G. Li, “Electronic post-compensation of WDM transmission impairments using coherent detection and digital signal processing,” Opt. Express **16**(2), 880–888 (2008). [CrossRef]

**12. **K. Kikuchi, “Electronic post-compensation for nonlinear phase fluctuations in a 1000-km 20-Gbit/s optical quadrature phase-shift keying transmission system using the digital coherent receiver,” Opt. Express **16**(1), 889–896 (2008). [CrossRef]

**13. **A.P.T. Lau and J. M. Kahn, “Signal design and detection in presence of nonlinear phase noise,” IEEE/OSA J. Lightwave Technol.25(10), 3008–3016 (2007). [CrossRef]

**14. **E. Ip and J. M. Kahn, “Feedforward carrier recovery for coherent optical communications,” IEEE/OSA J. Lightwave Technol. **25**(9), 2675–2692 (2007). [CrossRef]

**15. **D. S. Ly-Gagnon, S. Tsukamoto, K. Katoh, and K. Kikuchi, “Coherent Detection of Optical Quadrature Phase-Shift Keying Signals With Carrier Phase Estimation,” IEEE/OSA J. Lightwave Technol. **24**(1), 12–21 (2006). [CrossRef]

**16. **R. Noe, “Phase noise-tolerant synchronous QPSK/BPSK baseband-type intradyne receiver concept with feedforward carrier recovery,” IEEE/OSA J. Lightwave Technol. **23**(2), 802–808 (2006). [CrossRef]

**17. **S. J. Savory, G. Gavioli, R. I. Killey, and P. Bayvel, “Electronic compensation of chromatic dispersion using a digital coherent receiver,” Opt. Express **15**(5), 2120–2126 (2007). [CrossRef]

**18. **G. Charlet, M. Salsi, P. Tran, M. Bertolini, H. Mardoyan, J. Renaudier, O. Bertran-Pardo, and S. Bigo, “72×100 Gb/s transmission over transoceanic distance, using large effective area fiber, hybrid Raman-Erbium amplification and coherent detection,” in *Proceedings OFC/NFOEC*, San Diego, CA, 2009, Paper PDPB6.

**19. **W. Shieh and K.P. Ho, “Equalization-enhanced phase noise for coherent detection systems using electronic digital signal processing,” Opt. Express **16**(20), 15718 – 15727 (2008). [CrossRef]

**20. **C. Xie, “Local oscillator phase noise induced penalties in optical coherent detection systems using electronic chromatic dispersion compensation,” in *Proceedings OFC/NFOEC*, San Diego, CA, 2009, Paper OMT4.

**21. **C. Xie, “WDM coherent PDM-QPSK systems with and without inline optical dispersion compensation,” Optics Express **17**(6), 4815–4823 (2009). [CrossRef]