## Abstract

Using an alternative approach for evaluating the Bit-Error Rate (BER), we present a numerical and experimental investigation of the performance of phase-modulated optical communication systems in the presence of nonlinear phase noise and dispersion. The numerical method is based on the well known Karhunen-Loève expansion combined with a linearization technique of the Nonlinear Schrödinger Equation (NLSE) to account for the nonlinear interaction between signal and noise. Our numerical results show a good agreement with experiments.

©2009 Optical Society of America

## 1. Introduction

In recent years, we have seen an increasing interest in using new modulation schemes to increase the capacity of multi-channel optical communication systems. Differential phase-shift-keying (DPSK) is now one of the most used scheme and takes advantage of the mature technology of Mach-Zehnder modulators and balanced receivers to achieve record performance [1].

In long-haul optical communication systems, the main source of degradation is the accumulated amplified spontaneous emission (ASE) noise generated by the optical amplifiers. During the light propagation inside the fiber, the ASE noise interacts with the signal through the Kerr effect, which induces signal phase fluctuations. This is known in the literature as nonlinear phase noise [2]. It was shown experimentally that the benefit of using DPSK together with balanced detection can vanish as nonlinear phase noise becomes dominant over other impairments [3]. Therefore, the performance evaluation of DPSK systems should include the interaction between signal and noise in order to correctly assess the bit-error rate (BER), as well as system margins and reach.

The most straightforward method to evaluate the BER is the standard Monte Carlo simulation. It includes the interaction between signal and noise and is often used as reference for other simulation techniques. For BER’s smaller than 10^{-6}, the Multicanonical Monte Carlo method [4] is more efficient. However, if the BER has to be computed several times, the overall computational effort of both Monte Carlo methods is so large that it becomes unpractical to apply these methods for optimizing systems including nonlinear phase noise.

The simulation time can be drastically reduced by deriving simplified models or closed-form expressions for the BER. For instance, if the distribution of the differential phase noise in DPSK systems is approximated as Gaussian, then a differential phase Q-method [5] can be used to qualitatively estimate the BER in the presence of nonlinear phase noise. In absence of dispersion, the statistics of the noise at the receiver and, therefore, the BER can be exactly calculated [6–8] using an optical matched filter. A simplified method developed by Ho [9], was verified experimentally [10], where a good agreement between simulation and experiment was achieved. However, in the presence of dispersion, the method overestimates the impact of nonlinear phase noise. Using a perturbational analysis, Ho and Wang also investigated the effect of dispersion on nonlinear phase noise [11], but they used second-order statistics as a measure of the performance instead of the BER.

Dispersion and nonlinear phase noise can be adequately taken into account by using techniques based on the linearization of the nonlinear Schrüdinger equation (NLSE). The computational effort can be significantly reduced at a cost of neglecting the nonlinear noise-noise interaction and intra-channel effects. The main idea of a linearization procedure is to separate the analysis of the signal and noise, i.e., the Split-Step Fourier method [12] accounts for nonlinear signal-signal distortion and the interaction between signal and noise is evaluated by solving the NLSE, where the signal is a non-modulated carrier and the noise acts as a perturbation of the general solution. In this case, the quadratic nonlinearity of the photodetector is kept and the linearization is only performed on the nonlinear term of the NLSE. A comprehensive review of these techniques can be found in Demir’s paper [13]. This approach has been extensively used to simplify the analysis of the NLSE in the presence of noise. The main difference between several methods based on linearization [14–20] is the use of a real/imaginary, in-phase/quadrature or amplitude/phase representation of the ASE noise. For example, Holzlöhner *et al*. [19] used a generalization of [21] and a real/imaginary representation of the noise to derive a covariance matrix method. In this case, timing and phase jitter were separated in order to accurately evaluate eye diagrams and BER’s for soliton systems. However, the method relies on the correct evaluation of a covariance matrix, whose computational cost equals that of a Monte Carlo simulation [20]. Moreover, the separation of phase and timing jitter may not be applicable to phase-modulated systems.

In this paper, we first present in detail an extended Karhunen-Loève expansion method based on a system transmission matrix approach to evaluate the impact of nonlinear phase noise in DPSK systems. In order to validate our simulation method and results, an experiment was performed, where the effect of dispersion on nonlinear phase noise was investigated. In our theoretical development, we use the eigenfunction expansion method for calculating the BER [22] and the transmission matrix approach [14] to evaluate the nonlinear signal-noise interaction. The computational effort of the method is negligible when compared to Monte Carlo methods and its simple modular structure makes it suitable for implementation in commercial simulation softwares.

This paper is organized as follows. In section 2, we derive a system transmission matrix through the linearization of the NLSE, in section 3 we present the method for calculating the BER, in section 4 the experiment is described, and in section 5 we present the results, section 6 concludes the paper.

## 2. Linearization of the NLSE

Noise enhancement due to fiber nonlinearities can be found in the literature under several names [14–16]. Modulation Instability, Parametric Gain, four-wave-mixing (FWM) between signal and noise are few examples. Despite different names, they all rely on the same mathematical formalism to evaluate the impact of amplified ASE noise on the system performance. In this section, we use a linearization technique of the NLSE in order to evaluate the noise enhancement due to fiber nonlinearities and also to define a system transmission matrix **W**(**f**), which accounts for the nonlinear interaction between signal and noise.

Consider an electrical field at the input of an optical fiber *A*(*z* = 0,*t*) = *s*(*z* = 0,*t*)+*a*(*z* = 0,*t*), where *A*(*z*,*t*) is the slowly varying complex envelope, *s*(*z* = 0,*t*) is the signal with peak power *P*
_{0} and *a*(*z* = 0,*t*) is the ASE of a single optical amplifier in one polarization. The NLSE in one polarization describes the propagation of this signal and noise inside the fiber and is given by

where *β*
_{2} and *β*
_{3} are the second and third order dispersion parameters, *γ* is the nonlinear parameter and *α* is the attenuation. If the signal *s*(*z*, *t*) is much stronger than the noise *a*(*z*, *t*), then NLSE can be linearized by neglecting the noise-noise beat term arising from the fiber nonlin-earity, i.e. terms involving *a ^{n}*(

*z*,

*t*), where

*n*> 1, are not taken into account. In the following, we consider a single fiber of length

*L*, then we extend the result for a multi-span system.

#### 2.1. Single fiber

Considering the signal *s*(*z* = 0,*t*) as a continuous wave with power *P*
_{0} and neglecting the noise
*a*(*z* = 0,*t*), i.e.*A*(*z*,*t*) becomes *A*(*z*), the NLSE (1) reduces to

$$\phantom{\rule{.2em}{0ex}}A\left(z\right)=\sqrt{{P}_{0}}\xb7{e}^{-\mathrm{j\varphi}\left(z\right)},$$

where *ϕ*(*z*) = *γP*
_{0} ∫_{0}
^{z}
*e*
^{-αz′}
*dz*′. If we add the noise *a*(*z*,*t*) to the solution as a perturbation, we have

Substituting in (1) and assuming that |*a*(*z*, *t*)|^{2} ≪ *P*
_{0}, we obtain in frequency domain *ω*,

In the time domain, the noise is given by *a*(*z*,*t*) = *a _{p}*(

*z*,

*t*)+

*ja*(

_{q}*z*,

*t*), where

*a*(

_{p}*z*,

*t*) and

*a*(

_{q}*z*,

*t*) are real functions. In frequency domain,

*ã*(

*z*,

*ω*) =

*ã*(

_{p}*z*,

*ω*) +

*jã*(

_{q}*z*,

*ω*), where

*ã*(

_{p}*z*,

*ω*) and

*ã*(

_{q}*z*,

*ω*) may be complex. Using these relations and 2

*ã*(

_{p}*z*,

*ω*) =

*ã*(

*z*,

*ω*)+

*ã*

^{*}(

*z*, -

*ω*), we obtain

$$\phantom{\rule{3em}{0ex}}-\frac{1}{6}{\beta}_{3}{\omega}^{3}{\tilde{a}}_{q}\left(z,\omega \right)=-\mathrm{j\gamma}2{P}_{0}{e}^{-\mathrm{\alpha z}}{\tilde{a}}_{p}\left(z,\omega \right).\phantom{\rule{10em}{0ex}}$$

Defining $\rho =\frac{{\beta}_{2}}{2}{\omega}^{2}\phantom{\rule{.2em}{0ex}}+\phantom{\rule{.2em}{0ex}}\frac{{\beta}_{3}}{6}{\omega}^{3},$, a set of equations can be written as

The solution is given by **ã**
_{out} = **M** · **ã**
_{in}, where **ã** = [*ã _{p}*(

*z*,

*ω*)

*ã*(

_{q}*z*,

*ω*)]

*, [·]*

^{T}^{T}is the transpose operator and the transmission matrix

**M**in absence of loss is given by

where $\delta =\sqrt{{\rho}^{2}+2\mathrm{\rho \gamma}{P}_{0}}$ The transmission matrix with *α* ≠ 0 can be obtained by dividing the fiber into many short pieces of length *dz*. The total transmission matrix is given by the ordered product of the transmission matrices for each segment of the fiber [14,23]. If *dz* ≪ *α*
^{-1} and *dz* = $\mathrm{dz}=\frac{L}{{N}_{\mathrm{sec}}}$ being *N*
_{sec} ∈ ℕ, a mean-field approximation can be applied in each section

where *δ _{i}*, and

*γ*for each segment are given by

_{dz}Note that this solution does not take into account the attenuation of the signal and noise during the propagation. The attenuation was included in the calculation as a factor which reduces the impact of nonlinear effects. In order to include the attenuation effect, the matrix **M** should be multiplied by exp (-*αL*).

Applying the Wiener-Kinchine theorem, the power spectral density matrix of the noise at the output of the fiber can be calculated as [15]

where **G**
_{1} = **G**
_{1}(*z*, *ω*), [·]** ^{H}** is the transpose-conjugate operator,

*τ*is a time window and

*E*{·} indicates statistical averaging. The power spectral density of the noise at the fiber output is given by Φ(

*z*,

*ω*) =

*G*+

_{pp}*G*=

_{qq}*E*{|

*ã*(

_{p}*z*,

*ω*)|

^{2}}+

*E*{|

*ã*(

_{q}*z*,

*ω*)|

^{2}} and at the input Φ(0,

*ω*) =

*E*{|

*ã*(0,

_{p}*ω*)|

^{2}} +

*E*{|

*ã*(0,

_{q}*ω*)|

^{2}} = Φ

_{ASE}, where Φ

_{ASE}is the power spectral density of the noise generated by the optical amplifier in one polarization. Thus, the power spectral density at the end of the fiber is given by

#### 2.2. Multi-span system

In Fig. 1, a multi-span system with different fibers, filters and amplifiers is depicted. **H**
* _{i}* is the filter transfer matrix and is given by

where the noise at its output is given by **ã**
_{out} = **H**
* _{i}* ·

**ã**

_{in}. The power spectral density matrix

**G**

_{N+1}(

*z*,

*ω*) can be calculated by passing the noise vector

**ã**through the devices shown in Fig. 1. Thus, the noise at the input of the receiver is given by

where **M**
* ^{i}* =

**H**

_{N+1}· ∏

^{N}

_{k=i}

*M*·

_{k}**H**

*and*

_{k}*N*is the number of spans. Note that

**H**

_{N+1}is the receiver optical filter and can be incorporated into the receiver structure or left in the link. Therefore, the power spectral density of the noise can be calculated as follows:

where Φ^{i}
_{ASE} is the power spectral density of the additive white Gaussian noise generated by the *i*-th amplifier. Finally, using (9), *G _{pp}*,

*G*,

_{pq}*G*and

_{qp}*G*can be evaluated in order to calculate the Φ(

_{qq}*z*,

*ω*) at the receiver input. Note that each term of the above sum has a product of the matrices

**M**

*. In order to go over the sum only one time, it is necessary to calculate the system transmission matrix backwards, i.e., the sum should be calculated from*

_{i}*i*=

*N*to 1. Several properties of the interaction between signal and noise can be evaluated using the power spectral density matrix

**G**

_{N+1}(

*z*,

*ω*), as reported in [24]. It is important to note that

*G*

_{N+1}(

*z*,

*ω*) is a positive semidefinite matrix and, therefore, it can be written as

**G**

_{N+1}(

*z*,

*ω*) = Φ

_{ASE}

^{N+1}· (

**W**̃ ·

**W̃**), where [15,25]

^{H}and the statistics of the received noise are equivalent to a AWGN noise with power spectral density Φ_{ASE}
^{N+1}· **I** filtered by **W**̃. Therefore, the noise at the receiver input is given by

The noise statistics of every **a**̃* _{i}*, was normalized by

**a**

_{N+1}and is included in

**W**̃. Note that in

*ã*(

*z*,

*ω*) =

*ã*(

_{p}*z*,

*ω*) +

*jã*(

_{q}*z*,

*ω*),

*ã*(

_{p}*z*,

*ω*) and

*ã*(

_{q}*z*,

*ω*) may be complex functions. In order to deal with system functions in the frequency domain, it is more convenient to define a noise vector

**ñ**(

*z*,

*ω*) = [

*ñ*

_{re}(

*z*,

*ω*)

*ñ*(

_{im}*z*,

*ω*)]

^{T}containing the real and imaginary parts of

*a*̃(

*z*,

*ω*):

By defining a discrete symmetric frequency vector **f** = [*f*
_{-M}
*f*
_{-M+1} … 0… *f*
_{M-1}
*f _{M}*]

*and using the relation*

^{T}*ω*= 2

*πf*, the vector

**n**̃

_{out}(

*z*,

*f*) can be written as a (4

*M*+2) × 1 column vector

**n**̃

_{out}(

**f**) containing the real and imaginary parts of

*a*̃

_{out}(

*z*,

*f*) at the frequencies

**f**and it is given by

where **D**
_{0} is a (2*M*+ 1) × (2*M*+ 1) anti-diagonal matrix and **a**̃_{N+1}(**f**) is a (4*M*+2) + 1 column vector. The terms **W**̃* _{mn}*(

**f**) with {

*m*,

*n*} ∈ {1,2} are (2

*M*+ 1) + (2

*M*+ 1) diagonal matrices, where each diagonal entry is the

*W̃*(

_{mn}*f*) element of the matrix

*W*̃. Consequently, the noise at the receiver input can also be written as

where **n**̃_{N+1} (**f**) = **B _{2}** ·

**a**̃

_{n+1}(

**f**),

**B**=

_{2}**B**and

_{2}**W**(

**f**) is the system transmission matrix given by

where the matrices **B _{1}** and

**B**are given by

_{2}and **I** is the identity matrix. The matrices **B _{1}** and

**B**are used to calculate the real and imaginary parts of

_{2}*a*̃

*(*

_{N+1}*z*,

*f*) and

*a*̃

_{out}(

*z*,

*f*). Note that

**n**〰

_{out}(

**f**) cannot be defined like

**n**〰

_{N+1}(

**f**) because the matrix

**W**〰(

**f**) may be non-symmetric. In fact, it can be seen from (9) that it is non-symmetric if

*β*

_{3}≠ 0. Moreover, the entries in the matrix

**W**(

**f**) corresponding to the frequency

*f*can be dropped if enough simulation bandwidth is available. The break on the matrix symmetry will not affect the result because no significant linear or nonlinear effect will occur at this frequency. On the other hand, the even number of samples will increase the speed of the Fast-Fourier Transform (FFT) algorithm. In the following sections, the system transmission matrix

_{M}**W**(

**f**) has a reduced size of 4

*M*+ 4

*M*and is used to evaluate the BER accounting for nonlinear phase noise.

## 3. Evaluation of the bit-error rate

In long-haul optical communication systems, the BER is basically determined by the accumulated noise generated by the optical amplifiers through spontaneous emission and by nonlinear fiber effects. The accurate computation of the BER is the most complex part in the receiver modelling. In phase-modulated systems with balanced detection, conventional methods that approximate the probability density function at the receiver to a Gaussian distribution or use the standard Q-factor analysis do not work even if the received noise is white and Gaussian [1,26]. If the received optical noise can be considered AWGN, the BER can be exactly evaluated using several methods based on the principle of Karhunen-Loève series expansion [1,22,27
,28]. If fiber nonlinearities are of concern, then the nonlinear interaction between signal and noise generates additional noise, known in the literature as nonlinear phase noise. It changes the statistics of the received signal and make the exact evaluation of the BER a very difficult task. Moreover, the received optical noise is no longer AWGN, however, under some assumptions it can still be considered Gaussian, for instance, if the system has enough dispersion, filtering [29] or even PMD. In this case, linearization techniques of the NLSE, where a simplified solution of the NLSE is used and the noise is treated as a perturbation, can be applied to calculate the BER with a very good approximation [15,30]. In this section, we use the method proposed in [1,22] and combine it with the system transmission matrix **W**(**f**), derived in the previous section, in order to calculate the BER in presence of nonlinear phase noise. This method can yield accurate BER results, taking into account ASE noise, pulse shaping, optical and electrical filtering, interferometer phase error and nonlinear phase noise. We start our derivation with the standard Karhunen-Loève method, where no interaction between signal and noise is considered, then we extend the result by using the system transmission matrix.

#### 3.1. Standard Karhunen-Loève method

Figure 2 shows a balanced receiver used for the demodulation of the DPSK signal. The evaluation of the BER depends on the knowledge of the probability density function (PDF) of the decision sample *I*(*t _{k}*). Assuming arbitrary optical and electrical filtering, it is very difficult to
find an analytical formula for the PDF. However, the use of numerical methods to calculate the PDF from the moment generating function (MGF) can give very accurate results. In this section, we use the Karhunen-Loève series expansion to write the decision sample as a sum of uncorrelated and independent random variables such that the MGF can be easily determined. The Karhunen-Loève series expansion is widely used in communications engineering to describe stochastic processes. The main idea is to find a set of orthonormal basis functions that make the expanded noise components uncorrelated and if the processes is Gaussian also statistically independent, which is the case of ASE noise.

In Fig. 3, a block diagram of the signal and noise propagation in the frequency domain is depicted. The signal *s*̃_{out}(*f*) is obtained by solving numerically the NLSE using, for example, the Split-Step Fourier method [12], which accounts for all signal distortions. The interaction between signal and noise is included using the system transmission matrix **W**(**f**), where only the power of the signal, noise variances and link parameters are considered. The block “C / V” transforms the complex signal into a vector where its real and imaginary parts are separated. Without interaction between signal and noise, **n**̃_{out}(**f**) is given by the sum of the noise generated by each amplifier in the link. Therefore, the received signal can be written back, for simplicity, in its complex scalar form *E*̃(*f*) = *s*̃_{out}(*f*) +*n*̃_{out}(*f*), where *n*̃_{out}(*f*) is AWGN. The decision variable at the output of the balanced receiver *I*(*t _{k}*) is given by [1]

where

*t _{k}* is the sampling time and

*H*(

_{e}*f*),

*H*(

_{o}*f*),

*H*

_{1}(

*f*) and

*H*

_{2}(

*f*) are the transfer functions of the electrical filter, optical filter, upper and lower branch of the delay interferometer, respectively [1]. A necessary and sufficient condition for the Karhunen-Loève series expansion to hold is that the eigenfunctions

*φ*(

*f*) should satisfy the second kind homogeneous Fredholm integral equation [22]:

The eigenvalue *λ _{m}* of the

*m*eigenfunction

^{th}*φ*(

_{m}*f*) is a real-valued constant. The eigenvalues are ordered as |

*λ*

_{m-1}|≥ |

*λ*|

_{m}*λ*| ≥ |

_{m}*λ*

_{m+1}|, where

*m*is a positive integer. All the eigenfunctions satisfy the following condition:

where *δ _{ml}* is the Kronecker delta. Equation (29) implies that the eigenfunctions form a complete set of orthonormal basis functions over [-∞, ∞]. With this set,

*E*̃(

*f*)

*e*can be expanded as

^{j2πft}where *c _{m}*(

*t*) = ∫

^{∞}

_{-∞}

*E*̃(

*f*)

*e*

^{j2πft}+

*φ*

_{m}^{*}(

*f*)

*df*is the time dependent coefficient. Since

*E*̃(

*f*) consists of signal and noise, we can define

*s*(

_{i}*t*) = 𝔉

^{-1}{

*s*̃

_{out}(

*f*) +

*φ*

_{i}^{*}(

*f*)} and

*n*(

_{i}*t*) = 𝔉

^{-1}{

*n*̃

_{out}(

*f*) +

*φ*

_{i}^{*}(

*f*)} as the time dependent signal and noise coefficients, respectively. Therefore, we obtain

Applying (31) to (26), we express *I*(*t _{k}*) as a series summation:

As the ASE noise is white Gaussian, the coefficients *n _{i}*(

*t*) are uncorrelated und, therefore, independent complex Gaussian random variables.

_{k}#### 3.2. Discrete analysis

In (26), the decision sample has been expressed with double integrals, which, for numerical convenience, can be written as a double sum within the Discrete Fourier Transform (DFT) grid. The points are equally spaced by Δ*f* and coincide with the discrete frequency vector of the DFT. The resulting summation is given by

where $2M=\frac{\mathrm{BW}}{\Delta f}$ is the total number of samples, BW is the simulation bandwidth and *e _{m}* and

*K*are defined as

_{ml}where *e _{S,m}* and

*e*are the signal and noise part of

_{n,m}*e*and

_{m}*f*(

*m,l*) = ((

*m*,

*l*) -

*M*- 1)Δ

*f*. Note that

*I*(

*t*) and

_{k}*e*are random variables and

_{m}*K*is a deterministic variable. Equations (26) and (28) can be easily rewritten in matrix format as

_{ml}where *q _{i,m}* =

*φ*(

_{i}*f*)√Δ

_{m}*f*. Equation (39) shows that the second kind homogeneous Fredholm integral becomes a standard eigenvalue problem [31], where the eigenvalues and eigenvectors of the matrix

**K**are used to expand

**e**and also define the MGF. The coefficients of the sum (32) are given by

*s*(

_{i}*t*) =

_{k}**q**

^{H}*·*

_{i}**e**

_{s}and

*n*(

_{i}*t*) =

_{k}**q**

_{i}**+**

^{H}**e**, where

_{n}*n*(

_{i}*t*),

_{k}*i*= 1,2,…, 2

*M*, are zero-mean complex Gaussian random variables with independent real and imaginary components of variance

*σ*

^{2}= Φ

^{out}

_{ASE}Δ

*f*/2. Thus, the decision sample

*I*(

*t*) can be expressed as quadratic form of Gaussian random variables and its MGF can be evaluated in a closed form as follows [1,28]:

_{k}where *α _{i}*, =

*λ*|

_{i}*s*(

_{i}*t*)|

_{k}^{2},

*β*, = 2

_{i}*λ*

_{i}*σ*

^{2}and

*ξ*= 1 or

*ξ*= 2 for polarized or unpolarized noise, respectively. The mean and variance of

*I*(

*t*) are given by

_{k}respectively. Finally, the BER can be calculated from the MGF using the Laplace inverse transform and saddlepoint integration or approximation methods, as described in [1, 28].

#### 3.3. Extended Karhunen-Loève method

The standard Karhunen-Loève method assumes that the noise at the input of the balanced receiver is white and Gaussian. Considering nonlinear phase noise the received noise is neither white nor Gaussian. However, if enough dispersion and filtering is present in the system, the real and imaginary parts of the received noise can be approximated as Gaussian, but not white. In fact, they are correlated and differently perturbed during the propagation in the fiber. By separating in frequency domain the real and imaginary parts of the signal and noise, the impact of nonlinear phase noise on the BER can be evaluated by combining the standard Karhunen-Loève method with the system transmission matrix **W**(**f**). As shown in Fig. 3, the received signal is given by **E**̃(**f**) = *s*̃_{out}(**f**)+**W**(**f**)**n**̃^{N+1}(**f**) and (38) and (39) can be rewritten as

where **e _{W}** and

**K**are a 4

_{W}*M*×1 column vector and a 4

*M*× 4

*M*matrix, respectively, given by

and

The coefficients of the sum (32) are here given by *s _{i,w}* (

*t*) =

_{k}**q**

_{i,w}**H**·

**e**

_{w,s}and

*n*(

_{iw}*t*) =

_{k}**q**

_{i,w}**+**

^{H}**e**, where

_{w,n}*n*(

_{i,w}*t*),

_{k}*i*= 1,2,…, 4

*M*, are zero-mean real Gaussian random variables with variance .

*σ*

^{2}= Φ

^{N+1}

_{ASE}Δ

*f*/2. The MGF of

*I*(

*t*)can be written in the same form as in (40) and the BER can be calculated using the Laplace inverse transform and saddlepoint integration or approximation methods, as described in [1,28]. If fiber nonlinearities are neglected, the result of the evaluation of the BER should be equal as in the previous section, however, the computational time will increase because the size of the vectors and matrices was doubled. Following we apply the extended Karhunen-Loève method to evaluate the performance of DPSK systems.

_{k}## 4. Simulation and experimental set-up

In order to validate the results obtained using the extended Karhunen-Loève method, we conducted an experiment, where nonlinear phase noise and self-phase modulation (SPM) are the dominant effects. For this purpose we consider two noise loading schemes, as shown in Fig. 4. First, with the ASE noise added at the receiver, the BER is measured for a non- (a) and high-(b) dispersive span. Then, the ASE noise is added at the transmitter and the BER is measured again for both cases. The accumulated dispersion at the end of each span was set to zero. If the ASE noise is distributed along the transmission link, then the results should lie between both cases considered here [32–35].

Depending on the number of spans *N*, the total input power at MP1 was set such that the accumulated mean nonlinear phase shift of the system set-up (a) and (b),

amounts to 0.9rad, where *P*
_{(*)} is the average fiber input power, *L*
^{(*)}
_{eff} is the effective length and *γ*
_{(*)} the nonlinear parameter. Therefore, by fixing the accumulated mean nonlinear phase shift, the amount of nonlinearity in each span is varied, which should give the same performance in absence of dispersion for any number of spans *N* [7]. In this configuration, only the variance of the nonlinear phase noise is evaluated in terms of BER and the effect of dispersion on nonlinear phase noise can be isolated.

Simulations and experiments were performed using a *R _{b}* = 20 Gbit/s RZ-50% DPSK signal at a fixed wavelength of 1556.555 nm for pseudo random binary sequences (PRBS) of length 2

^{7}- 1 (simulation) and 2

^{23}- 1 (experiment). In order to increase the accuracy of the simulation results, the signal power spectrum at the transmitter was measured and used to fit the simulated signal power spectrum, as shown in Fig. 5. The receiver optical filter transfer function and fiber parameters were also measured using standard procedures. Table 1 shows the fiber parameters used in the experiment and simulations, where L is the fiber length,

*α*is the attenuation, D is the chromatic dispersion parameter, S the slope and PMD the polarization-mode dispersion parameter. At the transmitter and in the loop, optical bandpass filters (OBPF) with bandwidth of 3 nm and 5 nm, respectively, were used to limit the noise bandwidth. Two polarization scramblers, at the receiver and in the loop, were used to average the effect of polarization dependent loss (PDL). The 3 dB bandwidth of the optical receiver filter was chosen to be 0.3 nm; the free spectral range (FSR) of the Mach-Zehnder delay interferometer was measured to be 24.84 GHz. The electrical filter was modelled as a fifth-order Bessel filter with a bandwidth of 0.75

*R*, which was determined from the back-to-back measurements. In Section 2, the system transmission matrix

_{b}**W**(

**f**) was derived assuming the signal as a continuous wave. In order to take into account the effect of signal modulation, instead of

*P*

_{0}, an effective fiber input power

*P*

_{eff}was used to calculate the system transmission matrix [15].

*P*is evaluated by filtering

_{eff}*P*(

*t*) = |

*A*(

*t*)|

^{2}with a proper window and sampling. For a RZ-DPSK signal with sinusoidal intensity profile,

*P*

_{eff}can be calculated analytically using the following equations [36 ]:

where *L*
_{eff} and *L _{D}* are the effective and dispersion lengths of the transmission fiber, respectively. For the DSF fiber,

*L*is zero, which results in a effective power equal the peak power. However, due to filtering and PMD a small memory is introduced to the system. This could be verified after comparing the measured and simulated power spectral densities. Therefore, in our simulations an effective fiber input power of 95% (DSF) and 63% (SMF+DCF) of the fiber input peak power was used to calculate

_{D}**W**(

**f**). In the fiber, the signal propagates in two orthogonal polarizations, which causes PMD and nonlinear coupling between both polarizations. The linear distortion due to PMD can be estimated by allowing a maximum ratio of DGD to symbol duration of 10% [16]. Using the parameters in Table 1, the mean DGD is given by DGD

_{DSF}= 0.87, 1.38, 3.1ps for

*N*= 2, 5, 25, respectively, and DGD

_{SMF+DCF}= 0.93, 1.32, 2.1ps for

*N*= 5, 10, 25, respectively. Therefore, the impact of PMD on the performance can be neglected for this set-up. Moreover, the nonlinear coupling between both polarizations in presence of PMD can be simulated using (1) as an averaged propagation equation [37], where the nonlinear parameter

*γ*is reduced to 91% of its maximal value. Under these assumptions we were able to match simulation and experimental results.

## 5. Results

For the system depicted in Fig. 4, the BER was measured and simulated at different values of optical signal-to-noise ratio (OSNR) at the receiver. The results are summarized in Fig. 6 and Fig. 7. The BER was first measured and simulated in a back-to-back configuration, which gives equal results for both noise loading schemes. With the noise added at the receiver, the signal was first transmitted through *N* = 2, 5 and 25 non-dispersive (DSF) spans. In this case, SPM dominates and the BER was accurately calculated using the extended Karhunen-Loève expansion method, as shown in Fig. 6(a). Since the mean nonlinear phase shift was kept constant, the BER is almost equal for different number of spans. After that the signal was transmitted *N* times over a fully dispersion compensated span (SMF+DCF) and BER curves were measured and simulated again for back-to-back, *N* = 5, 10 and 25 spans, as shown in Fig. 6(b). An improvement of 1dB from the DSF curve was observed for both low- and high-OSNR, as shown in Fig. 6. Therefore, using a high-dispersive span, the effect of dispersion reduces the impact of SPM.

In a second step, the noise was added to the signal at the transmitter and propagated again through *N* = 2, 5 and 25 non-dispersive (DSF) spans. The BER can now be calculated using the extended Karhunen-Loève expansion method. The SPM induced nonlinear phase noise generates an additional penalty comparing with the receiver noise loading scheme. The small difference between the simulated and measured curves for *N* = 2 and 5 is interpreted as a consequence of a small deviation of the system zero dispersion together with in-line optical filtering and ASE noise accumulation. For *N* = 25 the accumulated nonlinear phase noise seems to be so strong that the linearization procedure does not work anymore. This problem was also observed by Holzlöhner *et al*. [19,20] in long-haul soliton systems, where only one polarization was considered. The interaction between nonlinearities and PMD can also result in an additional penalty that can not be predicted by the linearization procedure. In this case, further investigations of the ASE noise evolution are required in order to correctly evaluate the BER.

Following, the signal was transmitted again *N* = 5, 10 and 25 times over a fully dispersion compensated span. In Fig. 7(b), the system performance is almost the same for 5, 10 and 25 spans, showing that the BER depends solely on the mean nonlinear phase shift, which so far was known only for systems without dispersion. Comparing Fig. 7(a) and Fig. 7(b), an improvement of 3dB and 2dB from the DSF curve was observed for low- and high-OSNR, respectively. Therefore, if enough dispersion or filtering is present in the link, which is the case of most WDM systems, the extended Karhunen-Loève expansion method agrees very well with the experimental results, as shown in Fig. 7.

## 6. Conclusion

In this paper, we presented in detail the modeling of nonlinear phase noise in phase modulated optical communication systems. We demonstrated numerically and experimentally that
the BER can be accurately calculated in presence of nonlinear phase noise by combining the standard Karhunen-Loève expansion method with the system transmission matrix **W**(**f**). Additionally, we showed that dispersion can improve the system performance and the gain is found to be larger for low-OSNR than for high-OSNR. It suggests that a system can have the same performance either employing compensation of nonlinear phase noise or FEC. We also verified that the performance of a fully dispersion compensated span is solely given by the mean nonlinear phase shift. This result gives the basic relation between reach-distance and power for a given system performance.

## Acknowledgment

The authors would like to thank Bernhard Goebel for the fruitful discussions during the development of this work. This work was funded by the German Ministry of Education and Research (BMBF) in the framework of the EIBONE project.

## References and links

**1. **A. H. Gnauck and P. J. Winzer, “Optical phase-shift-keyed transmission,” J. Lightwave Technol. **23**, 115–130 (2005). [CrossRef]

**2. **J. P. Gordon and L. F. Mollenauer, “Phase noise in photonic communications systems using linear amplifiers,” Opt. Lett. **15**, 1351–1353 (1990). [CrossRef] [PubMed]

**3. **H. Kim and A. H. Gnauck, “Experimental investigation of the performance limitation of DPSK systems due to nonlinear phase noise,” IEEE Photon. Technol. Lett. **15**, 320–322 (2003). [CrossRef]

**4. **R. Holzlöhner and C. R. Menyuk, “Use of multicanonical Monte Carlo simulations to obtain accurate bit error rates in optical communications systems,” Opt. Lett. **28**, 1894–1896 (2003). [CrossRef] [PubMed]

**5. **X. Wei, X. Liu, and C. Xu, “Numerical simulation of the SPM penalty in a 10-Gb/s RZ-DPSK system,” IEEE Photon. Technol. Lett. **15**, 1636–1638 (2003). [CrossRef]

**6. **A. Mecozzi, “Limits to long-haul coherent transmission set by the Kerr nonlinearity and noise of the in-line amplifiers,” J. Lightwave Technol. **12**, 1993–2000 (1994). [CrossRef]

**7. **K.-P. Ho, *Phase-Modulated Optical Communication Systems* (Springer Science+Business Media, Inc, New York, USA, 2005).

**8. **M. P. Dlubek, A. J. Phillips, and E. C. Larkins, “Nonlinear Evolution of Gaussian ASE Noise in ZMNL Fiber,” J. Lightwave Technol. **26**, 891–898 (2008). [CrossRef]

**9. **K.-P. Ho, “Performance of DPSK Signals With Quadratic Phase Noise,” IEEE Trans. Commun. **53**, 1361–1365 (2005). [CrossRef]

**10. **J.-A. Huang, K.-P. Ho, H.-K. Chen, S. K. Liaw, and H.-C. Wang, “Impact of nonlinear phase noise to DPSK signals: experimental verification of a simplified theoretical model,” IEEE Photon. Technol. Lett. **17**, 2236–2238 (2005). [CrossRef]

**11. **K.-P. Ho and H.-C. Wang, “Effect of dispersion on nonlinear phase noise,” Opt. Lett. **31**, 2109–2111 (2006). [CrossRef] [PubMed]

**12. **G. P. Agrawal, *Nonlinear Fiber Optics*, 3rd ed. (Academic Press, San Diego, USA, 2001).

**13. **A. Demir, “Nonlinear Phase Noise in Optical-Fiber-Communication Systems,” J. Lightwave Technol. **25**, 2002–2032 (2007). [CrossRef]

**14. **R. Hui, M. O’Sullivan, A. Robinson, and M. Taylor, “Modulation instability and its impact in multispan optical amplified IMDD systems: theory and experiments,” J. Lightwave Technol. **15**, 1071–1082 (1997). [CrossRef]

**15. **P. Serena, A. Orlandini, and A. Bononi, “Parametric-gain approach to the analysis of single-channel DPSK/DQPSK systems with nonlinear phase noise,” J. Lightwave Technol. **24**, 2026–2037 (2006). [CrossRef]

**16. **G. P. Agrawal, *Lightwave Technology: Telecommunication Systems* (John Wiley & Sons, Inc, New Jersey, USA, 2005).

**17. **A. V. T. Cartaxo, B. Wedding, and W. Idler, “Influence of fiber nonlinearity on the phase noise to intensity noise conversion in fiber transmission: theoretical and experimental analysis,” J. Lightwave Technol. **16**, 1187–1194 (1998). [CrossRef]

**18. **A. V. T. Cartaxo, B. Wedding, and W. Idler, “Influence of fiber nonlinearity on the fiber transfer function: theoretical and experimental analysis,” J. Lightwave Technol. **17**, 1806–1813 (1999). [CrossRef]

**19. **R. Holzlohner, V. S. Grigoryan, C. R. Menyuk, and W. L. Kath, “Accurate calculation of eye diagrams and bit error rates in optical transmission systems using linearization,” J. Lightwave Technol. **20**, 389–400 (2002). [CrossRef]

**20. **R. Holzlohner, C. R. Menyuk, W. L. Kath, and V. S. Grigoryan, “A covariance matrix method to compute bit error rates in a highly nonlinear dispersion-managed soliton system,” IEEE Photon. Technol. Lett. **15**, 688–690 (2003). [CrossRef]

**21. **D. Marcuse, “Derivation of analytical expressions for the bit-error probability in lightwave systems with optical amplifiers,” J. Lightwave Technol. **8**, 1816–1823 (1990). [CrossRef]

**22. **J.-S. Lee and C.-S. Shim, “Bit-error-rate analysis of optically preamplified receivers using an eigenfunction expansion method in optical frequency domain,” J. Lightwave Technol. **12**, 1224–1229 (1994). [CrossRef]

**23. **A. G. Green, P. P. Mitra, and L. G. L. Wegener, “Effect of chromatic dispersion on nonlinear phase noise,” Opt. Lett. **28**, 2455–2457 (2003). [CrossRef] [PubMed]

**24. **P. Serena, A. Bononi, and A. Orlandini, “Fundamental laws of parametric gain in periodic dispersion-managed optical links,” J. Opt. Soc. Am. B **24**, 773–787 (2007). [CrossRef]

**25. **G. Strang, *Linear Algebra and its Applications*, 3rd ed. (Saunders, 1988).

**26. **G. Bosco and P. Poggiolini, “On the Q factor inaccuracy in the performance analysis of optical direct-detection DPSK systems,” IEEE Photon. Technol. Lett. **16**, 665–667 (2004). [CrossRef]

**27. **M. Kac and A. Siegert “On the Theory of Noise in Radio Receivers with Square Law Detectors,” J. Appl. Phys. **18**, 383–397 (1947). [CrossRef]

**28. **E. Forestieri, “Evaluating the error probability in lightwave systems with chromatic dispersion, arbitrary pulse shape and pre- and postdetection filtering,” J. Lightwave Technol. **18**, 1493–1503 (2000). [CrossRef]

**29. **A. Orlandini, P. Serena, and A. Bononi, “An Alternative Analysis of Nonlinear Phase Noise Impact on DPSK Systems,” in *32nd European Conference on Optical Communication, ECOC 2006* (Cannes, France, 2006). [CrossRef]

**30. **L. D. Coelho, L. Molle, D. Gross, N. Hanik, R. Freund, C. Caspar, and E.-D. Schmidt, “Numerical and Experimental Investigation of the Effect of Dispersion on Nonlinear Phase Noise in RZ-DPSK Systems,” *in 33rd European Conference on Optical Communication, ECOC 2007* (Berlin, Germany, 2007). [CrossRef]

**31. **W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, *Numerical Recipes: The Art of Scientific Computing* (Cambridge University Press, Cambridge, U.K., 2007).

**32. **E. T. Spiller, W. L. Kath, R. O. Moore, and C. J. McKinstrie, “Computing large signal distortions and bit-error ratios in DPSK transmission systems,” IEEE Photon. Technol. Lett. **17**, 1022–1024 (2005). [CrossRef]

**33. **R. O. Moore, G. Biondini, and W. L. Kath, “A Method to Compute Statistics of Large, Noise-Induced Perturbations of Nonlinear Schrödinger Solitons,” SIAM Review **50**, 523–549 (2008). [CrossRef]

**34. **S. Kumar and L. Liu, “Reduction of nonlinear phase noise using optical phase conjugation in quasi-linear optical transmission systems,” Opt. Express **15**, 2166–2177 (2007). [CrossRef] [PubMed]

**35. **M. Ohm, R. J. Essiambre, and P. J. Winzer, “Nonlinear phase noise and distortion in 42.7-Gbit/s RZ-DPSK systems,” in *31st European Conference on Optical Communication, ECOC 2005* (Glasgow, Scotland, 2005). [CrossRef]

**36. **P. Serena, A. Orlandini, and A. Bononi, “A parametric gain approach to DPSK performance evaluation in presence of nonlinear phase noise,” in *30th European Conference on Optical Communication, ECOC 2004* (Stockholm, Sweden, 2004).

**37. **N. Hanik, “Modelling of nonlinear optical wave propagation including linear mode-coupling and birefringence,” Opt. Commun. **214**, 207–230 (2002). [CrossRef]