## Abstract

The noise statistics of optically demodulated signal of the recently proposed modulation format, namely differential polarization-phase-shift keying, are analyzed. It is found that, in the linear regime, the Gaussian approximation is reasonably accurate in estimating the BER. In the nonlinear regime, the noise of the demodulated signal is dominated by the effect of nonlinear phase noise and obeys chi-squared statistics with a degree of freedom of one. Gaussian approximation underestimates the BER in the nonlinear regime. A two-step BER estimation method accounting for both linear beat noise and nonlinear phase noise is proposed. The effectiveness of this two-step BER estimation method at all power levels has been established by comparison with direct error counting.

© 2005 Optical Society of America

## 1. Introduction

Direct detection differential polarization-phase-shift keying (DPolPSK), also called differential Jones-vector-shift keying (DJSK) has recently been proposed for high-spectral efficiency optical communications [1]. The schematic of quaternary DPolPSK transmitter and receiver is shown in Fig. 1(a). It is a constant-intensity modulation format that encodes information on both the polarization and phase of lightwave. DPolPSK differs from the conventional differential polarization-shift keying (DPolSK) in that DPolSK encodes information only on the polarization of lightwave [2]. Optical demodulation of DPolPSK is the same as differential phase-shift keying (DPSK). Dynamic polarization control is not required at the receiver. This is accomplished by using multilevel detection. This receiver simplification is especially important for wavelength-division multiplexing (WDM) systems, where dynamic polarization control should be performed on a per channel basis since the SOP at each wavelength is generally different. In [1], the conventional Gaussian approximation for BER estimation using demodulated signal is found to fairly accurate in the linear regime, but generally underestimate the BER in the nonlinear regime. However, no physical explanations were provided. In this paper, the underlying mechanism for the effectiveness of the Gaussian approximation in the linear regime and its failure in the nonlinear regime are explained by understanding the noise statistics in both regimes. Moreover, a two-step BER estimation method is proposed to accurately estimate the BER in all the regimes.

## 2. Noise statistics and BER estimation in the linear regime

Since the optical demodulator and receiver for DPolPSK is the same as that for DPSK, it is somewhat surprising that the Gaussian approximation works well for DPolPSK because it has been found to be not applicable for DPSK BER estimation when balanced receivers are used [3]. In this section, using chi-squared statistics model we present a rigorous analysis of the noise statistics for DPolPSK in the linear regime to understand the underlying mechanism for the effectiveness of Gaussian approximation. The chi-squared model was used for DPSK and proven to be accurate [4]. The mathematical model of DPolPSK optical demodulator and receiver with optical preamplification is shown in Fig. 1(b). The ideal optical bandpass filter has a bandwidth of *M*/*T*, where *T* is the symbol interval and *M* is assumed to be an integer for the simplicity. The electrical filter is an integration and dump circuit. The simple analytical solution can be obtained under these ideal assumptions so that it is viable to evaluate the Gaussian approximation at the low BER, say 10^{-9}. The more practical filters Gaussian optical filters and Bessel electrical filters have been used in [1] to evaluate the Gaussian approximation using direct error counting for BER larger than 10^{-5}. In a conveniently normalized form following that used in Eq. (2.116) in [4], for DPolPSK the decision voltage at the output is given by

where *n*_{p}
is the number of photons per bit, Γ and Λ denote outputs at the constructive and destructive ports of the delay interferometer (for DPSK, when the phase difference is zero, Γ=1 and Λ=0). *n*_{‖}
and *n*
_{⊥} are the complex ASE noise in the parallel and orthogonal polarization state; *n* and *n*′ stand for the noise at present and the previous symbol slot. In Eq. (1), the power ratio between the parallel and orthogonal polarization state is assumed to be 1:2. The four noise terms (*n*±*n*′)/2 are mutually independent zero-mean Gaussian random variables with independent in-phase and quadrature components of the same variance *σ*
^{2}=*M·n*_{sp}
/2, where *n*_{sp}
is the noise enhancement factor and is assumed to be 1 in this paper corresponding to an ideal optical amplifier. Under these assumptions, like for DPSK [4, Eq. (8.39)], the decision voltage, i.e. the demodulated DPolPSK signal at each photodetector, *v*
_{Γ} or *v*
_{Λ}, obeys (non-central) chi-squared distribution with probability density function

where the degree of freedom is *n*=2*pM*, variance is *σ*
^{2} and noncentrality parameter is ${m}_{c}^{2}$. When polarization filtering is used, *p*=1. Otherwise, *p*=2. For DPolPSK, *p* is always 2. *I*
_{n/2-1}() denotes the modified Bessel function of order *n*/2-1. For the upper level of the middle eye in the demodulated DPolPSK signal, the distributions of *v*
_{Γ} and *v*
_{Λ} are *pdf*
_{Γ} (*x*)=*pdf* (*x*|2
_{p}*M,n*_{sp}
/2,8*n*_{p}
/3) and *pdf*
_{Λ }(*x*)=*pdf* (*x* | 2*pM*, *n*_{sp}
/2,4*n*_{p}
/3), which correspond to constructive interference of orthogonal polarization state and destructive interference of parallel state at the Γ port. The lower level of the middle eye is symmetric with the upper level. Using balanced detection, the symbol error rate (SER) of middle eye can be obtained by

Double integration is carried out to calculates the error probability that *v*
_{Γ} is less than v_{Λ}. When *p*=1 and *M*=1, the integration in (3) can be represented by the sum of a Marcum’s Q function and a modified Bessel function. The results calculated using the exact Eq. (3) and using the simple Gaussian approximation with the same mean and variance are compared and are shown in Fig. 2(a). In the figure, *p*=2 and *M*=2 (a realistic value) are used. The close agreement between chi-squared statistics and Gaussian approximation proves the effectiveness of Gaussian approximation for the middle eye.

The BER for the upper eye depends on the choice of the threshold. For a given threshold, *thre*, the SER of the upper eye can be obtained by

where *pdf*
_{Γ} (*x*) and *pdf*
_{Λ} (*x*) are the distributions of *v*
_{Γ} and *v*
_{Λ} for the lower level of the upper eye (i.e., upper level of the middle eye), and *pdf*
_{Γ}′ (*x*) and *pdf*
_{Λ}′ (*x*) are those for the upper level of the upper eye. *pdf*
_{Γ} (*x*) and *pdf*
_{Λ} (*x*) are the same as those in Eq. (3). *pdf*
_{′}Γ(*x*)=*pdf*(*x*|2
_{p}*M,n*_{sp}
|2,4*n*_{p}
) and *pdf*
_{′}Λ(*x*)=*pdf*(*x*|*pM,n*_{sp}
/2,0). Owing to symmetry, the result for the lower eye is the same as that for the upper eye. The optimal threshold for Gaussian approximation is different from that for exact chi-squared statistics. The calculated BER for the upper eye in DPolPSK using both methods with different optimal thresholds are surprisingly close, as shown in Fig. 2(b).

Despite the close relation between DPolPSK and DPSK, the above comparison between Gaussian approximation and chi-squared statistics model proves that the conventional Gaussian estimation method is fairly accurate for DPolPSK when beat noise is dominant or in the linear regime after transmission. The underlying mechanism for this difference is related to the nature of the signals at the constructive and destructive ports as follows. Differential detection recovers the data by comparing *v*
_{Γ} to *v*
_{Λ}. In DPSK, the deconstructive port is close to ‘0’, where the Gaussian statistics differ significantly from the chi-squared statistics. In contrast, for the middle eye in DPolPSK, both the constructive and destructive ports have significant power. In this case, the difference between chi-squared and Gaussian statistics is small and hence the error using Gaussian approximation is much smaller even if balance detection is used with a threshold at null. On the other hand, for the upper and lower eye, the decision thresholds are non-zero. The BER obtained using Gaussian approximation is again found to be reasonably accurate even though the optimal threshold for Gaussian approximation is different from the actual threshold based on chi-squared estimation. This is similar to what happened in OOK although OOK uses a single-ended receiver and DPolPSK uses a balanced receiver [5, 6].

According to the results shown in Fig. 2, the SER of three eyes are almost equal, which also validates the choice of the 1:2 power ratio. This choice of power ratio contradicts the common wisdom that the equal spacing is not optimal for multilevel detection when ASE-signal beat noise is dominant [7]. The reason is because each symbol in DPolPSK has the same intensity although multilevel detection is used and therefore the dominate signal-ASE beat noise is the same for all demodulated signal levels. The difference between demodulated signal levels comes from the difference between the SOPs of lightwave, not the intensity.

## 3. Noise statistics and BER estimation in the nonlinear regime

The discussion above assumes that ASE-signal beat noise is dominant, which is the case when the signal power is low. As transmission power increases, the impact of optical nonlinearity needs to be considered. In DPolPSK, nonlinear phase noise is expected to be the dominant nonlinear mechanism in the high-power regime. Nonlinear phase noise may substantially change the noise statistics of demodulated signal level. In this section, the noise statistics of demodulated signal level when nonlinear phase noise is dominant are analyzed. Using the more accurate statistics, a new approach is proposed to accurately estimate the BER at all power levels.

Nonlinear phase noise is the result of the interaction between SPM and optical intensity fluctuation due to ASE in optical amplifiers. In comparison with DPSK [8], nonlinear phase noise in DPolPSK is caused by the fluctuation of overall intensity of both polarization states. Since the magnitude of SPM does not depend on the SOP and each symbol in DPolPSK has the same intensity, the magnitude of nonlinear phase noise for every symbol is the same although the SOP of lightwave may change from symbol to symbol. As a result, the photocurrent at the output of balanced detector in DPolPSK can be written as

where *θ*
_{1}(*t*) and *θ*
_{2}(*t*) are 0 or *π, R* is the responsivity of detector, *I*_{A}
and *I*_{B}
are the optical intensity of adjacent symbols, and Δ*θ*_{D}
is the differential phase error of adjacent symbols due to nonlinear phase noise. Using Eq. (5), the photocurrent error due to nonlinear phase noise can be obtain as

The approximation in Eq. (6) is valid when Δ*θ*_{D}
is small. In other words, it is more accurate when the BER is low. Since the approach based on Eq. (6) is fairly accurate at BER=10^{-3} as shown below, it is reasonable to assume the approach is still effective at the lower BER. One immediate conclusion according to Eq. (6) is that the nonlinear phase noise has different impact on the four levels. The +3 and -3 levels are three times noisier than +1 and -1 levels and therefore, the SER of the upper and lower eye is higher than that of the middle eye in the nonlinear regime. This is in contrast to the linear regime where four levels are almost equally noisy. Therefore, the optimal power ratio in the nonlinear regime is greater than 1:2. Additionally, if nonlinear phase noise Δ*θ*_{D}
is assumed to be a Gaussian random variable, Δ*i* of each individual level will obey chi-squared statistics with a degree of freedom of one. The tail of chi-squared probability density function with a degree of freedom of one drops much slower than that of Gaussian. This explains why the BER estimation using Gaussian approximation underestimated the BER in the nonlinear regime [1]. Since it has been shown that Gaussian statistics works fairly well in the linear regime, if the noise due to nonlinear phase noise and linear beat noise can be assumed independent, the decision voltage in the nonlinear regime can be better approximated by the sum of a chi-squared random variable with a degree of freedom of one and a Gaussian random variable.

Using the more accurate chi-squared plus Gaussian statistics model, we proposed a new BER estimation method for DPolPSK. We note that both chi-squared and Gaussian statistics itself can be sufficiently described by the mean and variance. To avoid sophisticate probability density function fitting which generally requires more data points, a simple two-step method is proposed. Assume that the Gaussian random variable, *g*, has a mean of <*g*> and a variance of
^{2}*σ*_{g}
and the chi-squared random variable, *c*, has a mean of <*c*> and a variance of
^{2}*σ*_{c}
. For chi-squared statistics with a degree of freedom of one, $<c>=\sqrt{{\sigma}_{c}^{2}\u20442}$. In the first step, the nonlinear coefficient of fiber is set to zero and only the linear beat noise is simulated. The variance obtained in the first step,
^{2}*σ*_{1}
is assumed to be
^{2}*σ*_{g}
. In the second step, the simulation is performed along with the fiber optical nonlinearity. The variance obtained in the second step is
^{2}*σ*_{2}
, and following the statistical independence assumption, ${\sigma}_{c}^{2}={\sigma}_{2}^{2}-{\sigma}_{1}^{2}.<g>={m}_{2}\pm \sqrt{{\sigma}_{c}^{2}\u20442}$, where *m*
_{2} is the mean obtained in the second step; and the positive (negative) sign is used when the two positive (negative) levels are considered. Using this approach, the statistics of chi-squared and Gaussian random variable can be obtained directly from mean and variance in the simulation, like in the traditional Gaussian BER estimation method. The extra simulation time in the first step is only marginal, since optical nonlinearity is not considered in the simulation. The results are summarized in Fig. 3. The transmission link in the simulation comprises 45 dispersion managed fiber spans. Each span includes 100 km standard single-mode fiber, dispersion compensating fiber and Erbium-doped fiber amplifiers between the fibers. The parameters of fiber and amplifier are the same to that used in Fig. 6 of [1]. As discussed before, Gaussian approximation is fairly accurate in the linear regime and underestimates the BER in the nonlinear regime. Similarly, if only chi-squared statistics of a degree of freedom of one is used, it is fairly accurate in the highly nonlinear regime, but always overestimates the BER in the rest of power level. The proposed two-step method accounting for both chi-squared and Gaussian statistics can fairly accurately estimate the BER at all power levels.

There are issues on the assumptions used in the proposed two-step method. For example, a more careful study shows that nonlinear phase noise in DPSK is close to, but not exactly Gaussian random variable [9]. In addition, to the author’s best knowledge, there is no solid proof that in DPSK nonlinear phase noise is independent to linear ASE noise. Moreover, in some sense, the interaction between optical nonlinearity and dispersion is assumed not to be dominant in the system so that the separate two-step can be used. To more accurately address the interaction between optical nonlinearity and dispersion, the probability density function fitting to a statistics of chi-squared plus Gaussian may be helpful. Nevertheless, the close agreement between the proposed two-step method and direct error counting validates the used assumptions in the practically interested system condition.

It should be noted that the proposed two-step method can also be used to estimate the BER based on eye diagram measurement in experiments. The contribution from linear phase noise in the nonlinear regime can be obtained by extrapolating the parameters from measurements in the linear regime. This is because experimentally measured mean and variance of the demodulated signal and OSNR in the linear regime are sufficient to calculate the mean and variance due to linear phase noise at high power levels for the demodulated signals. It is also noted that Eq. (6) is valid for DPSK as well. Therefore, when nonlinear phase noise is dominant, the demodulated signal in DPSK can also be directly used to estimate BER using chi-squared model as shown in Fig. 3, as opposed to using the not-measurable phase-Q [10].

## 4. Conclusion

To conclude, the noise statistics of demodulated signal of DPolPSK are analyzed. By comparison to the exact chi-squared model, the Gaussian approximation is found to be reasonably accurate in estimating the BER at the range of interest, 10^{-9} to 10^{-3}, in the linear regime. The physical mechanism for the effectiveness of Gaussian approximation is explained in comparison with DPSK for the middle eye in DPolPSK and OOK for the upper (lower) eye. In the nonlinear regime, the decision voltage noise due to nonlinear phase noise obeys chi-squared statistics with a degree of freedom of one and Gaussian approximation generally underestimates the BER. The two-step BER estimation method including both linear beat noise and nonlinear phase noise is proposed. By comparison to direct error counting at high BER (~10^{-3}), the two-step method is found to be effective at all power levels.

## References and links

**1. **Y. Han and G. Li, “Direct detection differential polarization-phase-shift keying based on Jones vector,” Opt. Express12, 5821–5826 (2004). http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-24-5821 [CrossRef] [PubMed]

**2. **S. Benedetto and P. Poggiolini, “Theory of polarization shift keying modulation,” IEEE Trans. Commu. **40**, 708–721 (1992). [CrossRef]

**3. **P. J. Winzer, S. Chandrasekhar, and H. Kim, “Impact of filtering on RZ-DPSK reception,” IEEE Photon. Technol. Lett. **15**, 840–842 (2003). [CrossRef]

**4. **G. Jacobsen, *Noise in digital optical transmission systems*, (Artech House, Boston, MA, 1994).

**5. **D. Marcuse, “Derivation of analytical expressions for the bit-error probability in lightwave systems with optical amplifiers,” J. Lightwave Technol. **8**, 1816–1823 (1990). [CrossRef]

**6. **P. A. Humblet and M. Azizoglu, “On the bit error rate of lightwave systems with optical amplifiers,” J. Lightwave Technol. **9**, 1576–1582 (1991) [CrossRef]

**7. **S. Walklin and J. Conradi, “Multilevel signaling for increasing the reach of 10 Gb/s lightwave systems,” J. Lightwave Technol. **17**, 2235–2248 (1999). [CrossRef]

**8. **J. P. Gordon and L. F. Mollenauer, “Phase noise in photonic communications systems using linear amplifiers,” Opt. Lett. **15**, 1351–1353 (1990). [CrossRef] [PubMed]

**9. **Keang-Po Ho, “Asymptotic probability density of nonlinear phase noise,” Opt. Lett. **28**, 1350–1352 (2003). [CrossRef] [PubMed]

**10. **X. Wei, X. Liu, and C. Xu, “Numerical simulation of the SPM penalty in a 10-Gb/s RZ-DPSK system,” IEEE Photon. Technol. Lett. **15**, 1636–1638 (2003). [CrossRef]