## Abstract

In this paper, as an attractive alternative to the conventional discrete Fourier transform (DFT) based orthogonal frequency division multiplexing (OFDM), discrete cosine transform (DCT) based OFDM which has certain advantages over its counterpart is studied for optical fiber communications. As is known, laser phase noise is a major impairment to the performance of coherent optical OFDM (CO-OFDM) systems. However, to our knowledge, detailed analysis of phase noise and the corresponding mitigation methods for DCT-based CO-OFDM systems have not been reported yet. To address these issues, we analyze the laser phase noise in the DCT-based CO-OFDM systems, and propose phase noise estimation and mitigation schemes. Numerical results show that the proposal is very effective in suppressing phase noise and could significantly improve the performance of DCT-based CO-OFDM systems.

© 2009 Optical Society of America

## 1. Introduction

Thanks to the inherent immunity to chromatic dispersion (CD) and polarization mode dispersion (PMD) in the optical fiber channel which cause severe inter-symbol interference (ISI), the coherent optical orthogonal frequency division multiplexing (CO-OFDM) is regarded as one of the most promising technologies for high speed optical fiber communications [1]–[3]. Till now, discrete Fourier transform (DFT) based CO-OFDM has been widely researched [3]–[5]. Another effective multi-carrier modulation scheme, which is named discrete cosine transform (DCT) based OFDM also gained interests from researchers due to some attractive advantages over its counterpart. DCT-based OFDM can achieve bandwidth advantage and has better BER performance than that of DFT-based OFDM systems in the presence of carrier frequency offset due to the energy compaction property of DCT [6, 7]. Particularly, DCT-based OFDM can completely avoid inphase/quadrature (IQ) imbalance problem in DFT-based OFDM systems when one-dimensional modulation formats, e.g. pulse amplitude modulation (PAM), are used. Moreover, the fast DCT algorithms proposed could provide fewer computational steps than fast Fourier transform (FFT) algorithm [8, 9]. Based on these features, DCT-based OFDM can be a promising transmission scheme for optical fiber communication. However, to the best of our knowledge, the DCT-based CO-OFDM system has not been carefully investigated yet. As is known, CO-OFDM systems are very sensitive to laser phase noise, so the studies of the phase noise and the corresponding mitigation methods for the DCT-based CO-OFDM system that are still left untouched, are very crucial.

In this paper, we study the DCT-based CO-OFDM for optical fiber communications, investigate the laser phase noise in the DCT-based CO-OFDM system, and propose phase noise estimation and mitigation scheme which can also perform channel compensation jointly. As far as we know, it is the first time that phase noise estimation and mitigation schemes are presented for the DCT-based CO-OFDM system.

## 2. DCT-based CO-OFDM system

The baseband equivalent model of the CO-OFDM transmission system analyzed is shown in Fig. 1. A DCT-based CO-OFDM system includes an OFDM transmitter, an optical link and an OFDM receiver, which is very similar to the DFT-based CO-OFDM system. However, in a DCT-based system, in stead of performing the inverse DFT and DFT at the transmitter and receiver, inverse DCT (IDCT) and DCT are performed to the OFDM data blocks. Here, all the linear effects in the fiber channel are considered and bold face letters are used to denote vectors and matrices. **d** stands for the transmitted data bits which are mapped into **a** according to the modulation format chosen. For the *i*th OFDM symbol,

where the superscript *T* stands for the transpose and *N* is the number of subcarriers. Perform IDCT to **a**
_{i} to generate digital domain OFDM signals and we can obtain

where **D** is the DCT matrix and **D**=[**d**
_{0}
**d**
_{1} ⋯ **d**
_{N-1}]. Here ${\mathbf{d}}_{m}={\left[\begin{array}{cccc}\multicolumn{1}{c}{\beta \left(0\right)}& \multicolumn{1}{c}{\beta \left(1\right)\mathrm{cos}\genfrac{}{}{0.1ex}{}{\pi \left(2m+1\right)}{2N}}& \multicolumn{1}{c}{\cdots}& \multicolumn{1}{c}{\beta \left(N-1\right)\mathrm{cos}\genfrac{}{}{0.1ex}{}{\pi \left(2m+1\right)\left(N-1\right)}{2}}\end{array}\right]}^{T}$, *m*=0,1, ⋯,*N*-1 and

Then guard sequence which does not carry useful information is inserted. Guard sequence is employed to make the DCT-based CO-OFDM system immune to ISI. In the DCT-based OFDM system, the guard sequence of length 2*ν* is inserted before and after the information symbol **x**
_{i} in order to eliminate ISI. At the receiver, the 2*ν* received samples corresponding to the guard sequence are discarded before DCT. Suppose that the channel memory length is *L* and in order to mitigate ISI completely, the length of the guard sequence must satisfy that *ν*≥ ⌈*L*/2⌊ where ⌈·⌊ means the integer no less than *L*/2. At the same time, in order to diagonalize the channel matrix by DCT, the guard sequence should be chosen as symmetric extensions of the information sequence on both ends [6], which means that the prefix and the suffix of length ν are added to **x**
_{i}. So the transmitted OFDM symbol after the insertion of the guard sequence is described as

where

*V _{ν}* is the

*ν*×

*ν*anti-identity matrix,

**I**

_{N}is the

*N*×

*N*identity matrix and

**0**

_{ν×(N-ν)}is the

*ν*×(

*N*-

*ν*) zero matrix. For example, let

**x**

_{i}=[

*x*(0)

_{i}*x*(1) ⋯

_{i}*x*(7)],

_{i}*ν*=2, then the transmitted OFDM symbol is [

*x*(1)

_{i}*x*(0)

_{i}*x*(0)

_{i}*x*(1) ⋯

_{i}*x*(7)

_{i}*x*(7)

_{i}*x*(6)]. According to the analysis about the guard sequence above, the structure of the DCT-based OFDM symbol transmitted is shown in Fig. 2. Let

_{i}*w*(

*t*) denote the amplified spontaneous emission (ASE) noise induced by the optical amplifiers (OAs) periodically placed in the optical link, which is considered as the additive Gaussian noise [10].

*ϕ*(

*t*) is the phase noise accounting for

*ϕ*(

_{T}*t*) introduced in the up-conversion at the transmitter and

*ϕ*(

_{R}*t*) introduced in the down-conversion at the receiver. Before the receiver, a prefilter should be employed to realize the diagonalization of the channel matrix by DCT, whose design will be discussed in the next section. Sample the

*i*th received OFDM symbol

*r*(

_{i}*t*) and we can obtain the baseband received vector

**r**

_{i}=[

*r*(0)

_{i}*r*(1) ⋯

_{i}*r*(

_{i}*N*-1)] with the elements presented as

where *h _{i}*(

*l*) is the

*l*th coefficient of the channel impulse response (CIR) which has a memory of 2

*ν*for the

*i*th symbol.

## 3. Proposed phase noise and channel compensation

By employing a proper prefilter and designing the guard sequence according to the method as shown in Eq. (5), the channel matrix can be diagonalized by DCT. The prefilter could be a finite impulse response (FIR) filter which is designed to make CIR symmetric. After the prefilter, the coefficients of CIR should satisfy

The design of the prefilter can be described as the following optimization problem [6, 11]

where MSE denotes the channel-filtering mean square error, **h̃**_{i}=[*h _{i}*(0)

*h*(1) ⋯

_{i}*h*(

_{i}*ν*)]

^{H},

**Ĩ**is the symmetric stacking matrix and defined as

**R**
_{i} is a positive-definite matrix determined by the original CIR as well as the noise variance [11] and the superscirpt *H* denotes Hermitian transpose. From (8), the optimum h̃i can be obtained, which is the eigenvector of **Ĩ**^{T}
**R**
_{i}
**Ĩ** corresponding to its minimum eigenvalue. Then the optimum symmetric CIR **h**
_{i}=[*h _{i}*(-

*ν*)

*h*(-

_{i}*ν*+1) ⋯

*h*(

_{i}*ν*)] is obtained by

**h**

_{i}=

**Ĩ**

**h̃**

_{i}. Finally, the optimum prefilter coefficients are determined as [11]

where **R**
^{yy}
_{i} and **R**
^{yx}
_{i} are the output-input cross-correlation and the output autocorrelation matrices for the *i*th OFDM symbol respectively, which both relate to the original channel matrix [11].

So, at the receiver, after removing the guard sequence and performing DCT on the received symbol by assuming a perfect DCT window and frequency synchronization, we obtain the symbol in the transform domain **y**
_{i}=[*y _{i}*(0)

*y*(1) ⋯

_{i}*y*(

_{i}*N*-1)] with the elements as

$$\times \mathrm{cos}\left(\genfrac{}{}{0.1ex}{}{\pi \left(2m+1\right)n}{2n}\right)+{W}_{i}\left(k\right),k=0,1,\cdots ,N-1.$$

where

and *H _{i}*(

*n*) (

*n*=0,1, ⋯,

*N*-1) are the main diagonal elements of the diagonal matrix Λ

_{i}=

**DH**

^{eqv}

_{i}

**D**

^{T}, i.e.,

*H*(

_{i}*n*)=Λ

*(*

_{i}*n,n*), where

**H**

^{eqv}

_{i}is the

*N*×

*N*equivalent channel matrix for the ith OFDM symbol which is determined by the coefficients of CIR [

*h*(-

_{i}*ν*) ⋯

*h*(0) ⋯

_{i}*h*(ν)] and the design of the guard sequence [6]. Define that

_{i}Then we obtain

$$+\genfrac{}{}{0.1ex}{}{\beta \left(k\right)}{2}\left[\sum _{n=0,n\ne k}^{N-1}\beta \left(n\right)\left({\Theta}_{i}\left(n+k\right)+{\Theta}_{i}\left(n-k\right)\right){H}_{i}\left(n\right){a}_{i}\left(n\right)\right]+{W}_{i}\left(k\right).$$

From Eq. (14), we can find that the phase noise causes a rotation of the signal constellation and a loss of orthogonality among the subcarriers, i.e. the inter-carrier interference (ICI). Let

which represents the ICI. Following [12], for a large number of subcarriers, i.e. a large *N*, *ξ _{i}*(

*k*) can be approximated as Gaussian noise.

The phase rotation in DCT-based CO-OFDM, which is different from that in DFT-based CO-OFDM, is no longer a common phase error (CPE), but varies from one subcarrier to another. According to Eq. (13), Θ_{i}(*k*)’s are the scaled DCT coefficients of the phase noise and for the lasers commonly employed in CO-OFDM systems, Θi(k) satisfies that when *k*≠0, Θ(k)≪Θ(0). So we can approximate the phase rotation with Θ_{i}(0), and rewrite Eq. (14) as

where *ζ _{i}*(

*k*)=

*ξ*(

_{i}*k*)+

*W*(

_{i}*k*). The matrix-vector form of Eq. (16) can be expressed as

where

Here, diag{**x**} is the diagonal matrix with the vector x as the diagonal. Use the first OFDM symbol as preamble. As an initialization, according to the least square (LS) criterion [13], utilize the preamble to estimate the diagonal elements of the matrix Γ_{1} by

Since the optical fiber channel varies very slowly (<10 KHz), Λ_{i} can be considered constant (=Λ_{1}) in an OFDM block. Define ΔΘ_{i}(0)=Θ_{i}(0)/Θ_{1}(0). Utilizing the received symbol **y**
_{i} and the pilots, ΔΘ_{i}(0) can be estimated as

where *k _{p}*=

*k*

_{0}+

*pd*(

_{p}*p*=0,1, ⋯,

*M*-1),

*k*

_{0}is the index of the first subcarrier which is used as a pilot and

*d*is the density of the pilots. Estimate the diagonal elements of Γ

_{p}_{i}by

Perform zero forcing equalization on the received symbol **y**
_{i} and derive

where the superscript * means conjugation, and then make decisions on *z _{i}*(

*k*)’s to form the detected sequence

**a**̂

_{i}=[

*a*̂

_{i}(0)

*a*̂

_{i}(1) ⋯

*a*̂

_{i}(

*N*-1)]

^{T}. After de-mapping,

**d**̂

_{i}is obtained finally.

## 4. Numerical results

Standard Monte Carlo simulation is employed to evaluate the performance of the proposed joint phase noise and channel compensation scheme. We consider a CO-OFDM system with a sampling-rate of 10 GS/s (Giga Samples per second) and utilizing 256 subcarriers at the transmitter. 32 samples are used as guard sequence per OFDM symbol and the modulation format is QPSK. So a total bit rate of 17.8 Gb/s is resulted in. 10 pilots are inserted equally in every OFDM symbol. CD is considered as the factor of inducing ISI in the optical fiber channel.

It is shown in section 3 that the phase rotation varies from one subcarrier to another for the DCT-based CO-OFDM system. To simplify the estimation as well as mitigation of the phase noise, we use a CPE-like approximation for the phase rotation. Here we analyze the scaled DCT coefficient Θ(*k*) further to prove the validity of this approximation. Figure 3 shows 20log_{10}(*E*(‖Θ(*k*)‖/‖Θ(0)‖)) for *k*=0,1, ⋯,50 and different laser linewidths which produce mild, moderate and serious phase noise respectively. Here, *E*(·) denotes the average. We can find that for all the cases, Θ(*k*) satisfies when *k*≠0, Θ(*k*)≪Θ(0).

Figure 4 and Fig. 5 analyze the performance of the DCT-based CO-OFDM system employing the proposed joint phase noise and channel compensation scheme by presenting the curves of BER versus OSNR. We consider the cases that the overall end-to-end OFDM link has a memory length of 6 and 8 which correspond to a residual CD of 1600 ps/nm and 2000 ps/nm respectively. For both cases, the numerical results for the laser linewidths of 100 KHz, 400 KHz, 700 KHz and 1 MHz which induce mild, medium and serious phase noise respectively are given. The results that only channel compensation is performed for different cases are also provided for comparison. We can find that the phase noise could cause serious BER floor without compensation. But it can be mitigated effectively and significant improvement of BER performance can be achieved if the proposed algorithm is applied.

## 5. Conclusion

In this paper, DCT-based CO-OFDM is studied and an effective phase noise mitigation scheme is proposed based on the theoretical analysis of the phase noise in the transform domain, which can also realize channel compensation jointly. Numerical results demonstrate that the proposed algorithm can largely improve the BER performance of the DCT-based CO-OFDM system.

## Acknowledgment

The authors would like to thank the reviewers for their constructive suggestions that help improve the manuscript. This work was supported by China Postdoctoral Science Foundation under Grant 20080430279 and National Natural Science Foundation of China under Grant 60907029.

## References and links

**1. **I. B. Djordjevic and B. Vasic, “Orthogonal frequency division multiplexing for high-speed optical transmission,” Opt. Express **14**, 3767–3775 (2006).
[CrossRef] [PubMed]

**2. **W. Shieh, H. Bao, and Y. Tang, “Coherent optical OFDM: theory and design,” Opt. Express **16**, 841–859 (2008).
[CrossRef] [PubMed]

**3. **J. Armstrong, “OFDM for optical communications,” J. Lightw. Technol. **27**, 189–204 (2009).
[CrossRef]

**4. **Q. Yang, S. Chen, Y. Ma, and W. Shieh, “Real-time reception of multi-gigabit coherent optical OFDM signals,” Opt. Express **17**, 7985–7992 (2009).
[CrossRef] [PubMed]

**5. **S. L. Jansen, I. Morita, T. C. W. Schenk, N. Takeda, and H. Tanaka, “Coherent optical 25.8-Gb/s OFDM transmission over 4160-km SSMF,” J. Lightw. Technol. **26**, 6–15 (2008).
[CrossRef]

**6. **N. Al-Dhahir, H. Minn, and S. Satish, “Optimum DCT-based multicarrier transceivers for frequency-selective channels,” IEEE Trans. Commun. **54**, 911–921 (s).
[CrossRef]

**7. **F. Gao, T. Cui, A. Nallanathan, and C. Tellambura, “Maximum likelihood based estimation of frequency and phase offset in DCT OFDM systems under non-circular transmissions: algorithms, analysis and comparisons,” IEEE Trans. Commun. **56**, 1425–1429 (2008).
[CrossRef]

**8. **W. H. Chen, C. H. Smith, and S. C. Fralick, “A fast computational algorithm for the discrete cosine transform,” IEEE Trans. Commun. **25**, 1004–1009 (1977).
[CrossRef]

**9. **Z. D. Wang, “Fast algorithm for the discrete W transform and for the discrete Fourier transform,” IEEE Trans. Acoust., Speech, Signal Processing **32**, 803–816 (1984).
[CrossRef]

**10. **C. Yang, F. Yang, and Z. Wang, “Iterative minimum mean square error equalization for optical fiber communication systems,” IEEE Photon. Technol. Lett. **19**, 1571–1573 (2007).
[CrossRef]

**11. **N. Al-Dhahir and J. M. Cioffi, “Efficiently computed reduced-parameter input-aided MMSE equalizers for ML detection: a unified approach,” IEEE Trans. Inf. Theory **42**, 903–915 (1996).
[CrossRef]

**12. **L. Tomba, “On the effect of Wiener phase noise in OFDM systems,” IEEE Trans. Commun. **46**, 580–583 (1998).
[CrossRef]

**13. **S. M. Kay, *Fundamentals of Statistical Signal Processing V. 1. Estimation Theory*, (Englewood Cliffs, NJ: Prentice-Hall PTR, 1993).