## Abstract

By exploiting the electromagnetic wave’s four-dimensional signal space, we find that for the additive white Gaussian noise channel, the modulation format with best sensitivity to be an 8-level format with 1.76 dB asymptotic gain over BPSK, for uncoded optical transmission with coherent detection. Low-complexity modulators are presented for the format, as well as an interpretation in terms of quantum-limited sensitivity.

©2009 Optical Society of America

## 1. Introduction

The recent interest in coherent optical transmission technologies has resulted in a remarkable progress, such as demonstration of transmission at 10 Gbaud with 4 bits/symbol using of-fline [1, 2] and online [3] post-processing of the data. The modulation format used in [1–3] is referred to as dual-polarization quaternary phase-shift keying (DP-QPSK), which is a 16-level format that can be seen as independent binary phase-shift keying (BPSK) modulation in the four quadratures of the optical field.

Many, if not most, coherent optical systems of practical interest, are well modeled as an additive white Gaussian noise (AWGN) channel. This includes for example links limited by the amplified spontaneous emission (ASE) noise from optical amplifiers, or links limited by shot noise from the local oscillator [4, Ch. 4]. For such optical links, the BPSK format are often believed to be the most power efficient one—i.e., the modulation format requiring the lowest signal-to-noise ratio (SNR) per bit to reach a given bit error rate (BER). However, we show in this paper that there are modulation formats with better power efficiency than BPSK, and also, which is most significant, with comparable, or less, complexity than DP-QPSK. In particular, we will present an 8-level modulation format that is asymptotically 1.76 dB more power efficient than BPSK. Disregarding pulse position modulation (which may provide unbounded capacity but is unpractical in high-speed links [5]) we provide numerical evidence that this format gives the best possible sensitivity for the uncoded optical channel. Power-efficient modulation formats, such as the one presented in this paper, are of practical importance in optical communications by enabling increased nonlinear tolerances, as well as of fundamental importance by providing ultimate sensitivity limits for the optical channel.

## 2. Global comparison of performance

It is well known that the problem of finding the *M*-ary modulation format with the least average power requirement to reach a given BER in an AWGN environment is equivalent (in the limit of low BER) to the problem of placing *M* points so that their minimum distance is maximized under an average energy constraint. Alternatively, the minimum distance can be kept constant and the average energy minimized, which is in turn equivalent to packing *M* rigid spheres so that their average squared distance *E _{s}* from the origin is minimized. Furthermore, it can be shown that the bit error rate asymptotically becomes well approximated by the union bound [6, p. 195], and that the dominating term for the

*BER*depends on the signal power

*P*as $\mathrm{erfc}\left(\sqrt{P\gamma \left(R{N}_{0}\right)}\right)$, where

*R*is the bit rate and

*N*

_{0}is the (single-sided) noise spectral density. The

*asymptotic power efficiency*

*γ*is defined as [6, p. 220]

*γ*=

*d*

^{2}

_{min}/(4

*E*), where

_{b}*E*=

_{b}*E*/log

_{s}_{2}

*M*is the energy per bit, and

*d*

_{min}is the sphere diameter or, equivalently, the minimum (Euclidean) distance between constellation points. Observe that

*γ*, which is usually given in dB, depends on the constellation geometry only, not on the transmitted power. It is 0 dB for BPSK and QPSK, and it can therefore be interpreted as the

*sensitivity gain*over BPSK to transmit the same data rate. In this paper we will, for what we believe is the first time, present the maximum sensitivity gain for all constellation sizes

*M*≤32 and dimensions

*N*≤4.

Coherent systems have in the most general case a four-dimensional constellation space (*N*=4), corresponding to two quadratures in two polarization states. The constellation vectors are formed from the real and imaginary part of the electric field’s x and y polarization components as [7] (*E*
_{x,r},*E*
_{x,i},*E*
_{y,r},*E*
_{y,i}). As an example, the DP-QPSK format can be expressed (in normalized units) as the 16 levels *C*
_{1}={(±1,±1,±1,±1)}, allowing for any sign selection, and it has *d*
_{min}=2, *E _{s}*=4, and

*γ*=0 dB just as BPSK and QPSK.

Consider a constellation of *M* nonoverlapping spheres in *N*-dimensional space. To find the packing that minimizes the average squared distance from the origin is a geometric problem that can be solved by numerical optimization. One starts with *M* randomly positioned nonoverlap- ping spheres, which are then made to relax into a closely-packed cluster by suitable attractive and repelling forces. Unfortunately, there exist many packings that are locally optimal in this respect. Therefore the process is repeated for a large number of random initial conditions until the best packing emerges, which can be very time consuming. The sphere-packing problem has been addressed previously in the literature, mostly via such numerical optimization. Rigorous mathematical proofs of optimality have been obtained only in a few special cases. For example, optimum constellations for dimensions *N*=2 and *N*=3 were discussed in [8] and [9], respectively, and results for *N*=4 are available online [10]. We independently designed similar constellations ourselves, which support the results from these sources.

The results are expressed in Fig. 1, plotting the spectral efficiency *SE* vs. the sensitivity penalty 1/*γ* for the optimum constellations. Such a chart is the conventional way of comparing modulation formats [4, 6, 11] (possibly with a different normalization). Here we define the spectral efficiency to be the number of bits per symbol per polarization (i.e., per *dimension pair*, as suggested in [6, p. 219]), so that *SE*=log_{2}(*M*)/(*N*/2). This definition of *SE* will cause BPSK, QPSK, and DP-QPSK to have *SE*=2, since BPSK has dimension *N*=1. The leftmost points in this graph are thus the most power-efficient modulation formats, and we may note that for small *N* this occurs for *simplices*, i.e., the equilateral triangle (or 3-PSK format) for *N*=2 and the tetrahedron (*M*=4) for *N*=3. These modulation formats have received limited practical interest, due to the difficulty of (i) generating them and (ii) mapping bits to symbols when *M* is not a power of 2.

## 3. The PS-QPSK format

The first dimension for which the simplex is not the most power-efficient format is *N*=4. Instead, the overall optimum occurs for *M*=8, showing an improved asymptotic sensitivity of 1.76 dB (or 1.5 times) over BPSK. This 8-level modulation format consists of the levels *C*
_{2}={(±2,0,0,0), (0,±2,0,0), (0,0,±2,0), (0,0,0,±2)}. This normalization makes the amplitude the same as for the DP-QPSK format discussed above. This is the four-dimensional version of *biorthogonal signaling* [12, pp. 198–203], [13]. The constellation forms the vertices of a four-dimensional polytope known as the *cross-polytope*, or *16-cell*, since it is bounded by 16 tetrahedrons. It has been suggested previously to be used for signal modulation, see, e.g., [7,14], but it has so far not been recognized as the overall most power-efficient modulation format in four-dimensional space.

It is possible to directly implement *C*
_{2} in a conventional (see e.g. [2]) optical transmitter for DP-QPSK, although it would require three modulation levels (-2,0,2). However, we will consider also a few other representations of the *C*
_{2} format, that might give rise to simpler transmitter structures. By a 45° phase rotation, the constellation may be expressed as *C*′_{2}=√2{(±1,±1,0,0), (0,0,±1,±1)}, which is QPSK transmission in *either* the x *or* the y polarization. Thus, two bits are transmitted via QPSK and the third bit determines whether the x or y polarization is used. Therefore, we will refer to this format as polarization-switched QPSK (PS-QPSK). A schematic transmitter for PS-QPSK is shown in Fig. 2 (a), showing a standard QPSK transmitter followed by a polarization modulator. Moreover, a 45° polarization rotation gives another way of expressing the PS-QPSK format: *C*
_{2}″=±{(1,1,1,1), (1,1,-1,-1), (1,-1,1,-1), (1,-1,-1,1)}, revealing it to be a subset of the DP-QPSK (*C*
_{1}) levels; namely, those having an even number of minus signs. This means that the PS-QPSK format can be obtained from the conventional DP-QPSK transmitter by using two XOR gates, which will force the driving bits *b*
_{1},*b*
_{2},*b*
_{3},*b*
_{4} to have even parity, as shown in Fig. 2 (b).

## 4. Bit- and symbol error rates

We will now compare the PS-QPSK and DP-QPSK formats in terms of bit- and symbol error rates. The DP-QPSK constellation points form the vertices of a four-dimensional hypercube, and as it can be regarded as four parallel independent BPSK channels, its BER will be equal to that of BPSK, i.e., ${\mathrm{BER}}_{\mathrm{DP}-\mathrm{QPSK}}={\mathrm{BER}}_{\mathrm{BPSK}}=\mathrm{erfc}\left(\sqrt{{E}_{b}\u2044{N}_{0}}\right)\u20442$. The SER of PS-QPSK is [12, p. 201]

where Eq. (2) is an expression that facilitates numeric evaluation of the integral [13]. To get the BER we need to consider the bit-to-symbol mapping. The eight levels of the PS-QPSK format are not possible to Gray code, since each point has 6 nearest neighbors. The best one can do is to encode the levels so that the pairs that are furthest away from each other have inverted binary code words, which is achieved by the transmitter in Fig. 2 (b) (although not with the transmitter in Fig. 2 (a)). In such a situation, the six most likely symbol errors will have one or two bits wrong, of the transmitted three bits. Ignoring the seventh possible symbol error, which is much less probable, *BER*
_{PS-QPSK}≈*SER*
_{PS-QPSK}/2. (An exact expression is given in [12, p. 203].) The BER for PS-QPSK and BPSK/DP-QPSK is shown in Fig. 3. The required *E _{b}*/

*N*

_{0}at a BER of 10

^{-3}is 5.82 dB for PS-QPSK and 6.79 dB for BPSK, while at 10

^{-9}we have 11.04 dB for PS-QPSK and 12.55 dB for BPSK. As the BER decreases, the

*E*/

_{b}*N*

_{0}difference approaches 10log

_{10}(3/2)=1.76 dB.

## 5. Sensitivity limits

We have seen from the above that PS-QPSK can give up to 1.76 dB of improved sensitivity over BPSK. We will now consider how much this improves the ultimate quantum-limited sensitivity of a coherent transmission system. As a specific example, we consider a coherent optically amplified system limited by ASE noise from inline optical amplifiers. We also assume a homodyne receiver with phase and polarization diversity. Such a system has been shown to be well described by the AWGN model, with the SNR given by [4, Ch. 3.4, Table 1],

where *n _{b}* is the average number of received photons per bit,

*N*is the number of amplifiers in the link and

_{A}*n*is the spontaneous emission noise factor from the inline amplifiers. In fact, Eq. (3) holds for both heterodyne and homodyne receivers limited by ASE noise. Since

_{sp}*N*>1, we see that in the limiting case (a single amplifier with a 3 dB noise figure), the sensitivity in terms of number of photons per bit is given directly by

_{a}n_{sp}*E*/

_{b}*N*

_{0}. For

*BER*=10

^{-9}, this translates into the well-known [4, 11] sensitivity of 18 photons per bit for BPSK. However, from Fig. 3 we see that PS-QPSK improves this sensitivity to 13 photons per bit. At

*BER*=10

^{-3}, we get 4.5 photons per bit for BPSK (which was given in dB units in [4]) and 3.8 photons per bit for PS-QPSK.

Since the SNR in the shot-noise limit is 3 dB higher than for the ASE limit (assuming the same number of photons per bit and unity photodetector quantum efficiency [4, Erratum, Table 1]), the shot-noise limited sensitivity in terms of photons per bits is half of the above values. Since no more power efficient modulation formats are possible, according to Fig. 1, we believe the above values provide the ultimate quantum-limited sensitivities for optical coherent receivers without coding.

## 6. Conclusions

We have shown that the overall most power-efficient modulation format for uncoded, coherent optical systems is the PS-QPSK format, or four-dimensional biorthogonal signaling, which has an asymptotic gain relative to BPSK (and DP-QPSK) of 1.76 dB. This can be understood as follows: half the symbols of DP-QPSK are used, in such a way that the power can be decreased to half without reducing the minimum distance of the constellation, giving a factor of 2 of improved sensitivity. However, the reduced number of bits per symbol from 4 to 3 gives 3/4 of penalty, thus in total a gain of 3/2, or 1.76 dB. At a BER of 10^{-9}, this improves the ASE-limited sensitivity from 18 (for BPSK) to 13 (for PS-QPSK) photons per bit. We conclude that the PS-QPSK format has the best sensitivity attainable in optical systems, unless the constellation dimension is extended, e.g., by the use of error-correcting codes. Thus, the PS-QPSK format is the answer to the deceptively simple question posed in the title.

## Acknowledgements

MK wishes to acknowledge the hospitality of and discussions with S. Radic and N. Alic during a longer stay at UCSD, providing an opportunity to finish this work. He also wishes to acknowledge discussions with P. Andrekson and P. Winzer. Finally we like to acknowledge funding from the Swedish Strategic Research Foundation, SSF, as well as VINNOVA within the IKT grant.

## References and links

**1. **S. Tsukamoto, D. Ly-Gagnon, K. Katoh, and K. Kikuchi, “Coherent demodulation of 40-Gbit/s polarization-multiplexed QPSK signals with 16-GHz spacing after 200-km transmission,” in *Proceedings of Optical Fiber Communication and National Fiber Optic Engineers Conference*, OFC/NFOEC, vol. 6, p. PDP29 (2005).

**2. **G. Charlet, M. Salsi, J. Renaudier, O. Pardo, H. Mardoyan, and S. Bigo, “Performance comparison of singly-polarised and polarisation-multiplexed coherent transmission at 10 Gbauds under linear impairments,” Electron. Lett. **43(20)**, 1109–1111 (2007).
[CrossRef]

**3. **H. Sun, K. Wu, and K. Roberts, “Real-time measurements of a 40 Gb/s coherent system,” Opt. Express **16(2)**, 873–879 (2008).
[CrossRef]

**4. **E. Ip, A. Lau, D. Barros, and J. Kahn, “Coherent detection in optical fiber systems,” Opt. Express16(2), 753–791 (2008). Erratum vol. 16, no. 26, p. 21943, 2008.
[CrossRef]

**5. **J. R. Pierce, “Optical Channels: Practical Limits with Photon Counting,” IEEE Trans. Commun. **26(12)**, 1819–1821 (1978).
[CrossRef]

**6. **S. Benedetto and E. Biglieri, *Principles of Digital Transmission: With Wireless Applications* (Kluwer Academic Publishers, 1999).

**7. **S. Betti, F. Curti, G. De Marchis, and E. Iannone, “A novel multilevel coherent optical system: 4-quadrature signaling,” J. Lightwave Technol. **9(4)**, 514–523 (1991).
[CrossRef]

**8. **R. L. Graham and N. J. A. Sloane, “Penny-packing and two-dimensional codes,” Discrete and Computational Geometry **5(1)**, 1–11 (1990).

**9. **N. J. A. Sloane, R. H. Hardin, T. S. Duff, and J. H. Conway, “Minimal-Energy Clusters of Hard Spheres,” Discrete and Computational Geometry **14(3)**, 237–259 (1995).

**10. **N. J. A. Sloane, R. H. Hardin, T. S. Duff, and J. H. Conway, “Minimal-Energy Clusters, Library of 3-d clusters, Library of 4-d clusters,” (1997). URL http://www.research.att.com/~njas/cluster/.

**11. **J. Kahn and K.-P. Ho, “Spectral efficiency limits and modulation/detection techniques for DWDM systems,” IEEE J. Sel. Top. Quantum Electron. **10(2)**, 259–272 (2004).

**12. **M. K. Simon, S. Hinedi, and W. C. Lindsey, *Digital Communication Techniques: Signal Design and Detection* (Prentice-Hall, 1995).

**13. **L. Xiao and X. Dong, “New analytical expressions for orthogonal, biorthogonal, and transorthogonal signaling in Nakagami fading channels with diversity reception,” IEEE Trans. Wireless Commun. **4(4)**, 1418–1424 (2005).
[CrossRef]

**14. **D. Saha and T. Birdsall, “Quadrature-quadrature phase-shift keying,” IEEE Trans. Commun. **37(5)**, 437–448 (1989).
[CrossRef]