## Abstract

Advanced modulation formats combined with digital signal processing and direct detection is a promising way to realize high capacity, low cost and power efficient short reach optical transmission system. In this paper, we present a detailed investigation on the performance of three advanced modulation formats for 100 Gb/s short reach transmission system. They are PAM-4, CAP-16 and DMT. The detailed digital signal processing required for each modulation format is presented. Comprehensive simulations are carried out to evaluate the performance of each modulation format in terms of received optical power, transmitter bandwidth, relative intensity noise and thermal noise. The performance of each modulation format is also experimentally studied. To the best of our knowledge, we report the first demonstration of a 112 Gb/s transmission over 10km of SSMF employing single band CAP-16 with EML. Finally, a comparison of computational complexity of DSP for the three formats is presented.

© 2015 Optical Society of America

## 1. Introduction

The fast increasing bandwidth demand of data center and other optical interconnect applications requires optical transmission system with data rate up to 400 Gb/s or even 1 Tb/s. Using advanced modulation formats with coherent detection and digital signal processing, the capacity of long-haul transmission system has been increased significantly. However, intensity modulation with direct detection (IM/DD) is more suitable than coherent system for short reach applications in terms of cost, footprint and power consumption. The 100 Gb/s transmission of Ethernet frames has already been standardized using four parallel streams of 25 Gb/s transmission with OOK modulation. It is widely agreed that 4x100 Gb/s scheme based on existing 100GbE inexpensive and power efficient components is a promising way to realize 400 Gb/s short reach transmission system, which means a bit rate of 100 Gb/s per lane is essential.

A 103.125 Gb/s OOK signal has been generated using Mach-Zehnder modulator and detected as duo-binary signal [1]. However, this scheme requires high speed electronic components in order to generate the high baud rate binary signal. As a result, there has been significant interest in using advanced modulation formats and DSP-enabled transmitters and receiver for 100 Gb/s transmission in order to reduce the baud-rate and the bandwidth requirements for either electronic or optical components. The simplest method for reducing the baud-rate without reducing the data rate is through the use of pulse amplitude modulation (PAM). If the number of possible signal levels in a symbol is *N* and the required data rate is *R*, the symbol rate *D* can be reduced by a factor of *log _{2}N*, and

*D = R/ log*. However, with the number of pulse levels increasing, the signal will be more sensitive to the impairments from electric, optical devices and optical transmission channel. Comparing with PAM-8 and PAM-16, PAM-4 is regarded as a more attractive format for 100 Gb/s short reach applications in terms of system performance and implementation complexity. A 100 Gb/s transmission link was demonstrated using polarization division multiplexed PAM-4 signal [2]. 112 Gb/s short reach transmission using single polarization PAM-4 signal was demonstrated using SiP Mach-Zehnder modulator [3]. In our previous work, we have also successfully demonstrated a 112 Gb/s PAM-4 transmission system with commercial available 25G EML-TOSA and PIN-ROSA [4].

_{2}NAnother alternative scheme that may provide good system performance and high data rate using low-cost and bandwidth limited optical components is carrier-less amplitude and phase modulation (CAP). 10 Gb/s multi-level CAP for short reach communications was experimentally studied [5]. 40 Gb/s CAP-32 system was successfully demonstrated for low cost data communication links [6]. Higher order CAP signals were also experimentally demonstrated for short reach applications [7]. Using multi-band CAP, high-speed AWG and EML, 102 Gb/s signal transmission over 15km have been realized [8]. Subsequently, 400 Gb/s O band transmission over 20 km and 40 km of SSMF using multi-band CAP signal has been reported [9].

Discrete multi-tone (DMT) modulation also known as direct detection orthogonal frequency domain modulation (DD-OFDM), is another attractive scheme for low-cost and bandwidth limited short reach communication systems. DMT, as one kind of multi-carrier modulation techniques [10–14], shows high spectrum efficiency, flexible multi-level coding, and high tolerance to channel impairments. A 52.8 Gb/s DMT signal transmission over 20km of SSMF has been achieved with a distributed feedback laser [12]. A single wavelength, single polarization 101 Gb/s DMT signal generated with a 64 GSa/s AWG and a directly modulated laser has been transmitted over 10km of SSMF [13]. More recently, 4x117 Gb/s DMT signals have been successfully transmitted over 40 km of SSMF [14].

Although many researches and performance comparisons on PAM, CAP and DMT have been reported [15–17], there were no reported full system level comparisons among these modulation formats in 100 Gb/s short reach transmission systems. In this paper, we provide a comprehensive comparison of three typical advanced modulation formats for short reach transmission systems by using the same simulation model and experimental setup. They include PAM-4, CAP-16 and DMT. We focus on realizing a bit rate of 112 Gb/s with commercially available components using the modulation formats under study. The remainder of this paper is organized as follows. Section 2 presents the digital signal processing required different modulation formats at the transmitter and the receiver. In section 3, performance comparison is carried out using the same simulation setup for different formats in terms of received optical power, bandwidth of transmitter, relative intensity of noise (RIN) and thermal noise of the receiver. Section 4 presents experimental setup for different modulation formats to realize a data rate of 112 Gb/s and experimental results are presented and discussed. To the best of author’s knowledge, for the first time, a single-band CAP-16 signal with a bit rate of 112 Gb/s on single wavelength and single polarization was experimentally demonstrated for short reach transmission system. In section 5, computational complexity of DSP for different modulation formats is studied. Section 6 concludes this paper.

## 2. Digital signal processing for different modulation formats

Arbitrary waveform generator (AWG) and analogue to digital convertor (ADC) are used in the system, digital signal processing can be applied at both transmitter and receiver sides. In this section, we describe the digital signal processing required for different modulation formats at both ends of the transmission system.

#### 2.1 DSP for PAM-4 signal

Figure 1 shows the DSP flow chart for PAM-4 signal at the transmitter and the receiver sides. A 2^{16} de Bruijn bit sequence was used for bit to symbol mapping and the generation of PAM-4 signal. In order to achieve a bit rate of 112 Gb/s (100Gbit/s excluding Ethernet and FEC overhead), a baud rate of 56GB/s is required for PAM-4. In the experimental setup, the sampling rate of AWG is set to be 63 GSa/s. Therefore, the symbol stream was up-sampled by a factor of 1.125 (63/56). In order to avoid aliasing, a raised cosine (RC) pulse shaping filter with a roll-off factor of 0.12 was used for pulse shaping. Since the bandwidth of both the transmitter and the receiver is much smaller than signal bandwidth, the up-sampled signal was pre-emphasized by using an inverted linear filter.

At receiver side, the sampled signal was normalized and resampled to 2 samples per symbol. A training symbol aided least mean square (LMS) algorithm was first used to initialize the equalizer taps. After the taps are fully converged, the equalizer will switch to a decision directed mode. After symbol decision, the bit error rate was calculated by error counting.

#### 2.2 DSP for CAP-16 signal

Figure 2 shows the DSP flow chart for CAP-16 signal. At transmitter side, a 2^{16} de Bruijn bit sequence is used for bit to symbol mapping and the generation of in-phase and quadrature signals. After I/Q separation, the two signals are sent into two shaping filters. The impulse responses of two filters are expressed as ${h}_{I}(t)=g(t)\cdot \mathrm{cos}(2\pi {f}_{c}t)$ and ${h}_{Q}(t)=g(t)\cdot \mathrm{sin}(2\pi {f}_{c}t)$ in order to form a Hilbert pair. A square-root-raised-cosine shaping filter $g(t)$ with a roll-off coefficient of $\alpha =$0.06 is used as the baseband impulse response. The center frequency ${f}_{c}$is given by $(1+\alpha )\cdot B/2+\Delta f$. Here, *B* is the baud-rate and is 28 GB/s in order to achieve a bit rate of 112 Gb/s (100Gbit/s excluding Ethernet and FEC overhead) with CAP-16 format. $\Delta f$is the frequency offset and is set to be 0.1 GHz. After the shaping filter, the signal are resampled and pre-emphasized by an inverted linear filter.

At receiver side, the signal samples are first sent into two matched filters. The two filters are the time-reversed version of the shaping filters at the transmitter in order to separate the in-phase and quadrature components. Then the signal was resampled to be 2 samples per symbol. Here, a hybrid equalizer is used for channel equalization. A modified cascaded multi-modulus algorithm (MCMMA) is used for pre-convergence [5]. Decision directed least-mean-square (DD-LMS) is used as second stage fine equalization. After the equalization, the signal is decoded and the bit error rate is calculated by error counting.

#### 2.3 DSP for DMT signal

Figure 3 shows the DSP flow chart for DMT system. At Transmitter side, the DMT signals are encoded using a 2^{16} de Bruijn bit sequence with the following parameters: FFT size is 512 with Hermitian symmetry, cyclic prefix (CP) is 8, and the number of data subcarriers is 217. For every data frame, 21 training symbols are inserted at the beginning of each data frame, which consists of 1 symbol for symbol frame synchronization and 20 symbols for channel estimation. The total data rate for experiment is 111.8 Gb/s, which results a useful bit rate of 100Gbit/s excluding Ethernet and FEC overheads. The generated DMT signal is pre-emphasized by using an inverted linear filter and loaded into an AWG operated at 63 GSa/s with 8 bits resolution.

At receiver side, after normalization and resampling, symbol synchronization is realized by training symbols. Thus CPs are removed and the data symbols on each subcarrier are obtained by FFT operation. An one tap equalizer is implemented to compensate for channel distortions. Finally, the recovered symbols are decided and decoded to obtain the bit sequence. The bit error rate is calculated by error counting.

## 3 Simulation setup and results

Firstly, system simulation is carried out to evaluate the performance of different modulation formats regarding received optical power, TX bandwidth, relative intensity noise (RIN) and thermal noise of optical receiver. The schematic of simulation is shown in Fig. 4. The transmitter consists of a low pass filter (LPF) and an electric absorption modulated laser (EML). The output data is filtered by a 4th order Bessel low pass filter with a 3 dB bandwidth of 20 GHz in order to simulate the bandwidth limitation of the transmitter. The center wavelength of EML is 1310 nm with an output power of 3.5 dBm. The optical link includes standard single mode fiber (SSMF). The receiver is modeled with a photo-detector (PD) and a transimpedance amplifier (TIA) with a given equivalent input noise current density. An electric 4th order Bessel low-pass filter with a 3 dB bandwidth of 20 GHz is placed after PD in order to simulate the bandwidth limitation of the receiver. The received data is then processed by DSP. Main simulation parameters are shown in Table 1. For PAM-4 signal, the baud-rate is 56 GB/s, which gives a bit rate of 112 Gb/s. For CAP-16, the baud-rate is 28 GB/s and the root raised cosine pulse shaping has a roll off factor of 0.06. For DMT signal, the bit-rate is set to be 112 Gb/s, including 64QAM to QPSK signal for subcarriers. For simulation setup, no pre-emphasis is used at the transmitter side for all the formats.

The simulation results are shown in Fig. 5. Figure 5(a) shows the BER as a function of received optical power for different formats. The performance of DMT is the best among the three formats. PAM-4 is slightly better than CAP-16. Receiver sensitivities of −6.95 dBm, −6.4 dBm and −6.15 dBm at 7% FEC overhead limit of 3.8x10^{−3} are demonstrated for DMT, PAM-4 and CAP-16, respectively. Figure 5(b) shows the receiver sensitivity penalty as a function of transmitter bandwidth. The received optical power at BER = 3.8x10^{−3} for transmitter bandwidth of 35 GHz for each formats was used as the reference. It can be seen that PAM-4 is most sensitive to TX bandwidth. Because the baud rate of PAM-4 signal is 56 GB/s, which is much larger than that of the devices. Due to high spectrum efficiency and sharp pulse shaping, the bandwidth of CAP-16 is significantly reduced with respect to PAM-4 at the same data rate. Therefore, CAP-16 is more tolerant to TX bandwidth. Based on different characteristics of optical transmission system with different TX bandwidth, optimum bit loading can be designed in order to obtain the best performance while keeping the same data rate of 112 Gb/s. This gives DMT the best system performance when the TX bandwidth is extreme low. Figure 5(c) shows the receiver sensitivity penalty as a function of relative intensity noise (RIN) for different formats. PAM-4 and CAP-16 have similar tolerance to RIN. For DMT, since it is more like an analog signal, it is more sensitive to the RIN. At 1 dB penalty, the tolerance of RIN is −147 dB/Hz, −143 dB/Hz and −143 dB/Hz for DMT, PAM-4 and CAP-16, respectively. Figure 5(d) shows the receiver sensitivity as a function of thermal noise of the optical receiver. Again, the performance of PAM-4 and CAP-16 is similar. It can be seen that DMT is more tolerant to the thermal noise.

## 4 Experimental setup and results

In order to provide a fair comparison, all modulation formats are tested in the same experimental setup with a target bit rate of 112 Gb/s. The experimental setup is shown in Fig. 6. The pre-calculated data is loaded into the AWG with a 3dB bandwidth of 12 GHz to generate electric signal. Thus, the electric signal is amplified to a peak-to-peak voltage of 1.7 Vpp by a linear electric amplifier (EA) and used to drive a 25 Gb/s EML (electric absorption modulated laser). The 3 dB bandwidth of the EML is 20 GHz. The optimal bias voltage is optimized to be −2.3V for all the case. The center wavelength is 1296.2 nm, and the power of modulated optical signal is 3.5 dBm. A variable optical attenuator (VOA) is placed after 10 km SSMF to adjust the received optical power. At the receiver side, the optical signal is detected by a PIN + TIA receiver with a 3 dB bandwidth of 30 GHz. The detected signal is sampled by an 80 GSa/s real-time scope and is processed offline. Figure 7 shows the measured end-to-end channel frequency response by performing a discrete frequency sweep with the AWG. According to the measured results, the 3 dB bandwidth of the optical channel is 17 GHz.

Firstly, the performance of the 112 Gb/s PAM-4 transmission system was experimentally studied and the results are shown in Fig. 8. It should be noted that an inverted linear filter with an attenuation of 10 dB at 0 Hz is used for pre-emphasis at transmitter. Figure 8(a) shows the optical spectra for the back-to-back system and after 10km SSMF transmission. Figure 8(b) shows the BER as a function of received optical power for back-to-back system and after 10km transmission. It can be seen that receiver sensitivities at 7% FEC overhead for BER of 3.8x10^{−3} are −5.1 dBm and −5.6 dBm for back-to-back system and 10km transmission system, respectively. No power penalty is investigated from the transmission. A BER floor of 1.5x10^{−3} is shown for 112 Gb/s PAM-4 signal. The eye-diagram of recovered 112 Gb/s PAM-4 signal with a received power of −5 dBm in 10 km transmission system is shown in Fig. 8(c). The eye-diagram is obtained after the receiver DSP as shown in Fig. 1, which is after the LMS for channel compensation. To obtain multiple points within one symbol duration, we up-sample the 2 samples per symbol signal to 32 samples per symbol by padding zeros in the frequency domain. According to the results, we still can demonstrate certain eye-opening with a received optical power of −5dBm. The probability density function of 112 Gb/s PAM-4 signal after 10km transmission with a received optical power of −5 dBm is shown in Fig. 8(d). Four peaks can be obtained at each level of PAM-4 signal. The corresponding bit error rate is 3.6x10^{−3}.

Then, 112 Gb/s CAP-16 10 km transmission system was experimentally demonstrated with the same experimental setup. Figure 9(a) shows the optical spectra for back-to-back system and after 10 km SSMF transmission. The shape of the optical spectra is similar to those of PAM-4 signal as shown in Fig. 8(a). Due to the bandwidth limitation of AWG and other components in the optical link, an inverted linear filter with an attenuation of 12 dB at 0 Hz is used to compensate for the power attenuation at high frequencies. The electric spectrum of 112 Gb/s CAP-16 signal with pre-emphasis at the output of AWG is shown in the Fig. 9(b) (blue). The electric spectrum after PD is also shown in Fig. 9(b) (red). It can be seen that the frequency components lower than 20 GHz have a good flatness. The power of frequency components start to degrade from 20 GHz and experience large attenuation at higher frequency. However, no further pre-emphasis can be carried out due to the limited output power of AWG. Therefore, MCMMA algorithm combined with DD-LMS algorithm based hybrid equalization as shown in Fig. 2 (b) is used in receiver to eliminate performance degradation due to this filtering effect. Figure 9(c) shows the BER as a function of received optical power for 112 Gb/s CAP-16 back-to-back system and 10 km transmission system. It can be found that the performance of 10km transmission system slightly outperform back-to-back system. Receiver sensitivities of −4.9 dBm and −5.2 dBm at 7% FEC overhead limit of 3.8x10^{−3} were demonstrated for back-to-back system and 10km transmission system, respectively. A BER floor of 2x10^{−3} is shown for 112 Gb/s CAP-16 signal. The constellation of 112 Gb/s CAP-16 signal after 10 km transmission at a received power of −5 dBm is also shown in Fig. 9 (d) and the bit error rate is 3.2x10^{−3}.

Then, we conducted 111.8 Gb/s DMT transmission system with the same experimental setup in order to obtain a fair comparison. It should be noted that an inverted linear filter with an attenuation of 4 dB at 0 Hz is used for pre-emphasis at transmitter. Fig. 10 shows the experimental results for DMT transmission system. One of the advantages of DMT is that it can adapt the bit and power allocation for each subcarrier in order to maximize the bit rates and optimize the system performance. In this experimental, we use probing DMT signals with different modulation formats (including QPSK, 16QAM, 32QAM and 64QAM) to define the modulation format for each subcarrier. The target bit rate and bit error rate are 112 Gb/s and 3.8x10^{−3} after 10 km SSMF transmission with a received optical power of −5 dBm, respectively. The bit loading used in this experiment is shown in Fig. 10(a) with a total data rate of 111.8 Gb/s. The optical spectra for back-to-back transmission and 10 km SSMF transmission are shown in Fig. 10(b). Then, the BER as a function of received optical power is investigated for back-to-back system and 10 km SSMF transmission system as shown in Fig. 10(c). It can be found that the performance of DMT after 10 km transmission is similar to back-to-back system. Receiver sensitivities of −5 dBm and −5.1 dBm at 7% FEC overhead limit of 3.8x10^{−3} are demonstrated for back-to-back system and 10 km transmission system, respectively. No power penalty is investigated from transmission. A BER floor of 3x10^{−3} is shown, which is higher than that of PAM-4 and CAP-16. That is due to the high peak to average power ratio (PAPR) of the DMT signal, the limited ENOB of AWG and the nonlinearity of the driver and the EML. Additional nonlinearity compensation techniques are required to eliminate these effects, which will be discussed in our future work.

Finally, we compare the performance of different modulation formats. Figure 11(a) shows BER as a function of received optical power of different formats for back-to-back signals. It can be seen that PAM-4 slightly outperforms CAP-16. At low received optical power regime (<-5 dBm), DMT outperforms the two other modulation formats and this agrees with the simulation results. However, at high received optical power regime (>-5 dBm), the performance of DMT is the worst among the three modulation formats and a BER floor of 3x10^{−3} was observed for DMT signal. As pointed out before, this is due to the high PAPR of DMT signal, limited 8 bit resolution and nonlinearity of the driver and the EML. Receiver sensitivities of −4.9 dBm,-5 dBm and −5.1 dBm were obtained for CAP-16, DMT and PAM-4, respectively. Figure 11(b) shows BER as a function of received optical power of different modulation formats after 10km SSMF transmission. Same as the BTB cases, PAM-4 signal still outperforms CAP-16 signal. For DMT signal, the performance is best when the received power is lower than −6 dBm. However, at high received optical power region (>-5 dBm), the performance of DMT is worst among three modulation formats. Receiver sensitivities of −5.2 dBm,-5.1 dBm and −5.6 dBm are investigated for CAP-16, DMT and PAM-4, respectively.

## 5 Computational complexity analysis

The complexity of DSP directly affects the hardware costs and power consumption of the transmission systems. In this section, we provide a brief comparison of the computational complexity of digital signal processing used at both the transmitter and the receiver sides for three formats. Considering there are many common functions in the DSP for different modulation formats and some functions are simple to implement, only functions with high computational complexity are used to evaluate the computational complexity of DSP for each format. They are LMS based equalizer for PAM-4, two-stage equalizer based on MCMMA and DD-LMS for CAP-16 and FFT/IFFT for DMT signal. In Additional, the equalizer for PAM-4 and CAP-16 can be implemented either in time domain or frequency domain, which also are known as time-domain equalizer (TDE) and frequency domain equalizer (FDE), respectively. It is well known that the computational complexity of FDE is much lower than that of TDM [18]. In this section, we will include both TDE and FDE for PAM-4 and CAP-16 in the computational complexity comparison. Considering chip resource and power consumption of DSP in ASIC, the cost of a multiplier is much higher than an adder. Therefore, computational complexity of those functions is evaluated in terms of the number of real multipliers per bit.

For PAM-4, if TDE adapted by LMS with a tap spacing *T/2* of and tap length of *N* is used for equalization, we need *N* real multipliers for output calculation, *N* real multipliers for tap updating by LMS to obtain one symbol output from the equalizer. By putting these together, the computational complexity of TDE for PAM-4${C}_{PAM4}^{TDE}$ can be expressed as

*M*is the number of constellation points on the signal constellation, which is 4 for PAM4 signal. If FDE is implemented for equalization and overlap-saver method with 50% overlap is used, in order to obtain

*N/2*symbols output from the equalizer, we need N multiplications for output calculations of one block, N multiplications for tap updating,

*4Nlog*multiplications for 8 FFT/IFFT processes, which include 2 FFT for inputs, 1 IFFT for outputs, 1 FFT for error calculation and 4 FFT/IFFT for employing gradient constraint. For FFT implementation, the classical radix-2 algorithm is assumed to be used, which requires

_{2}(N)*Nlog*multiplications to execute FFT of

_{2}(N)/2*N*numbers. Therefore the computational complexity of FDE for PAM-4${C}_{PAM4}^{FDE}$ can be expressed as

*M*is the number of constellation points on the signal constellation, which is 4 for PAM4 signal.

For CAP-16 with TDE, the equalizer includes two stages: modified CMMA and DD-LMS. As for modified CMMA, the delay spacing is *T/2*, and the tap length is *N _{1}*. To obtain one symbol output from the equalizer, we need

*N*complex multipliers for output calculation,

_{1}*N*complex multipliers for tap updating. For DD-LMS, the delay spacing is

_{1}*T*, and the tap length is

*N*. Thus, to obtain one symbol output from the equalizer, we need

_{2}*N*complex multipliers for output calculation and

_{2}*N*complex multipliers for tap updating. Here, one complex multiplier equals to 3 real multipliers [19]. Therefore, the computational complexity of TDE for CAP-16${C}_{CAP16}^{TDE}$ can be expressed as

_{2}For CAP-16 with FDE, the equalizer also includes two stages: modified CMMA and DD-LMS. As for modified CMMA, the delay spacing is *T/2*, and the tap length is *N _{1}*. To obtain

*N*symbols output from the equalizer, we need

_{1}/2*N*complex multiplications for output calculations of one block,

_{1}*N*complex multiplications for tap updating,

_{1}*4N*complex multiplications for 8 FFT/IFFT processes, which include 2 FFT for inputs, 1 IFFT for outputs, 1 FFT for error calculation and 4 FFT/IFFT for employing gradient constraint. For the second stage equalizer by DD-LMS, the delay spacing is

_{1}log_{2}(N_{1})*T*, and the tap length is

*N*. Thus, to obtain

_{2}*N*symbols output from the equalizer, we need

_{2}*N*complex multiplications for output calculations of one block,

_{2}*N*complex multiplications for tap updating,

_{2}*4N*complex multiplications for 8 FFT/IFFT processes, which include 2 FFT for inputs, 1 IFFT for outputs, 1 FFT for error calculation and 4 FFT/IFFT for employing gradient constraint. Here, one complex multiplier equals to 3 real multipliers. Therefore, the computational complexity of FDE for CAP-16${C}_{CAP16}^{FDE}$can be expressed as

_{2}log_{2}(N_{2})For DMT, the computation complexities of FFT function and IFFT are same. For FFT implementation, the classical radix-2 algorithm is assumed to be used, which requires *Nlog _{2}(N)/2* complex multiplications to execute FFT of

*N*complex numbers. Thus, the computational complexity C

_{DMT}can be expressed as

*N*is the FFT size,

*N*is the number of data subcarriers and

_{sc}*M*is the average number of constellation points on the signal constellation.

Table 2 shows the comparison of computational complexity for the three modulation formats. Experimental parameters for each format are used to calculate the computational complexity. For PAM-4 signal, the tap length of equalizer is 301. For CAP-16, the tap lengths are 31 and 101 for the first-stage and second-stage equalizers, respectively. FFT size, number of subcarriers and average number of constellation points are 512, 217 and 16, respectively. From Table 2, it is clear that FDE requires much lower computational complexity than that of TDE. PAM-4 and CAP-16 have similar computational complexity. DMT offers lower complexity than the other two formats.

## 6 Conclusions

In this paper, a comprehensive study on the performance of PAM-4, CAP-16 and DMT for 100 Gb/s short reach optical transmission system was carried out both numerically and experimentally. Digital signal processing required for each formats was given in detail. Comprehensive simulations were carried out and the results show that DMT exhibits better performance than PAM-4 and CAP-16 in terms of received optical power, TX bandwidth and thermal noise. However, DMT is more sensitive to RIN than PAM-4 and CAP-16. The performance of three formats was also studied experimentally by using the same experimental setup. The feasibilities of using all three formats to achieve a bit rate of 112 Gb/s short reach transmission system are verified experimentally. To the best of our knowledge, we report the first demonstration of 112 Gb/s transmission over 10 km of SSMF employing single-band CAP-16 with EML. The experimental results show that PAM-4 slightly outperforms CAP16. DMT provides a better performance than the other two but suffers from a relative high BER floor due to the limited resolution of DAC/ADC and the nonlinearity of driver and EML. At last, a comparison of computational complexity of DSP for the three formats was presented. It shows that DMT offers much lower computational complexity than PAM4 and CAP16.

## Acknowledgment

The authors would like to acknowledge the financial support of project G-YJ84 and G-YL24 of the Hong Kong Polytechnic University, H-ZG46 of Huawei Technologies Co. Ltd, The Hong Kong Scholar Program (XJ2013026), National Natural Science Foundation of China (61401020), and Beijing Natural Science Foundation (4154080).

## References and links

**1. **J. Lee, N. Kaneda, T. Pfau, A. Konczykowska, F. Jorge, J.-Y. Dupuy, and Y.-K. Chen, “Serial 103.125-Gb/s transmission over 1 km SSMF for low-cost, short-reach optical interconnects,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2014), paper Th5A.5. [CrossRef]

**2. **R. Rodes, J. Estaran, B. Li, M. Muller, J. B. Jensen, T. Gruendl, M. Ortsiefer, C. Neumeyr, J. Rosskopf, K. J. Larsen, M. C. Amann, and I. T. Monroy, “100 Gb/s single VCSEL data transmission link,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2012), paper PDP5D.10. [CrossRef]

**3. **M. Chagnon, M. Osman, M. Poulin, C. Latrasse, J. F. Gagné, Y. Painchaud, C. Paquet, S. Lessard, and D. Plant, “Experimental study of 112 Gb/s short reach transmission employing PAM formats and SiP intensity modulator at 1.3 μm,” Opt. Express **22**(17), 21018–21036 (2014). [CrossRef] [PubMed]

**4. **K. P. Zhong, W. Chen, Q. Sui, M. J. Wei, A. P. K. Lau, C. Lu, and L. Zeng, “Low cost 400GE transceiver for 2km optical interconnect using PAM4 and direct detection,” in Asia Communications and Photonics Conference (ACP) (2014), paper, ATh4D.2. [CrossRef]

**5. **L. Tao, Y. Wang, Y. Gao, A. P. T. Lau, N. Chi, and C. Lu, “Experimental demonstration of 10 Gb/s multi-level carrier-less amplitude and phase modulation for short range optical communication systems,” Opt. Express **21**(5), 6459–6465 (2013). [CrossRef] [PubMed]

**6. **L. Tao, Y. Wang, Y. Gao, A. P. T. Lau, N. Chi, and C. Lu, “40 Gb/s CAP32 System With DD-LMS Equalizer for Short Reach Optical Transmissions,” IEEE Photon. Technol. Lett. **25**(23), 2346–2349 (2013). [CrossRef]

**7. **L. Tao, Y. Wang, Y. Gao, and N. Chi, “High Order CAP System Using DML for Short Reach Optical Communications,” IEEE Photon. Technol. Lett. **26**(13), 1348–1351 (2014). [CrossRef]

**8. **M. I. Olmedo, T. Zuo, J. B. Jensen, Q. Zhong, X. Xu, S. Popov, and I. T. Monroy, “Multiband Carrierless Amplitude Phase Modulation for High Capacity Optical Data Links,” IEEE Lightwave Technology, Journalism **32**(4), 798–804 (2014).

**9. **T. Zuo, A. Tatarczak, M. I. Olmedo, J. Estaran, J. Bevensee Jensen, Q. Zhong, X. Xu, and I. Tafur, “O-band 400 Gbit/s Client Side Optical Transmission Link,” in *Optical Fiber Communication Conference,(OFC)* (2014), paper M2E.4. [CrossRef]

**10. **Z. Li, T. Jiang, H. Li, X. Zhang, C. Li, C. Li, R. Hu, M. Luo, X. Zhang, X. Xiao, Q. Yang, and S. Yu, “Experimental demonstration of 110-Gb/s unsynchronized band-multiplexed superchannel coherent optical OFDM/OQAM system,” Opt. Express **21**(19), 21924–21931 (2013). [PubMed]

**11. **Y. Bao, Z. Li, J. Li, X. Feng, B. O. Guan, and G. Li, “Nonlinearity mitigation for high-speed optical OFDM transmitters using digital pre-distortion,” Opt. Express **21**(6), 7354–7361 (2013). [CrossRef] [PubMed]

**12. **T. Tanaka, M. Nishihara, T. Takahara, L. Li, Z. Tao, and J. C. Rasmussen, “50 Gbps class transmission in single mode fiber using discrete multi-tone modulation with 10G directly modulated laser,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2012), paper Oth4G. [CrossRef]

**13. **W. Yan, T. Tanaka, B. Liu, M. Nishihara, L. Li, T. Takahara, Z. Tao, J. C. Rasmussen, and T. Drenski, “100 Gb/s optical IM-DD transmission with 10G-class devices enabled by 65 Gsamples/s CMOS DAC core,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2013), paper OM3H.1. [CrossRef]

**14. **T. Tanaka, M. Nishihara, T. Takahara, W. Yan, L. Li, Z. Tao, M. Matsuda, K. Takabayashi, and J. C. Rasmussen, “Experimental demonstration of 448-Gbps+ DMT transmission over 30km SMF,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2014), paper M2I. 5. [CrossRef]

**15. **J. L. Wei, D. G. Cunningham, R. V. Penty, and I. H. White, “Feasibility of 100G ethernet enabled by carrierless amplitude/phase modulation and optical OFDM,” in *European Conference and Exhibition on Optical Communications (ECOC)* (2012), paper P6.05. [CrossRef]

**16. **Y. Kai, M. Matsuda, T. Tanaka, T. Takahara, L. Li, Z. Tao, B. Liu, J. C. Rasmussen, and T. Drenski, “Experimental comparison of pulse amplitude modulation (PAM) and discrete multi-tone (DMT) for short reach 400Gbps data communication,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2014), paper Th.1.F.3.

**17. **L. Tao, Y. Ji, J. Liu, A. P. T. Lau, N. Chi, and C. Lu, “Advanced modulation formats for short reach optical communication systems,” IEEE Netw. **27**(6), 6–13 (2013). [CrossRef]

**18. **M. S. Faruk and K. Kikuchi, “Adaptive frequency-domain equalization in digital coherent optical receivers,” Opt. Express **19**(13), 12789–12798 (2011). [CrossRef] [PubMed]

**19. **J. Leibrich and W. Rosenkranz, “Frequency Domain Equalization with Minimum Complexity in Coherent Optical Transmission Systems,” in *Proc. Conf. Optical Fiber Commun. Conf. (OFC)* (2012), paper OWV1.