## Abstract

Ultra-high-speed optical communication systems which can support ≥ 1Tb/s per channel transmission will soon be required to meet the increasing capacity demand. However, 1Tb/s over a single carrier requires either or both a high-level modulation format (i.e. 1024QAM) and a high baud rate. Alternatively, grouping a number of tightly spaced “sub-carriers” to form a terabit *superchannel* increases channel capacity while minimizing the need for high-level modulation formats and high baud rate, which may allow existing formats, baud rate and components to be exploited. In ideal Nyquist-WDM superchannel systems, optical subcarriers with rectangular spectra are tightly packed at a channel spacing equal to the baud rate, thus achieving the Nyquist bandwidth limit. However, in practical Nyquist-WDM systems, precise electrical or optical control of channel spectra is required to avoid strong inter-channel interference (ICI). Here, we propose and demonstrate a new “super receiver” architecture for practical Nyquist-WDM systems, which jointly detects and demodulates multiple channels simultaneously and mitigates the penalties associated with the limitations of generating ideal Nyquist-WDM spectra. Our receiver-side solution relaxes the filter requirements imposed on the transmitter. Two joint DSP algorithms are developed for linear ICI cancellation and joint carrier-phase recovery. Improved system performance is observed with both experimental and simulation data. Performance analysis under different system configurations is conducted to demonstrate the feasibility and robustness of the proposed joint DSP algorithms.

© 2013 OSA

## 1. Introduction

To meet the capacity requirements of future optical networks, the design of transport systems capable of supporting 1Tb/s and greater channel rates based on “superchannel” architectures requires a judicious choice of modulation formats, channel spacing and signal processing in order to achieve the maximum channel capacity without limiting reach or incurring unacceptable complexities. Nyquist-WDM [1, 2] and Coherent Optical-OFDM (CO-OFDM) [3–5] are two complementary approaches that may be used to obtain ultra-high spectral efficiency in superchannel coherent systems. Ideally they are purely orthogonal systems. However, the generation and maintenance of these ideal orthogonal superchannel signals is quite difficult, and any slight imperfection results in severe inter-channel interference (ICI) at such close channel spacing.

The subchannels of Nyquist superchannels are likely to be transported and routed as a bound group therefore linear ICI is likely to be created primarily at the transmitter during multiplexing or at the receiver during demux. At the transmitter side, the conventional approaches to mitigate the ICI include aggressive optical filtering [6], Nyquist-like optical filtering [1], and high-speed DAC enabled electrical spectral shaping [7]. At the receiver side, ICI may occur during subchannel separation. This occurs for both optical demux or, in the case of colorless receivers, the electrical demux. Ultimately however, the complexity of the generation and mux/demux process depends on the receivers’ capability to manage ICI and inter-symbol interference (ISI) impairments. To date coherent detection for Nyquist-WDM systems is accomplished strictly by operations on a single channel without use of any neighbor-channel information. It is likely however that future DSP implementations will include sufficient processing capacity to allow neighbor-channel information to be used in the demodulation of any given channel. Therefore, we propose and demonstrate a coherent receiver architecture for Nyquist-WDM systems that we call a “super receiver”, which detects and demodulates multiple channels jointly. By taking advantage of information from side channels using joint DSP to cancel ICI, we achieve greatly improved performance compared to conventional receivers which process each channel independently. The proposed architecture has similar receiver complexity as MIMO technologies proposed for crosstalk cancellation in few-mode fibers [8]. However, different from the MIMO approach which explores the mode-division multiplexing; the proposed “super receiver” architecture builds on conventional WDM systems and cancels the ICI that results from the *frequency* overlap of the neighboring subchannels. Additionally, in carrier phase-locked Nyquist-WDM systems, we propose a jointly estimated carrier-phase methodology that exploits the carrier information of the multiple subchannels.

In section 2, we introduce the background of Nyquist-WDM for superchannel coherent optical systems, and emphasize practical implementation issues. In section 3, the “super receiver” architecture is described. Section 4 explains our two joint DSP algorithms: adaptive linear-ICI cancellation and joint carrier-phase recovery. The experimental and simulation results of proposed algorithms under a variety of system configurations are shown and analyzed in section 5. Finally, we discuss the implications of the super receiver concept and identify future areas of investigation in section 6.

## 2. Background and practical issues of superchannels

As mentioned previously, the 1Tb/s ‘per channel’ goal can be realized by tightly grouping a number of “sub-carriers”, and this has become known as a *superchannel*. Nyquist-WDM and CO-OFDM are the two main approaches to generate superchannel signals optically. We note that electrical OFDM has features similar to CO-OFDM although the sub-carriers are initially generated in the electrical domain [9, 10]. Theoretically, both Nyquist-WDM and CO-OFDM can achieve baud-rate channel spacing without inducing inter-channel interference (ICI) or inter-symbol interference (ISI). However, in practical systems, neither of the two schemes is perfectly ICI or ISI free.

For Nyquist-WDM superchannel systems, the ideal spectral shape for each subchannel is rectangular with the channel spacing (Δf) equal to the baud rate B, Fig. 1. Therefore, no inter-channel interference (ICI) exists. On the other hand, due to the rectangular spectrum, the pulse within each subchannel is *sinc* shape in the time domain with zero-crossing points at integer multiples of the symbol period T, thus a Nyquist pulse (inter-symbol interference (ISI) free) is achieved. In contrast, for CO-OFDM system, each carrier is *sinc* shape in the frequency domain, and rectangular in the time domain. Therefore, theoretically both schemes can be ICI and ISI free simultaneously [7, 11].

However, in practical systems, neither of the two schemes is perfectly ICI and ISI free. For CO-OFDM systems, the generation and maintenance of rectangular time-domain pulses through the transmission system is very difficult, especially at higher baud rate, since the overall pulse shape is determined by the net bandwidth of all the electrical and optical components in the transmission link. Furthermore, carrier separation at the receiver side is not trivial, requiring an FFT function either in the electrical domain [3, 12] or optical domain [13, 14]. For the electrical carrier separation, a banded detection scheme is usually used by simultaneously detecting more than one carrier per digital sampling, which further reduces the achievable baud rate per subchannel due to the limitation of ADC sampling rate.

For Nyquist-WDM systems, practical issues are raised by the requirement of perfect rectangular spectral shaping. Since the carriers are closely packed at a channel spacing equal to the baud rate, there is no guard band and any slight spectral imperfection results in ICI and degrades the system performance. Conventional methods to mitigate ICI effects in Nyquist-WDM systems include pre-shaping the signal spectrum using optical [1, 14] or digital [7] filters to approach a near-rectangular spectrum. Alternatively, strong optical filters can be applied on each subchannel (thus eliminate ICI), followed by DSP to cancel the induced inter-symbol interference (ISI) [6]. The digital pre-filtering (digital pulse shaping) approach by using high-speed DACs is able to achieve more accurate spectral shaping required of true Nyquist-WDM systems. However, Nyquist-like systems (Δf slightly larger than B) may also provide good solutions allowing more cost-effective and power-efficient optical filtering. Transmitter-side approaches are less flexible and adaptable to the overall channel variation, and slight spectral imperfections result in strong ICI at tight channel spacing. A Rx-side solution reduces the filter requirement imposed on the Tx and more readily enables adaptation to changing channel conditions. Therefore, a “super receiver” architecture is proposed to jointly detect and demodulate multiple subchannels at the Rx side and mitigate the penalties associated with the limitations of creating ideal Nyquist-WDM spectra.

## 3. “Super receiver” principle and design

The “super receiver” architecture, Fig. 2, retains the optical functions of a conventional superchannel receiver, but performs the receiver-side DSP jointly on multiple subchannels. A superchannel transmitter produces multiple sub-carriers generated by independent laser sources [1] or by phase-locked methods [17]. In principle, the proposed joint ICI cancellation algorithm is effective for both multi-carrier generation methods. However, the proposed joint carrier-phase estimation algorithm is only applicable for phase-locked multi-carrier systems. These tightly spaced optical carriers are then independently modulated and optically multiplexed. Strong optical filtering is usually applied to each subchannel before multiplexing to shape the spectrum and minimize ICI. We examine the efficacy of our super receiver architecture by considering two cases: 1) without Tx optical filtering for individual subchannels and 2) with conventional Tx optical shaping filters. After the optical fiber transmission and optical demultiplexing, each subchannel is sent to its corresponding coherent receiver for O/E conversion, digital sampling and demodulation. As with the Tx, optical filtering associated with optical demultiplexer is optional. With no individual Rx optical filtering, the function of optical demultiplexer is a simple optical splitter or a wideband optical filter. This is a ‘colorless’ receiver where the carrier separation is done by mixing with an appropriate local oscillator (LO), and passing through subsequent electrical filters. The LOs for coherent receivers are generated in the same way as at the transmitter.

In order to capture the synchronized information across the subchannels for the subsequent joint signal processing, the optical and electrical path for each subchannel after demultiplexing needs to be the same length, and the digital sampling must be synchronized across all subchannels. However, this requirement can be relaxed if appropriate time-domain memory is applied in the joint DSP functions.

Information is available from multiple subchannels in the joint DSP block, thus enabling joint signal processing to compensate both linear and nonlinear impairments between the subchannels. In carrier phase-locked Nyquist-WDM systems, since all the subchannels experience the same carrier-phase variation, a more accurate carrier-phase estimation can be achieved based on the multiple copies of carrier-phase information from the “super receiver”.

## 4. Joint DSP algorithms

Two joint DSP algorithms that exploit the side subchannel information obtained from the “super receiver” architecture are developed and demonstrated: 1) linear-ICI cancellation based on adaptive LMS algorithm and 2) joint carrier-phase recovery scheme based on Viterbi-Viterbi (V-V) algorithm. As mentioned previously, the joint carrier-phase recovery requires phase-locked sources and LO’s.

#### 4.1 Adaptive linear-ICI cancellation

A three-subchannel dual-polarization (DP) QPSK superchannel system is considered, and the ICI equalization processing for the center subchannel (Ch.2) is evaluated. The joint DSP procedures are depicted in Fig. 3. After the superchannel signal is sampled (with synchronized ADCs), each subchannel independently follows conventional DSP processing in the following order: chromatic-dispersion compensation, polarization demultiplexing, timing recovery and carrier-phase estimation. After timing and carrier-phase recovery, an adaptive ICI equalizer is applied across the three subchannels for both X and Y polarizations separately based on an adaptive least-mean-squares (LMS) algorithm. The joint ICI cancellation can be done for the two polarizations separately after polarization demux, as long as the relative polarization states of the three subchannels remain fixed after demultiplexing. Specifically, the ICI can be considered to occur at the Rx when the subchannels are separated either optically or electrically. Thus as long as the “X” polarization of one subchannel has the same polarization as “X” in its neighbor channels then each polarization can be processed for ICI cancellation separately. In section 5.1, Fig. 8, we demonstrate experimentally that separate ICI processing of X and Y polarizations is an effective method. We emphasize that the relative polarization state among subchannels must be maintained only over a very limited distance; from the optical demux (or passive split) to the photodiodes of the coherent receivers. In practice, this can be readily accomplished by integrating multiple coherent receivers to detect a superchannel group. In addition, notice that the requirement of polarization tracking can be eliminated by joint processing both X and Y polarizations of all the neighboring subchannels, however, this doubles the complexity of the joint processing.

For each polarization, side subchannels (Ch.1 and Ch.3) are first shifted in the frequency domain by the amount of subchannel spacing to retrieve the spectral overlapping condition experienced during the coherent detection. In simulation, we determined that the precision of the frequency shift needs to be better than ~1MHz. This is more stringent than the required precision of the carrier-LO offset estimation, but it is reasonable since carrier-LO offsets result in a simple constellation rotation whereas inter-channel frequency offset errors result in an improper coherent addition to the impaired channel. This frequency shift requirement is most readily accomplished using phase-locked sources and LO’s, however it may be feasible to find and track the relative frequency shift between the neighboring channels through joint DSP. Then the signals from three subchannels are fed into an ICI filter (equalizer), where the filter coefficients (W_{12}, W_{22}, W_{32}) are jointly and adaptively updated based on the LMS algorithm. Each filter coefficient Wij represents the weighted crosstalk from subchannel i to subchannel j. As mentioned previously, each Wij is comprised of a number of time-domain taps to allow for the compensation of any timing skew between the subchannels. Intra-channel ISI can also be partially compensated by these taps. The number of taps depends on the timing offset between the subchannels, it is found that 10 ~20 taps are usually sufficient, which corresponds to ~5cm of path mismatch. After the ICI equalizer, ISI equalizers are applied to each polarization to compensate any residual ISI effects induced by optical or electrical filtering in the link. Note that the same procedure is applied to all subchannels of a superchannel signal with the edge subchannels experience only one-side crosstalk since we presume a small guard band between superchannels.

The additional complexity of joint DSP is scalable for two reasons: 1) For linear-ICI cancellation, only two adjacent subchannels are required for joint processing, independent of the number of superchannel signals within the group. 2) Since the joint ICI equalizer is at the end of DSP procedure, the prior independent DSP of each subchannel (pol-demux, timing recovery, and carrier phase recovery) are unchanged and the resulting signals can be reused for demodulation of adjacent subchannels. An example of 5-subchannel joint DSP is illustrated in Fig. 4. We calculated the number of real additions and multiplications required for the joint ICI cancellation. Assuming N is the number of taps in the joint ICI equalizer, M is the number of subchannels (M ≥ 2), and L is processing block length (the total number of samples with 2 samples per symbol), an M-subchannel joint ICI cancellation algorithm requires [(24N + 12)M-16N-8]L additional real additions and [(24N + 24)M-16N-16]L real multiplications. For the case of 5-subchannels with a 15 tap equalizer and block length of 10000, the additional complexity of the joint ICI cancellation algorithm is about 20% over the conventional single-channel based demodulation (including CD equalization, CMA pol-demux, timing and carrier phase recovery, and ISI equalization).

Since the LMS algorithm is used, the joint ICI algorithm can be applied to any type of modulation format (e.g. DP-16QAM) assuming other parts of the DSP algorithm are compliant. Our initial tests with 16QAM yield similar improvements as experienced with QPSK. This joint ICI cancellation algorithm is capable to cancel the linear crosstalk, and its effectiveness in the nonlinear transmission regime is left as future work. Based on the same super-receiver architecture, another approach by using maximum a posteriori (MAP) estimation for ICI cancellation can be realized and achieves the similar performance as the adaptive LMS ICI equalizer [19].

#### 4.2 Joint carrier-phase recovery

The joint carrier-phase estimation exploits the multiple versions of carrier-phase information available in carrier phase-locked superchannel systems. The initial DSP follows the previous joint ICI cancellation procedure up to carrier-phase recovery. After timing recovery, a joint carrier-phase recovery block is implemented based on a Viterbi-Viterbi (V-V) carrier-phase estimation algorithm. Again, we demonstrate the performance using a three-subchannel example, Fig. 5. The incoming complex baseband signals from each subchannel (both X-pol and Y-pol) are first raised to the 4th power to extract the phase information by removing the data dependencies [16]. Then, the three subchannel streams are averaged. For lasers with relatively narrow linewidth, the carrier-phase noise can be considered as a slow variation with time, and remains constant over several symbol periods. Since all the subchannels experience the same carrier-phase noise variation in a carrier phase-locked system, we average the estimated phase noise both in the time domain and across the subchannels to reduce the impact of ASE and other random noise sources. Also, inter-channel phase-noise impairments can be reduced by the cross-channel averaging process. After that, the argument of the output is divided by 4 and the phase is unwrapped. Finally, the jointly estimated phase error is applied to correct the carrier phase for all the subchannels. Thus the primary difference with the traditional Viterbi-Viterbi carrier-phase recovery algorithm is that one more average stage is added across the subchannels to have a better estimation of the carrier phase.

We found that the two outside subchannels always suffer less crosstalk than the inner subchannels, thus their carrier-phase estimates are more accurate. Therefore, by using the estimated phase information only from the two outside sub-channels, a more precise phase estimation can be obtained and applied to all the subchannels. We emphasize that the relative phase-locked condition must be maintained among the subchannels for joint carrier phase recovery. In practical systems, an integrated superchannel Tx design (with coherent comb generation, optical demux/mux, and data modulation) is preferred to minimize the phase mismatch generated at the Tx. After fiber transmission, we found in simulation that the phase-locked condition is well maintained once the deterministic chromatic dispersion is compensated.

After the joint carrier-phase recovery stage, the three-subchannel signals for both X and Y polarizations can be sent to the joint ICI equalizer. The joint carrier-phase recovery algorithm is compatible with any subsequent DSP including the joint ICI cancellation algorithm.

## 5. Experimental and simulation results and analysis

To demonstrate the “super receiver” architecture and the two proposed joint DSP algorithms, we performed a proof-of-concept experiment and a separate more complete simulation analysis. We considered two possible superchannel scenarios with different system configurations, as shown in Fig. 6(a).

For scenario 1, optical filtering of each subchannel is omitted throughout the link, and the Tx spectra are limited only by the bandwidth of the generating drivers and modulators. The sub-carrier separation is done strictly by the electrical filters at the receivers. Therefore, the ISI penalty from narrow optical filtering is avoided. However, due to the highly overlapped subchannel spectra, severe ICI exists. We examine this extreme case as a test of the robustness of the joint ICI cancellation algorithm.

Scenario 2 represents a more practical Nyquist-WDM superchannel setup, where each subchannel is optically filtered before multiplexing to minimize ICI. Electrical filters are used to separate the subchannels at the receiver side. The trade-offs between ICI and ISI effects require an optimal design of the optical filter shape and bandwidth, which is analyzed in our simulations. As mentioned, electrical pre-filtering may achieve more accurate spectral shaping, however, we consider the optical pre-filtering method as an example of the general Tx-side approaches, and study the ICI impairments induced by imperfect spectral shaping under different Tx filter configurations and investigate the performance benefits of the “super receiver” approach. Examples of Tx optical spectral shapes of 32GBaud QPSK signal under different optical filter bandwidths are shown in Fig. 6(b).

The joint ICI cancellation algorithm is validated under both system scenarios. Under scenario 1, we performed a proof-of-concept experiment with a two-subchannel setup. For scenario 2, a more complete simulation analysis under a three-subchannel configuration was conducted, and the optimal optical filter bandwidth was examined under different channel-spacing conditions. We also demonstrated the performance of the joint carrier-phase recovery algorithm, using simulations for both three-subchannel and five-subchannel configurations in a carrier phase-locked system of scenario 2.

#### 5.1 Joint adaptive ICI cancellation algorithm results (experimental and simulation)

We experimentally investigated scenario 1 using a two-subchannel system without Tx optical filtering, as depicted in Fig. 7.

Two independent lasers each carrying 28Gbaud (112Gb/s) DP-QPSK data were muxed together without optical filters. After back-to-back (BTB) transmission, the two-subchannel signals were split into two paths, and were received by two coherent receivers with two synchronized 40GSamples/s Agilent sampling oscilloscopes with a 3dB analog bandwidth (BW) of 16GHz and a 6dB BW of 18GHz. Channel spacing was varied from 50GHz to 25GHz.

The BER (bit-error ratio) vs. OSNR (optical signal-to-noise ratio) was measured for each channel spacing and the required OSNR to achieve a BER of 10^{−3} was determined, Fig. 8. The performance of both the conventional independent demodulation method and the LMS ICI cancellation algorithm is compared. The corresponding simulation results are shown for comparison. The conventional algorithm includes a long-memory ISI equalizer (15 taps) to mitigate the filtering induced ISI effects and is a typical algorithm used for Nyquist-WDM superchannel signal demodulation without joint DSP [17].

The required OSNR at large channel spacing, >2 × BW_{elec} (twice the electrical BW of the sampling scope), is ~15.5dB for both experiment and simulation indicating good performance for 28Gbaud DP-QPSK as well as good simulation accuracy in the low ICI regime. Decreasing the channel spacing to less than the electrical bandwidth results in increased ICI and increased required OSNR, Fig. 8. However, with the joint ICI equalizer applied, much of the ICI penalty is recovered. For this two-subchannel case, the 30GHz channel spacing has a 2dB penalty using the conventional independent process yet the joint ICI approach reduces this penalty to 1dB, demonstrating the capabilities of the joint ICI cancellation approach. We note that the experimental penalties are larger than the simulation penalties likely resulting from slightly different spectral shape and the high sensitivity to ICI. Note in this particular system setup, since no optical filter is applied at multiplexing, there is significant ICI and the conventional algorithm experiences large ICI penalties.

As mentioned, Scenario 1 is an extreme case without Tx spectral shaping and the two-channel setup does not capture all the crosstalk on both side of the subchannel. Therefore, a three-subchannel configuration of Scenario 2 was studied via simulation to test the joint DSP methods under more practical subchannel filtering conditions. We performed simulation analysis of 3 × 32Gbaud (3x128Gb/s) DP-QPSK superchannel system with different optical filters applied and optimized to reduce ICI while minimizing the induced ISI penalty. The optical filter shape was chosen as super-Gaussian with order 3.5, which is a reasonable model for most AWGs (Arrayed Waveguide Gratings) and WSSs (Wavelength Selective Switches) used in conventional DWDM systems. Therefore, no special design of either Tx optical shaping filters or electrical pulse shaping is required in the system examined. The optical filter bandwidth was varied from 24GHz to 50GHz. At the receiver side, three sampling receivers were synchronously triggered, with the digital sampling rate of 80GSample/s per subchannel. The electrical filters at the digitizer were 10th order Bessel filters with a 30GHz 3dB bandwidth. The relatively wide electrical bandwidth minimizes the fixed electrical filtering effects, emphasizing the optimization of the optical filter bandwidth. It is noted that for a given channel spacing and channel filtering condition, the equalizers in the DSP will adjust their filter shapes adaptively to yield an optimized performance. The overall optimal performance still needs to be examined under different combinations of optical filter BW and channel spacing conditions.

Firstly we tested the required OSNR to obtain both 10^{−2} and 10^{−3} BER vs. channel spacing. We compared the conventional method with our ICI cancellation algorithm under different optical filter bandwidth configurations, Fig. 9. For all the optical filter bandwidth cases ((a) to (d)), the absolute performance deteriorates as the channel spacing decreases. The joint LMS ICI cancellation algorithm always shows performance gain over conventional methods at narrow channel spacing in all the filter bandwidth cases. For the very narrow optical filter case (30GHz, Fig. 9(a)), the joint DSP method provides less benefits due to the reduced ICI from narrow filtering. However, very narrow filtering induces ISI penalties, which can be seen by comparing the performance floors in Fig. 9(a) and 9(d) at wide channel spacing cases (near 50GHz). In particular, the required OSNR to achieve BER = 10^{−3} for the 30GHz optical filter case (15.8dB) is about 1dB higher than for the 50GHz optical filter case (14.8dB) while both are in the ICI-free regimes (near 50GHz channel spacing). Narrower filtering results in dramatically larger ISI penalties. Therefore, a trade-off between ISI and ICI exists, which requires optimal design of the optical filter bandwidth for a given channel spacing.

To investigate the optimal filter bandwidth for a given channel spacing, we determined the required OSNR to obtain a specific BER (10^{−2} and 10^{−3}) vs. optical filter bandwidth under different channel-spacing conditions, Fig. 10.

At wide channel spacing case (50GHz; Fig. 10(a)), both methods approach the single-channel reference limit, since there is negligible ICI. At narrow optical filter bandwidth strong ISI penalties occur and the intra-channel equalizer in each method is equally effective. Thus, when there is negligible ICI, it is preferable to maintain as large an optical filter bandwidth as possible. As the channel spacing is reduced (35GHz; Fig. 10(b)), ICI penalties appear at wide optical filter bandwidths. ICI cancellation now yields a distinct advantage, reducing the penalty and producing the insensitivity to optical filter bandwidth. For Nyquist channel spacing (32GHz; Fig. 10(c)) and narrower (31.25GHz; Fig. 10(d)), stronger ICI penalties are observed for the conventional method. However, with the LMS ICI cancellation algorithm applied, the ICI penalties are significantly reduced. Again the performance is insensitive to filter bandwidth, in contrast to the conventional demodulation which exhibits a very strong dependence on optical filter bandwidth. For example, for the sub-Nyquist spacing of 31.25GHz, the optimal filter bandwidth for the conventional method is ~30GHz with the required OSNR of 20dB to achieve BER = 10^{−3}. The LMS ICI equalizer yields ~18dB OSNR requirement for any filter bandwidth. Thus, joint LMS ICI cancellation methods produce better absolute performance, and relax the optical filter bandwidth requirement.

Based on the results shown in Fig. 10, we chose an optical filter BW of 34GHz for the Nyquist spacing case (32GHz) and examined the performance over a 12-span transmission link. Three-subchannel signals were transmitted through 12 spans of 80km SSMF with launch power of 0dBm per subchannel. We compared the results with single channel reference and the back-to-back (BTB) case, Fig. 11(a). In order to successfully cancel the crosstalk through the LMS ICI equalizer, a joint chromatic-dispersion compensation is required in front of the joint DSP. This shifts the reference frequency of frequency-domain CD compensation of the side-subchannels to align with the center channel of interest. The detailed discussion of the joint CD compensation can be found in [20].

Firstly, the BTB performance is shown, demonstrating again a 2.3dB OSNR gain of the joint method compared to the conventional method at BER = 10^{−3}. After 12 spans (960km) of SSMF transmission, the single channel performance shows only a small penalty compared to the BTB case. On the other hand, the 12-span multi-channel conventional DSP case reveals a penalty >5dB with respect to the single-channel BTB case, and ~1dB penalty compared to the multi-channel BTB case. With the ICI equalizer applied, the penalty is reduced by ~1.5dB, which demonstrates the effectiveness of the joint DSP in both linear and nonlinear transmission regimes.

To study the effect of timing offset between subchannels due to the imperfect synchronization or path mismatch across the coherent receivers, we determined the OSNR gain with respect to the number of time-domain offset symbols. The OSNR gain is measured under the optimized system setup at 32GHz channel spacing with 34GHz optical filter bandwidth. Different ICI-filter tap lengths in the LMS ICI equalizer were tested as shown in Fig. 11(b). Note that the ICI equalizer operates with 2 samples per symbol, so filter length of 5 taps spans 2.5 symbols. As the results indicate, as long as the number of offset symbols is within the filter memory range, the ICI equalizer is effective. Thus by increasing the filter tap length, it is more tolerant to the timing offset. These results demonstrate that the requirement of perfect synchronized sampling and exact path matching among the subchannels can be relaxed if appropriate time-domain memory is applied in the ICI equalizer.

#### 5.2 Joint carrier-phase recovery algorithm results (simulation)

Joint carrier-phase recovery requires phase-locked sources and LO’s. Therefore we examined simulations with phase-locked carriers under system scenario 2. In the simulation, the phase-locked carriers were generated by a phase modulator, which was driven by a sinusoidal RF wave that determines the channel spacing between the carriers. Each subchannel carried 28Gbaud DP-QPSK data and was spectrally shaped by a 3.5th order super-Gaussian optical filter of 38GHz. At the receiver, the superchannel signal was passive split and fed into three coherent receivers, and the corresponding phase-locked LOs were generated in the same way as at the transmitter side. The digital sampling rate was 40GSample/s per subchannel, and the bandwidth of electrical filters was set to 19GHz. The linewidth of the lasers was fixed at 0.1MHz and the Tx laser and LO are assumed to be perfectly aligned in frequency domain. In practice, methods to track the relative wavelength drift between the sources and LO’s are feasible and are similar to the frequency-offset estimation for conventional single channel demodulation. Both three-subchannel and five-subchannel configurations were tested. The BER vs. OSNR results with and without joint carrier-phase recovery reveals a consistent ~0.5dB improvement at all channel spacing conditions, Fig. 12(a). Further investigations of a five-subchannel (channel spacing at 31.25GHz) simulation allowed a comparison of three different carrier-phase recovery algorithms: conventional V-V, joint V-V, and joint V-V using only outer-subchannels phase information, Fig. 12(b). For the last method, the estimated carrier phase of the two outer subchannels was averaged and the estimated phase information was used to update all the inner channels. It is found that this method works as well as the joint V-V case, demonstrating the effectiveness of using only the outer-subchannel carrier-phase information to recover all the inner-channel carrier phase. The improvement of joint V-V over conventional V-V is ~1/3dB for all OSNR cases. Since this joint V-V algorithm is compatible with the joint ICI equalizer based on the same “super receiver” architecture, it is beneficial to implement this joint V-V algorithm in phase-locked superchannel systems to provide additional performance gain.

## 6. Conclusions

We proposed a novel coherent receiver architecture for superchannel coherent optical systems. Based on this “super receiver” structure, two joint DSP algorithms for linear-ICI cancellation and joint carrier-phase recovery are developed and demonstrated through experimental and simulation results.

The ICI cancellation algorithm was shown to systematically improve performance whenever spectral overlap occurs between adjacent subchannels. Timing offset between subchannels was shown to be readily managed by increasing the time-domain memory size of the ICI equalizer. We also demonstrated that the proposed Rx-side approach can be optimized together with either electrical or optical spectral shaping at the Tx to create a more flexible solution for a dynamic network. Furthermore we showed that the joint ICI cancellation methods enable sub-Nyquist channel spacing with modest penalty. These methods will also benefit Nyquist-like systems where spectral shaping at the transmitter is performed exclusively by optical filters and there is ICI among the channels.

For the joint carrier-phase recovery algorithm, consistent performance improvement over the conventional Viterbi-Viterbi carrier-phase recovery algorithm was observed in carrier phase-locked systems, and the feasibility of using only outer-subchannel phase information for all the inner-subchannel carrier-phase recovery was successfully demonstrated.

As a result, the proposed “super receiver” architecture enables joint digital signal processing to compensate cross-channel impairments as well as more accurate estimation of transmission channel characteristics, and therefore greatly enhance coherent receiver performance for highly spectral-efficient superchannel systems.

## Acknowledgments

The authors recognize the continued support of the Georgia Tech Terabit Optical Networking Consortium members for making this research possible.

## References and links

**1. **E. Torrengo, R. Cigliutti, G. Bosco, G. Gavioli, A. Alaimo, A. Carena, V. Curri, F. Forghieri, S. Piciaccia, M. Belmonte, A. Brinciotti, A. La Porta, S. Abrate, and P. Poggiolini, “Transoceanic PM-QPSK terabit superchannel transmission experiments at baud-rate subcarrier spacing,” ECOC 2010, We.7.C.2.

**2. **Y. Cai, J. X. Cai, C. R. Davidson, D. Foursa, A. Lucero, O. Sinkin, A. Pilipetskii, G. Mohs, and N. S. Bergano, “High spectral efficiency long-haul transmission with pre-filtering and maximum a posteriori probability detection,” ECOC 2010, We.7.C.4.

**3. **X. Liu, S. Chandrasekhar, B. Zhu, P. J. Winzer, A. H. Gnauck, and D. W. Peckham, “Transmission of a 448-Gb/s reduced-guard-interval CO-OFDM signal with a 60-GHz optical bandwidth over 2000 km of ULAF and five 80-GHz-grid ROADMs,” OFC 2010, PDPC2.

**4. **S. Chandrasekhar and X. Liu, “Experimental investigation on the performance of closely spaced multi-carrier PDM-QPSK with digital coherent detection,” Opt. Express **17**(24), 21350–21361 (2009). [CrossRef] [PubMed]

**5. **A. Sano, E. Yamada, H. Masuda, E. Yamazaki, T. Kobayashi, E. Yoshida, Y. Miyamoto, R. Kudo, K. Ishihara, and Y. Takatori, “No-Guard-Interval coherent optical OFDM for 100-Gb/s long-haul WDM Transmission,” J. Lightwave Technol. **27**(16), 3705–3713 (2009). [CrossRef]

**6. **Y. Cai, J. X. Cai, C. R. Davidson, D. Foursa, A. Lucero, O. Sinkin, A. Pilipetskii, G. Mohs, and N. S. Bergano, “Achieving high spectral efficiency in long-haul transmission with pre-filtering and multi-symbol detection,” ACP 2010, page 349–350.

**7. **R. Schmogrow, M. Winter, M. Meyer, D. Hillerkuss, S. Wolf, B. Baeuerle, A. Ludwig, B. Nebendahl, S. Ben-Ezra, J. Meyer, M. Dreschmann, M. Huebner, J. Becker, C. Koos, W. Freude, and J. Leuthold, “Real-time Nyquist pulse generation beyond 100 Gbit/s and its relation to OFDM,” Opt. Express **20**(1), 317–337 (2012). [CrossRef] [PubMed]

**8. **R. Ryf, S. Randel, A. Gnauck, C. Bolle, A. Sierra, S. Mumtaz, M. Esmaeelpour, E. Burrows, R. J. Essiambre, P. J. Winzer, D. Peckham, A. McCurdy, and R. Lingle Jr., “Mode-division multiplexing over 96km of few-mode fiber using coherent 6 x 6 MIMO processing,” J. Lightwave Technol. **30**(4), 521–531 (2012). [CrossRef]

**9. **Y. Ma, Q. Yang, Y. Tang, S. Chen, and W. Shieh, “1-Tb/s per channel coherent optical OFDM transmission with subwavelength bandwidth access,” OFC 2009, Paper PDPC1.

**10. **R. Dischler and F. Buchali, “Transmission of 1.2 Tb/s continuous waveband PDM-OFDM-FDM signal with spectral efficiency of 3.3 bit/s/Hz over 400 km of SSMF,” OFC 2009, Paper PDPC2.

**11. **G. Bosco, A. Carena, V. Curri, P. Poggiolini, and F. Forghieri, “Performance limits of Nyquist-WDM and CO-OFDM in high-speed PM-QPSK systems,” IEEE Photon. Technol. Lett. **22**(15), 1129–1131 (2010).

**12. **S. Chandrasekhar and X. Liu, “Terabit superchannels for high spectral efficiency transmission,” ECOC 2010, Tu.3.C.5.

**13. **D. Hillerkuss, M. Winter, M. Teschke, A. Marculescu, J. Li, G. Sigurdsson, K. Worms, W. Freude, and J. Leuthold, “Low-complexity optical FFT scheme enabling Tbit/s all-optical OFDM communication,” ITG Symposium on Photonic Networks, 2010Paper 5.

**14. **K. Takiguchi, M. Oguma, T. Shibata, and H. Takahashi, “Optical OFDM demultiplexer using silica PLC based optical FFT circuit,” OFC 2009, OWO3.

**15. **G. Gavioli, E. Torrengo, G. Bosco, A. Carena, V. Curri, V. Miot, P. Poggiolini, M. Belmonte, F.Forghieri, C. Muzio, S. Piciaccia, A. Brinciotti, A. La Porta, C. Lezzi, S. Savory, and S. Abrate, “Investigation of the impact of ultra-narrow carrier spacing on the transmission of a 10-Carrier 1Tb/s superchannel,” OFC 2010, OThD3.

**16. **A. J. Viterbi and A. M. Viterbi, “Nonlinear estimation of PSK-modulated carrier phase with application to burst digital transmission,” IEEE Trans. Inf. Theory **29**(4), 543–551 (1983). [CrossRef]

**17. **Z. Dong, J. Yu, H.-C. Chien, N. Chi, L. Chen, and G.-K. Chang, “Ultra-dense WDM-PON delivering carrier-centralized Nyquist-WDM uplink with digital coherent detection,” Opt. Express **19**(12), 11100–11105 (2011). [CrossRef] [PubMed]

**18. **H.-C. Chien, J. Yu, Z. Dong, Z. Jia, “Offset RZ pulse shape for 128 Gb/s PDM-QPSK subject to aggressive filtering,” OFC 2012, Paper OM3H.3.

**19. **J. Pan, C. Liu, T. Detwiler, A. Stark, Y.-T. Hsueh, and S. E. Ralph, “Inter-channel crosstalk cancellation for Nyquist-WDM superchannel applications,” J. Lightwave Technol. **30**(24), 3993–3999 (2012). [CrossRef]

**20. **C. Liu, J. Pan, T. Detwiler, A. Stark, Y.-T. Hsueh, G.-K. Chang, and S. E. Ralph, “Joint digital signal processing for superchannel coherent optical system: joint CD compensation for joint ICI cancellation,” ECOC 2012, Paper Th.1.A.4.