1. INTRODUCTION

A. Emerging Expectations for DCX

As cloud services are expanding, there have been increasing efforts to relocate data centers previously concentrated in urban areas to suburban areas with abundant power and infrastructure space to equalize regional energy supplies. Sovereign clouds, which keep data within a designated geographic area to improve data security and privacy, are also becoming more prevalent. In addition, the growing use of private 5G technology has increased the need for high-speed, low-latency optical links between remote locations. In particular, there are increasing cases where data owned in local data centers are connected to cloud services and mobile edge computing (MEC) using dark fiber and high-capacity optical links. However, these demands are met with significant challenges, such as the time it takes to procure dark fiber and start using the network, and the complexity and cost of the network increases as the number of locations to be connected increases.

Against this backdrop, there are growing expectations for data center exchange (DCX) services that directly connect any DCs distributed in a metro area via optical fiber to quickly establish high-capacity optical paths between each company’s cloud services and MECs and the users connected to them [1].

Figure 1 shows an example of a DCX configuration. Urban data centers (UDCs) are located in the gray area in the center of Fig. 1, and they are connected by carrier links (CLs) owned by the carrier, which provides a point of presence (POP) in each UDC. User equipment can connect from a suburban data center (SDC) to the POP in the UDC via alien access links (AALs) and directly to a remote cloud MEC or other user equipment without electrical conversion. Here, the user equipment can be located at either an SDC or UDC, and if it is at a UDC, it connects directly to the POP without an access line.

 

Fig. 1. Example of DCX physical deployment.

Download Full Size | PDF

B. Service Requirements for DCX

Several challenges still remain in the development of the DCX services that require real-time WDM provisioning to quickly fulfill user demand, and protocols to enable user–carrier cooperation automatically. Also, when a failure occurs in a user transceiver (TRx) installed outside the carrier network, the carrier needs to isolate the root cause and perform restoration work. The service requirements for DCX are broken down as follows:

Here, the target response time required for (1) and (2) is set to be around 10 min. [For (2), the case where the quality of transmission (QoT) of multiple optical links is degraded at once is included, this target is defined as the time required to restore one end-to-end (EtE) optical path between a particular user’s muxponder.] The latency in (3) is assumed to be within 2 ms, dominated by fiber propagation delay. This delay corresponds to a metro area of roughly 200 km in radius, depending on the fiber installation conditions.

C. Importance of Open Interface/Specification

An open architecture that enables fast WDM provisioning between various devices must be designed to address requirements (1)–(4) described in Section 1.B. The architecture should allow DCX operators to exchange information to calculate the QoT of each link in real time, independent of the vendor or implementation of each device. This requires accurately estimating the generalized signal-to-noise ratio (GSNR) that indicates the signal quality at the physical layer in all possible transmission channels between the user sites.

For requirements (4) and (5) described in Section 1.B, interoperability is crucial among users, clouds, and MECs because each player uses their preferred devices which are adapted to their system and operation.

Standardization organizations such as the Optical Internetworking Forum [2], OpenROADM MSA [3], and OpenZR+ MSA [4] have defined common hardware specifications for data plane interoperability. For control plane interoperability, there is no common specification that addresses the service requirements for DCX within the optical transport industry, while the OpenROADM MSA, Telecom Infra Project Open Optical & Packet Transport (TIP OOPT) [5–7], and IOWN Global Forum (IGF) [8] provide architectures and open interfaces that partially meet these needs identified here.

D. Fast WDM Provisioning Approach Based on Open Architecture

In this paper, we propose a fast WDM provisioning approach for WDM links, including two types of commercial TRxs implemented with indium phosphide (InP) and silicon photonics (SiP) modulators. Our approach uses a technique to estimate the GSNR of all possible EtE optical paths quickly and accurately by only using a probe WDM channel that measures QoT link by link at measurable, convenient arbitrary modulation formats and modulators.

We have verified a minimal probing method and an integrated Linux-based device software architecture that utilizes open interfaces, specifications, and architectures defined by the OpenROADM MSA, TIP OOPT, and IGF. We leverage OpenROADM MSA-compliant coherent TRxs to ensure data plane interconnectivity. For the hardware abstraction interface and network operating system (NOS) that controls user TRxs, we applied the TAI architecture and the Goldstone NOS that are under development openly in TIP OOPT. The architecture proposed in the IGF was adopted for the network architecture of user TRx and POP. We implement software libraries for fast WDM provisioning, and monitoring/configuration of the user’s TRx to provide control plane interoperability. To estimate the EtE GSNR accurately and design the optimal optical path in a short time, it was necessary to automatically obtain the back-to-back TRx characteristic because this is the most critical parameter for improving accuracy in metro areas. The proposed device software architecture utilizes server-based technologies such as containers and a hardware abstraction interface for coherent modules, and then solves the issue of automatic obtaining of the back-to-back TRx characteristic currently not supported by standard YANG models, such as the OpenROADM YANG model and the OpenConfig YANG model.

In addition, we experimentally verified the accuracy of this approach using three different routes of 32 km/72 km/122 km in the C-band. For the operational EtE GSNR measurements, the accuracy estimated from the sum of the measurements for each link was 0.6 dB, and the wavelength-dependent error was about 0.2 dB. Then, using field fibers deployed in an urban area, a Linux-based transmission device software architecture, and coherent transceivers with different optical frequency ranges, modulators, and modulation formats, the fast WDM provisioning of an optical path was completed within 6 min.

The remainder of this paper is organized as follows. After introducing related work in Section 2, we introduce our approach to estimating the GSNR of all possible EtE optical paths quickly with high accuracy in Section 3. Section 4 presents a Linux-based muxponder device software architecture that includes a DCX operator agent as a container. The architecture also allows for optimization of different settings of coherent TRxs from multiple vendors with different optical frequency ranges and modulators. In Section 5, we evaluate and compare the estimation error of this method and the penalty due to wavelength dependence and wavelength assignment of the device against the actual EtE GSNR measurements. In Section 6, we report the time measurements for the quality estimation of each link, EtE optical path margin design, and remote configuration of user muxponders using an experimental setup containing field fibers. Section 7 discusses future challenges for improving accuracy and expanding system flexibility, and Section 8 concludes the paper.

 

Fig. 2. Example network configuration of DCX. (a) Configuration of OSaaS. (b) Configuration of DCX. (c) POP configuration example.

Download Full Size | PDF

2. RELATED WORK

A converged inter/intra data center network architecture with an autonomous control plane for flexible bandwidth allocation has been introduced as a way to quickly establish high-capacity optical paths between distributed DCs in a metro area [9]. The architecture includes two types of physical layer connections (background and dynamic) to meet stringent bandwidth and delay requirements. Subsequently, Ju et al. targeted the provisioning of disaster-resilient cloud services using distance adaptive modulation formats by simultaneously considering content placement, modulation-format adaptive routing, path and content protection, and spectrum allocation [10]. They investigated frequency utilization with different traffic parameters and different network topologies by numerical simulations. However, neither of these prior studies considered WDM provisioning.

Kaeval et al. have conducted a number of studies on transmission mode optimization approaches for WDM provisioning [11–13]. They proposed an approach for provisioning WDM using QoT probe channels in a transparent optical path connecting two endpoints in a single- or multi-domain optical network, referred to as optical spectrum as a service (OSaaS). Kaeval et al. estimated the GSNR in EtE between the TRxs of OSaaS users [11,12] and proposed an approach to estimate the GSNR EtE using the estimated GSNR for each link that constitutes the EtE route [13]. In both cases, the GSNR is estimated for each wavelength using a probe channel with the same wavelength as the signal light in operation, which requires time for provisioning in the case of multi-wavelength multiplexed transmission. Furthermore, while OSaaS is a P-to-P connection, DCX is a many-to-many connection deployed on a two-dimensional plane, as shown in Fig. 2(b), so the model is more complex than that in [13], with many combinations of optical path routes and wavelengths. The differences between the models will be discussed in detail in Section 3. OSaaS does not provide an architecture for users and carriers to work together when setting WDM paths. Furthermore, a single vendor device is applied to the transmitter and receiver, and no study or experiment has been conducted to set up optical paths between optical transmission devices with various optical frequency ranges, modulators, and modulation formats installed at each connection point, such as DCX. This will be discussed in detail in Section 4.

An architecture for users and carriers to work together using optical transmission devices with optical frequency range and modulation format has been proposed in [14]. In the proposed architecture and protocol, a mode catalog, including the modulation format and characteristics of the coherent TRx, is generated at the user site, the QoT of the link is estimated, and these are sent to the carrier’s controller to optimize and configure the user site equipment. In [15], it was determined that the back-to-back characteristics of coherent TRxs can be accurately modeled for different types of coherent implementations and modulators. However, [14,15] do not mention network topology, specific steps for mode optimization, or a device software architecture for user TRx remote control, and only laboratory experiments have been conducted.

3. FAST AND ACCURATE GSNR ESTIMATION OF ALL POSSIBLE EtE OPTICAL PATHS

This section presents an approach to WDM provisioning and selecting the optimal mode in a short time, which is essential for satisfying service requirements (1)–(4).

Figure 2(a) shows the OSaaS configuration [11], which consists of a multi-domain open line system and can accommodate many users by applying DWDM. Figure 2(b) shows an example of a network forming a DCX, in which a functionally disaggregated ROADM device [8] consisting of an optical switch and optical aggregation is applied to the POP in Fig. 2(c). As in Fig. 2(a), DCX can efficiently accommodate many user groups with DWDM technologies.

Figure 3(a) shows possible patterns in which two user sites can be connected via DCX. If the number of POPs between user sites is limited to three to minimize latency, there are four patterns as shown in Fig. 3(a) (two POPs: blue line, three POPs: three red lines). When connecting the five sites shown in Fig. 2(b), there are more than 40 possible patterns. To satisfy service requirements (1)–(4) presented in Section 1, the EtE GSNR between each site must be determined accurately. The optimal mode must be selected in a short time (here, we assume that existing techniques can be applied to routing and wavelength assignment and focus our discussion in this paper on the fiber propagation design of the WDM optical path). We have previously determined that the back-to-back characteristics of coherent TRxs can be accurately modeled following Eqs. (1) and (2) for various modulation formats, baud rates, and modulator types [15]. Doing so allows the Gaussian noise approximation to be applied in the metro region as well as the long haul. Equation (1) relates the bit error rate (BER) to the back-to-back GSNR as a function of each modulation format, such as 16QAM or QPSK, and Eq. (2) shows two examples of error functions. We also previously found that the reciprocal of the GSNR is simply the sum of the reciprocal contributions of three SNR terms originating from three noise sources as Eq. (3): intrinsic back-to-back TRx characteristics, spontaneous emission of the amplifier, and fiber nonlinear optical effects [15]. In the previous study, we used the BER and GSNR as QoT metrics. The ${{Q}^2}$-factor, another widely used QoT metric, has the following one-to-one relationship with the BER: ${\rm BER} = \frac{1}{2}{\rm erfc}\left({\sqrt {\frac{{{Q^2}}}{2}}}\right)$. Thus, we can use the BER and ${{Q}^2}$-factor interchangeably; this paper mainly uses the BER:

 

Fig. 3. Approach to selecting the optimal mode with high accuracy in a short time. (a) Routes between two user sites (maximum three POPs). (b) Link-by-link, multi-mode/modulator probing, and optimal mode selection.

Download Full Size | PDF

In [15], we also experimented with two types of commercial TRxs implemented with InP and SiP modulators and verified that Eqs. (1)–(3) were valid for both modulators. Furthermore, we assume that the reciprocal of the EtE GSNR is equal to the reciprocal of back-to-back TRx characteristics ${{\rm SNR}_{{\rm TRx}}}$ and the sum of the reciprocal of the ${{\rm GSNR}_{{\rm LS},n}}$ of each link, as in Eq. (4). Although it has been reported in [13] that this equation is valid with an accuracy of $\pm {1.4}\;{\rm dB}$ for 3116 km transmission as in Fig. 2(a), no experiments have yet been performed to verify the accuracy in the case of Fig. 2(b):

Using these results, the EtE GSNR of a potential path can be estimated in a short time without directly measuring the QoT, regardless of the modulation format and modulator type of the TRx. Figure 3(b) shows the steps to measure QoT on a link-by-link basis for one of the routes in Fig. 3(a) using a measurably convenient mode and multiple types of modulators and then calculating the EtE GSNR to select the operational mode. For example, ${{\rm GSNR}_1}$ is measured between TRxs with InP and SiP modulators. Since the QoT is relatively high for a single span, the BER is measured with the 16QAM 64Gbaud mode and converted to the GSNR (the method for obtaining ${{\rm SNR}_{{\rm TRx}}}$ will be discussed in the last paragraph of Section 4). ${{\rm GSNR}_{\rm{EtE}}}$ is then calculated by summing the link-by-link reciprocal GSNRs, and the optimal mode can be selected after adding the operational margin [16]. In this example, ${{\rm GSNR}_{{\rm LS},n}}$ is estimated using 16QAM 64Gbaud, and QPSK 64Gbaud is selected by adding an operational margin to the calculated ${{\rm GSNR}_{\rm{EtE}}}$. Since this approach uses common parameters (BER, modulation format, and baud rate), it can be applied to all coherent TRxs without being restricted to a specific device.

 

Fig. 4. Obstacles in user TRx remote control.

Download Full Size | PDF

4. DEVICE SOFTWARE ARCHITECTURE

The approach presented in Section 3 makes it possible to select a feasible and optimal transmission mode for fiber propagation design. However, there are still issues to be addressed in satisfying requirements (4) and (5) in Section 1.B, which will be discussed in this section. In particular, the back-to-back TRx characteristic described in Section 3 is critical for accurately estimating the EtE GSNR; in the case of DCX services, the DCX operator needs to extract this characteristic from the TRx at the user site to design the optical path accurately. Later in this section, we introduce a device software architecture and procedure to improve the accuracy of the GSNR and automatically optimize the mode in a short time without changing existing open interfaces and MSAs.

Figure 4 illustrates the drawbacks of when the carrier remotely controls the user TRx as a DCX servicer.

Difficulty of root cause analyses: If a failure occurs at a user site after the service has been launched, the carrier, as a DCX service provider, is required to isolate the failure on its own. The approach for the carrier to monitor and control the TRx at the user site would be to manage the TRx from the controller via the NOS using an open YANG model or implementing a remote monitoring and control function within the TRx. However, no open specification has been defined to monitor and control the TRx installed in user sites by utilizing vendor-proprietary fault inspection or investigation functions. Though there have been efforts to enhance traditional standardized YANG models for network design and operation automation [17], no interface has been defined for utilizing vendor-proprietary functions on these models.

Interoperability issue: Accidents caused by device connectivity should be prevented to ensure the safe operation of network services. For example, when using TRxs from multiple vendors, accidents can occur due to specification mismatches between the NOS and the TRx, causing the optical power input from the user device to exceed the regulated value or signals with an unexpected optical frequency being input into the network during network connection. Even for TRxs with a specified MSA, there are cases where vendors implement specifications outside the MSA to meet their customers’ requirements, as well as cases where the units of parameters differ between vendors in terms of dBm or linear power (mW). Also, since each vendor TRx has a different optical frequency range, the optical frequency corresponding to f1 defined by the vendor differs for each one. For these reasons, it is difficult for carriers to remotely restore the system when a failure occurs due to a specification mismatch between the NOS and TRx at a user site.

Utilization of TRx characteristics: As shown by Eq. (3) in Section 3, estimating the EtE GSNR requires TRx characteristics, but a new mechanism will be needed when the carrier controller designs an optical path between the TRxs at the user sites.

In this paper, we propose a device software architecture to remove the obstacles shown in Fig. 4. (Note that the physical network connection method for the control signal between the controller and the user muxponder and the user authentication method is outside the scope of this paper.) Figure 5 shows the device software architecture used in this experiment, which enables the carrier to monitor and control a TRx in the user muxponder. The hardware is Phoenix [5], as specified in the TIP OOPT, and the TRx is based on the OpenROADM MSA specification [3]. Two CFP2-DCOs from vendors (A and B in Fig. 5) are implemented. The IGF’s disaggregated architecture described in the Open All-Photonics Network Functional Architecture [8] is used for the user muxponder and ROADM network configuration. The device software used is the NEC NOS, developed on the basis of Goldstone NOS [6] at the TIP OOPT GitHub site. Gearbox and coherent optics control functions are implemented as containers. In this experiment, the carrier’s coherent optics controller and carrier agent (red) are added as containers in addition to the group of containers initially installed on the user muxponder (blue). The TAI architecture hides the differences between various TRx form factors or vendors. It consists of a library called libTAI which absorbs these differences and an open interface, which enables TAId to monitor and control TRx A/B agnostically in regard to form factor type or vendor type. We developed LibTAI for TRx A/B and used it in this experiment, and then performed mode optimization between various devices using the common parameters (BER, modulation format, and baud rate). Since only common parameters are used for mode optimization, it can be done on any standard TRx.

 

Fig. 5. Example of container-based device software architecture.

Download Full Size | PDF

TAI is designed to add attributes for using vendor-proprietary functions as well as attributes for standard functions. The carrier can remotely control these containers and directly access the user’s TRx registers with TAI and an in-house library to ensure the safe operation and robustness of the DCX. For example, the CFP MSA defines a module vendor private register [18]. Vendors use this to provide their proprietary functions, conducting pre-shipment inspections and root cause investigations for unexpected faults. If carrier controllers can take advantage of this capability with containers, they can use this information to conduct more advanced root cause investigations without requesting the actions of the user. Furthermore, carriers and users do not need to disclose the source code to each other when utilizing containers with executable binaries, and operational management, such as firmware updates, can be performed immediately and independently. Note that the automatic firmware upgrade of the coherent module has been demonstrated in a multi-vendor environment [19].

Figure 6 shows an example of the user muxponder controlled by the carrier. When the user muxponder connects to the DCX, it obtains and installs the necessary container from the carrier controller and sends TRx data such as vendor name, firmware version, and parameters to the carrier controller. The carrier controller then checks to see whether the TRx is certified and requests information on the TRx characteristics from the user muxponder if it has been executed. We assume that an executed nondisclosure agreement is in place between the carrier and the TRx vendor [20]. There are several possible approaches for vendors to provide data on the TRx characteristics. For example, some TRx vendors store back-to-back BER versus OSNR and other characteristics in registers in the TRx at the shipment. In this experiment, we stored these characteristics as a JSON file in the user muxponder. The time required to implement and complete the sequence shown in Fig. 6 was measured to be 90 s, which was determined to be sufficiently short.

 

Fig. 6. Example of the carrier procedure for monitoring and controlling the TRx in the user muxponder.

Download Full Size | PDF

5. EtE GSNR MEASUREMENT RESULTS AND ESTIMATION ERRORS

This section presents the evaluation results of the errors from the link-by-link measurement approach introduced in Section 3, as well as wavelength dependence and Q-value variation, using commercial open systems and field fibers.

 

Fig. 7. Equipment in the COSMOS testbed. (b) Field trail setup. (c) Field fibers.

Download Full Size | PDF

A. Experimental Setup

Figures 7(a) and 7(b) show the field trial setup using the NSF COSMOS testbed, a city-scale programmable testbed deployed in Harlem, New York City. The COSMOS testbed provides a programmable optical networking environment, including optical space switches and Lumentum ROADM-20 units, where many experiments have been conducted [21–23]. We experimentally constructed two user sites, two POPs, AAL1, AAL2, and CL, with three different paths (32 km/72 km/122 km) in the C-band by utilizing commercial optical transmission devices and field fibers. We deployed white box muxponders, which comply with TIP’s Phoenix requirements [5], and installed NEC’s NOS, which is based on the TIP Goldstone NOS [6]. These muxponders utilize OpenROADM compliant 400G CFP2-DCO pluggable transceivers from Fujitsu Optical Components and Lumentum. Figure 7(c) shows the Manhattan dark fiber routes we used in this experiment., i.e., 32 km loop-back field fibers between Columbia University and a colocation data center at 32 Avenue of the Americas (provided by Boldyn), and an 8 km loop-back field fiber between Columbia University and the City College of New York (provided by CrownCastle). The first AAL, AAL1, consists of a ROADM and a 32 km field fiber and connects to a CL. The second AAL, AAL2, consists of a ROADM and 40 km spool fiber. The CLs have different lengths and losses for different link conditions, with a total of 25 WDM background channels inserted to emulate existing in-service channels (this will be described in detail in Section 5.C). All input powers to the fiber links were +4 dBm/channel. Two muxponders are placed at the POPs, interfacing the AAL and CL for pre-FEC BER probing measurements. The controller interacts with the muxponders at the user sites and POPs through a pre-established secure channel using gRPC.

B. Link-by-Link Measurement Error

To determine the error of the approach shown in Fig. 3(b), we compared the measured EtE GSNR (${{\rm GSNR}_{\rm{EtE}}}$) and estimated the GSNR with link-by-link measurement (${{\rm GSNR}_1}/{{\rm GSNR}_0}/{{\rm GSNR}_2}$), as shown in Fig. 7(b). The results are plotted in Fig. 8, where GSNR_all on the vertical axis corresponds to the entire GSNR, including the GSNR of the TRx and link.

 

Fig. 8. Measured EtE GSNR and estimated GSNR with link-by-link measurement. (a) Frequency 191.5 THz. (b) Frequency 194.6 THz.

Download Full Size | PDF

Figure 8(a) and 8(b) show the difference between the two GSNRs at 191.5 THz and 194.6 THz, respectively. We changed the CL with the optical space switch and evaluated it in three route patterns: short, middle, and long. There were no significant differences in wavelength or CL distance, with errors ranging from 0.3 to 0.6 dB.

C. Wavelength-Dependent Error

Figure 9 shows the experimental system configuration with an inset showing the WDM spectrum assignment. To measure the wavelength dependence of the link system, we established probing channels listed in the table in Fig. 9 between muxponder 2 and muxponder 3. We changed the CL with the optical space switch and evaluated it in three patterns: short, middle, and long, with 25 WDM background channels and 100 GHz spacing (see the background channels table in Fig. 9) inserted to emulate existing in-service channels. To verify if $\lambda {a}$ (191.5 THz, at the edge of the C-band) can be utilized as a probe signal for channels of other wavelengths, we compared the difference between the GSNR measured at $\lambda {a}$ and that measured at different wavelengths (b, c, d, e). The wavelength-dependent error on GSNR probing caused by the line system was 0.05 dB, 0.11 dB, and 0.22 dB at the short, center, and long wavelengths, respectively. Although $\lambda {c}$ is located in the center of the background channels, no significant difference from $\lambda {a}$ and other wavelengths was observed. This suggests that the waveform degradation due to fiber nonlinear effects was relatively small for the system compared to the noise contribution of the TRx back-to-back characteristics, as expected for this metro-scale system.

 

Fig. 9. Experimental setup for wavelength-dependent error measurement.

Download Full Size | PDF

D. GSNR Fluctuation of System

The measured value of the GSNR in the experimental system constantly fluctuates. Since the estimation error cannot be smaller than this fluctuation, we evaluated the amount of fluctuation. We measured the GSNR every 5 s at three links (AAL1, AAL2, CL1) and the EtE path (EtE) and calculated the amount of fluctuation of the 5-min moving average. We define the amount of GSNR fluctuation as the difference between the 5th and 95th percentile of the 5-min moving average. The measurement duration was 7 days.

The GSNR varied every 5 s by about 0.2–0.4 dB over time (Fig. 10). Since we usually use the average value over several observations, we evaluated the fluctuating amount of the moving average. We set the average window period as 5 min. The 5-min moving average fluctuated from about 0.1 to 0.2 dB over time (Fig. 10). The amount of GSNR fluctuations for AAL1, AAL2, CL, and EtE was 0.15, 0.10, 0.07, and 0.07 dB, respectively (Fig. 10), overall ranging from about 0.1 to 0.2 dB.

 

Fig. 10. Time variations of the GSNR (top) and histograms of the 5-min moving average (bottom).

Download Full Size | PDF

E. Summary of the Estimated Error

Table 1 shows the estimated error of our experimental setup in Fig. 7. The overall margin required for fast WDM provisioning using the link-by-link approach and probe light of any wavelength was 0.7 dB. This value is the same order of magnitude as the system’s GSNR fluctuation, roughly 0.1–0.2 dB, so any available wavelength can be used for the approach shown in Fig. 3 with a small margin.

Table 1. Estimated Error

View Table | View all tables in this article

6. RESULTS OF FAST WDM PROVISIONING EXPERIMENTS USING FIELD FIBERS

In this section, we implement the proposed architecture and evaluate the time required for estimating the quality of each link, designing the EtE optical path margin, and remotely configuring the user muxponder, for two routes (short and long) using the field fibers shown in Fig. 7(c).

Figure 11(a) shows the experiment’s network configuration. The route shown was used for AAL1, CL, and AAL2 in the experimental configuration shown in Fig. 7. Two dark fiber routes (32 and 8 km) shown in Fig. 7(c) were used. The short CL (32 km field fiber) and long CL (32 km field fiber + 90 km spool fiber) have different lengths and losses for different link conditions, with a total of 25 WDM background channels inserted to emulate existing in-service channels (the same as in Fig. 9). Figure 11(b) shows an overview of fast WDM provisioning and the elapsed time. The controller gathers JSON-formatted TRx characteristics from all muxponders and then measures the BER of AAL1, AAL2, and CL with the muxponders. We used $\lambda {a}$ (191.5 THz) as a probing signal. The controller calculates the margins using the approach in Fig. 3(b) and selects the optimal operational mode. Finally, the controller configures the operation mode to the user muxponders and measures the EtE BER. The total provisioning time was 351 s, broken down as follows. Link BER measurements took 270 s, margin design and mode selection took under 1 s, and user muxponder configuration and BER measurement took 80 s.

 

Fig. 11. Fast WDM provisioning experiment. (a) Network configuration. (b) Fast WDM provisioning overview and elapsed time.

Download Full Size | PDF

We provisioned two routes, short and long, assuming a 400G bandwidth request from a user. Table 2 shows the measured BER and converted GSNR for AAL1, CL, and AAL2. After combining these measured GSNR values, our controller estimated the EtE GSNR and provisioned the optical paths (Table 3). For the short route, our controller successfully selected the single 400G DP-16QAM path using $\lambda {b}$ with a 5.2 dB secured GSNR margin. For the detour route, our controller also successfully selected the two 200G DP-QPSK paths using $\lambda {b}$ and $\lambda {c}$ as 400G DP-16QAM is not error-free due to the low EtE GSNR. The secured margins for these two paths were 5.1 and 4.9 dB. In all cases, the differences between the estimated and measured $Q$-factor (BER) were less than 0.7 dB. Most of the time is spent changing the transceiver configurations during the BER measurements. We administratively halted the transceiver for every configuration to ensure a stable transition, which required about a minute each.

Table 2. Measured BER (GSNR)

View Table | View all tables in this article

Table 3. Estimated GSNR EtE, Selected Mode, Secured Margin, Estimated Q-Margin, and Measured Q-Margin

View Table | View all tables in this article

7. FUTURE CHALLENGES

Although the wavelength dependence and fiber nonlinear effects were relatively small in our experimental system with the C-band, other line systems such as C+L multi-band and fibers with various characteristics will likely be routed when providing DCX services. In such cases, techniques for estimating the wavelength dependence of amplifier gain and noise figure and nonlinear effects caused by the fiber will be essential. Wavelength-dependent issues may be solved by applying the latest technology, such as transfer learning, to predict complex amplifier gain profiles [24].

DCX operators will be able to provide extended support services for their users with applying the device software architecture based on containers and TAI for streaming telemetry from TRxs to carrier controller.

We used TAI as the abstraction interface and CFP2-DCO as the coherent TRx, but the DCX needs to accommodate other form factors, such as QSFP-DD, which are widely used by data center operators. Since the source code for QSFP-DD control has been opened in SONiC [25], it should be utilized instead of TAI. An architecture that efficiently accommodates different MSAs such as the CFP MSA [18], the common management interface specification (CMIS) [26], and coherent-CMIS [27] will be required.

8. CONCLUSION

We proposed a technique to quickly and accurately estimate the GSNR of all possible EtE optical paths using a QoT probe channel that measures link by link at measurable, convenient, and arbitrary modulation formats and modulators. We also proposed an approach to select the optimal mode in a short time based on Gaussian noise approximation with a device software architecture that utilizes server-based technologies such as containers and the hardware abstraction interface for coherent modules. Since this approach uses common parameters (BER, modulation format, and baud rate), it can be applied to all coherent TRxs without being restricted to a specific device. These techniques enable DCX services that directly connect many-to-many user sites deployed in a two-dimensional plane via optical paths to respond to sudden traffic demands and quickly reconstruct optical path routes in the event of a disaster. We identified issues related to user support/responsibility and interoperability/network robustness and proposed a Linux-based muxponder device software architecture with a DCX operator agent as a container. We experimentally constructed three different paths by utilizing commercial optical transmission devices and field fibers; the accuracy estimated from the sum of the measurements for each link was 0.6 dB, and the wavelength-dependent error was about 0.3 dB. We combined WDM provisioning with the device software architecture in which users and carriers cooperate to optimize optical path transmission modes. Finally, we evaluated the time required to estimate the link quality for designing the EtE optical path margin and remotely configuring user muxponders. We provisioned two routes utilizing field fibers at 32 and 122 km distances, assuming a 400G bandwidth request from a user. Our controller successfully selected the single lambda 400G DP-16QAM path for the short route, and the two lambda 200G DP-QPSK paths for the long route within 6 min automatically. The estimated and measured $Q$-factor (BER) differences were less than 0.7 dB.