I. Introduction

Thanks to the widespread adoption of coherent technology, optical communication has significantly progressed in recent years and now offers a plethora of design parameters for lightpath deployment. Several choices such as modulation format, baud rate, forward error correction (FEC) coding, single/multicarrier transmission, adaptive channel spacings, and flex-grid network technologies, among others, offer a variety of “degrees of freedom” to system and network engineers, thus making the number of possible combinations for lightpath deployment grow dramatically.

In this context, predicting the lightpath quality of transmission (QoT) prior to deployment is essential to discern the most effective solution and for an optimized design and planning of the optical network. As of today, existing (pre-deployment) estimation techniques for lightpath QoT can be roughly classified into two categories. On the one hand, sophisticated analytical models (e.g., split-step Fourier method [1]) capturing different physical layer impairments can be used to estimate with great precision the bit error rate and reach of a given lightpath, but they impose high computational requirements that are not compatible with real-time prediction and are not scalable to large network topologies and dynamic network operation. On the other hand, approximated formulas (e.g., simplified power budget with nonlinear-impairment estimations based on a Gaussian model [2]) introduce higher link margins in the calculation of the lightpath budget to compensate for model inaccuracies, thus leading to an underutilization of network resources [3].

An alternative approach to QoT prediction relies on sensing the QoT of already deployed lightpaths by means of optical performance monitors (OPMs) [4] installed at the receiver side and on exploiting the knowledge extracted from field data to predict the QoT of unestablished lightpaths [5]. To this aim, different machine-learning (ML) techniques have been recently investigated, e.g., network kriging [6], case-based reasoning [7], and neural networks [8,9].

In this paper, we investigate and apply a ML-based classifier to predict the probability that the bit error rate (BER) of a candidate lightpath will not exceed the system tolerance threshold, using as features the traffic volume to be served, modulation format, lightpath total length, length of its longest link, and number of lightpath links. To train the classifier, we assume that either BER measurements over already-deployed lightpaths are provided by field OPMs¹ (or directly by optical transceivers) or that, in the absence of real field data, a BER estimation tool (E-Tool) is used to generate synthetic data. In the remainder of this paper, we opt for the latter approach due to the difficulty of retrieving field data. The classification output is meant to be provided to a routing and spectrum assignment (RSA) algorithm that will make the final deployment decision.

In our performance assessment, we provide a specific focus on how classification performance is influenced by the choice of the training data: relying on historical BER measurements obtained by observing the lightpaths deployed during normal network operations might not suffice to achieve good results. Therefore, it might be necessary to deploy lightpath probes to evaluate in the field the BER of lightpath configurations that would normally not be adopted to serve user traffic.

The remainder of the paper is structured as follows: in Section II we briefly overview the related literature and in Section III we provide some background notions on ML classification. Section IV describes the assumed transmission model and our E-Tool, Section V illustrates the proposed ML binary classifier, and Section VI assesses the classifier performance. Future research directions are discussed in Section VII and conclusions are drawn in the Section VIII.

II. Related Work

The adoption of ML techniques as decisional support tools for the design and planning of optical networks has recently gained considerable attention in the scientific community. A few works have already appeared, which apply ML approaches in both physical and networking layers.

At the physical layer, techniques such as Bayesian filtering have been proposed for parameter estimation in models for laser amplitude and phase noise characterization [10,11], whereas a ML detector based on the distance-weight $k$-nearest neighbors (kNN) algorithm has been proposed to overcome system impairments (e.g., non-Gaussian symmetric noise, laser phase noise, and nonlinear phase noise) in zero-dispersion and dispersion-managed links [12]. Ridge and kernelized Bayesian regression models have been employed for the characterization and mitigation of power excursions in gain-controlled erbium-doped fiber amplifiers [13].

At the network layer, ML-based frameworks for the control and management of optical networks have been proposed: in Ref. [14], artificial neural networks are used to predict the evolution of the network traffic, whereas, in Ref. [15], reinforcement learning techniques are employed by a resource-allocation agent and incorporated into a cognitive architecture-on-demand control plane.

Coming to the specific target of this study (QoT prediction of a candidate lightpath prior to establishment), some ML-based approaches have been investigated: regression models such as network kriging and least-squares minimization with $l_2$-norm regularization have been applied in Refs. [6,16] to estimate the QoT of multiple lightpaths in terms of BER, by relying on the measurements obtained from a limited number of “active lightpaths” (i.e., lightpaths that carry dummy traffic and are instead used as measurement probes). In our paper, we assume that the BER of an already established lightpath is measured by means of OPMs installed at the lightpath termination nodes, as described in Ref. [4]. A neural network fed with either synthetic or field data is used in Ref. [9] to evaluate the $Q$-factor of multicast connections, whereas a case-based reasoning technique (i.e., an artificial intelligence method which makes decisions based on previously observed data stored in a knowledge database) is proposed in Refs. [7,17] to decide whether the BER of an unestablished lightpath will be above or below a given system threshold. In this paper, we address a QoT-prediction problem as in Refs. [6,9,16], but our approach is significantly more complex, as we assume that different combinations of routes and modulation formats can be used for the candidate lightpath, and our proposed ML classifier returns the most suitable combinations (i.e., the ones having the highest probability of ensuring a BER below threshold). Note that a short, summarized version of this study can be found in Ref. [18], but in this extended version, we use a more accurate BER-calculation model for the generation of synthetic data which takes into account nonlinear effects; we consider a more realistic procedure for the generation of training datasets, which emulates the evolution of a dynamic routing and spectrum assignment with first-fit criterion; and we provide a much more extensive performance assessment, considering different network topologies and various sets of classification features.

III. Background on Machine Learning Classifiers

This section provides some background on binary classifiers based on ML and describes the performance metrics used to evaluate their effectiveness.

A. Basic Principles

We consider in the following an instance (or “sample”) as a set of numerical and/or categorical values (or “features”) representing an instantiation of our problem. In the context of QoT prediction of unestablished lightpaths, the features characterize the lightpath we want to deploy: an example of a numerical feature is the volume of the traffic request we want to serve, whereas an example of a categorical feature is the modulation format we want to use for transmission. A set of instances, which are considered independent of each other, is named a dataset.

We associate with each instance a class which is described by a binary value: 1 if it satisfies a given rule (positive instances), 0 otherwise (negative instances). For example, considering the BER as a QoT metric, we associate 1 with an instance if the BER of the lightpath characterized by the features constituting the instance is below a certain threshold $T$, 0 otherwise.

In this work, we want to build a ML algorithm that, given an instance, predicts its class; this is a classification problem and such an algorithm is called a binary classifier (when the problem is instead to estimate a real-valued target, the problem is called regression and is solved by means of a regressor). One can consider a classifier as a function mapping a point of the space of features to a real number. Such a real number is the score of the instance, i.e., its probability of belonging to class True.

Note that features should be chosen in order to contain information that is useful to discriminate the class of an instance. Non-informative features—i.e., features that show no correlation to the class of the instance—are known to reduce the performance of classifiers, even though different classifiers have different sensitivity to this issue. In this regard, we deem as an important contribution of this paper the identification of a methodology for the identification of the most informative features among those initially selected for the QoT classification problem (see Subsection VI.C).

Before it can be used, a classifier needs to be trained by means of a training dataset, i.e., a set of instances whose class is known. In this phase, the classifier learns a mapping between the space of features and the class. Many different classification algorithms have been proposed in the literature [19] and are routinely used by ML practitioners; algorithms differ in terms of achievable accuracy, scalability to datasets with many instances and/or features, computational effort required for training and for testing, sensitivity to outliers, and interpretability of the resulting models. Random forests, logistic classifiers, support vector machines (SVMs), kNN, and neural networks are among the most frequently adopted techniques in recent works.

Once a classifier is trained, it can be used to test an instance that was not part of the training set. Given the numerical features belonging to such a test instance, the output produced by the classifier is the predicted probability ${\widehat{P}}_{\mathrm{pos}}$ that the instance belongs to the positive class. This probability is the output score of the classifier: it will be very close to 1 for instances that are very likely to be positive, and close to 0 for instances that are very likely to be negative; a classifier may also return scores close to 0.5 for instances that are difficult to classify and may belong to either class. Since in practical applications one often needs a single well-defined output, the output score is typically binarized and the test instance is classified as positive if and only if its score is greater than or equal to a threshold $\gamma =0.5$. Therefore, if for a given testing instance the classifier will produce a score equal to 0.95—which indicates that the instance is very likely positive—one will classify such an instance as positive, and the same would happen if the classifier produces a score equal to 0.51, indicating a large uncertainty in the prediction.

B. Performance Evaluation Metrics

Given a testing dataset, in the following we consider two measures to evaluate the performance quality of a trained classifier: accuracy and area under the receiver operating characteristic curve (AUC) [19].

The accuracy corresponds to the fraction of the test instances that are correctly classified. This measure is very easy to interpret and understand, but, as a classifier performance metric, suffers from a number of drawbacks:

The AUC is the second metric we report, which obviates these issues and is widely used in the ML literature. Given a trained classifier and a testing set, one may set an arbitrary threshold $\gamma$ and divide the testing instances into four groups:

Note that TP + FN corresponds to the number of positive samples in the testing dataset, and TN + FP corresponds to the number of negative samples in the testing dataset. We define the True positive rate (TPR) as the fraction of all positive instances that are classified as such, i.e., $\mathrm{TPR}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$.

Conversely, the false positive rate (FPR) is the fraction of all negative instances that are incorrectly classified as positive: $\mathrm{FPR}=\frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}}$. Note that both the TPR and FPR are in the [0,1] range. An ideal classifier has $\mathrm{TPR}=1$ and $\mathrm{FPR}=0$.

By increasing the value of $\gamma$, we reduce the number of instances that we classify as positive and increase the number of samples that we classify as negative. This has the effect of decreasing TP while correspondingly increasing FN, and increasing TN while correspondingly decreasing FP. This reduces the TPR and also reduces the FPR.

The receiver operating characteristic (ROC) curve represents the FPR (on the horizontal axis) and the TPR (on the vertical axis) for different values of the threshold $\gamma$. For $\gamma =1$, all instances are classified as negative (except those for which the classifier returned exactly 1.0, which we assume are very few); therefore $\mathrm{TPR}\approx 0$ and $\mathrm{FPR}\approx 0$: this lies on the bottom left of the ROC space. At the opposite end, for $\gamma =0$, all instances are classified as positive; therefore $\mathrm{TPR}=1$ and $\mathrm{FPR}=1$.

The ROC curve always connects these two extremes. Any classifier that ignores the value of features (e.g., the classifier that always returns the most frequent class, or a classifier that returns answers at random) yields a ROC curve on the diagonal, regardless of the accuracy they can achieve. The ideal classifier (or any classifier that perfectly separates the two classes for at least one value of $\gamma$) yields a ROC curve connecting (0,0), (0,1), and (1,1). Classifiers that capture some useful information yield a ROC curve above the diagonal and approach but do not reach the point (0,1) on the ROC space.

The AUC is used as an effective and robust metric for the performance of binary classifiers that does not depend on the specific choice of $\gamma$. It ranges from 0.5 (for a useless classifier) to 1 (for an ideal classifier). According to Ref. [20], the value of the AUC is preferable to accuracy when evaluating the quality of classifiers, and it has a very useful intuitive interpretation as follows. Pick a negative and a positive sample at random from the testing dataset, and score both samples with the trained classifier; the AUC of the classifier can be interpreted as the probability that the classifier returns a larger score for the positive sample than for the negative sample. Therefore, for any choice of a negative and positive sample, a classifier with $\mathrm{AUC}=1$ will score the former lower than the latter, which implies that there exists a threshold $\gamma$ which perfectly separates negative and positive samples. Conversely, a classifier that returns random scores will have an AUC close to 0.5.

IV. Bit Error Rate Estimation Tool for Synthetic Data Generation

In this section, we discuss the assumptions on the system model and present the E-Tool we developed for the estimation of the BER once a candidate lightpath is deployed.

A. System Model

We assume that optical channels are multiplexed in a flexible grid with standard slice width of 12.5 GHz [21] and elastic transceivers operating at 28 Gbaud with optical bandwidth of 37.5 GHz (i.e., three slices). Superchannels with multiple adjacent transceivers are used to serve traffic demands exceeding the capacity of a single transceiver. We consider transparent links of dispersion uncompensated standard single-mode fibers where the signal power is restored by identical optical amplifiers equally spaced over the links (100 km), with gain $G=20\mathrm{dB}$ and noise figure $F=5\mathrm{dB}$.

At the receiver, we consider a processing as in Ref. [22] for each transceiver: after coherent detection and analog-to-digital conversion, chromatic dispersion is electronically compensated, and an adaptive equalizer tackles other potential linear channel effects; finally, error counting for determining the pre-FEC BER is performed.

B. BER E-Tool

For the generation of synthetic data, we propose an E-Tool that, on input of a candidate lightpath and modulation format, calculates an estimate of the uncoded BER at the input of the FEC soft decoder (called pre-FEC BER in the following). Modern FEC codes have a threshold behavior: roughly speaking, if the pre-FEC BER is below a value (determined by the FEC code properties), then the BER at the output of the FEC is, with high probability, able to satisfy the BER system requirement. Provided that there exists a FEC code that bridges the gap between the pre-FEC BER and the required system BER, the pre-FEC BER takes the role of the target BER. A typical value for the BER target is $T=4·{10}^{-3}$ [22], which we adopt in the remainder of the paper.

In linear optical communication systems, i.e., those affected by chromatic dispersion and additive white Gaussian noise (AWGN) only, the pre-FEC BER depends on the pre-FEC signal-to-noise ratio (SNR) through a function determined by the modulation format. Hence, once the target BER is fixed, we can compute the required SNR. For a specific lightpath, the pre-FEC SNR can be estimated by a link budget that takes into account the transmitted power $P_{\mathrm{in}}$, gains, and losses. If the pre-FEC SNR exceeds the required SNR, then the lightpath can be established.

C. Link Budget With Weak Nonlinear Propagation

When weak nonlinear propagation effects start to appear, a typical assumption is that the system behaves as a linear one, where the interference due to nonlinear interactions is treated as an independent contribution of AWGN with power $P_{\mathrm{NLI}}$, as discussed in Ref. [2]. An estimate of the value of $P_{\mathrm{NLI}}$ due to inter- and intra-channel interference can be assessed thanks to the analysis in Refs. [23,24]: in particular, it turns out that $P_{\mathrm{NLI}}$ depends on the transmitted powers and the modulation formats of all channels. The analysis in Refs. [23,24] enables a fairly accurate estimation of the nonlinear interference power without resorting to computationally expensive simulations based on slip-step Fourier methods.

To simplify the analysis, in order to decouple the value of $P_{\mathrm{in}}$ from the modulation formats of all neighboring channels, we assume that all channels transmit at the same power $P_{\mathrm{in}}$: a conservative value for $P_{\mathrm{in}}$ is determined using the Gaussian modulation format assumption of [22], where

Once $P_{\mathrm{in}}$ is found by Eq. (1), the nonlinear interference power $P_{\mathrm{NLI}}$ should be numerically computed, taking into account the actual modulation formats and spectrum occupancy of all channels. However, considering the channel of interest (central channel) and its neighboring channels along the whole lightpath, since the neighbors may change modulation format and bandwidth occupancy due to routing in the network and the flex-grid architecture, for simplicity we compute a conservative value of $P_{\mathrm{NLI}}$ by considering the effect along the entire path of the modulation format and the channel bandwidth of the spectrally nearest neighbor that the channel encounters along its path. This is done separately for its left and right neighbor. We claim that, in this way, we compute a conservative value of interference because the closer the neighbor channel, the larger the nonlinear interference. The value of $P_{\mathrm{NLI}}$ is numerically assessed using the approach provided in Ref. [24].

The nonlinear interference power has to be converted into a penalty to be accounted for in the link budget. The nonlinear penalty term $L_{\mathrm{NLI}}$ is computed as the difference (in dB) between $P_{\mathrm{in}}$ and the launch power $P_{\mathrm{in}}^{\mathrm{lin}}$ the transceiver would need to obtain the same reach if $P_{\mathrm{NLI}}$ were zero (linear system case). In formulae we have

If $X$ is negative, then $L_{\mathrm{NLI}}$ is set to an arbitrary large value, e.g., 50 dB, meaning that the link cannot be established. In Eq. (3), ${\mathrm{SNR}}_{\mathrm{req}}$ is the required SNR to reach the target BER, and $L_{\text{other}}$ is a term that accounts for the following penalties:

Finally, the SNR at the input of the FEC decoder is estimated as

V. Proposed ML Classifier for QoT Classification

A. Classifier Description

As depicted in Fig. 1, our proposed classifier considers the following five features:

 

Fig. 1. Classifier structure.

Download Full Size | PDF

Note that none of these features accounts for cross-channel nonlinear effects. We can also consider six additional features in case complete knowledge of the lightpaths already deployed in the networks is available²:

These six additional features capture information on cross-channel nonlinear effects.

The target variable that the classifier tries to predict is a binary variable, which is true if and only if the lightpath BER is lower than the system threshold $T=4·{10}^{-3}$. Note that the BER value is affected by other factors than those captured by the classification features (e.g., time-varying penalties). Therefore, it may occur that two dataset entries whose set of features are exactly the same exhibit different BER values and in turn different values of the target variable, i.e., the association between feature values and BER value is not deterministic. The classifier is trained on a training dataset and quantitatively evaluated on a separate testing dataset (see Fig. 1).

Since the classifier requires feature values that are numeric and that have comparable ranges to avoid numerical instability, we pre-process features as follows: i) the modulation format feature, which can take one of six possible categorical values, is replaced by six distinct binary features (one for each possible format); for each instance, the feature corresponding to the modulation format will take value 1, whereas the other five will take value 0; ii) the values of each feature are offset and rescaled to ensure that their distribution in the whole training set has mean 0 and standard deviation 1. At training time, we estimate the offset and scaling parameters for each feature. When the classifier is applied to test samples, the feature values are rescaled using such parameters.

On input of a test instance, the output produced by our classifier is the predicted probability ${\widehat{P}}_{\text{pos}}$ that the instance belongs to the positive class. The instance is then classified as positive if and only if such probability is greater than or equal to a threshold $\gamma =0.5$.

B. Dataset Generation

For the dataset generation, we rely on synthetic data simulating measurements obtained from the field. We assume that the network is operated by a first-performing lightpath RSA according to margined calculations; then, when the lightpath is deployed, its actual (not margined) BER is used for training purposes. In the following, we first describe how RSA is performed, and then the procedures used to generate the training and testing datasets.

VI. Numerical Assessment

Numerical assessment has been performed using the Japan and National Science Foundation (NSF) topologies, depicted respectively in Figs. 2 and 3.

 

Fig. 2. Japan network topology.

Download Full Size | PDF

 

Fig. 3. NSF network topology.

Download Full Size | PDF

For our experiments we generate three different (training and testing) datasets (A, B, and C, as described in Table I) and assess the classification performance by evaluating the accuracy and the AUC.

TABLE I. Dataset Description

View Table | View all tables in this article

A. Comparison of Learning Methods for Classification

Using dataset A, we have compared three kNN classifiers [19] with $k=\mathrm{1,5},25$ and five Random Forest (RF) classifiers [27] with 1, 5, 25, 100, 500 estimators. As a benchmark, we have additionally trained a dummy classifier which learns the most frequent class in the training set and always returns such a class for any testing instance, disregarding the feature values. The classifiers are trained with instances including all 11 features listed in Subsection V.A. In Table II, for each classifier, we report the training time and time required to evaluate one testing instance using a standard i5 processor: with the exception of the 1-nearest-neighbor classifier and the RF with one estimator, other options all perform comparably. Therefore, in the remainder of the paper, we adopt the RF classifier with 25 estimators for the next experiments since it provides a good tradeoff between performance and computational time.⁴

TABLE II. Performance Comparison of the Nine Considered Classifiers

View Table | View all tables in this article

B. Impact of the Training Set Size

We now evaluate the impact of the number of training instances on the classification performance. Figure 4 shows the ROC curves obtained for datasets A [Fig. 4(a)] and B [Fig. 4(b)] when the classifier is trained on subsets of the training set composed of 10, 100, 1000, and 10,000 randomly sampled instances (we consider values up to 1000 to be realistic for field datasets), as well as the ROC curve of the classifier trained on the whole training dataset. Results obtained for both network topologies with 1000 training instances closely approach those obtained using the whole training dataset. Therefore, in the following, results will be obtained by training the classifier with only 1000 samples.

 

Fig. 4. ROC curve for different training set sizes, evaluated over datasets (a) A and (b) B. (a) Japan topology. (b) NSF topology.

Download Full Size | PDF

We now focus on analyzing the classification output: it is worth noting that results for each test scenario (i.e., a triplet composed of source, destination, and traffic amount) can be arranged in a table, where each cell corresponds to one setting (i.e., a possible choice of modulation format and route for that triplet). Table III exemplifies the output provided by the classifier and compares the true $P_{\text{pos}}$ of each setting (top) to the corresponding predicted ${\widehat{P}}_{\text{pos}}$ (bottom) for a traffic request of 500 Gbps from node 8 to node 7. Since the ${\widehat{P}}_{\text{pos}}$ of a given setting indicates the predicted probability that, when transmitting over the lightpath with the modulation format indicated by the considered setting, the BER will not exceed the threshold $T$, the closer ${\widehat{P}}_{\text{pos}}$ approaches 1, the safer the choice of that setting would be from a network design perspective. This output can be exploited to make the final decision about the deployment of the new lightpath by any RSA method: if the predicted probability is close to 0, the setting should not be adopted. Conversely, if ${\widehat{P}}_{\text{pos}}$ approaches 1, the network engineer can decide whether to adopt such a setting for the lightpath deployment based on the risk he/she accepts from a design perspective.

TABLE III. Example of Classification Output for a Traffic Request of 500 Gbps from Node 8 to 7 in the Japan Network

View Table | View all tables in this article

TABLE IV. Considered Feature Subsets

View Table | View all tables in this article

C. Analysis of Feature Relevance

We have so far considered 11 features in the proposed ML classifier. An important question at this point is which features are more important to achieve good accuracy and AUC. This question is of high practical value, as collecting more or less features poses higher or lower burdens in terms of monitor deployment and control complexity. In principle, removing irrelevant features would make the system less costly and complex to manage. Hence, we now evaluate the usefulness of each feature by comparing the classification performance after training the classifier over training datasets A and B, considering seven different subsets (S1 to S7) of the 11 features listed in Subsection V.A. The considered subsets are listed in Table IV. Moreover, for each subset of features, we test the classifier over the full testing dataset and over a subset of its instances with BER in the range $[4·{10}^{-4},4·{10}^{-2}]$, i.e., focusing on the test samples which are near to the threshold $T$ and, thus, more “difficult” to classify. The obtained results are reported in Fig. 5. In the case of the NSF topology we first notice that, when focusing on test instances “near to threshold,” both metrics decrease w.r.t. the values obtained over the full testing dataset, whereas, for the Japan topology, such decrease is much less pronounced. This shows that the classification performance is still acceptable even for “difficult” test instances (i.e., those with BER values closely approaching $T$).

TABLE V. AUC Comparison of Probing Approaches

View Table | View all tables in this article

 

Fig. 5. Accuracy and AUC, depending on the feature set selection, for datasets (a) A and (b) B. (a) Japan topology. (b) NSF topology.

Download Full Size | PDF

Results also show that, in both topologies, training the classifier with the feature sets S1, S2, and S5 leads to the highest and comparable AUC values. Note that S1 includes all 11 features, whereas S2 excludes the features characterizing the nearest-neighbor lightpaths. However, if we focus on the AUC of “near to threshold” instances, the results obtained in scenario S1 are slightly higher, which leads us to conclude that information on the closest neighbors does provide some insight into classification of the instances with BER close to $T$, as intuition would suggest.⁵

In particular, S5 includes only three attributes (total lightpath length, traffic volume, and modulation format) which suggest that information on the number of links and length of the longest link are not very useful if the previous three features are used. Indeed, in the transmission model implemented in our E-Tool, transmission impairments due to the traversal of intermediate nodes are on the order of 0.1 dB per node and thus have negligible impact on the BER computation. Similarly, knowing the length of the longest link of the lightpath does not bring much additional information on system impairments once the number of links and the lightpath length are known: this is due to the fact that both linear and nonlinear penalties are mainly determined by the two latter attributes.

However, if we further remove either the traffic volume or the lightpath length from set S5 (as in subsets S6 and S7, respectively), classification performance degrades both in terms of AUC and accuracy, especially for “near to threshold” instances. Results similar to those achieved with feature set S6 are also obtained for the feature set S4, which also includes the number of links and the length of the longest link to the features already included in S6. Performance degradation becomes extremely severe when eliminating the modulation format from the feature set, as done in S3 (which contains only the traffic volume and the lightpath characteristics). With such training features, the AUC is slightly higher than 0.6 for the Japan topology, meaning that the improvement w.r.t. a random classification (which would return 0.5) is scarce.

Note, however, that our choice of modeling time-varying penalties with a random variable with exponential distribution (due to the lack of information about the true statistical distribution of such penalties, as discussed in Subsection IV.C) is the most conservative one and that different distributions (e.g., the uniform distribution with bounded support, which was adopted in Ref. [18]) would lead to better performance.

D. Impact of the Approach Adopted During the Training Phase to Collect Training Data (Historical, Random Probes, Selective Probes)

Finally, we focus on dataset C, and we first evaluate the classification performance after removing from the training set all the instances obtained by probing lightpaths carrying dummy traffic (i.e., we include only historical data as described in Subsection V.B) and randomly sampling a set of 1000 instances, then repeat the experiment by replacing either 50, 100, or 500 instances with randomly chosen instances among the ones obtained via selective probing. AUC results averaged over 50 trials are reported in Table V and compared to results obtained by training the classifier over 1000 randomly chosen instances from dataset A. Results show that, as expected, relying exclusively on historical data leads to low AUC values, as the vast majority of them belong to the class of positive instances. Including instances obtained from selective probes in the training dataset (which mostly belong to the class of negative instances) improves the AUC: the highest improvement is obtained with 250 probes (increase by more than 0.12 when evaluating the classifier over all the instances of the testing set, and of almost 0.05 when focusing on test instances with BER in the range $[4·{10}^{-4},4·{10}^{-2}]$). Additional increase of the probing instances up to 500 did not lead to further performance improvements, and even caused performance degradation above 500. However, the overall performance is still lower than the one obtained when using training instances drawn from dataset A, which is constituted of almost 95% of instances obtained by random probes, thus ensuring a more exhaustive coverage of the whole feature space. It follows that a good classification performance can be achieved only at the price of an extensive deployment of random probe lightpaths.

E. Quantifying Potential Resource Savings

From the point of view of network design, quantifying the benefits of ML-aided QoT prediction in terms of resource savings (e.g., spectrum occupation and number of installed transceivers) is the main—yet unanswered—question. Resource savings depend on how the probabilistic output of the QoT classifier is integrated into the RSA: such a research direction is still largely unaddressed in the literature, and we briefly discuss the topic in the following.

A potential role of the classifier is to identify the lightpaths for which margined formulas were too conservative, i.e., where the next modulation format (the one with twice the number of constellation points) would still have led to a below-threshold BER. Therefore, for each given lightpath and corresponding modulation format obtained with margined formulas (conservative option), we construct a classification instance considering the next modulation format (aggressive option); such an instance may yield $\mathrm{BER}<T$ (and therefore the aggressive option would allow saving resources over the conservative option) or not (which implies that a costly reconfiguration is necessary if the aggressive option is chosen). An RSA decisional algorithm may adopt the aggressive option if ${\widehat{P}}_{\text{pos}}\ge \gamma$, and the conservative option otherwise. In the case of a false positive, we would wrongly choose the aggressive option and incur reconfiguration costs; in the case of a false negative, we would waste resources using the conservative option when the aggressive option would have been more efficient. By using large values of $\gamma$, we minimize the risk of false positives but have to accept more false negatives.

We compute an upper bound on the potential savings by considering the 2161 selective probe instances included in dataset C, and computing the resource savings ignoring reconfiguration costs, i.e., ignoring the impact of false positives. For values of $\gamma$ ranging from 0.5 to 1, Table VI reports the percent savings in the number of installed transceivers and overall spectrum occupation; the classification accuracy; and the false positive rate, i.e., the fraction of below-threshold BER instances which were incorrectly classified as positive. We observe that savings are limited to 1% when $\gamma =1$, i.e., the case with no false positives. By lowering $\gamma$, we are more prone to choosing the aggressive option: savings can reach 17% when $\gamma$ is set to 0.5, at the expense of a larger false positive rate ($\mathrm{FPR}=0.13$). Future work must be devoted to identifying the best tradeoff between classification uncertainty and reduction of resource utilization, as well as identifying the technologies (e.g., bandwidth variable flexible transceivers, probabilistic constellation shaping) that could take most advantage of the adoption of ML-based QoT prediction techniques. More importantly, note that the assumptions used to generate our dataset are conservative in both the calculation of nonlinear impairments and fast time-varying penalties, and hence we expect resource savings to be more significant with realistic measurement datasets.

TABLE VI. Upper Bounds on Resource Savings Enabled by the Proposed QoT Classifier

View Table | View all tables in this article

VII. Future Research Directions

Future work should address the application of online ML mechanisms, which are specifically designed for scenarios where data becomes available in a sequential order: whenever a new datum arrives, it is used to improve the current prediction model implemented by the ML algorithm, and the newly acquired knowledge will be adopted to make decisions at the next step. This online mechanism naturally fits dynamic RSA approaches, where traffic requests are generated at different moments in time and must be routed and allocated in a commensurate spectrum portion upon arrival. Moreover, active ML techniques [28] could be adopted to mitigate the issue of installing probe lightpaths: active ML algorithms are capable of interactively querying the user, asking to observe data with specific characteristics. This way, the number of samples to build an accurate predictor may be reduced. Therefore, if the process of generating data is costly (as in the case of probe lightpath deployment) active learning is a candidate approach to reduce the cost of dataset generation. However, when considering a real optical network scenario, it may be impossible to satisfy some of the queries of a ML active algorithm. For example, the algorithm may ask to observe the measurements obtained over a 1300 km long lightpath, but the deployment of such a lightpath may be impossible due to the structure of the network topology (i.e., a succession of consecutive links with total length of 1300 km may not exist). Therefore, a thorough investigation of the effectiveness of active ML techniques in reducing the training dataset size while taking into account the constraints imposed by the network structure and topology is necessary. Furthermore, considering that the consequences of classification errors may be catastrophic in terms of violation of Service Level Agreements stipulated with customers and content providers, and could lead to unacceptable QoS degradation, cost-sensitive ML learning approaches [29] should be investigated; such approaches allow for the definition of misclassification costs to penalize specific types of prediction errors. Costs caused by different kinds of errors can be arbitrarily defined, and the learning objective is to minimize the expected costs.

VIII. Conclusion

This paper proposes a machine-learning method to predict the quality of transmission of optical lightpaths prior to deployment: based on the lightpath characteristics (total length, length of the longest link, and number of lightpath links), modulation format used for transmission, and traffic volume to be served, the proposed algorithm predicts whether the bit error rate of the candidate lightpath will exceed a given system threshold. The performance of the classification algorithm is evaluated over a wide set of simulation scenarios. Results show that high values of accuracy and AUC can be achieved, though at the price of extensive deployment of probing lightpaths necessary to evaluate in the field the BER of lightpath configurations that would normally not be adopted to serve user traffic. Based on the reported results, our proposed classifier can be considered a useful component for integration into RSA decision tools.