1. Introduction

Reservoir Computing is a human brain-inspired machine learning scheme that is versatile, has fast training times, and utilises the intrinsic information-processing capacities of dynamical systems [1,2]. The simple training scheme that is employed in reservoir computing, which involves only training the output layer, avoids difficulties that typically arise in the training of recurrent neural networks, such as the vanishing gradient in time [3]. This makes reservoir computing particularly suited for hardware implementation [4–8]. In the context of optimising computation speeds and power consumption, optical implementations are of particular interest and were made feasible by the introduction of so-called delay-based reservoir computing in [9]. In [9] the authors drew the connection between time-delayed systems and networks to show that a single non-linear node with time-delayed feedback could serve as a reservoir in which the reservoir responses are multiplexed in time rather than in space. Various experimental implementations of this delay-based scheme have been realised, including in opto-electronic [9–13], optical [6,8,14–21] and electromechanical [22] systems. Delay-based reservoir computers have demonstrated good performance in applications including time-series-predictions [18,20,23–25], fast word recognition [26], hand-written digit recognition [19] and signal conditioning [8,21,27,28], while new proposals emphasize the future potential for applications in cyber-security, wireless communications [29] or telecommunication [30].

Despite the extensive research that has been carried out on delay-based reservoir computers since their inception in 2011 [9], there has been very little focus on the influence of the delay-times on the reservoir computing performance. In most studies one of two fixed values are chosen for the delay-time; either resonance with the input clock-cycle [9,18,31,32] or slightly larger than the input clock-cycle, commonly referred to as the desynchronised regime [11,15]. Furthermore, it is often stated that it is necessary for the delay-based reservoir computing concept, that the delay-time is chosen as one of these values. However, this is an unnecessary constraint that stems from viewing delay-based reservoirs as networks. In this article, we will introduce the concept of delay-based reservoir computing in a manner that demonstrates that there is no predefined relationship between the delay-time of the reservoir and the input clock-cycle. We will review delay-based reservoir computing in the context of the influence of the delay-time, highlighting the effect of resonances between the reservoir delay-time and the input clock-cycle on the computing performance. We start by reviewing the effect of tuning the delay in single-delay reservoir computing systems and subsequently show, how adding more delays allows for an even better tunability of the reservoir properties and can thus lead to performance improvement.

This work is structured as follows. First, we will give a brief introduction to the general reservoir computing concept in Sec. 2. In Sec. 3.1 we will introduce delay-based reservoir computing by first introducing the concept of time-multiplexed reservoir computing and then explaining why time-delay systems serve as suitable time-multiplexed reservoirs. Following this, we will review the existing literature on aspects of this topic in Sec. 3.2–3.4, Sec. 4 and 5. Then, in Sec. 6, we present new results for the two delay-coupled oscillators, where we show how the memory capacity can be tuned, and how choosing the correct delays can lead to a substantial performance improvement for the NARMA10 task.

2. Reservoir computing concept

There are many articles and reviews that give good and thorough introductions to the reservoir computing concept. Here we will give a brief introduction and refer the reader to [1,33–36] for more in-depth discussions of the concepts and mathematical foundations.

A reservoir computer consists of three parts; the input layer, the reservoir, and the output layer. Figure 1 shows a sketch of this layout for the case of one-dimensional input and output layers (the concept can easily be extended to high-dimensional inputs and outputs, for example, as shown in [35]). In this sketch, we have depicted the reservoir as a recurrent neural network, as this was the original system for which the reservoir computing concept was conceived and allows for a direct comparison between reservoir computing and more traditional machine learning algorithms. However, we would like to emphasize that the reservoir does not need to be a network for reservoir computing to work. It is the training method that sets reservoir computing apart from other machine learning methods. Specifically, only the weights connecting the reservoir with the output layer are trained (connections highlighted in green in Fig. 1). The input weights (connections between the input layer and the reservoir) and any internal properties of the reservoir are kept fixed during training. Typically, the input weights are chosen randomly, as are the internal weights when the reservoir is a recurrent neural network.

 

Fig. 1. Sketch of a reservoir computer based on a recurrent neural network (echo state network). The green lines represent the connections that need to be trained, while the rest of the network remains unchanged. Concept taken from [1].

Download Full Size | PDF

In the training phase, the reservoir is fed $K_\textrm {tr}$ successive inputs and the response of the reservoir is sampled $N$ times for each input. These responses (node states in the case of a neural network reservoir) are written into an $K_\textrm {tr}\times \left (N+1\right )$-dimensional state matrix $\underline {{\underline{\mathbf S}}}$, the last column of which is filled with a bias term of one. The training step is then to find the weights ${\underline {\mathbf W}}^{\textrm{out}} \in \mathbb {R}^{N+1}$ that minimise the difference between the output $\underline{{\mathbf o}}=\underline{{\underline{\mathbf S}}}^{~}{\underline{\mathbf W}}^{\textrm{out}} \in \mathbb {R}^{K_\textrm {tr}}$ and the target output $\hat{{\underline{\mathbf o}}}\in \mathbb {R}^{K_\textrm {tr}}$, i.e. the ${\underline{\mathbf W}}^{\textrm{out}}$ that solves

2.1 Performance measure

To quantify the quality of the prediction, $\underline{{\mathbf o}}=\underline{{\underline{\mathbf S}}}^{~}{\underline{\mathbf W}}^{\textrm{out}}$, there are various error measures (see for example [37,38]). One commonly used measure is the normalised root-mean-square error (NRMSE), defined as

3. Delay-based reservoir computing

3.1 Concept

We will introduce the concept of delay-based reservoir computing in a different manner to the usual approach. We do this to clarify a misconception about the relationship between the delay-time $\tau$ of the reservoir and the input clock-cycle $T$, the misconception being that these quantities must be equal, or more generally, that there is any predefined relationship between these quantities. In order to do this, we will first introduce the concept of time-multiplexed reservoir computing independently of delay-based systems and will subsequently show why time-delayed systems are suitable reservoirs for time-multiplexed reservoir computing.

3.1.1 Time-multiplexed reservoir computing

One of the main concepts underlying delay-based reservoir computing is that rather than multiplexing in space, as is done in conventional reservoir computing, one multiplexes in time. This means that the response of the reservoir is sampled multiple times as a function of time rather than as a function of space. In such a scheme, each piece of input data must be fed into the reservoir for a certain time interval, which we will refer to as the clock-cycle $T$. During that interval, the response of the system must be sampled a number of times. A sketch of an example input sequence and the corresponding reservoir responses are shown in Fig. 2(a) and Fig. 2(c), respectively. The number of times that the system is sampled (blue circles in Fig. 2(a)) gives the output dimension, commonly referred to as the virtual nodes $N_v$ in delay-based reservoir computing. It is analogous to the output dimension $N$ in the general reservoir computing scheme described in Section 2.

 

Fig. 2. Visualisation of time-multiplexing (sequential sampling indicated by blue dots) and masking. Left and right panel show unmasked and masked input (top row) while the respective system response is shown below. Time is normalized to the input cycle $T$.

Download Full Size | PDF

In principle, any dynamical system can be used as the reservoir for such a time-multiplexed scheme, however, the dynamics and characteristic time-scales of the system play an important role in the performance that can be achieved. It has been shown that systems generally perform well when the dynamics of the underlying autonomous system (i.e. the system not driven by the input data) are steady state dynamics but the system is close to a bifurcation leading to chaotic or other high-dimensional dynamics ([39,40]). As indicated in Fig. 2(c), if the reservoir relaxes back to its steady state within an input clock-cycle, multiple identical system responses are sampled. As these responses are not linearly independent, the effective output dimension of the reservoir is reduced. One way to mitigate this problem is to apply a mask to the input data. An example of a masked input is shown in Fig. 2(b). By disturbing the system within one input cycle $T$, the response can be diversified. If the length of the mask steps, typically labelled $\theta$, is chosen appropriately, the system can be prevented from relaxing to its steady state.

Typically, a sequence of step functions with randomly selected step heights, that repeat for each clock-cycle, are used for the masking procedure, as depicted in the example shown in Fig. 2(b). However, work has also been done on the performance of chaotic, multilevel, binary, or pattered masks [24,41–43]. In the example shown in Fig. 2(b), the system is sampled at the end of each mask step. The choice of sampling position is arbitrary, however, it must be the same in each clock-cycle. It has also been shown that it can be beneficial to vary the lengths of the mask step and the sampling positions within one clock-cycle [44–46]. Which type of mask results in the best performance is both system and task-dependent.

3.1.2 Time-multiplexed RC with a delay system

Having introduced the concept of time-multiplexed reservoir computing, we will now address why time-delayed systems are suitable as reservoirs in this scheme. For reservoir computing to yield good results, the responses of the reservoir to sufficiently different inputs must be linearly separable. For this to be possible, the state-space of the reservoir must be sufficiently high-dimensional. This can be achieved by having a large network of coupled nodes or by using one nonlinear node with time-delayed feedback. For delay-based reservoir computing, delay differential equations of the following form are often considered (see for example [9,26,35,47,48]):

In Fig. 3, we show a sketch of a time-delayed reservoir. The system is fed a masked input sequence and the response of the system is sampled at predefined time intervals. Here we would like to emphasise that the time-multiplexing procedure can be viewed completely independently of the time-delayed system, and there is no predefined relationship between the delay time $\tau$ and the clock-cycle $T$. However, resonances between $\tau$ and $T$ can have a significant impact on the performance that can be achieved, as will be discussed in Section 3.3.

 

Fig. 3. Sketch of a delay-based reservoir computer realized with a laser subjected to optical feedback, masked input, and time-multiplexed output.

Download Full Size | PDF

3.2 Viewing delay-based reservoirs as networks

When the delay-based reservoir computing scheme was introduced in [9] the authors introduced the concept of virtual nodes equidistantly distributed along the delay-line. They chose $T=\tau =\theta N_v$, where $N_v$ is the number of virtual nodes, and described how this system can be considered as a network. They considered the two cases $\theta < T'$ and $\theta >>T'$, where $T'$ is the characteristic timescale of the nonlinear node. In the first case, $\theta < T'$, the system is perturbed by each subsequent input step before it can relax to a steady state. This means that the transient state of the system for one input interval depends on its state in the previous intervals. This can be viewed as a unidirectional coupling between neighbouring virtual nodes. Figure 4(a) shows a sketch of the network that such a system emulates. In addition to the coupling via the nonlinear node dynamics, choosing $\tau =\theta N_v$ means that in such a network view of the system, the virtual nodes are also coupled with their state in the previous clock-cycle, as indicated by the self-feedback loops in Fig. 4(a).

 

Fig. 4. Network topologies as they arise in delay-based reservoir computing systems when the delay is restricted to $\tau =n \theta$ ($n\in \mathbb {N}$). In (a,b) the input cycle $T$ is chosen resonant to the delay, $\tau =T$, while $\tau =T+\theta$ in (c). The characteristic system time $T'$ is larger than $\theta$ in (a) while it is smaller than $\theta$ in (b,c).

Download Full Size | PDF

In the second case, $\theta >>T'$, the nonlinear node relaxes to its steady state within one $\theta$ mask interval, this means that there is no dependence on the state of the system during the previous mask step and the system can be described by a set of independent nodes with self-feedback (see Fig. 4(b)). In this scenario, for reservoirs of the form given by Eq. (4), the delay-differential equation describing the reservoir can be reduced to a set of recursive equations that describe the state of the virtual nodes. Let $x_k$ be $x\left ( t\right )$ at the end of the $k^{\textrm{th}}$ $\theta$ interval of the piece-wise constant masked input function $J\left ( t\right )$, i.e. $x_k=x\left ( t\right )$ for $t=\left ( k+1\right )\theta$ with $J\left ( t\right )=J_k$ for $k\theta < t \leq \left ( k+1\right )\theta$. Then, if the dynamics of the underlying autonomous system are steady state dynamics, for $\theta >>T'$, Eq. (4) can be written as

The equivalences between time-delayed systems and networks, and more generally the now well-known equivalence between time-delayed and spatially extended systems [50], lead to the inception of delay-based reservoir computing. However, continuing to view delay-based reservoirs as networks puts unnecessary constraints on the relationship between the delay-time $\tau$ and the clock-cycle $T$. Mainly because the direct equivalence is only applicable when $\tau =n\theta$ for $n\in \mathbb {N}$ and $\theta =T/N_v$ [48]. It is not necessary to view delay-based systems as networks for reservoir computing purposes and letting go of this idea could lead to improved performance. This is because the delay-time plays an important role in determining the memory of the reservoir and allowing this parameter to be tuned more freely could lead to better fulfilment of task specific memory requirements [47,51,52].

3.3 Delay and clock-cycle resonances

It is generally known that resonances between various characteristic time-scales can have an important influence on the dynamics of a system. This is a phenomenon that is observed in a multitude of systems, including neural oscillators [53,54], mode-locked lasers [55] and chemical oscillators [56], among many more. Sometimes these resonances are beneficial to the desired outcome and other times they have a detrimental effect. In the case of delay-based reservoir computing, resonances between the delay-time of the reservoir and the clock-cycle generally have a detrimental effect on the performance. This has been demonstrated in a number of publications [17,47,48,51,52,57–59]. An example of $\tau$-$T$ resonances is shown in Fig. 5, wherein the total memory capacity and the NARMA10 NRMSE (see the Supplemental Material for the definitions of these two commonly used benchmarking tasks) are plotted as a function of $\tau$ and $T$. In Fig. 5(a), dips in the memory capacity are found at the main resonance $\tau =T$ and at higher order resonances $p\tau =qT$ for $p,q\in \mathbb {N}$. In Fig. 5(b) increases in the NRMSE are found at some of these resonances for the NARMA10 task. By inspecting the evolution of the performance in Fig. 5(b), it also becomes clear that the optimal NARMA10 performance is not achieved at the commonly used relation of $\tau =T+\theta$. Note that the detrimental effect of resonances affects different parts of the memory capacity differently. Since memory requirements are different for each task, the performance for some tasks may therefore not be reduced by the resonant case.

 

Fig. 5. Total memory capacity (a) and performance of NARMA10 prediction (NRMSE) (b) plotted as a function of clock-cycle $T$ and delay-time $\tau$. Dashed lines indicate resonances of rational ratio between the two timescales, i.e. $n T = m \tau$, with $n,m \in \mathbb {N}$. System and parameters as in [51].

Download Full Size | PDF

Two explanations for this resonance phenomenon were given in [48] and [52]. The first, taken from [48], translates the delay-based reservoir computer via an Euler-step method into an equivalent spatially extended network. This method works for a constrained set of delay differential equations. Looking at the connections of this equivalent network system, one sees a reduction of node interconnection for resonant setups. Thus, a resonant case yields a badly connected network, lowering the reservoir computation capabilities.

The second explanation, given in [52], uses an eigenvalue analysis and describes how the reservoir reacts to the inputs in the regime where the input is a small perturbation. They show that resonances between the delay-time $\tau$ and clock-cycle $T$ yield a multitude of overlapping half-circle rotations in the many complex planes of the delay-based reservoir computers infinite-dimensional phase space. As mentioned above, practically, the reservoir only has a finite number of accessible eigendirections, as all but a finite number of eigendirections decay infinitely fast. Thus, the remaining finite eigendirections yield the usable phase space of the reservoir to project the input information into. In the case of a long delay-time $\tau$ (long in comparison to the local timescale of the reservoir, which is the case in typical reservoir computing settings), the imaginary parts of all eigenvalues $\mu _{k}$ are approximately given by

3.4 Hardware implementation

There have been various hardware implementations of photonic delay-based reservoir computing showing good performance on a range of benchmarking tasks. We refer the reader to [34] for a general review of photonic reservoir computing, [35] for a review specifically about delay-based photonic reservoir computing, and [36] for a general review on the hardware implementation of reservoir computing, including time-delayed systems. Here we shall mention only selected papers.

Opto-electronic and all-optical implementations of delay-based reservoir computing have demonstrated good performance in benchmarking tasks such as spoken digit recognition and non-linear channel equalisation [11,14,15], at data processing rates up to 1 Gbyte/s [14]. However, in these experimental setups, results have only been reported for fixed feedback delay-times that are either equal to the clock-cycle [14] or chosen according to the so-called desynchronised setup with $\tau =T+\theta$ [11,15]. The results of the theoretical studies discussed in Section 3.3 indicate that further improvement in the performance could be achieved by freely tuning the feedback delay-time.

In [18] a reservoir consisting of a semiconductor laser with feedback from a short external cavity is studied. In this work, the authors aim to increase the number of virtual nodes under the constraint of viewing the reservoir as a network, meaning that they restrict the clock-cycle to $T=n\tau$ for $n\in \mathbb {N}$. This restriction, which is unnecessary for the time-multiplexed reservoir computing concept, is detrimental to the memory capacity of the reservoir. There are two factors that lead to decreased memory capacity when $T=n\tau$ for $n\in \mathbb {N}$. Firstly, the resonance between $T$ and $\tau$ decreases the memory capacity, as discussed in Section 3.3 and shown in [48]. Secondly, with each roundtrip in the feedback cavity the influence of past inputs decreases. Therefore, as $n$ is increased, the memory of previous clock-cycles decreases, as is demonstrated by the low memory capacities above the main resonance ($T=\tau$) in Fig. 5(a). The authors of [18] mitigate the low memory of their setup by including multiple past inputs in the input signal. A similar approach of using multiple delayed inputs was recently investigated theoretically in [59], showing promising results for task-specific performance enhancement in hardware implemented reservoirs. The authors of [20] experimentally realized an integrated setup of a delay-based reservoir computer, also with a short feedback delay which restricts the number of virtual nodes, and post-processing methods are investigated to improve the performance. In this study there is a small mismatch between clock-cycle $T$ and delay-time $\tau$.

4. Reservoir computing with multiple delays

An extension of the delay-based reservoir computing setup reviewed in the previous section is to include additional delay-lines. So far, there have only been a few works on this topic, which are mostly theoretical. However, one of the earliest works was an experimental study using an opto-electronic reservoir [60]. In [60] 15 time-delayed feedback lines were included. The delay-times were chosen as integer multiples of the virtual node separation, $\tau _n=n\theta$ for $n\in \mathbb {N}$, with the largest delay being equal to the clock-cycle, $\tau _{\textrm {max}}=T=N_v\theta$. The feedback weights were randomly selected and the equivalent network had sparse connectivity. The authors described their approach as a method of providing enhanced dynamical connectivity within a network view of the reservoir. Once again, the restrictions placed on the feedback delay-times in the setup used in [60] do not allow for tasks specific delay-tuning of the memory capacity.

In [57] a theoretical investigation of the influence of a second time-delayed feedback term is done. The first delay is chosen resonant with the clock-cycle, $\tau _1=T$, and the second is given by $\tau _2=n\theta$ for $n\in \mathbb {N}$, where $n$ is varied to find task specific optimal values. At resonances between $\tau _1$ and $\tau _2$, the authors find poor performance for the NARMA10 task. This holds with the results of [48] since $\tau _1=T$ in [57].

In [17] a semiconductor laser with dual optical feedback is simulated. For the first delay-line the desynchronised scheme, $\tau _1=T+\theta$, is chosen and the second delay is tuned. In this work, a strong dependence on $\tau _2$ is shown for the Santa Fe time series prediction task, as well as resonance effects between $\tau _1$ and $\tau _2$. Similarly, in [12] an opto-electronic dual feedback scheme is simulated, also with $\tau _1=T+\theta$ and tuned $\tau _2$. Here, a strong $\tau _2$ dependence and delay resonance effects are also demonstrated.

In each of the studies mentioned above, at least one of the delay-times is restricted to $\tau =T$ or $\tau =T+\theta$. Furthermore, the ranges over which the second delay-time is tuned do not exceed $4T$ in these studies. However, task specific memory requirements could demand much larger delays, as will be demonstrated for the NARMA10 task in Section 6. and as is indicated by the task-specific dependence of the performance on delayed inputs recently shown in [59].

5. Reservoir computing with multiple delay-coupled nodes

So far we have considered delay-based reservoirs consisting of one non-linear node with feedback, with the multiplexing done in time. Hybrid spatial- and time-multiplexing approaches can also be realised, for example by delay-coupling multiple non-linear nodes. The advantage of such an approach is that the output dimension can be increased without the need for increasing the clock-cycle ($\theta$ can not be made arbitrarily small due to the finite time needed for a dynamical system to respond to the input). There are a number of theoretical studies on this topic [31,32,61–64], however in none of these studies is the influence of the coupling delay-times considered as a tunable parameter, rather the focus is on increasing the output dimension or investigating the influence of system parameters other than the delay-times.

In [31] a comparative study is carried out on the performance of two delay-coupled non-linear nodes and two uncoupled non-linear nodes with self-feedback where the outputs are collected into the same state vector. Various parameters of their chosen nonlinearity, as well as input and feedback scaling parameters, are scanned to find areas of optimal performance for various benchmarking tasks. However, the delay-times are kept fixed and equal to the clock-cycle. The authors find that it is both parameter and task dependent as to whether the performance is better in the coupled or the uncoupled system.

Variants of two semiconductor lasers delay-coupled in a ring configuration are considered in [62–64], with either resonant delay-times ($\tau =T$) or the desynchronised case ($\tau =T+\theta$). In [63] the influence of slightly varying one of the delay-times is considered, however, only over a small range with the aim of showing that the performance does not degrade if it is not possible to make the two delays exactly equal in an experimental setting.

In [32] the authors study the performance of multiple uncoupled semiconductor lasers with self-feedback, where, as in [31], the outputs are collected into one state vector. The purpose of this work is to demonstrate a method of increasing the output dimension. All delay-times are chosen equal and resonant to the clock-cycle in this study. An experimental realisation of such a setup, consisting of two uncoupled semiconductor lasers, is demonstrated in [65]. Here the delay-times are also kept fixed; one equal to the clock-cycle and the other slightly larger. The authors report improved performance compared with a single laser configuration.

Finally, in [61] the effect of mixing spatial- and time-multiplexing was investigated. The authors set the total output dimension to a fixed value and varied the ratio of real and virtual nodes. They considered various delayed ring-coupling configurations for the real nodes. For all configurations, they found optimal performance for the NARMA10 task when the number of virtual nodes is larger than the number of real nodes. In this study the number of delay-lines and the length of the delay-lines varied with the proportion of virtual to real nodes, however, this was done to keep the ratio of the number of virtual nodes and the delay-times constant rather than to investigate the influence of varying delay-times.

As of yet, the potential performance enhancement that could be achieved by introducing multiple time scales via the delay-times, in multi-node systems, has not been thoroughly investigated.

6. Memory capacity tuning in systems with multiple time-delays

We will now motivate a promising direction for future research in delay-based reservoir computing which involves more than one delay-line and their respective tuning. Our results show that this tuning crucially influences the memory capacity and the performance for certain tasks. For demonstrating the impact of the delay-times we use a system of two coupled Stuart-Landau oscillators. We look at the influence of the delay-times introduced via the coupling of the oscillators, as well as via self-feedback. The results pertaining to the additional self-feedback are also applicable for a reservoir consisting of a single non-linear node with multiple feedback terms.

6.1 Coupled Stuart-Landau oscillators - model and simulation details

To investigate the influence of multiple delays and coupled nodes, we use a system of two delay-coupled Stuart-Landau oscillators. The Stuart-Landau oscillator is a generic system describing the behaviour near a Hopf bifurcation and is therefore suitable for investigating the influence of delayed coupling and delayed feedback in a general context. The corresponding equations of motion are

 

Fig. 6. Scheme of two oscillators (blue circles) delay-coupled in a ring-topology without (a) and with (b) self-feedback. The coupling is unidirectional, with coupling strength $\kappa _j$ and delay $\tau _j$ ($j \in \left [1,2\right ]$). Both nodes are externally driven with constant pump rate $\lambda _j$ and time dependent input $I_j(t)= g_j(t)u(t)$ scaled with $\eta _j$.

Download Full Size | PDF

Simulations were done in C++ using standard libraries and the linear algebra library "Armadillo" [66]. The delayed Stuart Landau equations Eq. (9) were integrated using a Runge-Kutta 4th order algorithm with $\Delta t=0.01$. The matrix inverse was calculated via the Moore-Penrose-Pseudoinverse from the C++ linear algebra library "Armadillo". For all simulations, the reservoir was initialised by first simulating without input for an integration time of 1000$\tau _1$ and then simulating with input for a buffer time of 1000 input clock-cycles. In the training stage, 5000 input values were used for the NARMA10 task and 40000 for the memory capacity. In both cases, the testing procedure involved 5000 inputs. Before feeding the input into the reservoir, it was normalised via a linear transformation, such that $u(t)\in [-1,1]$. This increases comparability between the memory capacity calculations and NARMA10, and ensures that inputs do not become too large. To avoid over-fitting, for the NARMA10 task regularisation by noise was included. This means that the Tikhonov regularisation parameter introduced in Eq. (2) was set to $\lambda =0$ and instead all entries of the state matrix were disturbed by additive Gaussian white noise with standard deviation $D=10^{-8}$. Regularisation by noise has been shown to be equivalent to Tikhonov regularisation [67]. For the masks, random binary values in $\{0,1\}$ were chosen. The default parameters used for simulation are given in Table 1.

Table 1. Default Dimensionless Parameter Values for the Numerical Simulations of the Reservoir Computing Performance of Two Oscillators (Index $j$) Described by Eq. (9).

View Table

6.2 Impact of coupling-delay and clock-cycle in ring-coupled systems

In this section, we will investigate the influence of a second feedback delay-time in a system of two ring-coupled nodes by allowing the delay-times $\tau _1$ and $\tau _2$ to be different. This setup corresponds to Fig. 6(a), i.e. Equation (9) with $\kappa ^{self}_{1,2}=0$.

As discussed in Section 3.3, previous works show that resonances between the delay-time $\tau$ and the clock-cycle $T$ decrease the memory capacity in systems with one delay [48,51]. These resonances occur at $m\tau =nT$ for $n,m\in \mathbb {N}$ (see Fig. 5(a)). Introducing a second delay via the ring-coupling scheme gives the same resonance structure when $\tau _1=\tau _2$. However, if we introduce a mismatch between the delays, i.e. $\tau _1\neq \tau _2$, resonances are found at rational multiples between the mean delay, $\tau _\textrm {mean}=\left (\tau _1+\tau _2\right )/2$, and the clock-cycle: $m\tau _\textrm {mean}=nT$ for $n,m\in \mathbb {N}$. This resonance structure is shown in Fig. 7. The resonances are especially pronounced at the points, where both delays are in resonance with the clock-cycle (intersection points of horizontal and vertical lines in Fig. 7), with the largest losses in the memory capacity being found at the resonances points where both delays are equal, i.e. $\tau _1=\tau _2=nT$ for $n\in \mathbb {N}$.

 

Fig. 7. Total memory capacity of two mutually-coupled oscillators (see Fig. 6(a) and Eq. (9) with $\kappa _j^{self}=0$), color coded as a function of the ring-delays $\tau _1$ and $\tau _2$. Dashed lines indicate $1/2(\tau _1+\tau _2)=n/m T$ (mean delay resonance-lines) with $n,m \in \mathbb {N}$, $\tau _{1,2}=n/m T$ and $\tau _1=\tau _2$, respectively. Resonances are especially pronounced at the intersections where both delays are in resonance.

Download Full Size | PDF

In order to understand why the resonances occur at the mean delay, it is useful to show how the delay-mismatched system can be rewritten in terms of a system with two identical delays. A system of $N$ oscillators in a delayed ring-topology can generically be described by

For the equivalent system, Eq. (12), the input term is shifted from $I_j(t)$ to $I_j(t-t'_j)$. This implies a general correspondence between ring-coupled systems with multiple delays and systems with delayed inputs. In [59] it was recently shown that an appropriately chosen additional delayed input can improve the reservoir computing performance of a delay-based reservoir for various benchmark tasks [59]. In order to show how a similar effect can be achieved in the ring-coupled system, it is necessary to look at the dependence on the delay-times of the individual contributions to the total memory capacity.

Figure 8 shows the linear memory capacity as a function of the mean delay and the number of steps into the past. In agreement with previous work, increasing the delay-time of a reservoir while keeping the clock-cycle constant, increases the long-time memory capabilities [51]. However, as the delay-times become much larger than the clock-cycle and the characteristic timescale of the non-linearity, gaps in the memory capacity begin to form. The input regions that can be remembered appear as oblique stripes in Fig. 8. If the delays are equal (Fig. 8(a)), their position is given by multiples of the ratio between delay and clock-cycle. More precisely, if $\tau \in [nT,(n+1)T[,n\in \mathbb {N}$, then the first areas with high memory (dark blue regions in Fig. 8) are found at $u_{-n}$, since each input is fed into the system for a time interval of length $T$. The width of those areas is determined by the internal timescale of the system.

 

Fig. 8. Impact of delay-mismatch: Linear memory capacity $C_{u_{-steps}}$, color coded as a function of mean delay and steps into the past, averaged over 10 masks. (a) Identical delays $\tau _1=\tau _2$, (b) Mismatched delays $\tau _1=\tau _2+400$,

Download Full Size | PDF

The gaps can be reduced by mismatching the delay-times, as can be seen by comparing Fig. 8(a), where $\tau _1=\tau _2$, with Fig. 8(b), where $\tau _1=\tau _2+400$. For increasing mean delays, more long-time memory contributions are present, with smaller gaps in the mismatched case. This phenomenon is valid for the linear and non-linear capacities (corresponding scans for the quadratic terms are shown in the supplemental material). In [69] a similar reduction of gaps in the memory capacity was achieved by coupling systems with multiple delays, where the authors used a so-called deep reservoir computing scheme. With our setup, we are able to achieve similar results in the less complex ring-coupling scheme.

The filling of the gaps in the memory capacity for mismatched delays can be understood qualitatively. We first note that the system without delay ($\tau _\textrm {mean}=0$) has some capability to memorise inputs for short distances into the past, due to the non-zero relaxation time of the dynamical nonlinearity. For the sake of simplicity, this contribution is neglected in the following argumentation. When a delay is introduced, each delay line $\tau _j$ "stores" information about previous inputs and possibly nonlinear transformations of them. For instance, if we consider a ring of two oscillators with $\tau _1=\tau _2=3T$, then the ring will store information about inputs $u(t-3T),u(t-6T),u(t-9T)$ and so on. Since $T$ is the input clock-cycle, this corresponds to inputs $u_{-3},u_{-6},u_{-9},\ldots$, clearly leaving some gaps of previous inputs, that cannot be remembered. On the other hand, if we choose a system with equal mean delay, but a mismatch in the delay, e.g. $\tau _1=2T,\tau _2=4T$, we obtain information about $u_{-2},u_{-4},u_{-6},u_{-8}$ and so forth, filling some of the gaps in the equal delay case.

Aside from being able to fill gaps in the memory capacity by mismatching the delays, Fig. 8 shows that the delays can be chosen such that the system fulfils particular memory requirements that could be beneficial for specific tasks. However, in the ring-topology, varying one of the delay-times always influences the mean delay, meaning that the effect of large differences in the two delays can not be fully utilised. Therefore, in Section 6.3 we include an additional self-feedback delay-line that allows for more tunability of the memory of the system.

6.3 Manipulating the range of accessible past inputs via additional delay-lines

In this section, we show how an additional self-feedback term can be used to tune the memory capacity of our system of two ring-coupled nodes and how this can be used to improve the performance of the NARMA10 task. The setup corresponds to Fig. 6(b), i.e. Equation (9) with $\kappa ^{self}_{1,2}\neq 0$. For simplicity, we choose parameters identical to the default ring-topology parameters for all nodes and set the self-feedback coupling parameters to the values of the ring-coupling. The self-delays are chosen equal, $\tau ^{self}=\tau _1^{self}=\tau _2^{self}$, and $\tau ^{self}$ is treated as a free parameter. By further choosing $\tau _1=\tau _2$, there are effectively two independent time-scales introduced by the delays. This setup is mathematically equivalent to a single nonlinear node with two delayed-feedback terms. Therefore, all delay-dependent effects shown below are also applicable to such a system.

Figure 9(a) shows the linear memory capacities, as a function of the number of steps into the past and the self delay-time $\tau ^{self}$ (the mean-time of the ring-coupling is constant). Dark regions indicate high memory. First, for small steps into the past, the memory capabilities do not change with $\tau ^{self}$ and appear as a vertical stripe with a width given by the ring-delay time (see Fig. 8(a)). Second, there are tilted regions in the Fig. 9(a) that are found at multiples of the self-delay time. Their position is completely determined by the value of the self-delay. Note that in the supplemental material, the quadratic memory capacities are presented, which show similar trends.

 

Fig. 9. Impact of self-delay: Linear memory capacity $C_{u_{-steps}}$ color coded as a function of the self-delay $\tau ^{self}$ and the steps into the past, averaged over 10 mask realisations for (a) $\tau _{1}=\tau _{2}=210$, and (b) $\tau _{1}=\tau _{2}=620$.

Download Full Size | PDF

The width of the regions with high memory can be tuned via the value of the ring-delay (higher ring-delays yield larger width). Nevertheless, the manipulation of the width is limited, because as we have seen in the previous section, high ring-delays lead to gaps in the memory capacity (see Fig. 8(a)). Such a case is depicted in Fig. 9(b), where a ring-delay above the threshold, where gaps in the memory capacity begin to occur, was chosen. Note, that for fundamental reasons, a reservoir computer cannot remember all previous inputs (fading memory property) but it is possible to tune which ones contribute to the total memory capacity.

Now, even if the memory capacity of a system is completely known, it still depends crucially on the task if high or low performance is achieved. As an example for a demonstration of the complex connection between both, we use the NARMA10 time series prediction task. In Fig. 10(a) the NARMA10 normalised root-mean-square error (NRMSE) is shown as a function of the self-delay. As can be seen, if the delays are chosen equal, $\tau ^{self}=\tau _{1}=\tau _2$, the NRMSE starts with moderate values of around 0.3. Increasing the self-delay, the NRMSE decreases to slightly above 0.1. This is to our knowledge unprecedented in the reservoir computing literature for this number of virtual nodes (200) and demonstrates that the delay-times are an important tuning parameter for improving task specific performance.

 

Fig. 10. (a) NARMA10 computing error (NRMSE) as a function of the self-delay time $\tau ^{self}$ in the all-to-all coupled topology displayed in Fig. 6(b), averaged over 10 mask realisations (error bars indicate standard deviation). (b) Memory capacity corresponding to input terms $u_{-i}u_{-i-9}$ as they occur in the NARMA10 series Eq. (13) as a function of the self delay-time and the recall step.

Download Full Size | PDF

Further, increasing the self-delay above $\tau ^{self}=2000$, which is 10 times the clock-cycle, leads to a sudden performance drop-off which we can explain by the specific memory requirement of the NARMA10 task where the input terms $u_{-i-9},u_{-i}$ have a dominant role in the defining series:

7. Conclusion

In this article, we have reviewed delay-based reservoir computing with a focus on the influence of feedback and coupling delay-times. We aim to bring to the attention of the wider reservoir computing community that the delay-times have the potential to play a far more important role in improving the performance of delay-based reservoir computers than they have up until now. Furthermore, we highlight that, although it was important for the conception of delay-based reservoir computing, there is no need to view delay-based reservoirs as networks. Viewing delay-based reservoirs as networks has, in fact, led to misconceptions about the relationship between the clock-cycle and the delay-times of the reservoir. By introducing the concept of time-multiplexed reservoir computing independent of time-delayed systems, we demonstrate that there is no predetermined relationship between the clock-cycle and the reservoir delay-times.

Various recent studies have shown that choosing the delay-time resonant with the clock-cycle is generally detrimental to the computing capabilities of the reservoir. The commonly used desynchronised case, where delay and clock-cycle only differ by one virtual node, also does not lead to task specific optimal performance. If instead, the delay-times of the reservoir are freely tuned, after fixing the input clock-cycle, delay-times can be chosen to match the memory requirements of a particular task. This does not affect the computational speed and could be of particular interest, when designing experiments targeting tasks with specific memory requirements.

Additionally, we have demonstrated the potential of a second delay line for optimizing the performance by considering a system of two delay-coupled oscillators. We have shown how varying these delay-times influences the linear memory capacity by filling in gaps in the linear memory capacity. By adding a self-feedback term, the memory properties can be tuned even more freely to achieve both long- and short-term memory. For a task such as NARMA10, which has very specific memory requirements, correctly tuning the delay-times leads to a significant improvement in the performance.