1. Introduction

Photonic accelerators [1] have been gaining attention in recent years, and a variety of implementations and applications have now been explored [2–9]. These advancements can be attributed to a growing awareness of the saturating speed of performance improvements in conventional computational systems [10], despite the soaring demands for information processing in an extensive range of applications, especially in machine learning. Reinforcement learning [11] is a subfield of machine learning that involves optimizing computer outputs or actions to maximize the reward function. Its applications are now essential to our daily lives, ranging from self-driving vehicles [12] and targeted advertising [13] to wireless networking [14], and there is now a strong demand for computational acceleration. Specifically, what we focus on here is decision-making. Environments in which agents make decisions can be uncertain or ever-changing, depending on the problem.

The multi-armed bandit (MAB) problem [15] is a fundamental problem setting of decision-making, in which a player repeatedly selects from multiple slot machines with unknown hit probabilities, aiming to maximize the total reward. An efficient strategy for the problem requires balancing two contradicting operations: exploration, in which a player tries to identify the most high-paying slot machine by taking various options, and exploitation, in which a player selects only the estimated best slot machine. The competitive multi-armed bandit (CMAB) problem [16] extends the MAB problem to a multi-player setting, in which when two or more players simultaneously select the same slot machine, the reward is equally divided among them, resulting in a loss of opportunities for benefiting from other slot machines. Therefore, in the CMAB problem, avoiding selection collisions among players is important to maximize total team rewards, besides exploration and exploitation. Recent studies have explored employing the distinct properties of photonic phenomena to address the MAB problem [17–23]. Collective decision-making with a laser network, proposed in [24], stands out as a promising method to tackle the CMAB problem in terms of its experimental viability and scalability. It leverages the chaotic behavior of multiple optically connected lasers: the spontaneous exchange of the leader–laggard relationship and zero-lag synchronization.

A leader–laggard relationship signifies the similarities observed in the temporal waveforms of optical intensity between two mutually coupled lasers [25], where one of the lasers (referred to as the ‘leader’) oscillates so as to precede the other (referred to as the ‘laggard’) with an offset given by the coupling delay time between the two. A previous study [26] revealed that the relation switches spontaneously during low-frequency fluctuation (LFF) dynamics [27]. The LFF dynamics are characterized by quasi-periodic fluctuations on a MHz time scale superimposed on chaotic oscillations on a GHz time scale in the optical intensity, observed under conditions of intense optical coupling and low pump current. In decision-making systems based on the exchange of leader–laggard roles in the LFF dynamics [21,24,28], physical semiconductor lasers have a one-to-one relationship with virtual slot machines in the MAB problem, and a strategy is adopted to select the slot machine corresponding to the leader laser at a given moment. Consequently, a player alternately selects multiple slot machines, achieving exploration in the context of the MAB problem.

In zero-lag synchronization [29–31], conversely, a set of lasers in a network oscillates in synchrony without any delay. This non-trivial synchronous phenomenon has been demonstrated in theory and has been observed at a reduced level in experiments. One approach to predicting the formation of zero-lag synchronization in a laser network involves calculating the power of its adjacency matrix [30]. An unweighted adjacency matrix represents how information is transmitted from one node to another in the network, allowing for a qualitative assessment of whether each laser is zero-lag synchronized to the others. Exploiting this matrix-based inference, the previous work [24] introduced a four-laser network that exhibits zero-lag synchronization, forming two clusters of two lasers each, to address the CMAB problem with two players and two slot machines. The zero-lag synchronization enables each player to have information about the others’ slot machine selections without directly measuring the optical outputs of the opponents’ lasers at a remote location and to avoid selection collisions, thus attaining cooperative decision-making in the CMAB problem.

However, the previous study dealt only with the CMAB problem in which the number of players and the number of slot machines are the same, and each player selects slot machines in equal proportions. More general configurations should be implemented for practical applications where there can exist more options than players or where players want to take specific options more frequently than the alternatives. In addition, the previous work did not fully explore the networks, and there might be other possible networks [32] that are effective for a collective decision-making system.

In this study, first, we examine other candidates of laser networks that exhibit equivalent behaviors to the previously proposed network, and are capable of solving the competitive multi-armed bandit (CMAB) problem with a configuration of two players and two slot machines. The synchronous states in the possible networks are evaluated by performing a stability analysis. The need for this arises because the argument grounded in an unweighted adjacency matrix relied solely on conceptual observations and empirical rules without quantitative synchronization assessment. Subsequently, we demonstrate asymmetric slot machine selections by players both in numerical simulations and experiments, extending the collective decision-making system with a laser network to a broader and more practical range of problems. Our study reinforces the potential and feasibility of decision-making based on laser chaos and photonic accelerators.

2. Network configuration: requirements and stability analysis for verification

2.1 Candidate networks for decision-making

First, we reintroduce a decision-making system [24] for addressing the competitive multi-armed bandit (CMAB) problem in a 2-player, 2-slot-machine situation. Figure 1 illustrates the concept of the conflict-avoiding decision-making with a laser network. Lasers 1A and 1B are allocated to Player 1, and Lasers 2A and 2B are allocated to Player 2. Lasers 1A (2A) and 1B (2B) respectively correspond to Slot A and B selected by Player 1 (2). In this laser network, there exists a leader–laggard relationship between Lasers 1A and 1B and between Lasers 2A and 2B, where the oscillation of one of the laser leads that of the other, and the leader spontaneously switches in the LFF regime [26]. Meanwhile, Lasers 1A and 2B and Lasers 2A and 1B are, respectively, in zero-lag synchronization, where two lasers’ oscillations synchronize without delay. Each player selects a slot machine represented by the leader laser among the lasers assigned to them. When Laser 1A is the leader, Player 1 selects Slot A. Simultaneously, when Laser 2B, which is zero-lag synchronized with Laser 1A, becomes the leader, Player 2 selects Slot B. The same holds when Laser 1B (2A) is the leader. Therefore, players autonomously avoid conflicts between them without explicitly knowing the other’s slot machine selection or the behaviors of the lasers, and they achieve cooperative decision-making.

 

Fig. 1. Schematic illustration of decision-making system to solve the competitive multi-armed bandit (CMAB) problem with two players and two slot machines proposed in Ref. [24]. Zero-lag synchronization between Lasers 1A and 2B and between Lasers 2A and 1B contribute to conflict avoidance in decision-making.

Download Full Size | PDF

We characterize the synchronization behavior of the laser network for this problem as follows. $\mathrm {i}$) The four lasers are separated into two clusters of two lasers each. $\mathrm {ii}$) Lasers 1A and 2B (2A and 1B) are in zero-lag synchronization. $\mathrm {iii}$) Lasers 1A and 1B (2A and 2B) are not in zero-lag synchronization. Therefore, we can utilize networks that exhibit such synchronization for the collective decision-making system. This type of synchronization is called cluster synchronization in the literature [33,34], but we focus on more specific cases where the number of nodes in a cluster is the same. One necessary condition for such cluster synchronization is that the total optical injection into each laser in one cluster is uniform.

We define cluster 1 to consist of Lasers 1A and 2B and cluster 2 to consist of Lasers 2A and 1B. Here, we also need to discuss the following constraints on the laser networks. First, the coupling delay time of light $\tau$ is identical for all couplings. $\tau$ becomes a typical time scale in the dynamics of the leader–laggard relationship [26]. The effective frequency of decision-making is governed by $\tau$ [21,24], and our concern is establishing a uniform strategy for all players. Second, the coupling strength from a laser in cluster 1 to one in cluster 2 is uniform, and we denote this value as $\kappa _1$. Similarly, we represent the coupling strength from a laser in cluster 2 to one in cluster 1 as $\kappa _2$. Third, there is no connection between lasers within the same cluster. These assumptions are reasonable to avoid unnecessary complexity in our discussion, and we primarily focus on the networks satisfying those three conditions. However, the cluster synchronization can also be observed in the more general networks [34]. We will discuss cluster synchronization in networks with complicated couplings in the Appendix A, relaxing the second and third conditions.

Taking into account the factors described so far, possible networks are a complete bipartite graph shown in Fig. 2(a), and ones obtained by cutting some of this network’s paths while maintaining the cluster synchronization. There are in total six candidate networks obtained by removing some links of the complete bipartite network, when we consider ones that coincide by color exchange, rotation, or reflection symmetry, as illustrated in Fig. 2(b). Note that the network (IV) is identical to the one already proposed in the literature.

 

Fig. 2. Network configurations that can be used for cooperative decision-making. (a) Complete bipartite network of four lasers. (b) Six candidate networks that meet constraints and have solutions of desired cluster synchronization.

Download Full Size | PDF

Our model of semiconductor lasers in the networks (I)-(VII) is described by Lang–Kobayashi equations as follows [35]:

In all seven networks shown in Fig. 2, identical synchronous solutions corresponding to the cluster synchronization, $E_{\rm {1A}}(t) = E_{\rm {2B}}(t)$, $N_{\rm {1A}}(t) = N_{\rm {2B}}(t)$, and $E_{\rm {2A}}(t) = E_{\rm {1B}}(t)$, $N_{\rm {2A}}(t) = N_{\rm {1B}}(t)$ exist under the condition that $\omega _{\rm {1A}} = \omega _{\rm {2B}} (\equiv \omega _1)$ and $\omega _{\rm {2A}} = \omega _{\rm {1B}} (\equiv \omega _2)$. Note that a global synchronous solution for four lasers, $E_{\rm {1A}}(t) = E_{\rm {1B}}(t) = E_{\rm {2A}}(t) = E_{\rm {2B}}(t)$ and $N_{\rm {1A}}(t) = N_{\rm {1B}}(t) = N_{\rm {2A}}(t) = N_{\rm {2B}}(t)$, which is a specific case of the cluster synchronization, can also exist in the seven networks with some additional assumptions, e.g., $\omega _1 = \omega _2$. If all four lasers are completely synchronized, players cannot decide which slot machine to select because neither laser becomes a leader. This can be a problem for decision-making with the laser network. Therefore, the stability of the solutions for both cluster synchronization and global synchronization should be discussed.

2.2 Stability analysis of synchronous solutions

Although we have confirmed in the previous section that cluster synchronous solutions in the laser networks exist, their stability is not guaranteed. Also, there should be no global synchronization so that the decision-making strategy remains valid. First, we evaluate the stability of the desired cluster synchronization in the laser networks, (I)-(VII), by numerically calculating conditional Lyapunov exponents of the solutions of the cluster synchronization. For networks that exhibit stable cluster synchronization, we further compute conditional Lyapunov exponents of global synchronization to examine the instability of the undesired global synchronization. Conditional Lyapunov exponents characterize the synchronous behavior between multiple dynamical systems capable of dealing with chaotic systems [36–38]. The exponents quantify how small displacements between trajectories along a specific direction expand or decay on average over time. A negative maximum conditional Lyapunov exponent indicates asymptotic stability of the synchronous solution, resulting in complete synchronization between the systems. On the other hand, a positive value suggests that the solution is unstable, so that identical synchronization is not observed, with different initial conditions for each system.

In this section, we assume that $\omega _{\rm {1A}} = \omega _{\rm {2B}} = \omega _1$ and $\omega _{\rm {2A}} = \omega _{\rm {1B}} = \omega _2$ for the existence of the cluster synchronous solutions. Therefore, the optical phase difference terms are limited to two types: $\theta _1(t) \equiv (\omega _2 - \omega _1)t - \omega _2\tau$ and $\theta _2(t) \equiv (\omega _1 - \omega _2)t - \omega _1\tau$. Furthermore, we also assume that $\omega _1 = \omega _2$ for simplicity and to ensure the consistency of the analysis of cluster synchronization with that of global synchronization. Notably, frequency detuning between clusters, $\Delta f \equiv \frac {1}{2\pi }(\omega _1 - \omega _2)$, has little influence on the synchronous states in two clusters. Additionally, we adequately configure $\kappa _1$ and $\kappa _2$ depending on each network as follows: for (I) $\kappa _1 = \kappa _2 = \kappa /2$, for (II) and (III) $\kappa _1 = \kappa /2, \kappa _2 = \kappa$, and for (IV), (V), (VI), and (VII) $\kappa _1 = \kappa _2 = \kappa$. We focus on equal total coupling strength of the injected light into one laser for all networks, and we can consider the synchronization under dynamics equivalent among lasers.

First, we discuss conditional Lyapunov exponents for the cluster synchronization. Here we introduce variables spanning synchronized manifolds, $E_{\rm {S1}} = (E_{\rm {1A}} + E_{\rm {2B}})/2$, $N_{\rm {S1}} = (N_{\rm {1A}} + N_{\rm {2B}})/2$, $E_{\rm {S2}} = (E_{\rm {2A}} + E_{\rm {1B}})/2$, $N_{\rm {S2}} = (N_{\rm {2A}} + N_{\rm {1B}})/2$, and others spanning anti-synchronized, $E_{\rm {AS1}} = (E_{\rm {1A}} - E_{\rm {2B}})/2$, $N_{\rm {AS1}} = (N_{\rm {1A}} - N_{\rm {2B}})/2$, $E_{\rm {AS2}} = (E_{\rm {2A}} - E_{\rm {1B}})/2$, $N_{\rm {AS2}} = (N_{\rm {2A}} - N_{\rm {1B}})/2$, focusing on the stability of the synchronization between Lasers 1A and 2B, and that between Lasers 2A and 1B. If the cluster synchronization is asymptotically stable, for instance, $E_{\rm {1A}}$ and $E_{\rm {2B}}$ converge to $E_{\rm {S1}}$, $E_{\rm {2A}}$ and $E_{\rm {1B}}$ to $E_{\rm {S2}}$, and $E_{\rm {AS1}}$ and $E_{\rm {AS2}}$ to 0. On the contrary, if the synchronization is unstable, $E_{\rm {AS1}}$ and $E_{\rm {AS2}}$ exponentially expand over time. The same holds for the carrier densities.

We can obtain differential equations of the variables for synchronized manifolds and those for anti-synchronized manifolds with Eq. (1) and (2). Regarding the equations of the variables for synchronized manifolds, we assume $E_{\rm {AS1}}(t) = E_{\rm {AS2}}(t) = 0$ and $N_{\rm {AS1}}(t) = N_{\rm {AS2}}(t) = 0$ to compute complete synchronous trajectories. Derived equations are shown below (a detailed derivation is provided in the Supplement 1).

These equations are the same among the networks. As for the equations for anti-synchronized manifolds, on the other hand, we treat the variables $E_{\rm {AS1}}$, $N_{\rm {AS1}}$, $E_{\rm {AS2}}$, and $N_{\rm {AS2}}$ as tiny values and linearize the equations in terms of these variables to evaluate Lyapunov exponents. The linearized equations are described as follows. Here, we substitute $\Delta _{E1}$, $\Delta _{N1}$, $\Delta _{E2}$, and $\Delta _{N2}$ for $E_{\rm {AS1}}$, $N_{\rm {AS1}}$, $E_{\rm {AS2}}$, and $N_{\rm {AS2}}$, for clarity.

We set parameters of Lang–Kobayashi equations to typical values used in Refs. [21,24,28], as shown in Table 1. The time step $h$ used to calculate conditional Lyapunov exponents is set to 5 ps. We computed the time evolution of the delay-history vector, $\boldsymbol {\Delta}(t) = [\Delta _{E1}(t), \Delta _{E1}(t-h), \Delta _{E1}(t-2h),\ldots, \Delta _{E1}(t-\tau ), \Delta _{N1}(t),\ldots, \Delta _{N2}(t-\tau )]$, using Eq. (4)–(9) and employing the fourth order Runge–Kutta method, over the calculation period of $T = {1000}\;\textrm{ns}$ after waiting for a sufficiently long transient of 10 000 ns. The norm of the delay-history vector extended or shrank over time approximately at a pace given by $|\boldsymbol {\Delta}(t)| = e^{Lt}|\boldsymbol {\Delta}(0)|$, where we define $L$ as the maximal conditional Lyapunov exponent of the cluster synchronization. Technically, we normalized the delay-history vector to its initial norm every 10 steps, i.e., 50 ps, to prevent numeric overflow or underflow, and evaluated the time-average value of $L$.

Table 1. Parameters of Lang–Kobayashi equations.

View Table | View all tables in this article

Figure 3(a) illustrates the maximum Lyapunov exponents of the cluster synchronization in the networks, computed with varying settings of $\kappa = {0}\;\textrm{ns}^{-1}$ to 30 ns^-1, in increments of 1 ns^-1. The exponents for the networks (I), (II), (III), (IV), and (V) are negative in the region where $\kappa \geq {8}\;\textrm{ns}^{-1}$, represented by the red and blue curves, which suggests asymptotic stability of the cluster synchronization. For the networks (VI) and (VII), on the contrary, the exponents are positive with any value of $\kappa$, indicating that the cluster synchronization is unstable in the two networks. Therefore, the networks (I), (II), (III), (IV) and (V) remain candidates for the proposed cooperative decision-making system, while (VI) and (VII) are not.

 

Fig. 3. Maximum Lyapunov exponents numerically calculated with coupling strength $\kappa = {0}\;\textrm{ns}^{-1} \;\textrm{to}\; {30}\;\textrm{ns}^{-1}$. (a) The exponents of the cluster synchronization. (b) The exponents of the global synchronization.

Download Full Size | PDF

Next, we discuss conditional Lyapunov exponents for the global synchronization. We introduce variables spanning synchronized manifolds, $E_{\rm {S'}} = (E_{\rm {1A}} + E_{\rm {1B}} + E_{\rm {2A}} + E_{\rm {2B}})/4$, $N_{\rm {S'}} = (N_{\rm {1A}} + N_{\rm {1B}} + N_{\rm {2A}} + N_{\rm {2B}})/4$, and others spanning anti-synchronized manifolds, $E_{\rm {AS1'}} = (E_{\rm {1A}} + E_{\rm {1B}} - E_{\rm {2A}} - E_{\rm {2B}})/4$, $N_{\rm {AS1'}} = (N_{\rm {1A}} + N_{\rm {1B}} - N_{\rm {2A}} - N_{\rm {2B}})/4$, $E_{\rm {AS2'}} = (E_{\rm {1A}} - E_{\rm {1B}} + E_{\rm {2A}} - E_{\rm {2B}})/4$, $N_{\rm {AS2'}} = (N_{\rm {1A}} - N_{\rm {1B}} + N_{\rm {2A}} - N_{\rm {2B}})/4$, $E_{\rm {AS3'}} = (E_{\rm {1A}} - E_{\rm {1B}} - E_{\rm {2A}} + E_{\rm {2B}})/4$, $N_{\rm {AS3'}} = (N_{\rm {1A}} - N_{\rm {1B}} - N_{\rm {2A}} + N_{\rm {2B}})/4$. Similarly to the case of the cluster synchronization, we obtain differential equations of the newly introduced variables using Eq. (1) and (2), and then derive equations for complete synchronous trajectories and linearized equations to calculate Lyapunov exponents for the networks (I), (II), (III), (IV) and (V), under the corresponding assumptions. The equations are provided in the Supplement 1. Notably, the delay terms in the linearized equations depend on each network. Parameters and other computational conditions are the same as the ones used for the exponents of the cluster synchronization.

Figure 3(b) shows the maximum Lyapunov exponents of the global synchronization in the networks (I)-(V), calculated with $\kappa = {0}\;\textrm{ns}^{-1}\;\textrm{to}\;{30}\;\textrm{ns}^{-1}$. The exponents for all five networks are positive with any $\kappa$, which demonstrates that the undesired global synchronization is unstable for the networks. Therefore, the networks (I), (II), (III), (IV) and (V) are suitable for the decision-making system with sufficiently strong coupling strength.

3. Asymmetric preferences in cooperative decision-making

3.1 Numerical simulation

Considering specific problem settings, the symmetric strategy of slot machine selection is not enough, and the need to introduce asymmetric preferences of players arises. For example, when two players select among three slot machines, one of which is much lower-paying than the others, players are encouraged to select only the two good ones while avoiding conflict between their choices. Another instance arises when we want one of the players to select a particular slot machine more frequently so that they get more (less) reward than the others. Therefore, the capability to manipulate slot machine selection ratios, while maintaining conflict avoidance, is essential for the CMAB problem solver implementation.

Our interest lies in investigating how to control the proportions of slot machine selection through a fundamental modification of the lasers’ dynamics. In the proposed method, manipulating the balance of slot machine selection ratios corresponds to changing the probabilities of individual lasers leading the others. In this context, we introduce two functions to quantify the leader–laggard relations between lasers, as previously discussed in the literature [21,24,26]. One of these functions is a cross-correlation function, which is defined as follows:

The previous study revealed that the leader probabilities of lasers, as determined by STCC, can be controlled by adjusting the balance of coupling strength in situations involving two mutually coupled lasers [21], as well as configurations with three or more lasers in unidirectional ring setups [28]. Building on this understanding, we anticipate that leader probabilities in the proposed laser network can also be manipulated by altering the ratios of optical injection. In this numerical simulation, we choose the network (IV) based on its ability to achieve zero-lag synchronization in the subsequent experiment. As mentioned in Section 2.1, we assume that the coupling strength from Laser 1A to Laser 2A is identical to that from Laser 1A to Laser 1B, denoted as $\kappa _1$. Similarly, the coupling strength from Laser 2A to Laser 1A should be equal to that from Laser 2A to Laser 2B, termed as $\kappa _2$. In the numerical simulation, $\kappa _1$ and $\kappa _2$ are varied in the following manner, with the detuning of coupling strength $\Delta \kappa \equiv \kappa _1 - \kappa _2$:

Notably, other operational parameters, such as frequency detuning between clusters (discussed in the previous research [26] for two lasers) or injection current, can also significantly impact the leader probabilities. See the Supplement 1 for the result of numerical simulations to investigate the potential effects of frequency detuning and imbalanced injection currents between clusters. However, we focus on the coupling strength in this paper primarily because coupling strength is related to optical fibers accessible to both players with appropriate system design. Frequency detuning and injection current are, on the contrary, associated with individual lasers, making them unsuitable for operational parameters in considering the expansion toward decision-making in the CMAB problem.

With the configurations described so far, we generate the temporal waveforms of laser intensity $I_k(t) ~ (k = \rm {1A}, \rm {1B}, \rm {2A}, \rm {2B})$. Subsequently, we calculate cross-correlation values between Lasers 1A and 2B and those between Lasers 2A and 1B, and short-term cross-correlation values. We systematically vary $\Delta \kappa$ from −20 ns^-1 to 20 ns^-1, in increments of 1 ns^-1. Apart from coupling strength, the parameters applied in these simulations for the Lang–Kobayashi equations are consistent with those used in Section 2.2, as detailed in Table 1.

As representative instances, the results for $\Delta \kappa = {0}\;\textrm{ns}^{-1} (\kappa _1 = \kappa _2 = {30}\;\textrm{ns}^{-1})$, $\Delta \kappa = {5}\;\textrm{ns}^{-1} (\kappa _1 = {30}\;\textrm{ns}^{-1}, \kappa _2 = {25}\;\textrm{ns}^{-1})$, and $\Delta \kappa = {10}\;\textrm{ns}^{-1} (\kappa _1 = {30}\;\textrm{ns}^{-1}, \kappa _2 = {20}\;\textrm{ns}^{-1})$ are illustrated in Fig. 4. Figure 4(a), (b), and (c) exhibit the temporal laser intensity waveforms for $\Delta \kappa = {0}\;\textrm{ns}^{-1}$, 5 ns^-1, and 10 ns^-1. Figure 4(d), (e), and (f) respectively present the outcomes after applying a low-pass filter (the cutoff frequency is 60 MHz) to (a), (b), and (c), for visual clarity of laser intensity dropouts. In Fig. 4(d), we observe sudden dropouts followed by gradual intensity recoveries, typical phenomena in LFF dynamics. Moving to Fig. 4(e), the dropouts are less conspicuous compared to Fig. 4(d), and their intervals become irregular. The waveform becomes even more chaotic in Fig. 4(f). Meanwhile, zero-lag synchronization between Lasers 1A and 2B, as well as that between Laser 2A and 1B, persist, judging from the synchronous waveforms in Figs. 4(a)-(f), and the cross-correlation functions $\hat {C}_{\rm {1A,\ 2B}}$ and $\hat {C}_{\rm {2A,\ 1B}}$ having a peak at 0 ns, with a value of exactly 1.0 for any $\Delta \kappa$ configuration.

 

Fig. 4. Numerical simulation results - temporal laser intensity waveform, and low-pass-filtered intensity for visual clarity of intensity dropouts, calculated for different coupling strength settings corresponding to different targets of leader probabilities. (a), (d) $\Delta \kappa = {0}\;\textrm{ns}^{-1} (\kappa _1 = \kappa _2 = {30}\;\textrm{ns}^{-1})$. (b), (e) $\Delta \kappa = {5}\;\textrm{ns}^{-1} (\kappa _1 = {30}\;\textrm{ns}^{-1}, \kappa _2 = {25}\;\textrm{ns}^{-1})$. (c), (f) $\Delta \kappa = {10}\;\textrm{ns}^{-1} (\kappa _1 = {30}\;\textrm{ns}^{-1}, \kappa _2 = {20}\;\textrm{ns}^{-1})$.

Download Full Size | PDF

Figure 5(a), (b), and (c) illustrate the STCC waveforms for $\Delta \kappa = {0}\;\textrm{ns}^{-1}$, 5 ns^-1, and 10 ns^-1, respectively. In Fig. 5(a), frequent switching between $C_{\rm {1A}}$ and $C_{\rm {1B}}$, and that between $C_{\rm {2B}}$ and $C_{\rm {2A}}$ are observed, with their timing being precisely synchronized. In Fig. 5(b), the periods during which $C_{\rm {1B}} > C_{\rm {1A}}$ $(C_{\rm {2A}} > C_{\rm {1B}})$ appear to be slightly longer than those during which $C_{\rm {1A}} > C_{\rm {1B}}$ $(C_{\rm {2B}} > C_{\rm {2A}})$. In Fig. 5(c), the spontaneous exchanges between $C_{\rm {1A}}$ and $C_{\rm {1B}}$, and those between $C_{\rm {2B}}$ and $C_{\rm {2A}}$ are not remarkable anymore. The leader probabilities $L_{\rm {1A}}$, $L_{\rm {1B}}$, $L_{\rm {2A}}$, and $L_{\rm {2B}}$, calculated with the STCC waveforms, are shown in the lower left of Figs. 5(a), (b), and (c). Computing the probabilities, we compare $C_{\rm {1A}}$ and $C_{\rm {1B}}$, and $C_{\rm {2A}}$ and $C_{\rm {2B}}$ every 1 ns, and $T_{\rm {valid}}$ is 10 000 ns for (a), (b), and (c).

 

Fig. 5. Numerical simulation results - short-term cross-correlation (STCC) values and leader probabilities. (a) $\Delta \kappa = {0}\;\textrm{ns}^{-1} (\kappa _1 = \kappa _2 = {30}\;\textrm{ns}^{-1})$. (b) $\Delta \kappa = {5}\;\textrm{ns}^{-1} (\kappa _1 = {30}\;\textrm{ns}^{-1}, \kappa _2 = {25}\;\textrm{ns}^{-1})$. (c) $\Delta \kappa = {10}\;\textrm{ns}^{-1} (\kappa _1 = {30}\;\textrm{ns}^{-1}, \kappa _2 = {20}\;\textrm{ns}^{-1})$.

Download Full Size | PDF

We repeat the computation 40 times while changing initial states of the lasers randomly and take an average for each $\Delta \kappa$ value. The leader probabilities of each laser are shown in Fig. 6. With a positive value of $\Delta \kappa$, $L_{\rm {1A}}$ is greater than $L_{\rm {1B}}$, and similarly, $L_{\rm {2B}}$ is greater than $L_{\rm {2A}}$. As $\Delta \kappa$ reaches 20 ns^-1, $L_{\rm {1A}}$ and $L_{\rm {2B}}$ nearly converge toward 1. Conversely, for a negative value of $\Delta \kappa$, $L_{\rm {1B}}~(L_{\rm {2A}})$ is higher than $L_{\rm {1A}}~(L_{\rm {2B}})$ and as $\Delta \kappa$ approaches −20 ns^-1, $L_{\rm {1A}}$ and $L_{\rm {2B}}$ almost converge towards 0.

 

Fig. 6. Leader probabilities of the four lasers changing with the difference of the coupling strength $\Delta \kappa$.

Download Full Size | PDF

Here, we define the collision rate (CR) as the number of points at which the two players select the same slot machine simultaneously. With these configurations, CR $= (T_{\rm {bothA}} + T_{\rm {bothB}}) ~/~ T_{\rm {valid}}$, where $T_{\rm {bothA}}$ represents the periods during which $C_{\rm {1A}} < C_{\rm {1B}}$ and $C_{\rm {2A}} < C_{\rm {2B}}$ hold and the reverse is satisfied for the notation $T_{\rm {bothB}}$. In the numerical simulation, CR is 0 for any $\Delta \kappa$, indicating that two players always select the slot machines separately. This is based on the fact that the cluster synchronization is stable in the range of ${-20}\;\textrm{ns}^{-1} \leq \Delta \kappa \leq {20}\;\textrm{ns}^{-1}$, i.e., $\kappa _i \geq {8}\;\textrm{ns}^{-1}$ holds for both $i = 1,2$, following from the discussion in Section 2.2. Consequently, we demonstrate asymmetric preferences as well as conflict avoidance in the $2\times 2$ CMAB by changing the coupling strengths of the laser network.

Note that we assume uniformity in frequency of all four lasers in the numerical simulations. Frequency detuning is inherently unavoidable in experiments, and possible detuning within a cluster, i.e., non-zero differences between $\omega _{\mathrm {1A}}$ and $\omega _{\mathrm {2B}}$, and between $\omega _{\mathrm {2A}}$ and $\omega _{\mathrm {1B}}$, potentially leads to the collapse of perfect cluster synchronization. However, it is possible to experimentally observe states that are ‘almost synchronized,’ characterized by high cross-correlation values between laser intensities or optical frequency mutually pulled closer, as long as the values of the frequency detuning $\Delta f_1 \equiv \frac {1}{2\pi }(\omega _{\mathrm {1A}} - \omega _{\mathrm {2B}})$ and $\Delta f_2 \equiv \frac {1}{2\pi }(\omega _{\mathrm {2A}} - \omega _{\mathrm {1B}})$ are small (See Appendix B, for the numerical simulation to investigate the influence of the detuning within a cluster). Therefore, we consider that our collective decision-making principle remains effective for experimental settings as long as the near-synchronized states hold. The hypothesis is supported by the previous work [24], which has experimentally verified a four-laser network in near-synchronized states in two clusters for a fair situation. Our numerical simulation results, including unfair scenarios, will be validated in Section 3.2.

Moreover, there are discrepancies in the intrinsic parameters of the four lasers, e.g., the relaxation oscillation frequency, in experiments, which also deteriorate synchronization states, increasing the collision rate. We will discuss the impact of different laser parameters on our collective decision-making principle in Appendix C.

3.2 Experiment

To validate our numerical simulations, we aim to observe a set of intensity waveforms that yield STCC values corresponding to different leader probabilities, including an equal situation, while achieving low conflict. Similarly to the simulations, we adopt the network (IV). The experimental setups are illustrated in Fig. 7, and the equipment is described in detail in Table 2. We use four distributed-feedback (DFB) semiconductor lasers without isolators, enabling optical injection. Two lasers, referred to as Lasers 1A and 2A, are mutually coupled through separate unidirectional optical paths established using optical circulators and isolators. Then, half of the light from Lasers 1A and 2A is individually injected into the other laser, referred to as Laser 1B and 2B, respectively. Injection current thresholds for Lasers 1A, 1B, 2A, and 2B are 11.0 mA, 11.8 mA, 11.6 mA and 11.7 mA, respectively. Injection currents of Lasers 1A, 1B, 2A, and 2B are set to 12.1 mA, 12.4 mA, 12.7 mA and 12.9 mA, respectively, corresponding to approximately 1.1 times the thresholds. The injection current distribution enables the LFF dynamics and ensures an equivalent optical power level for each laser when uncoupled. We adjust the temperature to achieve the peak wavelength of 1547.0 nm in the optical spectrum for the lasers, and set the detuning of solitary optical frequencies to 0 Hz for all lasers.

 

Fig. 7. Experimental setup to validate the change in leader probability through manipulation of coupling strength. OSC: oscilloscope, PD: photodetector, ISO: optical isolator, CIRC: optical circulator, FC: fiber coupler, ATT: attenuator.

Download Full Size | PDF

Table 2. Experimental apparatus.

View Table | View all tables in this article

Here, we introduce $\kappa _{k,l}$, which represents the optical amplification rate of the light from Laser $k$ to Laser $l$ (when $\kappa _{k,l} = {1}$ the light from Laser $k$ is transmitted to Laser $l$ without any amplification or attenuation, and when $\kappa _{k,l} = {0}$, no light passes through). In the numerical simulation, we have assumed that all four lasers have identical properties and that $\kappa _{\rm {1A},\rm {2A}} = \kappa _{\rm {1A},\rm {1B}} = \kappa _1$ and $\kappa _{\rm {2A},\rm {1A}} = \kappa _{\rm {2A},\rm {2B}} = \kappa _2$ should be satisfied for cluster synchronization. In contrast, in the experimental setups, each laser has distinct features, which makes it necessary to scale the coupling strength $\kappa _{k,l}$. We insert electronic variable optical attenuators into each path and tune $\kappa _{k,l}$ by changing the applied voltage to the attenuators. We adjust the coupling strength, aiming at several different values of leader probabilities and maximal precision of zero-lag synchronization. Part of the laser output is directed into ultrafast photodetectors and is then transmitted to an oscilloscope configured with a sampling rate of 50 GSample /s. The coupling delay time of the four optical paths $\tau = {75.38}\;\textrm{ns}$ with errors less than 1 ns, estimated from the peaks of $\hat {C}_{\mathrm {1A,2A}}(s)$, $\hat {C}_{\mathrm {2A,1A}}(s)$, $\hat {C}_{\mathrm {1A,1B}}(s)$, and $\hat {C}_{\mathrm {2A,2B}}(s)$.

It is challenging to systematically alter coupling strengths, observe zero-lag synchronized intensity waveforms, and compute leader probabilities, as done in the numerical simulations. Therefore, we change the values of $\kappa _{\rm {1A},\rm {2A}}$, $\kappa _{\rm {1A},\rm {1B}}$, $\kappa _{\rm {2A},\rm {1A}}$, and $\kappa _{\rm {2A},\rm {2B}}$ to five different coupling strength setups named [A]-[E], each of which aiming at different resultant leader probabilities, as shown in Table 3. Our goal is to experimentally demonstrate different leader probabilities while avoiding collisions. Then, we acquire laser intensities and calculate the leader probabilities under each configuration. Laser intensities, low-pass-filtered intensities, and STCC waveforms are shown in Figs. 8, 9, and 10, respectively. For each setup [A]-[E], the coupling strength is configured to maximize the values of $\hat {C}_{\rm {1A,\ 2B}}(0)$ and $\hat {C}_{\rm {2A,\ 1B}}(0)$, and to equalize $L_{\rm {1A}}$ and $L_{\rm {2B}}$, as well as $L_{\rm {2A}}$ and $L_{\rm {1B}}$, thus reducing the collision rate (CR). This results in non-uniform coupling strengths, as opposed to our more simplistic numerical simulations. This extra step of tuning is required to counteract the inherent differences of the individual lasers in the experiment.

 

Fig. 8. Experimental results - temporal laser intensity waveform, measured with different coupling strength settings. (a) to (e) correspond to [A] to [E], respectively. Coupling strength corresponding to each setup [A]-[E] are shown in Table 3.

Download Full Size | PDF

 

Fig. 9. Experimental results - low-pass-filtered intensity for visual clarity of laser intensity dropouts, with (a) to (e) corresponding to different coupling strength settings [A] to [E], calculated from laser intensity waveforms shown in Fig. 8.

Download Full Size | PDF

 

Fig. 10. Experimental results - short-term cross-correlation (STCC) values, leader probabilities, and collision rate (CR), with (a) to (e) corresponding to different coupling strength settings [A] to [E], and (f) for an enlarged view of STCC values for setup [C].

Download Full Size | PDF

Table 3. Coupling strength setups, each aiming at different resultant leader probabilities: [A] and [B] favoring lasers in cluster 2 as leaders, [C] for fairness, and [D] and [E] favoring lasers in cluster 1.

View Table | View all tables in this article

Setup [C] aims at a fair situation, corresponding to the $\Delta \kappa = {0}\;\textrm{ns}^{-1}$ case in the numerical simulation. In Fig. 8(c) and Fig. 9(c), we observe sudden dropouts and gradual recovery of intensities, which are typical behaviors in the LFF dynamics. Setup [D] and [E] are designed to achieve $\Delta \kappa > {0}\;\textrm{ns}^{-1}$ conditions. Especially in Fig. 8(e) and Fig. 9(e), quasi-periodic dropouts and recoveries are also observed, showing that lasers were still in the LFF regimes. However, the dropouts become more frequent compared to (c), which is a sign of transient to pure chaotic dynamics. Similar tendencies are observed in Fig. 8(a) and Fig. 9(a), where setup [A] and [B] are designed to attain $\Delta \kappa < {0}\;\textrm{ns}^{-1}$ situations. Cross-correlation values with time shift being 0 ns, as provided in Table 4, indicate that we experimentally observe states that Lasers 1A and 2B and Lasers 2A and 1B are respectively almost synchronized for the five coupling strength setups. In our experimental configurations, perfect zero-lag synchronization characterized by $\hat {C}_{\rm {1A,2B}}(0) = \hat {C}_{\rm {2A,1B}}(0) = 1$ is not observed, due to mismatches in laser internal properties, physical discrepancies between the coupling delay time of fibers, and thermal instabilities in the environment.

Table 4. Cross-correlation values with a peak at a time shift of 0 ns, calculated from laser intensity waveforms shown in Fig. 8.

View Table | View all tables in this article

Figure 10 shows STCC waveforms and the corresponding leader probabilities and collision rate (CR). Similarly to the literature [24], we extract dropout parts of STCC for the calculation of the probabilities and CR because switches of leaders between lasers are stable only during the dropouts in our experimental configurations, as shown in yellow and purple lines each representing $C_{\rm {2A}}$ and $C_{\rm {2B}}$ in Fig. 10(f). We define that dropouts started when all the STCC values $C_{\rm {1A}}$, $C_{\rm {1B}}$, $C_{\rm {2A}}$, and $C_{\rm {2B}}$ fall below 0.45 and end when one of them rises above 0.9. The area enclosed by a dashed rectangle in Fig. 10(f) gives an example of the part extracted from the STCC waveform for setup [C]. We perform comparisons between $C_{\rm {1A}}$ and $C_{\rm {1B}}$, and between $C_{\rm {2A}}$ and $C_{\rm {2B}}$ every 10 ns, and $T_{\rm {valid}}$ is [A] 1690 ns, [B] 1380 ns, [C] 1250 ns, [D] 1240 ns, [E] 1610 ns, respectively. As described in Fig. 10, the ratio of the leader probabilities of Laser 1A (2B) to Laser 1B (2A) range from approximately [A] 30%-70% to [C] 50%-50% and [E] 70%-30%, which partly shows the experimental controllability of the leader probabilities. An attempt to achieve further asymmetric situations brought about desynchronization in our experiment, explained by the unstable cluster synchronization due to the reduced coupling strength, as discussed in Section 2.2.

Last but not least, the collision rate (CR) is approximately 0.15 and exhibits a minor dependency on the settings of coupling strength in the region. If two players independently and randomly select two slot machines evenly at 50%, CR is about 0.5. If Player 1 selects Slots A and B at a ratio of 30% : 70%, and Player 2 selects A and B at a ratio of 70% : 30%, isolated decision-making results in CR of approximately 0.42. Our result is even much lower than that value; therefore, we consider that we have experimentally demonstrated modifying slot selections of players while maintaining low conflict.

4. Conclusion

We examined crucial aspects of a cooperative decision-making system based on chaotic lasers in a network configuration to solve the competitive multi-armed bandit (CMAB) problem. The discussion, validated with quantitative evaluation of synchronization, revealed that, in total, five networks derived from the bipartite graph exhibit cluster synchronization indispensable for the proposed collective decision-making. Note that the stability analysis provided insights into the minimal coupling strength for cluster synchronization, which was not discussed in the argument based on an unweighted matrix [30]. Our discussion also revealed some implications of essential network structures for cluster synchronization: first, a network, obtained by removing some paths of the multipartite graph, should include a loop with lasers corresponding to the number of options (= slot machines). Second, a network should still be (unilaterally) connected. Among the seven candidates illustrated in Fig. 2, the network (VI) does not meet the latter condition, and (VII) does not meet the former. Their verification will be required in scaled problem settings, e.g., the CMAB with two players and three slot machines.

Furthermore, we extended the decision-making function to include an exploitation mechanism in the CMAB problem by demonstrating, both in simulations and experiments, the controllability of the leader probabilities of lasers, which correspond to slot machine selection proportions in the suggested system. Specifically, in the experiments, we achieved five different leader ratios ranging from 30%-70% to 70%-30% while attaining a collision rate of approximately 0.15, demonstrating coordination of the decision-making compared to asynchronous and independent selection. On the other hand, the decision-making in practical problem configurations, for instance, a scenario where two players narrow down to two high-reward slot machines among three, remains to be implemented. This proof of principle for fulfilling players’ asymmetric preferences, supported by the stability analysis of networks for cooperative decision-making, opens the door to the application of laser networks and other photonic-based systems for machine learning.

Appendix

A. Cluster synchronization in laser network with complicated couplings

Previous research [34] has discussed the existence of cluster synchronization in networks with more complex topologies. Our primary focus remains on the networks (I)-(VII), satisfying the three constraints: identical coupling delay time, uniform coupling strength between clusters, and no coupling within a cluster. However, we recognize the value of discussing cluster synchronization in networks with complicated couplings.

As mentioned in Section 2.1, one necessary condition for cluster synchronization is that the total optical injection into each laser in a cluster is uniform. We accordingly configure laser connections as shown in Fig. 11(a). For instance, the sum of coupling strength from Lasers 1A and 2B into 2A and that into 1B are both 20 ns^-1. The same is true for the connections from lasers in cluster 1 to those in cluster 1, cluster 2 to cluster 1, and cluster 2 to cluster 2. Note that the coupling delay time of all 16 connections is 5 ns. Figure 11(b) illustrates the coupling relationship between clusters 1 and 2 in a simplified manner.

 

Fig. 11. Schematic diagrams of a laser network with complicated couplings. Numbers represent coupling strength [ns^-1] of corresponding connections. (a) Four-laser network configuration exhibiting cluster synchronization. (b) Simplified diagram of (a), Lasers 1A and 2B are reduced to ‘1’ and Lasers 2A and 1B are reduced to ‘2.’

Download Full Size | PDF

We investigate the dynamics of the laser network illustrated in Fig. 11(a) in numerical simulations. Parameter settings besides optical connections are consistent with Table 1. The temporal waveforms of laser intensity and corresponding short-term cross-correlation values are shown in Fig. 12. As shown in Fig. 12(a), the intensity of Lasers 1A and 2B, and that of Lasers 2A and 1B, is respectively perfectly zero-lag synchronized, demonstrating cluster synchronization in a laser network with complicated couplings. STCC values calculated from the intensity are shown in Fig. 12(b), and the resulting leader probabilities are as follows: $L_{\mathrm {1A}}=L_{\mathrm {2B}}=0.80$ and $L_{\mathrm {2A}}=L_{\mathrm {1B}}=0.20$. Interestingly, despite coupling from cluster 2 to cluster 1 stronger than that from cluster 1 to cluster 2, lasers in cluster 1 are more likely to be leaders because the total coupling strength into a laser in cluster 1 is smaller than that into a laser in cluster 2.

 

Fig. 12. Numerically simulated dynamics of the laser network illustrated in Fig. 11(a). (a) Temporal laser intensity waveforms. (b) STCC values.

Download Full Size | PDF

As discussed above and illustrated in Fig. 11(b), the coupling relationship between clusters 1 and 2 is not in equilibrium. Technically, it should be unbalanced for the benefit of our decision-making principle; otherwise, four lasers would be in global synchronization.

In numerical simulations, uniformity in the parameters of lasers and fibers is assumed, and it is not difficult to configure the distribution of coupling strength so that the sum of optical injection into each laser in a cluster is uniform. However, in experiments, the coupling distribution optimal for cluster synchronization may be scaled depending on lasers, and it is impractical to employ such complicated networks for decision-making in the CMAB problem.

B. Near-synchronized states with frequency detuning within a cluster

As discussed in Section 3.1, the perfect cluster synchronization characterized by the cross-correlation values $\hat {C}_{\rm {1A,\ 2B}}(0)$ and $\hat {C}_{\rm {2A,\ 1B}}(0)$ of exactly 1.0 is observed in the numerical simulations, with the assumption of uniform optical frequency of all four lasers. However, detuning within a cluster, for instance, that between Lasers 1A and 2B, potentially leads to the collapse of the perfect cluster synchronization, and such type of detuning is inherently unavoidable in experiments.

We conduct numerical simulations to observe the near-synchronized states with frequency detuning within a cluster in the laser networks. Here, we focus on the network (IV) as an example. The detuning of Laser 1A from Laser 2B, i.e., $\Delta f_1 = \frac {1}{2\pi }(\omega _{\mathrm {1A}} - \omega _\mathrm {{2B}})$, is varied from −1.0 GHz to 1.0 GHz in 0.025 GHz increments, while the detuning of Laser 2A from Laser 1B, i.e., $\Delta f_2 = \frac {1}{2\pi }(\omega _{\mathrm {2A}} - \omega _\mathrm {{1B}})$, is established at 0 GHz. The coupling strength of all four optical fibers is fixed at 30 ns^-1. Other parameter settings are consistent with Table 1. Figure 13 shows changes in the peak values at 0 ns shifts of the cross-correlation functions $\hat {C}_{\rm {1A,\ 2B}}$ and $\hat {C}_{\rm {2A,\ 1B}}$. As predicted, $\hat {C}_{\rm {1A,\ 2B}}(0)$ decreases as the detuning magnitude increases, reaching approximately 0.92 at $\Delta f_1 = {\pm}1.0\;\textrm{GHz}$. We would consider this state still near-synchronized, albeit it is clear that the frequency detuning within the cluster has a deteriorating effect. On the contrary, $\hat {C}_{\rm {2A,\ 1B}}(0)$ is 1.0 in the region from −1.0 GHz to 1.0 GHz of detuning within cluster 1, suggesting that frequency detuning within one cluster has negligible influence on synchronization in the other cluster.

 

Fig. 13. Cross-correlation values $\hat {C}_{\rm {1A,\ 2B}}(0)$ and $\hat {C}_{\rm {2A,\ 1B}}(0)$ changing with the optical frequency detuning within cluster 1, $\Delta f_1 = \frac {1}{2\pi }(\omega _{\mathrm {1A}} - \omega _\mathrm {{2B}})$.

Download Full Size | PDF

C. Impacts of different intrinsic laser parameters on selection collisions

As mentioned above, we assume the uniform constants in the Lang-Kobayashi equations among lasers in the stability analysis and the numerical simulations. This assumption arises from the difficulty in accurately determining intrinsic parameters, such as relaxation oscillation frequency $f_r$, in experimental settings. They are generally different for each laser, and such parameter disparities potentially deteriorate synchronization states.

We perform numerical simulations to investigate their impacts on the collision rate, our evaluation metric for collective decision-making. We focus on the network (IV) as an example. We select normalized injection current $J/J_{th}$ and carrier lifetime $\tau _s$ as controlling parameters. The relaxation oscillation frequency is represented as $f_r = \frac {1}{2\pi }\sqrt {\frac {J/J_{th} - 1}{\tau _s\tau _p}(1 + G_NN_0\tau _p)}$, so it can indirectly be changed through variations in $J/J_{th}$ and $\tau _s$.

For the result in Fig. 14(a), the relative error of $J/J_{th} - 1$ of Laser 1A from that of the other three lasers $(J/J_{th} - 1 = 0.1)$ varies from −5.0% to 5.0%. The coupling strength is fixed at 30 ns^-1. Other parameters are the same as the ones in Table 1. Figure 14(a) shows that the collision rate increases almost linearly with respect to the absolute value of the relative error of $J/J_{th} - 1$, reaching approximately 0.08 at the relative error of $\pm 5.0{\%}$. For the result in Fig. 14(b), the relative error of the carrier lifetime $\tau _s$ of Laser 1A from that of the other three lasers $(\tau _s = 2.04{\times}10^{-9}\;\textrm{s})$ varies from −5.0% to 5.0%. The coupling strength is established at 30 ns^-1. Other parameter settings are consistent with Table 1. Figure 14(b) shows that the collision rate increases as the absolute value of the relative error of $\tau _s$ grows, reaching roughly 0.08 at the relative error of $\pm 5.0{\%}$, similarly to Fig. 14(a).

 

Fig. 14. Collision rate changing with the relative error of the intrinsic parameters of Laser 1A. (a) $J/J_{th} - 1$ of Laser 1A from the prescribed value $(J/J_{th} - 1 = 0.1)$. (b) Carrier lifetime $\tau _s$ of Laser 1A from the prescribed value ($\tau _s = 2.04{\times}10^{-9}\;\textrm{s}$).

Download Full Size | PDF

As demonstrated, discrepancies in laser internal parameters disrupt synchronization to some extent. However, we believe the collective decision-making remains effective with a laser network of four lasers with disparate parameters, supported by the low collision rate in the experimental results in the previous work [24] and in Section 3.2 of this paper.

Funding

Japan Society for the Promotion of Science (JP22K21269, JP23K16961, JP20H00233, JP22H05197); Core Research for Evolutional Science and Technology (JPMJCR17N2).

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 for supporting content.

References

1. K. Kitayama, M. Notomi, M. Naruse, et al., “Novel frontier of photonics for data processing—photonic accelerator,” APL Photonics 4(9), 090901 (2019). [CrossRef]  

2. J. Hardy and J. Shamir, “Optics inspired logic architecture,” Opt. Express 15(1), 150–165 (2007). [CrossRef]  

3. L. Larger, M. C. Soriano, D. Brunner, et al., “Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing,” Opt. Express 20(3), 3241–3249 (2012). [CrossRef]  

4. D. Brunner, M. C. Soriano, C. R. Mirasso, et al., “Parallel photonic information processing at gigabyte per second data rates using transient states,” Nat. Commun. 4(1), 1364 (2013). [CrossRef]  

5. Y. Shen, N. C. Harris, S. Skirlo, et al., “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11(7), 441–446 (2017). [CrossRef]  

6. A. N. Tait, T. F. De Lima, E. Zhou, et al., “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7(1), 7430 (2017). [CrossRef]  

7. J. Bueno, S. Maktoobi, L. Froehly, et al., “Reinforcement learning in a large-scale photonic recurrent neural network,” Optica 5(6), 756–760 (2018). [CrossRef]  

8. T. Inagaki, Y. Haribara, K. Igarashi, et al., “A coherent ising machine for 2000-node optimization problems,” Science 354(6312), 603–606 (2016). [CrossRef]  

9. J.-H. Han, F. Boeuf, J. Fujikata, et al., “Efficient low-loss InGaAsP/Si hybrid MOS optical modulator,” Nat. Photonics 11(8), 486–490 (2017). [CrossRef]  

10. B. Marr, B. Degnan, P. Hasler, et al., “Scaling energy per operation via an asynchronous pipeline,” IEEE Trans. VLSI Syst. 21(1), 147–151 (2013). [CrossRef]  

11. R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction, (MIT University, 1998).

12. D. Isele, R. Rahimi, A. Cosgun, et al., “Navigating occluded intersections with autonomous vehicles using deep reinforcement learning,” in 2018 IEEE international conference on robotics and automation (ICRA), (IEEE, 2018), pp. 2034–2039.

13. D. Agarwal, B.-C. Chen, and P. Elango, “Explore/exploit schemes for web content optimization,” in 2009 Ninth IEEE International Conference on Data Mining, (IEEE, 2009), pp. 1–10.

14. S. Wang, H. Liu, P. H. Gomes, et al., “Deep reinforcement learning for dynamic multichannel access in wireless networks,” IEEE Trans. Cogn. Commun. Netw. 4(2), 257–265 (2018). [CrossRef]  

15. H. Robbins, “Some aspects of the sequential design of experiments,” Bull. Amer. Math. Soc. 58(5), 527–535 (1952). [CrossRef]  

16. L. Lai, H. El Gamal, H. Jiang, et al., “Cognitive medium access: Exploration, exploitation, and competition,” IEEE Trans. on Mobile Comput. 10(2), 239–253 (2011). [CrossRef]  

17. M. Naruse, W. Nomura, M. Aono, et al., “Decision making based on optical excitation transfer via near-field interactions between quantum dots,” J. Appl. Phys. 116(15), 154303 (2014). [CrossRef]  

18. M. Naruse, M. Berthel, A. Drezet, et al., “Single-photon decision maker,” Sci. Rep. 5(1), 13253 (2015). [CrossRef]  

19. M. Naruse, Y. Terashima, A. Uchida, et al., “Ultrafast photonic reinforcement learning based on laser chaos,” Sci. Rep. 7(1), 8772 (2017). [CrossRef]  

20. R. Homma, S. Kochi, T. Niiyama, et al., “On-chip photonic decision maker using spontaneous mode switching in a ring laser,” Sci. Rep. 9(1), 9429 (2019). [CrossRef]  

21. T. Mihana, Y. Mitsui, M. Takabayashi, et al., “Decision making for the multi-armed bandit problem using lag synchronization of chaos in mutually coupled semiconductor lasers,” Opt. Express 27(19), 26989–27008 (2019). [CrossRef]  

22. R. Iwami, T. Mihana, K. Kanno, et al., “Controlling chaotic itinerancy in laser dynamics for reinforcement learning,” Sci. Adv. 8(49), eabn8325 (2022). [CrossRef]  

23. K. Morijiri, K. Takehana, T. Mihana, et al., “Parallel photonic accelerator for decision making using optical spatiotemporal chaos,” Optica 10(3), 339–348 (2023). [CrossRef]  

24. H. Ito, T. Mihana, R. Horisaki, et al., “Conflict-free joint decision by lag and zero-lag synchronization in laser network,” Sci. Rep. 14(1), 4355 (2024). [CrossRef]  

25. T. Heil, I. Fischer, W. Elsässer, et al., “Chaos synchronization and spontaneous symmetry-breaking in symmetrically delay-coupled semiconductor lasers,” Phys. Rev. Lett. 86(5), 795–798 (2001). [CrossRef]  

26. K. Kanno, T. Hida, A. Uchida, et al., “Spontaneous exchange of leader-laggard relationship in mutually coupled synchronized semiconductor lasers,” Phys. Rev. E 95(5), 052212 (2017). [CrossRef]  

27. T. Sano, “Antimode dynamics and chaotic itinerancy in the coherence collapse of semiconductor lasers with optical feedback,” Phys. Rev. A 50(3), 2719–2726 (1994). [CrossRef]  

28. T. Mihana, K. Fujii, K. Kanno, et al., “Laser network decision making by lag synchronization of chaos in a ring configuration,” Opt. Express 28(26), 40112–40130 (2020). [CrossRef]  

29. I. Fischer, R. Vicente, J. M. Buldu, et al., “Zero-lag long-range synchronization via dynamical relaying,” Phys. Rev. Lett. 97(12), 123902 (2006). [CrossRef]  

30. M. Nixon, M. Friedman, E. Ronen, et al., “Synchronized cluster formation in coupled laser networks,” Phys. Rev. Lett. 106(22), 223901 (2011). [CrossRef]  

31. M. Nixon, M. Fridman, E. Ronen, et al., “Controlling synchronization in large laser networks,” Phys. Rev. Lett. 108(21), 214101 (2012). [CrossRef]  

32. J. Ohtsubo, R. Ozawa, and M. Nanbu, “Synchrony of small nonlinear networks in chaotic semiconductor lasers,” Jpn. J. Appl. Phys. 54(7), 072702 (2015). [CrossRef]  

33. T. Dahms, J. Lehnert, and E. Schöll, “Cluster and group synchronization in delay-coupled networks,” Phys. Rev. E 86(1), 016202 (2012). [CrossRef]  

34. L. M. Pecora, F. Sorrentino, A. M. Hagerstrom, et al., “Cluster synchronization and isolated desynchronization in complex networks with symmetries,” Nat. Commun. 5(1), 4079 (2014). [CrossRef]  

35. R. Lang and K. Kobayashi, “External optical feedback effects on semiconductor injection laser properties,” IEEE J. Quantum Electron. 16(3), 347–355 (1980). [CrossRef]  

36. L. M. Pecora and T. L. Carroll, “Synchronization in chaotic systems,” Phys. Rev. Lett. 64(8), 821–824 (1990). [CrossRef]  

37. L. M. Pecora and T. L. Carroll, “Driving systems with chaotic signals,” Phys. Rev. A 44(4), 2374–2383 (1991). [CrossRef]  

38. A. Pikovsky and A. Politi, Lyapunov exponents: a tool to explore complex dynamics (Cambridge University, 2016).