1. Introduction

With the growing demand for computational power resulting from interest in more complex machine learning tasks, specifically designed computing hardware platforms such as graphical processing units (GPUs), tensor processing units (TPUs), and field programmable gate arrays (FPGAs), have been developed. On the other hand, optics, due to its speed and energy efficiency, has attracted attention as a potential platform for the next generation computing hardware [1–3]. For example, free-space diffractive networks [4,5], which utilize complex diffraction patterns of wave propagating through scattering layers, have been proposed for image classification. However, free-space computing typically requires a relatively large volume for light propagation, and the realization of a versatile and ubiquitous reprogrammable platform [6], akin to those in silicon photonics, remains a challenge. Photonic integrated circuits can manipulate light on the surface of a chip like electronic computing hardware [2,7], which becomes an emerging computing platform as an energy-efficient, compact, and high-speed general-purpose chip. Programmable photonic circuits have been demonstrated to perform linear operations for a variety of applications such as an optical neural network for deep learning [1,2,8] and quantum information processing [9,10].

Many programmable photonic circuits are based on Mach-Zehnder interferometers (MZIs). MZIs provide arbitrary control over the power splitting ratio and relative phase shift between input and output ports by varying the phase-shifting control elements via a thermo-optic or electro-optic effect. By cascading multiple directional couplers and phase shifters with specific mesh configurations [11,12], an MZI-based architecture can perform any linear transformation among multiple ports. Together with optic-electro-optic nonlinearity [2], or optical-modulator-based reprogrammable nonlinearity [13,14], the MZI-based architecture has shown its capability in performing complex machine learning tasks with advantages in processing speed [2]. However, in order to achieve large phase tuning ranges, the driving voltage of an MZI is relatively high [15] and the device length is on the order of $100~\mu$m. In large-scale on-chip integrated circuits for complex applications, the device footprint and power consumption then become major considerations. A natural idea is to employ resonant structures which can increase the light-matter interactions and thus reduce the device footprint, driving voltages, and power consumption [15–18]. Microring resonators have been proposed to program real-valued weights with “broadcast-and-weight” protocol [19] as a continuous-time recurrent neural network [20]. By programming weights at the connected waveguide between two microring resonators using phase-change materials, a photonic tensor core has been demonstrated as a dot-product engine [21]. Almost all proposals with microring resonators so far are based on wavelength-division multiplexing input signals and incoherently add up signals at the photodetectors. Coherence networks fully utilize the wave nature of the electromagnetic fields and may provide new opportunities in the design of optical neural networks [22,23].

In this paper, we propose a coherent optical neural network constructed with microring resonators, which shows advantages in device footprint and energy efficiency when compared with existing optical neural networks constructed using MZI-based architectures. The linear matrix multiplication layer is constructed by cascading multiple linear units, each consisting of a serially coupled double ring resonator [18,24–26] for signal mixing of different ports and a single ring resonator for phase tuning [15,26]. The nonlinear unit, which performs element-wise activation at each port, is constructed using microring modulators with electrical signal processing [13,14] and can be programmed to achieve an arbitrary nonlinear activation function. The linear and nonlinear components proposed in this paper maintain the coherent properties of input signals, which represent a complex-valued neural network [23]. Each layer can be cascaded directly on the same chip without intermediate digital-to-analog conversions, which reduces the latency and the energy waste of signal conversions. We describe the input and output relationship of our architecture using a transfer function and directly train the tunable parameters with automatic differentiation [27,28]. Our design and training algorithms are not limited to the ring-based microring resonator design and are applicable to different tunable systems. As a representative example, we demonstrate the operation of the network for information processing tasks such as operating as an Exclusive OR (XOR) gate or performing handwritten digit recognition on the MNIST dataset [29].

This paper is organized as follows. In Section 2, we introduce the setup of the ring-based optical neural network (RONN). In Section 3, we discuss the training process for the RONN using a model that is based on the transfer function of each component. We perform two-dimensional full-wave simulations of each component to fit the transmission amplitude and phase response as a function of tuning parameters such as refractive index. We then train the tunable parameters of the RONN to perform various tasks using automatic differentiation. In Section 4, we analyze and compare the energy consumption and footprint with the MZI setup. We conclude in Section 5.

2. Ring-based coherent optical neural network architecture

A standard feed-forward neural network cascades multiple layers to increase the approximation capability [30], where information moves in one direction from input to output data without cycles. For the $l^{\text {th}}$ layer, the input and output relationship can be represented as [30]:

In Eq. (1), $\textbf {x}_l$ and $\textbf {x}_{l+1}$ are, respectively, the input and output vectors comprising signal amplitudes. $W_l$ is a matrix that performs a linear transformation on the input vector. $f_l$ is an element-wise nonlinear activation function. Different from fully-connected neural networks as discussed in [2], $\textbf {x}_l$, $\textbf {x}_{l+1}$ and $W_l$ are all complex values in Eq. (1) since our proposed RONN implements a complex-valued neural network [23]. In this section, we discuss the configuration of our proposed ring-based programmable coherent optical neural network, as shown in Fig. 1, where Figs. 1(a) and (b) depict the basic components for performing the linear transformation as described by the matrix $W_l$, and Fig. 1(c) is the component that performs nonlinear activation function. All of these components consist of waveguides (referred to as “bus waveguides” below) coupled to ring resonators. Here, we assume that all of the ring resonators have the same diameter, whereas the separation distances between rings and waveguides can vary depending on the functionality. Additionally, we assume all components are working under continuous wave conditions at a single operating frequency, $\omega _0$, such that we can control the phase and amplitude of transmitted signals by tuning the refractive index of each component.

 

Fig. 1. Layout of a single layer, three-port, ring-based coherent optical neural network. Coherent light sources are injected from the left ports and the transmitted signals at the right ports will be detected or become the input of the next layer. (a) Tunable all-pass single ring resonator acting as a phase tuning component. (b) Tunable serially-coupled double ring resonators as a signal mixing component between ports. (c) Nonlinear activation unit to convert input signal $x_n$ to $f(x_n)$, where $f$ is a nonlinear function ($n=3$ in the example). The black ring of the nonlinear activation unit is used as a directional coupler to route the $\alpha$ portion of the optical energy for electrical signal processing. The diode is a photo-detector. The blue ring provides modulation to the signal. M denotes an electronic circuit, which takes the electronic output from the photo-detector to generate a modulation signal for the right ring. (d) Transmission and phase responses of a bus waveguide side-coupled with a ring as functions of phase detuning, $\Delta \phi$, for the critically-coupled and over-coupled cases. The over-coupled case is used for the phase tuning components colored in green. The nonlinear activation ring (colored in blue) requires critical coupling in order to get a larger amplitude tuning range. (e) An example transmission and phase response of the coupled double ring as a signal mixing component. The ring-waveguide coupling coefficient, the ring-ring coupling coefficient, and the single round trip amplitude transmission are $r_{rw} = 0.85$, $r_{rr} = 0.987$, and $a=1$, respectively.

Download Full Size | PDF

2.1 Linear layer

To achieve arbitrary unitary linear transformation [12,18] for the linear layer, we need phase tuning components and signal mixing components, as shown in Fig. 1(a) and (b), respectively.

2.1.1 Phase tuning components

We first consider a ring resonator coupled with a bus waveguide, as shown in Fig. 1(a). Ideal optical components do not have back-reflection [31]. The transfer function of the input and output signals in the bus waveguide after passing through the ring resonator is [26,32]

In Eq. (2), $a$ is the single round trip amplitude transmission in a ring and is determined by the material and bending losses. A coupling coefficient, $r$, may be controlled by the separation distance between the ring and the bus waveguide. Additionally,

In order to behave as a phase tuning component, the resonator-waveguide system should operate as an all-pass filter [26]. Thus, the system should be in the strongly over-coupled regime ($r<a$). As an example of an all-pass filter, the green line in Fig. 1(d) shows the transmission and phase response as a function of phase detuning $\Delta \phi$, as determined by Eq. (2) with $a=1$ and $r=0.9$. We see that the transmission remains unity, while the phase varies continuously and smoothly, as the detuning changes. Thus, by changing $\Delta \phi$, we can achieve a desired phase shift without affecting the amplitude of the signal.

2.1.2 Signal mixing components

The phase tuning components as discussed in the previous section control the relative phase differences among different ports independently. In order to connect and mix signals of different ports, we use a serially coupled double ring resonator [18,24–26], as shown in Fig. 1(b). Such kind of serially coupled ring resonators have been used as high-order filters to reduce the crosstalk between different wavelength channels in wavelength division multiplexing networks [24,33,34]. Here, we use double ring resonators to couple signals in different waveguides that propagate along the same direction and at the same wavelength. This configuration makes it easier to cascade multiple components with existing mesh setups [11,12] for multiple-port linear transformation while keeping the bus waveguide straight without the need for waveguide bend or crossing.

For simplicity, we assume the two rings are identical. They losslessly couple with each other with a coupling coefficient $r_{rr}$. Two waveguides are coupled to the upper and lower rings respectively with the same separation distance and have the coupling coefficient $r_{rw}$. The transfer matrix of the configuration consisting of two waveguides coupling with a double ring is [25,26]

Here, $\Delta \theta$ is the tunable phase parameter in the structure which can be calculated by

In order to obtain efficient signal mixing between ports, we would like the transmission coefficients, i.e. the matrix elements of $t_{double}$, to have large tunable ranges when $\Delta \theta$ varies. For the lossless case, where $a=1$,

So, $|t_{12}|^2$ reaches a maximum of $1$ when

Thus, in order for $|t_{12}|^2$ to reach unity, $r_{rr}$ and $r_{rw}$ should satisfy:

And the minimum of $|t_{12}|^2$ is $\frac {(1-r_{rw}^2)^2(1-r_{rr}^2)}{(r_{rw}^2+2r_{rw}r_{rr}+1)^2}$ , obtained when $\cos (\Delta \theta )=-1$. When $r_{rw}$ and $r_{rr}$ are close to $1$ and satisfy the condition of Eq. (9), we have $\min {|t_{12}|^2}\approx 0$ and $\max {|t_{12}|^2}=1$. The structure can therefore achieve all possible intensity transmission coefficients, which is important to the implementation of an arbitrary unitary matrix using the network structure shown in Fig. 1. The detailed transmission lineshape is not important as long as a wide range of transmission coefficient can be achieved by varying $\Delta \theta$. Here, we choose $r_{rr} = \frac {2}{(r_{rw}+1/r_{rw})}$ in later sections as a demonstration setup. The training results of the optical neural network are determined by many other factors such as the initialization, the gradient updates, and the choice of objective functions.

2.1.3 Linear matrix multiplication

We construct a 2-port linear transformation unit that can perform arbitrary $U(2)$ transformation [11,12,18] by combining a phase tuning component and a signal mixing component. An $N$-port device that can perform any $N\times N$ unitary linear transformation can be constructed by arranging multiple such $2$-port units in various mesh configurations [11,12].

If we apply a 2-port linear transformation unit between ports $n$ and $n+1$, the transfer matrix, when considering all ports, can be expressed as

2.2 Nonlinear layer

For a feed-forward neural network, nonlinear activation is indispensable in order to increase the complexity and approximation power of the neural network [30]. And for different tasks, one may prefer different nonlinear activation functions [36,37]. One way to achieve arbitrary nonlinear transformation is through the optic-electro-optic nonlinearity [2]. After passing through the linear layer, the optical signal at each port is detected and converted to an electric signal. One then performs a nonlinear function on the electric signal, and then regenerates the corresponding optical signal to inject into the next layer. However, the optical signal detection and regeneration process is slow and energy inefficient. Here, we employ the optical-modulator-based design concept proposed in [13]. In this design, a small portion of the optical signal is tapped off using a directional coupler from the bus waveguide and converted to the electric signal. One then performs a nonlinear transformation of the electric signal to generate a voltage that is applied on a modulator in order to influence the transmission of the optical signal in the bus waveguide. In this paper, we use ring resonators as directional couplers and optical modulators, as shown in Fig. 1(c), which will further reduce the device footprint and energy consumption compared to the original design in [13].

Suppose the amplitude of the input signal before entering the nonlinear activation unit is $x_n$ at port $n$. The signal first passes through the first ring resonator acting as a directional coupler to split a portion $\alpha$ of the energy (Fig. 1(c)). The routed-out optical signal is detected by a photo-detector as an electric current signal:

The design with the MZI modulator can be used to achieve nonlinear activation functions but is bulky. Here, we utilize a ring resonator modulator for a more compact design. For the ring resonator modulator, the transfer function $t_{modulator}(\Delta \varphi )$ is $t_{single}(\Delta \varphi )$ as in Eq. (2). In order to have a large tunable range, we would like the ring resonator to be critically coupled to the bus waveguide as shown by the Fig. 1(d) blue lines.

Depending on how the modulation phase response function $\Phi$ is configured, the nonlinear unit can be constructed in two modes: 1) the fixed-circuit mode; and 2) the lookup table mode [13,14]. We compare the nonlinear responses for MZI-based and ring-based designs working under different modes, as shown in Fig. 2.

 

Fig. 2. Nonlinear activation functions for MZI and ring-based structures under fixed-circuit mode and lookup table mode. (a) Transmission spectra for MZI and ring resonator as a function of phase detuning $\Delta \varphi$. The ring is critically coupled $(r=a=0.99)$ to the bus waveguide. (b) Nonlinear responses with fixed-circuit design at different biasing phases. (c) Target modReLU response curve as a function of input intensities. (d) Lookup table of phase requirements to achieve target modReLU activation function for MZI and ring modulators, respectively.

Download Full Size | PDF

For the fixed-circuit mode, the current signal detected from the photo-detector is converted to a voltage signal via a transimpedance amplifier. The voltage signal can be transformed with different circuit designs or can be applied directly to the modulator. For simplicity, we employ the second case and thus the modulation voltage is

In general, the nonlinear functions that can be achieved with the fixed-circuit mode are limited to specific sets based on the internal transfer function properties and voltage converter designs. To achieve arbitrary nonlinear responses, we can use a microcontroller that records the lookup table for arbitrary target functions. As an example, we implement the modReLU activation function [38] with the threshold 0.2, as shown in Fig. 2(c). We can calculate the corresponding phase requirements by solving Eq. (2) given input and output intensities for $\Delta \varphi$, for MZI and ring modulators, respectively, as shown in Fig. 2(d). Due to the resonance behaviour, the ring modulator based nonlinear activation unit requires much smaller phase shifts compared to the MZI modulator based structure, as shown in Fig. 2(a) and (d). The smaller phase shifts will lead to smaller modulation voltages, which will have smaller energy consumption for charging modulators [16,27].

Combining the linear and nonlinear units proposed above forms one layer of the RONN as shown in Fig. 1. The output optical signals after passing through one layer are still coherent, which will be directly used as the input of the next layer. We can implement multiple layers by simply connecting the bus waveguides (that is, connecting all bus waveguides at the right side of the $l$-th layer with the left side of the $l$+1-th layer in Fig. 1, assuming that all signals propagate from left to right side.) for deep neural networks without intermediate optic-electro-optic conversions between layers. This design reduces the potential latency and energy consumption associated with detecting and regenerating when signals propagate inside the chip between layers [2]. The output signals from the final layer are detected with photodetectors at designed ports.

Another source of latency occurs for the digital-to-analog conversion of the input data preparation and the analog-to-digital conversion of the output data detection. This latency is common to existing implementations and proposals of optical neural networks, and has been analyzed in details [39–42]. As a simple estimate, we consider the matrix-vector multiplication process of a single input vector with a dimension $N$. For the input data preparation, the information is transferred from the electronic to the optical domains using optical modulators. For a given input vector, this transfer process can be carried out in parallel for each element of the vector. Hence the latency for the input data preparation is independent of the length of the input vector and instead is entirely controlled by a time scale $\tau _{modulation}$ determined by the modulator. Similarly, for the output data detection, the latency is also independent of the length of the output vector and instead is controlled by a time scale $\tau _{detection}$ determined by the detector. The total latency due to the input/output process is then $\tau _{modulation}+\tau _{detection}$. The length of the optical mesh typically scales linearly as $N$. Thus the total latency is $\tau _{modulation}+\tau _{detection}+CN$, where the last term here is the transit time through the mesh. Since, in electronic processing, the time it takes for performing matrix-vector multiplication scales as $N^2$, the advantages of optical neural networks over the electronic processing appears when the input vector has a dimension $N$ is sufficiently large.

Based on the design described in this section, we can now train the tunable parameters in the RONN for neural network applications.

3. Neural network training based on transfer matrix method

A ring-based optical neural network can be constructed by cascading the linear and nonlinear devices as discussed in the previous section. Since the input-output relations of all these devices can be described by transfer functions, to train the optical neural network, it is sufficient to construct a model of the network in terms of transfer matrices. We then train the network by adjusting the tunable elements in the linear devices in order to achieve a certain performance objective. In such training, one can directly obtain the required physical tuning parameters that are applied on the tunable elements, such as voltages for electro-optic devices [43–45], temperatures for thermo-optic devices [46,47], or displacements for mechanical devices [48].

We summarize the general training process in Fig. 3. The training process involves two stages:

We use the methods described above to fit the transfer function of each component using the intensity and phase responses obtained from full-wave simulations and train the ring-based optical neural network architecture based on the fitted models.

 

Fig. 3. Training process for tunable systems. For each tunable component, a transfer function can be used to describe its optical response as a function of its tunable parameter. Then, algorithms are designed to train the tunable parameters such as temperatures, voltages, and forces for designed programmable network structures using automatic differentiation. Finally, the trained parameters are printed onto each component and measurements are performed with different input data.

Download Full Size | PDF

3.1 Component analysis

As an example, we numerically demonstrate the component analysis process for the signal mixing component (Fig. 1(b)) to get its characterization equation for training. One can use a similar method to get the characterization equations using experimental data instead.

For the signal mixing component, we perform a two-dimensional simulation with transverse electric polarization, as shown in Fig. 4. Two rings are coupled with each other with an edge-to-edge distance $d_{rr}=410$ nm, and they couple to the bus waveguides with the same edge-to-edge distance $d_{rw}=200$ nm. The device has a relative permittivity of $\epsilon _r=12$ and is surrounded by air. The waveguide has a width of $250$ nm and is single-moded at $\omega _0=2\pi \times 197.889$ THz supporting only the fundamental even mode. All ring resonators designed in this section have the same diameter $d_{ring}=6.45~\mu$m measured from the center of the ring to the waveguide center and are on resonance at $\omega _0$ before tuning. Suppose a signal at $\omega _0$ is incident at port 1 of the coupled ring structure, and the transmission and phase responses are measured at port 2, as shown in Fig. 4. The transmission and phase spectra can be tuned by changing the refractive index of the rings. The refractive index change can be directly related to the tunable elements such as the temperatures and voltages in experiments. Meanwhile, we can relate the tunable phase parameter $\Delta \theta$ with the refractive index change $\Delta n$ as

 

Fig. 4. Full wave simulations and spectra fitting results. (a) $E_z$ field distribution inside the signal mixing component. (b) Transmission as a function of refractive index changes. (c) Phase change as a function of refractive index changes.

Download Full Size | PDF

For the single ring phase tuning component, we use a single ring with the same diameter and separation distance to the waveguide as the signal mixing component such that $a=1, r=r_{rw}$ in Eq. (2). Then, in Eq. (2), the only undetermined parameter is $\Delta \phi$, which will be our training parameter in each phase tuning component.

The nonlinear component consists of an add-drop filter [26] acting as a directional coupler and a critically-coupled ring resonator as a modulator. The phase introduced by the directional coupler in the bus waveguide can be adjusted by adding delay lines. The splitting ratio and the phase of a directional coupler are fixed at a given frequency once designed. For simplicity, we assume that a lossless add-drop filter splits an $\alpha =0.1$ portion of the power in the bus waveguide to a detector, and the amplitude transmission coefficient that determines the signal remaining in the bus waveguide is then $\sqrt {1-\alpha }$. For the modulator part, we can add loss in the ring resonator such that $a=r=0.9$ in Eq. (2) to get a critically-coupled ring for the maximal tunability.

3.2 Neural network design

With the component analysis, we obtain all structure-dependent variables in the characterization equations which describe the properties of each component as functions of the training parameters, $\Delta \theta$ and $\Delta \phi$. We can then cascade the components to set up the network architecture with Clement’s mesh [12] for training. We note that Clement’s mesh has the capability to generate arbitrary unitary operators, but this capability is in fact not necessary for training of a coherent optical neural network. Removing the phase tuning components (i.e., green rings in Fig. 1) can still provide reasonable training results, as shown in later sections.

With the characterization equations and the mesh setup, we program Eq. (11) and Eq. (14) with JAX [28], which is a widely-used package that provides automatic differentiation to functions for high-performance machine learning research. Based on the input data and our target function (i.e. loss function), we can get the gradients with respect to each training parameter and gradient optimization methods can be chosen to search for the desired training parameters. Here, we employ the Adam gradient descent method [50].

3.2.1 XOR gate

Our first example is a 3-bit Exclusive OR (XOR) gate. A single layer RONN structure for the 3-bit XOR gate is shown in Fig. 1. The XOR gate consists of two such layers. We encode the 3-bit binary array with no phase difference among three ports as input signals. The input signal is normalized to a total power of 1. We detect the field intensity at the lowest port after the signal passes through the designed 2-layer RONN as the output signal for the 3-bit XOR gate. The target intensity of our XOR gate is defined to be 0.2 for the consideration of loss from the nonlinear layer after passing through the RONN. We define the loss function to be the mean square error between the target values and the detected intensities. The 2-layer RONN is trained by computing in each epoch the loss function that takes into account all possible combinations of the 3-bit binary arrays and by adjusting the training parameters of the network using a gradient descent method. The loss as a function of epoch is shown in Fig. 5(a). After training, we can obtain the tunable parameters $\Delta \theta$ and $\Delta \phi$ required to be applied to each component.

 

Fig. 5. 3-bit XOR gate training results. (a) The training loss for gate with and without nonlinearity. (b) Performance of the trained 2-layer RONN as a 3-bit XOR gate with and without nonlinearity.

Download Full Size | PDF

Figure 5(b) shows the performance of the neural network before training and the performance after training of the neural network with or without nonlinear components. We observe that a simple linear mesh is not enough for a 3-bit XOR gate and nonlinear components are indispensable.

3.2.2 MNIST dataset

To show the scalability of the RONN, we train a larger 16-port 2-layer RONN with the MNIST dataset. We follow the pre-processing steps as introduced in [13,51]. The original $28\times 28$ image is transformed into Fourier space and a mask of $4\times 4$ is applied to keep only the low frequency Fourier components. We then flatten the $4\times 4$ components to a vector with length $16$ and take the complex vector components as the amplitude of our input signals. The MNIST dataset has 10 classes, so we detect the intensities at 10 output ports and select the one that has the largest intensity as the predicted class. For the training, we use the categorical cross-entropy loss [30] and Adam gradient descent method [50] to determine the optimal values of the tunable parameters.

We train the different RONN structures including: 1) a complete RONN (i.e., with all rings, as shown in Fig. 1, referred to as the “Complete” type in Table 1), 2) an RONN without diagonal tuning components in each layer (i.e., without $\mathcal {D}_l$ rings in Fig. 1, referred to as the “Double-single” type in Table 1), and 3) an RONN without any phase tuning components (i.e., without any green rings in Fig. 1, referred to as the “Double-only” type in Table 1). Also, our nonlinear components have several hyper-parameters (e.g., splitting ratio $\alpha$, biasing phase $\varphi _b$, and the modulation function $\Phi$) which may be selected based on experience or treated as training parameters. For different network architectures, we perform experiments on linear, fixed activation, trained activation, and modReLU cases, respectively. The fixed activation case is specified with a fixed splitting ratio $\alpha =0.1$, and biasing phase $\phi _b=0$, as described in Sec. 2.2. For the trained activation case, we take the splitting ratio $\alpha$ and biasing phase $\phi _b$ as training parameters. We take a modReLU case as a demonstration of the look-up table method, as introduced in Sec. 2.2. Each setup is repeated ten times, and we calculate the mean and standard derivations of the classification accuracy with different random initialization settings.

Table 1. Classification accuracy of test datasets for different optical neural network architectures and different activation functions in terms of percentage.

View Table

The training results with different network architectures and different nonlinearity settings are summarized in Table 1. For different architectures, trained activation methods outperform other cases for the Fourier-truncated MNIST dataset. With experiments on different nonlinearity cases, different architectures have similar performance in terms of the classification accuracy. We observe that removing phase tuning components does not significantly degrade the performance for a practical optical neural network. This observation can help reduce the number of components in an optical neural network. As a summary, not all ring resonators are necessary for optical neural networks in small classification tasks and training the parameters of the nonlinearity components can improve the performance of an optical neural network. We explore different nonlinear functions here to demonstrate the reconfigurability of our nonlinear components since the choice of nonlinear functions can be different for different applications. One may observe that the test accuracy for nonlinear activation functions in Table 1 have slightly different performance but with limited improvements comparing with the RONN with only linear elements. This is because we use the Fourier-truncated MNIST dataset with only 16 elements to represent an image. The linear network already shows a high test accuracy and the room for further improvements is limited by the dataset. To show the scalability our RONN, we keep a larger number of Fourier components with a 2-layer 36-port RONN. The test accuracy of the trained RONN increases to 91.83% with the linear RONN, and 96.42% with the fixed nonlinearity configuration, respectively.

The RONN architectures discussed above use lossless structures. Optical loss is usually non-negligible in experiments [52] and a lot of efforts have been made to build low-loss integrated photonic devices [15,53,54]. In the presence of loss, the linear transformation is not unitary anymore [12]. However, for the photonic neural network applications, it is not necessary to use just unitary linear transformation, especially when the loss can be included in the characterization equation since we are training the tunable parameters directly. To show the performance of the RONN with lossy ring resonators, we change $a=0.98, r_{rw}=0.55$ and $r_{rr}=0.84$ in the characterization equation where all others remain the same as the “Complete” type, which is referred to as “Lossy RONN” in Table 1. We observe that, the prediction accuracy after training is comparable to the lossless case, which indicates that our proposed design process and the training algorithms are general for different cases, with or without loss.

Since the design parameters like $r_{rr}$ and $r_{rw}$ can be measured after the fabrication, our proposed design and training procedures can tolerate variations in the fabrication process. Another concern for the resonant-based architecture is its sensitivity towards resonant wavelength shifts in experiments due to temperature variations. Solutions including athermal designs and control-based feedback designs have been proposed to improve thermal stabilities [55].

4. Discussions

In this section, we discuss several aspects of our design. We first compare the footprint and energy consumption of ring-based and MZI-based architectures. One of the biggest advantages of using the resonator-based architecture is the reduction of the device footprint. Comparing to an MZI-based architecture, where each MZI is on the order of hundreds micrometers [2,15,18] in size, a ring resonator has a typical diameter of a few tens of micrometers, which results in a nearly 10-fold reduction in the overall device footprint area.

In terms of energy consumption, we mainly consider the charge and discharge of the modulation part in the nonlinear component since it is the fundamental active power consumption and is independent of imperfections under different experimental conditions like fabrication errors. For the linear part, once the network is trained, we assume that the phase shifter is fixed and there is no extra energy consumption. We note that it may require non-negligible energy to stabilize the microring resonators based on the materials and tuning mechanisms [26,56]. We ignore the possible energy cost for stabilization since it highly depends on the experimental conditions and can be improved with different structure designs, material systems, and fabrication techniques [15,55–57]. For the nonlinear activation component, signals will be routed out and detected at each port, and the modulation strength will change accordingly based on the detected intensity. So, there will be charge and discharge energy consumption inside the modulation section. For the energy consumption per bit, we assume both MZI and microring resonators are designed with the same materials. The energy consumption per bit is then determined by capacitance and the area of the modulation region [16]. The microring resonator has a much smaller footprint which results in a lower charge/discharge energy consumption per bit. On the other hand, the total power consumption is related to the operation speed and the operation speed will be limited by the quality factor of the modulated ring resonators.

We assume the optical neural network operates at the optical communication wavelength of $1.55~\mu$m, and the modulated ring resonator operates at its highest operation speed which is inversely proportional to the quality factor of the microring resonator, as shown in Fig. 6(a). The higher the operation speed is, the lower the quality factor needs to be designed for the ring resonator modulator. To compare with the energy consumption of the MZIs and ring-based modulators, we assume the effective waveguide modal index changes for both cases is the same by applying the same voltage [15]. We calculate the dynamic energy consumption [16] per bit and the power consumption for MZIs with various lengths ranging from $0.5$ mm to $2$ mm and microring resonators with different diameters as shown in Figs. 6(b) and (c). We observe that the energy consumption of the nonlinear components increases as a function of operation speed, and the ring architectures can reduce the dynamic energy consumption at least by a factor of 10.

 

Fig. 6. Energy consumption comparisons for ring-based and MZI-based nonlinear components. (a) Quality factor as a function of operation speed for the modulated ring resonators. (b) Energy consumption per bit of the ring resonator and MZIs. The shaded region represents the energy consumption per bit for an MZI with various lengths ranging from $0.5$ mm to $2$ mm. The solid lines are for ring resonators with different diameters. (c) Power consumption of the ring resonator and MZIs as a function of operation speed.

Download Full Size | PDF

In our design we assume weak back-reflection from all the components in the network. In principle, this can be achieved with suitable design of each component. Other approaches to design for structures that suppress back-reflection, such as the recently proposed concept of a reflectionless programmable signal routers [58], may also be of interest for this purpose.

5. Conclusion

In this paper, we introduce an on-chip ring-based optical neural network (RONN) architecture. We study the properties and transfer functions of different components including the phase tuning components and signal mixing components for matrix multiplications and modulated ring resonators for nonlinear activation. With the parameterized transfer functions, we train the RONN for different tasks such as a XOR gate and MNIST handwritten classification. This method can be extended to different physical computing platforms with arbitrary tunable transfer functions such as electric, thermal, or mechanical control systems.

While we are preparing this paper, we notice a related and independent preprint [59] that experimentally demonstrates a coherent programmable photonic circuits using MZIs for linear matrix multiplication and ring modulators for nonlinear activation. Our design considers all-ring based architecture which provides a more compact platform for future programmable photonic circuits.