1. Introduction

Advanced information systems depend on the extensive deployment of radio-frequency (RF) signals. The raised bandwidth and carrier frequency of RF signal can support high-resolution detection and high-capacity communication [1]. However, the RF signal processing is also under increased pressure. First, the high-speed and high-accuracy analog-to-digital conversion (ADC) generates large amounts of data from the raw waveform [2]. The data interleaving and storage require additional time cost. Second, the digital algorithms consume much time for complex processing tasks, such as classification and recognition [3]. For example, the computation-intensive artificial intelligence (AI) algorithms rely on massive multiply-and-accumulate (MAC) operations. The processing latency is limited by repeated data movement between the memory and processing engine [4]. Therefore, the inefficient data manipulation inevitably worsens the latency performance of main-stream digital solutions to RF signal processing, and hinders the potential applications like real-time automatic target recognition [5].

Emerging processing paradigms are expected to shed light on time-efficient RF signal processing. Recently, the photonic implementation of a simplified recurrent neural network (PRNN) is proposed to reduce the latency of RF waveform classification on the strength of parallel analog computing [6]. However, the required data pre-processing for precise trimming and synchronization of each transmission induces unwanted processing latency. Actually, compared to the massive raw data, the key information is at several-bit level (i.e., classification labels). The spiking provides a promising way to acquire sparse and task-relevant representations of the analog signal, instead of digitization [7]. This unique paradigm excels at analog signal processing by parallel and binary spike train with low latency, as studied in speech recognition [8]. In neuromorphic electronics, making use of high frequency oscillations (HFO) feature in EEG signals, the spike-based real-time detection of epilepsy has been demonstrated [9]. On the other hand, the photonic spiking neuron is characterized by leaky integration, threshold behavior, and spike emission at µs/ns-level [10]. The proposed implementations mainly include light-source-based modules [11–16] and micro-ring resonators [17–20]. Recent advances in photonic spiking processing concern image edge detection [21], image RGB encoding [22], and handwritten digits recognition [23], where the data of image pixel is encoded and loaded onto the photonic spiking neuron to perform computation tasks.

Here, we propose a photonic-spiking-based scheme for RF signal processing in fully-analog domain. The comparison of the conventional digital processing and the proposed scheme is conceptually illustrated in Fig. 1. The photonic spiking neuron integrates the input RF signal in a given time window and decides whether to generate a spike or not according to the threshold level. Then, the asynchronously-encoded photonic spike trains are processed through the photonic spiking neural network (PSNN). We investigate the effects of threshold level and time constant on the encoded spike number. Based on appropriate parameter configuration, distinct encoded spike numbers are experimentally observed for two classes of waveforms emitted by real RF devices. Furthermore, we construct a PSNN with 16 encoding neurons and 24 output neurons to carry out the waveform classification task. It is shown that a layer of encoding neurons with Gaussian parameter distribution can provide useful high-dimensional representations enabling a classification accuracy of 92%. Its processing latency is only limited by light transmission delay that is as low as ∼80 ns in the experiments. For comparison, we also implement a K-nearest neighbor (KNN) algorithm on a CPU platform to classify the time-frequency features extracted from the waveforms, achieving an accuracy of 94% and a running time of 0.7 s. The processing latency of photonic-spiking-based scheme is expected to be still lower than that of the KNN accelerator with elaborated field-programmable gate array (FPGA) by two orders of magnitude [24] (see Supplement 1). The photonic spiking response can directly map the waveform feature to classification results in a time-efficient manner, which highlights the merged benefits of low-latency photonics and asynchronous neuromorphic processing paradigms.

 

Fig. 1. Conceptual illustrations of conventional radio-frequency (RF) processing scheme and the proposed photonic spiking RF processing scheme. (a) High-speed and high-accuracy analog-to-digital conversion (ADC) provides reliable digital copy of the input RF waveform for flexible storage and digital processing. The processing latency is introduced by data operations in ADC and repeatedly data movement between processing engine and memory. (b) Subject to the leaky-integrate-and-fire (LIF) model, the photonic spiking response asynchronously converts the input RF waveform into sparse spike trains. The encoded spike train can be processed by a trained photonic spiking neural network (PSNN) for a given task. Overall, the processing latency consists of the spike-triggering latency (∼ns) and light transmission delay (∼tens of ns).

Download Full Size | PDF

2. Photonic analog-to-spike encoding

2.1 Threshold level

The threshold level determines the sensitivity (i.e., the possibility to emit a spike) of a photonic spiking neuron to a given input intensity. Its excitable dynamics is theoretically investigated through rate equations of optical injection locking (OIL) process [25] (see details in Supplement 1). With a negative frequency detuning of −11 GHz, the slave laser is located at the boundary of locking range at a low injection ratio of 0.02. A phase perturbation on the injected light is applied by a Gaussian pulse with 0.25 ns full-width-at-half-maxima (FWHM). In the OIL state, the master and slave laser oscillate at the same frequency with a relative phase difference determined by their original frequency detuning [26]. The dynamics is described by a saddle-node-on-limit-cycle model [27]. A strong phase perturbation above a certain threshold can push the OIL system to go over the unstable saddle node, then experience a limit cycle and end at the attractor. On the other hand, the OIL system with weak phase perturbation can not go over the saddle node and quickly go back to the attractor. At an above-threshold input amplitude of 0.27, the simulated phase portrait of output electrical field is manifested in Fig. 2(a). A large trajectory is observed in the complex plane, where nodes A and B respectively represent the start point and end point. The detailed phase portraits with varying input amplitudes are highlighted in Fig. 2(b). As the amplitude of the input pulse increases from 0.07 to 0.37, distinct photonic spiking responses are observed. First, the photonic spiking neuron maintains the locked state and hardly responds to the weakest perturbation. Second, a more intensive perturbation leads to a distinguishable response, and the photonic spiking neuron quickly returns to the locked state. Third, the photonic spiking neuron is pushed away from the locked state and enter the large trajectory at an input amplitude of 0.27, as shown by the pink solid-line alongside the arrow. Fourth, the analogous dynamics is also observed with the highest input amplitude. Actually, the large trajectory corresponds to a photonic spike triggering with the relative phase rotation for multiple times of 2π in the output electrical field. In conclusion, the switch of the photonic spiking response reflects the binary nature of excitable dynamics. The photonic spiking response is only triggered by above-threshold input amplitude as 0.27 and 0.37, while the below-threshold input amplitude leads to a much weaker one. Similar dynamics are observed by further simulations with a fixed input amplitude, in which the threshold level is lowered by increasing the frequency detuning. Consequently, increasing input amplitude or decreasing the threshold value can both facilitate the photonic neuron to spike.

 

Fig. 2. Impact of the threshold level in analog-to-spike encoding with photonic spiking neurons. (a) Theoretical phase portraits for an above-threshold input amplitude of 0.27. From the start point (node A), a large trajectory is elicited and ends at the attractor (node B). (b) Comparison of detailed phase portraits for below-threshold and above-threshold inputs. No evident response is observed at an input amplitude of 0.07 or 0.17. The large trajectories for the input amplitude of 0.27 and 0.37 correspond to photonic spikes. (c-f) Experimental results of photonic spiking responses to 2-GHz input pulse train. As the relative frequency detuning f_detu decreases from −1 to −3, the output spike number (i.e., the number of above-threshold input pulses) in one period increases from one to three, as correspondingly shown in (c,d). The double-spike response is observed for once (e) and three times (f) at a f_detu of −5 and −6, respectively. The output spike number acts as an indicator of the input intensity.

Download Full Size | PDF

To experimentally demonstrate the threshold behavior and multi-pulse excitability, a phase-perturbed OIL experimental setup is implemented (see details in Supplement 1). The master laser is located at the red-side locking boundary of the slave laser. We apply a 2-GHz input Gaussian pulse train with continuously-varying amplitude and a fixed FWHM of 0.125 ns. The lasers are equipped with commercial temperature control modules with an accuracy of 0.01 °C. As the master laser temperature is set to 21.5 °C, 21.7 °C, 21.9 °C, and 22 °C, responses of the photonic spiking neuron are depicted in Figs. 2(c-f), respectively. The frequency detuning is accordingly increased (absolute value) while the threshold level decreases. The relative frequency detuning f_detu is labeled for each case, as an indicator of the threshold level. The f_detu is calculated with a reference value corresponding to 21.4 °C. In Fig. 2(c), one period of the input pulse train consists of eight pulses. The triggering of one output spike implies that the highest input pulse is above the threshold level, as indicated by the red arrow. In Fig. 2(d), the output spike number in each input period increases to three. The above-threshold input pulses are around the center position within one period. Note that the consistency of the spike shape and amplitude is the signature of optical excitable dynamics observed in simulations. As f_detu further decreases to −5, a double-spike response is elicited as highlighted by the purple dot box in Fig. 2(e). It corresponds to the repeated transition dynamics along the large trajectory induced by the highest input pulse. In total, four input pulses individually trigger four spikes, one for double spikes, and the others for no spikes. At last, in Fig. 2(f), the double-spike response and single-spike response are respectively triggered three times, while the other input pulses remain at below-threshold levels. Consequently, the photonic spiking response can encode the intensity information of the input signal by the output spike number.

2.2 Time constant

The time constant determines the integration efficiency (i.e., the peak level of the carrier density change) of a photonic spiking neuron for input signal within a certain timescale. In a photonic spiking neuron, the carrier density can serve as the state variable, like the membrane potential in a biological spiking neuron. Actually, the leaky-integration process of the input signal dynamically updates the carrier density. The photonic spiking neuron senses each part of the input signal on the scale of time constant. Once the carrier density reaches a threshold level, photonic spikes will be triggered, and the carrier density returns to its equilibrium level. Basically, the photonic spiking neuron is more sensitive to the detailed waveform variation on the temporal scale that is comparable to its time constant. The waveform variations on other timescales contribute less to a spike emission.

To study the impact of the time constant in photonic analog-to-spike encoding, we select two classes of RF waveforms emitted by two brands and models of smartphones (labeled as ip6 and xm), as described by a RF waveform dataset [28]. The distinct waveform features are proved to support the smartphone model recognition, including the transient and steady-state feature. Each waveform set contains 150 samples (see details in Supplement 1). In the OIL-based spiking neuron setup, the master and slave bias currents are set as 25 mA and 32 mA, respectively. Note that the laser temperatures are both kept at 25 °C as default. The wavelength detuning is approximately 0.1 nm for a red-side locking state, which is observed by the optical spectrum analyzer (OSA) with a resolution of 0.02 nm. The injection-locked optical spectrum peaks at 1548.90 nm. The time constant of an OIL-based spiking neuron is at several-ns level, which is correlated with the carrier lifetime [29]. Here, we adjust the generating rate of the waveform with an arbitrary-waveform-generator (AWG) to tune the input signal duration (i.e., temporal scaling). As a result, we can investigate the photonic spiking responses when different ratios of time constant and input signal duration are applied, as shown in Fig. 3. We select a random sample (here, the sixth) in each waveform set as the input to AWG working in the repeat generation mode. In the pre-processing stage, the waveforms are DC-filtered and amplitude-normalized to remove trivial characteristics, which ensures that the photonic spiking neuron can encode the intrinsic features of each waveform class. The first and second rows of Fig. 3 correspond to the cases of class 1 (i.e., ip6) and class 2 (i.e., xm) waveforms, respectively. In Fig. 3(a), at a generating rate of 30 GSa/s, the input signal duration is ∼80 ns. No spike is triggered in the photonic spiking neuron output. The output fluctuations reflect the carrier density change in the laser cavity. When the input signal duration reduces to ∼50 ns with a generating rate of 50 GSa/s, a series of photonic spikes are elicited in each input period, as shown in Fig. 3(b). Furthermore, we scan the generating rate from 26 GSa/s to 52 GSa/s and record the output spike numbers corresponding to different input signal durations. In Fig. 3(c), no photonic spike is triggered when the input signal duration is longer than ∼60 ns (generating rate lower than 40 GSa/s), thus the analog-to-spike encoding fails. It reflects that the peak value of leaky-integration result is lower than the threshold level, in response to a long-duration input signal. The results of class 2 waveform are shown in Figs. 3(d-f), where the generating rates are 30 GSa/s and 44 GSa/s for the unsuccessful encoding case in Fig. 3(d) and the successful encoding case in Fig. 3(e), respectively. It is observed that the output fluctuations in Fig. 3(d) are more intensive than those in Fig. 3(a), indicating different integration efficiencies for the two classes of RF waveforms. In Fig. 3(f), the output spike number decreases to zero at an input signal duration of ∼80 ns, which means that the class 2 waveform can be encoded in a lower generating rate than that of class 1 waveform.

 

Fig. 3. Experimental results of photonic analog-to-spike encoding dependent on input signal durations. The generating rates of class 1 waveform (a-c) and class 2 waveform (d-f) are adjusted to vary the input signal duration. The ratio of time constant of photonic spiking neurons and input signal duration impacts the encoded results. The cases at a generating rate of (a) 30 GSa/s (no-spiking response) and (b) 50 GSa/s (spiking response) of class 1 waveform. (c) There is no spike in the encoded results once the input signal duration is above ∼60 ns. The cases at a generating rate of (d) 30 GSa/s (no-spiking response) and (e) 44 GSa/s (spiking response) of class 2 waveform. (f) The output spike number decreases to zero at an input signal duration of ∼80 ns.

Download Full Size | PDF

The experimental results emphasize the critical role of time constant in analog-to-spike encoding with photonic neurons. As the generating rate increases, the waveform is scaled into short duration while keeping up its temporal structures. Therefore, the waveform variation is on a faster timescale, which means the input activity per unit time is stronger and easier to induce a spike. The appropriate selection of time constant can filter out unwanted waveform details. Moreover, it is noted that the noise of AWG induces the input waveform variations and leads to the variation of the spike timings and numbers in each input period. In practice, the noise robustness depends on the decision boundary in a specific task. Here, we evaluate the encoding results based on whether there are spikes or not, corresponding to different waveform classes.

3. Responsivity difference with photonic spikes

It has been shown that the photonic analog-to-spike encoding result of a RF signal can be determined under a certain threshold level and time constant. In fact, both of the spike number and spike timing carry the information. In this section, we focus on the distribution difference of encoded photonic spike number for the two classes of RF waveform sets, instead of a single waveform. The last 50 samples in each waveform set of 150 samples are picked up to test the photonic spiking neuron. The aim is to improve the discrimination of two classes of waveforms according to the responses of one photonic spiking neuron. Hence, appropriate parameters of photonic spiking neuron should be configured. Note that the waveform generating rate is tuned to study the encoding results with varying ratios of time constant (∼ns level) and signal duration (40∼90 ns) in the experimental condition. Based on the results in Fig. 3(c) and Fig. 3(f), a generating rate of 40 GSa/s is applied, at which the class 1 waveform triggers no spike and the class 2 waveform triggers tens of spikes.

The responsivity difference of the two-class waveforms with a photonic spiking neuron is experimentally demonstrated, as shown in Fig. 4. The injection locking condition and pre-processing method is the same as described in the previous section. Specifically, all 100 input waveforms are divided into 10 batches for a moderate acquisition time window on the oscilloscope (OSC). The first five input batches contain all class 2 waveforms, and the others for class 1 waveforms. In Fig. 4(a), each waveform of the fourth input batch leads to a densely-distributed photonic spiking response. The responses to the sixth input batch are shown in Fig. 4(b), where a variety of class 1 waveforms fail to trigger any spike with only one exception. In the red-dashed box, it can be clearly observed that the photonic spiking response and basic response of carrier fluctuation overlap. Note that the amplitude of photonic spiking response is about three times higher than the basic response. Besides, the different amplitudes of the basic responses again highlight varying responsivities to waveforms. When the leaky-integrated carrier density reaches the threshold level, the photonic spike will be triggered accompanied by a relatively stronger basic response, as shown in the red-dashed box. In Fig. 4(c), the bar chart describes the distribution of photonic spiking responses over the two-class waveforms. In experiments, five class 2 waveforms and three class 1 waveforms are invalid to be generated by the AWG. Among the actual 45 input waveforms of class 2, 39 waveforms trigger at least one spike. On the other hand, 42 waveforms of class 1 fail to trigger any spike. Consequently, the photonic spike number offers a useful indicator for distinct classes of RF waveforms.

 

Fig. 4. Experimental responses to the two classes of RF waveforms with distinct photonic spike numbers. (a) In the fourth input batch of 10 class 2 waveforms, a series of photonic spikes are elicited by each waveform. (b) In the sixth input batch of 10 class 1 waveforms, nearly all waveforms fail to trigger spikes in addition to one with a photonic spiking response. The red-dashed box highlights that the photonic spiking response and basic response overlaps. (c) The bar chart of the responsivities with the photonic spike number. The responsivity to class 2 waveform is much higher than that of class 1 waveform.

Download Full Size | PDF

Furthermore, the ability of a photonic spiking neuron to distinguish RF signal benefits from its unique response mechanisms. First, the leaky-integration can not only tolerate the uncertainty in the analog link, but also build up a decision boundary for robust responses to the waveform variation in the same class. It contributes to the special hybrid encoding paradigm with binary spike output and continuous integration of the waveform details in an appropriate timescale. In contrast, the digital processing scheme pays a heavy price to apply high sampling rate matched to the waveform generation rate. Second, the asynchronous spiking response does not require temporal positioning operation, considering the variation of the waveform starting points (see Supplement 1). Third, the spike latency is at ∼ns level, hardly increasing the whole processing latency compared to the light transmission delay in the fiber-based experimental setup.

4. RF waveform classification by PSNN

In the previous section, it has been demonstrated that one photonic spiking neuron can provide distinct representations corresponding to different classes of input waveforms. To further study the processing capability, a classification task of waveforms is implemented by constructing a software-based PSNN, which can process parallel encoded spike trains. Based on PyTorch, we apply a fully-connected PSNN in the framework of BindsNET [30]. The architecture of the PSNN includes an input layer with a single input node, an encoding layer with 16 spiking neurons, and a classification layer with 24 spiking neurons, as shown in Fig. 5(a). First, the two-class waveforms are input to the single node and fanned out into every encoding neuron. Second, the time constant of the encoding layer is subject to a Gaussian distribution with a mean value of 90 and a variance of 10. Note that the time unit here corresponds to one data point length. Altogether, the input waveforms have 2501 data points. Considering a signal duration of 60 ns, the mean value of time constants in encoding layer is ∼2.1 ns. It agrees well with the experimental condition of OIL-based photonic spiking neuron. Third, the Gaussian distribution of time constants in classification layer has a high mean value of 500 (i.e., ∼12 ns) and a low variance of 1. The classification layer is divided into two groups, each with 12 spiking neurons, corresponding to the labels of the input waveforms. Additionally, the training set and test set contain 200 and 100 waveforms, respectively. The PSNN is trained in a supervised manner with an epoch of 10. For every training waveform, one random neuron in the label-corresponded group in classification layer is clamped to spiking [30]. On the whole, more encoding neurons provide distinct spiking responses for each waveform based on their time constants. The PSNN can make full use of abundant encoding spike trains and strengthen the information flow according to the given classification label. Consequently, the test waveform can be classified based on the spiking response in the PSNN output.

 

Fig. 5. The waveform classification task carried out by a photonic spiking neural network (PSNN). (a) The architecture of the PSNN, including one input neuron, 16 encoding neurons, and 24 output neurons. (b) The accuracy curve in the training phase with an input batch size of 10 waveforms. (c) The confusion matrix of the classification results for the test waveform set. The accuracy is 92% over 100 test waveforms. (d) An example of the spiking raster of encoding neurons. The variation of the spike timing is induced by different dynamics of membrane potential, which are shown in (e).

Download Full Size | PDF

The results of waveform classification task are shown in Figs. 5(b-e). In the training phase, the convergence of the accuracy curve is observed in Fig. 5(b). An overall accuracy of 92% is achieved on the training set. Hence, the PSNN links the input waveform to its label by recombining the spatio-temporal spiking response between the layers. The confusion matrix of the test results is indicated by Fig. 5(c). A random order of input sample is adopted for both the training and the test phases. The PSNN achieves an accuracy of 92% on the two-class waveform classification task. Predictions of the RF source labels can be consequently made based on the spiking responses of the PSNN. In Fig. 5(d), an example of the spiking raster of encoding neurons is illustrated. The time step corresponds to the data point length. Three neurons make spiking responses at a time step between 750 and 1000. Then nearly all neurons emit a spike at a time step of 1750. It is demonstrated that the diversified time constants lead to distinct spiking responses of every encoding neuron. The spike timing variation is caused by the dynamics difference of the neurons. Their membrane potential with input-waveform-driving dynamics is shown in Fig. 5(e). After a spike emission, the membrane potential is reset to its equilibrium level, which ensures the independent response to the subsequent part of input waveform.

The PSNN operates in analog domain with photonic spiking neurons and tunable weights. Hence, the processing latency consists of the light transmission delay between the adjacent layers and inherent spiking latency in a single neuron level, among which the major portion is the light transmission delay of ∼80 ns in the fiber-based experimental condition. For the digital processing scheme, the adopted KNN algorithm achieves an accuracy of 94% and 0.7 s processing latency in the same task (see Supplement 1). It is executed in MATLAB environment on a CPU platform (Intel Core i7-7700 CPU @ 3.60 GHz, 8 cores). The potential on-chip implementation can further reduce the interconnection length and enable more time-efficient PSNN-based RF signal processing.

5. Conclusion

We have demonstrated a photonic-spiking-based scheme to enable the time-efficient RF signal processing without the ADC and burdened digital processing. At first, the effects of threshold level and time constant on analog-to-spike encoding with a photonic neuron are studied. Then, it is proved that the configured photonic neuron makes distinct spiking responses dependent on the input waveform classes. Lastly, the diversity of time-constant-varying encoding neurons is found to contribute to high-accuracy waveform classification capability by a PSNN. Most importantly, the processing latency of the photonic-spiking-based scheme is mainly introduced by the light transmission delay at tens-of-ns level.

Three features are highlighted for the photonic-spiking-based RF signal processing. First, unlike the neuromorphic photonic computing for image processing, where the time constant is expected to be as low as possible, here we demonstrate the critical role of diversified photonic spiking neurons with varying time constants. How to effectively configure the parameter distribution for different RF processing task is worth exploring. Second, as in the waveform classification task, the timescale of task-relevant features (i.e., time constant of the photonic spiking neuron) is often longer than the original timescale of waveforms (i.e., the inverse of generation rate 40 GSa/s). This contrast highlights the motivation of the photonic-spiking-based scheme. Unlike the inefficient data-by-data manner of digital-processing-based scheme, the photonic spiking neuron presents the feature distribution after the inspection of each part of input waveforms, namely in a feature-by-feature manner. The obtained feature collects the contribution from all waveform details and can support high-accuracy and low-latency waveform classification. Third, the readout of the processing results only requires low-accuracy electronics devices because of the binary nature of spikes. Compared to other paradigms that are capable of complex RF signal processing (e.g., reservoir computing), the photonic-spiking-based scheme can present decision results in a more direct manner with binary spikes instead of extra linear regression.

For future researches, the integration of high-performance photonic spiking neuron should be the first step toward the on-chip photonic-spiking-based RF signal processing. Moreover, the novel implementation of photonic spiking neurons with tunable time constant can improve the diversity of the analog-to-spike encoding. At the same time, it is necessary to develop a robust and accurate threshold-level-control scheme for photonic spiking neuron. The real-time detuning monitoring and physical feedback link may provide a solution, as discussed in optical communication filed [26]. Besides, the weighted connection and layout of the PSNN remain a big challenge for the scale of tens of neurons. In conclusion, the photonic spiking, merging the asynchronous spiking response and ultra-fast photonics, has potential to enable real-time RF signal processing for critical scenarios, such as transmitter identification [3] and burst-mode transmission [31].