1.
Introduction
In the past few decades, the computing capability of conventional digital computers based on complementary metal oxide semiconductor (CMOS) transistors has been increased greatly as predicted by Gordon Moore[1]. With the rapid development of artificial intelligence (AI) and internet of things, an exponential growth in the amount of data has led to urgent requirements for high efficiency and ultralow power consumption for the data-centric applications. In a standard digital computer based on the von Neumann architecture, the memory and processor are physically separated. Significant data movement between memory and processor is costly in terms of time and energy in today’s von Neumann systems. Meanwhile, Moore’s law has been slowing down significantly in recent years[2]. Thus, innovative non-von Neumann computational approaches are highly desired in the More-than-Moore era.
The human brain, which is believed to be the most complex intelligent system in the universe, exhibits ultralow power consumption, massive parallelism, robust fault tolerance, self-adaptation, and self-learning ability. The architecture of the human brain differs from that of a digital computer. Generally, a biological neural network is composed of roughly 1011 neurons and 1015 synapses, and is normally represented by spiking neural network (SNN)[3]. In the brain, the information is represented and transmitted by action potential, i.e., spike[4, 5]. The spike signals transmitted between the neurons connected through synapses with synaptic plasticity. Spike timing-dependent plasticity (STDP) is one of the most widely studied synaptic plasticity mechanism[6-8].
Inspired by the network architecture and principles of the human brain, the neuromorphic computing system has drawn tremendous attention in the next generation of computing technology. Nowadays, both the digital and analog hardware paradigms based on the CMOS technologies have been fully developed[9, 10]. In addition, extensive efforts have also been made to mimic the functions of biological neurons and synapses by various electronic neuromorphic devices. In recent years, remarkable progress has been made from materials to devices, circuits, and architectures in the field of electronic neuromorphic computing[9-12]. However, the operation speed of the electronic neuromorphic computing system is restricted due to the bandwidth–connection density trade-off[13].
As a complementary approach, the photonics platform has gained increasing attention for hardware neuromorphic computing, due to the fascinating advantages such as high speed, wide bandwidth, and massive parallelism. The photonics neuromorphic computing shows great promise in the applications which require low latency, low power consumption, and high bandwidth. Nevertheless, the photonics neuromorphic computing is still in its infancy compared to the electronics counterpart. Nahmias et al. proposed a photonic leaky integrate-and-fire (LIF) neuron based on vertical-cavity surface-emitting lasers with saturable absorber (VCSELs-SA)[14]. With the help of Yamada model, the neuron-like spiking dynamics was reproduced in the VCSEL-SA model. On the other hand, there are also some proposed photonics synapses. For instance, Gholipour et al. developed photonic synapses based on amorphous gallium lanthanum oxysulphide microfibers[15]. Cheng et al. fabricated and demonstrated an on-chip photonic synapse via phase-change materials combined with integrated silicon nitride waveguide[16]. Specifically, Feldmann et al. implemented an all-optical SNN with self-learning capacity based on a nanophotonic chip, and successfully demonstrated supervised and unsupervised learning in the optical domain[17]. For more detail on the spiking neurons and synapses based on photonic devices, please refer to Refs. [13, 18–21].
Generally, the photonics neurons and synapses are studied separately with different devices, therefore leading to the need for further developing high-performance photonics neuron and synaptic devices. In addition, algorithm is another key issue that limits the progress of photonic SNN development.
Herein, we review some recent progress on the devices, architecture, and algorithm of photonic neural computing in our research groups. First, we introduce the photonic neuron at the device level. Then, we review the progress on the photonic STDP. Subsequently, we focus on several photonic neural networks at the system level. Finally, we summarize the challenges and opportunities faced by photonic neural computing, and propose promising solutions and perspectives.
2.
Photonic neuron
Photonic devices operating in the excitable regime are dynamically analogous to the biological neurons exhibit spiking dynamics. While the operating speed of photonic devices are many orders of magnitude faster than their biological counterparts. There are various optical neurons reported experimentally and numerically in recent years[14, 22-39]. Here, we focus on the lasers-based optical neuron. Some recent progress on the optical neurons based on conventional VCSEL, VCSEL-SA, as well as distributed feedback laser (DFB) are reviewed.
2.1
Optical neuron based on VCSEL-SA
A VCSEL-SA can be employed as a photonics neuron because it possesses the excitability property behaves analogously to the LIF neuron model. The excitability properties and the spike latency of the VCSELs-SA are shown in Fig. 1[39]. Fig. 1(a) shows that the VCSEL-SA could generate excitatory neuron-like spike and realize the temporal encoding. The spike latency, defined as the interval between the start time of stimulus pulse and the timing corresponding to the maximum value of spikes generated by VCSEL-SA, varied with different stimuli strengths and central timings. As presented in Fig. 1(b), when the stimuli pulse power exceeds a threshold value, the VCSEL-SA can emit a spike. In addition, such threshold behavior can be tuned by different bias current. Fig. 1(c) indicated that the spiking timing can be continuously adjusted by different stimulus strength.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-1.jpg'" class="figure_img" id="Figure1"/>
Download
Larger image
PowerPoint slide
Figure1.
(Color online) (a) Temporal output of the spike encoding based on the modeling-based photonic neuron. (b) Threshold-like response and (c) spike latency property of the modeling-based photonic neuron. ? [2020] IEEE. Reprinted with permission from Ref. [39].
Note, the inhibitory dynamics is also important for the neural information processing. We further revealed the inhibitory dynamics based on the polarization mode competition effect in a VCSEL-SA with two coexisting polarization-resolved modes[33]. The schematic diagram of inhibitory photonic neuron based on VCSEL-SA is presented in Fig. 2(a). The inhibition behavior was characterized by the spike amplitude and first spike latency. As presented in Fig. 2(b), the neuron-like inhibition behavior could be achieved thanks to the polarization mode competition. Furthermore, as presented in Fig. 3, such inhibition dynamics could also be used to realize the spike-based XOR in a single step with a single VCSEL-SA[34]. Additionally, the XOR operation was also achieved successfully with two pseudo-random return-to-zero sequences.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-2.jpg'" class="figure_img" id="Figure2"/>
Download
Larger image
PowerPoint slide
Figure2.
(a) Schematic diagram of inhibitory neuron based on VSCEL-SA. Reprinted from Ref. [31]. (b) The results of inhibition in a photonic neuron. Reprinted with permission from Ref. [33]. ? The Optical Society.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-3.jpg'" class="figure_img" id="Figure3"/>
Download
Larger image
PowerPoint slide
Figure3.
(Color online) (a) Schematic diagram of all-optical exclusive OR (XOR) operator based on a single VCSEL-SA. (b) XOR output for different sets of inputs. (c) The inputs and outputs of XOR for two RZ sequences. Reprinted with permission from Ref. [34]. ? The Optical Society.
2.2
Optical neuron based on VCSEL
The VCSELs have become promising candidates for artificial neuronal models due to the polarization switching or optical injection induced nonlinear dynamics. Recently, the controllable and reproducible excitation and inhibition behaviors at sub-nanosecond speeds for a commercially available VCSEL subject to the successive external stimulus have been experimentally and theoretically demonstrated[22, 24-30, 36]. Under an external injection characterized by a constant optical power level with sudden perturbation in the form of short power drops, the VCSEL-neuron can transmit from the injection-locking state into the unlocking state after the arrival of the perturbation due to the broken equilibrium and a spiking event is triggered. Moreover, the number of fired spikes can be controlled through adjusting the temporal duration of incoming perturbations, as shown in Figs. 4(b1)–4(b5)[26, 28]. Also, under the external injection in the form of a constant level with sudden power raises for increasing perturbation duration, the spiking regime could be entirely suppressed during the perturbation period, as shown in Figs. 4(c1)–4(c5)[30]. Moreover, the communication of excitatory and inhibitory spiking signals at sub-nanosecond speeds between two coupled VCSEL-neurons have also been demonstrated experimentally and numerically[26-28, 30].
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-4.jpg'" class="figure_img" id="Figure4"/>
Download
Larger image
PowerPoint slide
Figure4.
(Color online) (a) The experimental setup for spiking firing and inhibition of VCSEL-based neuron. ? [2018] IEEE. Reprinted with permission from Ref. [30]. (b) Time series of fired spiking responses. Reprinted with permission from Ref. [26]. ? The Optical Society. ? [2017] IEEE. Reprinted with permission from Ref. [28]. (c) Time series of suppressed spiking response. ? [2018] IEEE. Reprinted with permission from Ref. [30].
2.3
Optical neuron based on DFB
The DFB exhibits similar processing characteristics with the graded-potential-signaling-based neuron observed in the nervous system, such as temporal integration and pulse facilitation. A commercially available DFB was demonstrated to play a role in three applications of neuromorphic information processing for pattern recognition, single-wavelength STDP implementation, and sound azimuth measurement, as shown in Fig. 5[37]. To further investigate the spatiotemporal processing potential of DFB neurons, a network architecture was proposed to be equipped with N tunable weights on each input branch for complex pattern recognition[38]. The schematic of the network is illustrated by Fig. 6(a). Successful pattern recognition was demonstrated among the input sets when N = 3 and 4, as given in Figs. 6(b) and 6(c), respectively. Additionally, the network can learn a target pattern by the assistance of the STDP learning module also realized by a DFB, which is convenient for the potential integration.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-5.jpg'" class="figure_img" id="Figure5"/>
Download
Larger image
PowerPoint slide
Figure5.
(Color online) (a) The experimental setup for graded-potential-signaling-based neuromorphic processing applications with the optical neuron based on DFB, including (b, c) pattern recognition, (d, e) single-wavelength implementation of STDP, and (f, g) sound azimuth measurement[37]. Reproduced with permission. ? 2020 Springer Nature.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-6.jpg'" class="figure_img" id="Figure6"/>
Download
Larger image
PowerPoint slide
Figure6.
(Color online) (a) The DFB-based spatiotemporal pattern recognition network with STDP learning module. The network output for patterns with (b) 3 and (c) 4 input branches, respectively. Reprinted with permission from Ref. [38]. ? The Optical Society.
3.
Photonic synaptic plasticity
In brains, synaptic plasticity is believed to be closely related to the learning and memory. In this section, we review the emulation of the synaptic function based on the semiconductor optical amplifier (SOA) and vertical-cavity semiconductor optical amplifier (VCSOA). The photonic STDP is focused on.
3.1
Photonic STDP based on SOA
STDP is a long-term synaptic plasticity observed experimentally in biological synapses by Bi and Poo[6]. Thanks to the cross-gain modulation in SOA, the implementation of optical STDP scheme based on SOA was demonstrated[40-43]. The result in Fig. 7 shows that the STDP curve is similar to that measured in biological experiments, but at a much faster time scale[41]. Moreover, the height and width of the measured STDP learning window decrease as the SOA current increases.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-7.jpg'" class="figure_img" id="Figure7"/>
Download
Larger image
PowerPoint slide
Figure7.
(Color online) (a) Optical implementation of STDP. (b) The measured learning window of Optical STDP with different SOA driving current. Reprinted with permission from Ref. [41]. ? The Optical Society.
3.2
Photonic STDP based on VCSOA
Note that, for the photonic STDP based on SOA, the operation current of SOA is relatively high, i.e., several tens of or hundreds of mA. When operating below threshold, the VCSEL can also be regarded as VCSOA. The photonic STDP based on the VCSOA was proposed and demonstrated experimentally and numerically[44]. The schematic diagram is illustrated in Fig. 8(a). In our experiment, the VCSOA was biased at 1.42 mA, the optical pulse power (width) was 70 μW (100 ps). The energy consumption for biasing the VCSOA and triggering the STDP function could be estimated as several femtojoules per spike, which is much lower than the microelectronic counterparts[45, 46]. The experimental measurements are shown in Fig. 8(b). When two optical pulses with different time interval were injected into the VCSOA, the lagged pulse experienced different gain due to the carrier depletion caused by the leading pulses. With the increase of time interval, the output power of the lagged pulse increased gradually. The numerical results are presented in Figs. 8(c) and 8(d), which agree well with the experimental measurements. Fig. 8(e) shows that the STDP curve can be achieved in VCSOA with low bias current. The calculated STDP window was about 1 ns, indicating that the STDP operation rate was nearly 1 GHz, which is higher than the conventional electronics[45, 46]. Furthermore, such photonic STDP curve can also be achieved in real-time with a single VCSEL with dual-polarized pulsed optical injection[47].
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-8.jpg'" class="figure_img" id="Figure8"/>
Download
Larger image
PowerPoint slide
Figure8.
(Color online) (a) Schematic diagram of photonic STDP based on VCSOA. (b) The experimental measured output pulse train corresponding to the input pulse pairs with different time interval. (c) Simulated input pulse. (d) Simulated output pulse. (e) The calculated STDP curve. Pulse 1 (Pulse 2): the optical pulse injection beam; VODL: variable optical delay line, OC: optical coupler; Circulator: optical circulator; VCSOA: vertical-cavity semiconductor optical amplifier. Bias and TEC: The bias current and temperature controller for VCSOA;
4.
Photonic neural computing
In this section, we review the progress on the optical neural computing at the system level. We consider several network architectures and algorithms including photonic SNN, photonic convolutional neural network (CNN), photonic matrix computation, photonic reservoir computing (RC), and photonic reinforcement learning.
4.1
Photonic SNN
To design the algorithm for a photonic SNN, we proposed a novel framework of a fully VCSEL-based all-optical SNN and developed a self-consistent unified neuron-synapse-learning model that allows a complete learning-to-inference workflow[35, 39]. The unsupervised learning was implemented in a photonic SNN consisting of VCSELs-SA[35]. The input pattern is encoded into spikes at different timings by different VCSELs-SA. As shown in Fig. 9, the photonic SNN with photonic STDP enables the POST neuron to spike at the first spike timing of the input pattern in an unsupervised manner. In addition, supervised learning was also realized in a photonic SNN[39, 48]. As illustrated in Fig. 10, a supervised spike sequence learning task was implemented in a two-layer fully-connected photonic SNN consisting of excitable VCSELs-SA[48]. In this photonic SNN, the classical remote supervised method (ReSuMe) learning algorithm based on the photonic STDP is adopted to train the POST to fire the desired spike sequence. Fig. 10(c) illustrates the learning process in a typical run. The corresponding spike sequence distance (SSD) as a function of the learning epoch shown in Fig. 10(e) indicates that the learning process was convergent. Therefore, the photonic SNN successfully reproduced a desirable output spike sequence in response to a spatiotemporal input spike pattern by means of the iteration algorithm to update synaptic weights continuously.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-9.jpg'" class="figure_img" id="Figure9"/>
Download
Larger image
PowerPoint slide
Figure9.
(Color online) (a) Schematic diagram of photonic SNN based on VCSELs and VCSOAs. (b) PST as a function of the learning cycle. (c) Synaptic weights evolution during the learing process. n photonic presynaptic neurons and one postsynaptic neuron are conneted with optical STDP synapses. VCSEL1–VCSELn: photonic presynaptic neurons; VCSELn+1: photonic postsynaptic neuron; T: variable delay line; Wi (i = 1, 2, …, n): variable synaptic weight device connecting VCSELi and VCSELn+1; STDP array: optical STDP synapses realized by VCSOAs; C: optical coupler. The red dashed box represents the ex-situ approach for updating the synaptic weight. ? [2019] IEEE. Reprinted with permission from Ref. [35].
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-10.jpg'" class="figure_img" id="Figure10"/>
Download
Larger image
PowerPoint slide
Figure10.
(Color online) (a) Schematic diagram of a photonic SNN based on VCSEL-SAs for the supervised spike sequence learning. (b-e) Illustration of the spike sequence learning of a typical run. ? [2020] IEEE. Reprinted with permission from Ref. [48].
The spatiotemporal pattern classification based on supervised learning was further demonstrated based on the spatiotemporal design of the photonic SNN shown in Fig. 11(a)[39]. Optical character numbers were trained and tested by the proposed SNN. As shown in Fig. 11(b), the network was trained with a clean character image, and then, the inference was tested with a set of noisy patterns. The results show that this all-optical SNN can recognize ten numbers by a supervised learning algorithm. Besides, the training convergence can be optimized by using different bias current of VCSOA as presented in Fig. 11(c). As illustrated in Fig. 11(d), the accuracy rate of the trained network was robust to small noise strength of the optical digital character.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-11.jpg'" class="figure_img" id="Figure11"/>
Download
Larger image
PowerPoint slide
Figure11.
(Color online) (a) Architecture of the proposed all-optical SNN. (b) An example of a pattern classification task. The network is trained with (b1) a clean character image, and then, the inference was tested with a set of (b2) noisy patterns. (c) Comparison of convergence performance for supervised learning with different I of VCSOA. (d) Accuracy rate of the trained network as a function of the noise strength of the optical digital character. ? [2020] IEEE. Reprinted with permission from Ref. [39].
In addition, sound azimuth detection was emulated in a fully connected photonic SNN consisting of VCSELs-SA[49]. Fig. 12(a) shows the schematic diagram of the proposed photonic SNN which composed of two presynaptic neurons (PREs) considered as the ear sensors, and two postsynaptic neurons (POSTs). The difference between the precise spike timing of two POSTs (
m o}}$
m o}}$
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-12.jpg'" class="figure_img" id="Figure12"/>
Download
Larger image
PowerPoint slide
Figure12.
(a) Schematic structure of a 2 × 2 photonic SNN architecture to detect the sound azimuth, and two PREs correspond to the right ear and left ear, respectively. (b) Responses of POST1 and POST2 when
Besides, the winner-take-all mechanism was also achieved successfully in the photonic SNN consisting of VCSELs-SA as shown in Figs. 13(a) and 13(b)[50]. Two information processing tasks including pattern recognition and max-pooling operation were performed as shown in Figs. 13(c)–13(f). The results hold great promise for the development of energy-efficient and high-speed photonic SNN.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-13.jpg'" class="figure_img" id="Figure13"/>
Download
Larger image
PowerPoint slide
Figure13.
(Color online) (a) Schematic diagram of WTA based on VCSELs-SA. (b) The output of VCSELM,A,B-SA for WTA mechanism. (c) Schematic diagram of pattern recognition based on the WTA machine. (d) The inputs and results of pattern recognition. (e) Schematic diagram of max-pooling operation. (f) The results of max-pooling operation. ? [2020] IEEE. Reprinted with permission from Ref. [50].
Furthermore, associative learning and forgetting processes were emulated in a photonic SNN[51]. The schematic diagram of the photonic associative learning network is shown in Fig. 14(a). Fig. 14(b) shows that both the associative learning and forgetting processes could be achieved thanks to the photonic STDP rule. The pattern recall based on the associative learning was further demonstrated in the photonic SNN presented in Fig. 14(c). Complete pattern and incomplete pattern of number 8 are shown in Figs. 14(d1) and 14(d2), respectively. Fig. 14(d3) [Fig. 14(d4)] shows the initial output (final output) of number 8 before [after] associative learning process. The evolution of synapse weight corresponding to the pattern recall of number 8 is presented inFig. 14(e1). Without loss of generality, Figs. 14(d5, d6, d7, d8) and 14(e2) show the pattern recall process of number 5 and the corresponding weight evolution. Obviously, the incomplete pattern can be recovered and pattern recall was realized based on the photonic associative learning network.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-14.jpg'" class="figure_img" id="Figure14"/>
Download
Larger image
PowerPoint slide
Figure14.
(Color online) (a) Schematic diagram of associative learning and forgetting processes based on VCSELs and STDP. (b) The emulation of associative learning and forgetting processes. (c) Schematic diagram of pattern recall. (d) Complete and incomplete patterns of number 8 and 5 respectively, visualization initial and final outputs of number 8 and 5 respectively. (e) The change processes of synaptic weight for number 8 and number 5. ? [2020] IEEE. Reprinted with permission from Ref. [51].
4.2
Optical convolutional neural network
The optical implementation of CNN with fast operation speed and high energy efficiency is appealing due to its outstanding feature extraction ability[52-54]. The high-accuracy optical convolution unit (OCU) with cascaded acousto-optical modulator arrays is illustrated in Figs. 15(a)–15(c)[52]. The input data and convolutional kernel were fed into the modulator arrays to carry out the operation. With the hardware reusing scheme, complicated CNNs can be conducted by the units. In Fig. 15(d), convolution results on the digital computer and the proposed OCU are shown to support the feasibility.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-15.jpg'" class="figure_img" id="Figure15"/>
Download
Larger image
PowerPoint slide
Figure15.
(Color online) (a) The architecture of the optical convolution unit (OCU) by modulator arrays. (b) The transmission rate versus the modulation voltage of the single modulator. (c) An illustration of the serialization method. (d) The convolution results of MNIST-handwritten numbers and Fashion-MNIST data sets. Reprinted with permission from Ref. [52]. ? The Optical Society.
A more complete optical CNN implementation incorporating patching scheme was demonstrated in[53]. In Figs. 16(a) and 16(b), the introduction of optical delay lines to execute data manipulations promises low latency and power consumption. Meanwhile, wavelength-division-demultiplexing (WDM) was used to improve the computational capacity by parallel wavelength channels. Illustrated by Figs. 16(c)–16(f), the serialized input waveforms of digit 2 and 4 were successful to form delayed copies as the experimental demonstration of optical patching. Consequently, the scale of input modulator arrays was largely cut down.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-16.jpg'" class="figure_img" id="Figure16"/>
Download
Larger image
PowerPoint slide
Figure16.
(Color online) (a) The conceptual layout of the optical patching scheme with optical delay lines and wavelength-division-demultiplexing (WDM). (b) The experimental setup of the proposed scheme. Delayed copies of the input waveforms corresponding to (c, d) digit 2 and (e, f) 4, respectively. Reprinted with permission from Ref. [53]. ? The Optical Society.
4.3
Optical matrix computation
Matrix computations form the most widely used computational tools in science and engineering, and are the basic components of neural networks for deep learning. While the electronic matrix computations suffer from limited bandwidth. Alternatively, the optical methods offer a high-speed and low-loss solution. Optical matrix computation is also essential in the optical neural computing.
The Mech-Zehnder interferometer (MZI) mesh had been demonstrated for optical matrix computation[55, 56]. As shown in Fig. 17(a), the optical matrix computing processor can perform fundamental matrix computations including XB = C, AB = X and AX = C, where A, B, C are known matrices, and X is the matrix to be solved. Fig. 17(b) shows an example to self-configure a transmission matrix. An optical PageRank algorithm was further demonstrated based on the matrix computing processor. Furthermore, the optical matrix computation core could be applied for polarization processing[57, 58]. The micrograph of chip is presented in Fig. 17(a) and some experimental results are depicted in Fig. 18. The polarization processor could implement multiple polarization processing functions, including polarization multiple-input-multiple-output (MIMO) descrambler, polarization controller and polarization analyzer, which are the basic building blocks of polarization processing. More functions could be realized by using an additional two-dimensional output grating. A numerical gradient descent algorithm was employed to self-configure and self-optimize these functions. Recently, the 4-port general linear network has been applied for multi-channel optical switching, MIMO descramble and tunable filter by learning strategy[59]. Fig. 19 shows the typical demonstration of the three functions respectively. The works suggested great potential for chip-scale reconfigurable and fully programmable photonic computing and optical signal processors with artificial intelligent algorithm.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-17.jpg'" class="figure_img" id="Figure17"/>
Download
Larger image
PowerPoint slide
Figure17.
(Color online) Optical matrix computation and the application for polarization processing. (a) Special-purpose processors for optical matrix computing and polarization processing respectively. (b) Self-configuring example for the smart processors. ? [2020] IEEE. Reprinted with permission from Ref. [55].
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-18.jpg'" class="figure_img" id="Figure18"/>
Download
Larger image
PowerPoint slide
Figure18.
(Color online) Experimental results for photonic polarization processor chip. (a) Polarization MIMO descrambler. (b) Polarization controller. (c) Polarization analyzer. Reprinted with permission from Ref. [58]. ? The Optical Society.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-19.jpg'" class="figure_img" id="Figure19"/>
Download
Larger image
PowerPoint slide
Figure19.
(Color online) Experimental results for self-configuring optical signal processor. (a) Multichannel optical switching. (b) Optical MIMO descramble. (c) Tunable optical filter. Reprinted with permission from Ref. [59]. ? 2017 American Chemical Society.
4.4
Photonic reservoir computing
RC is a brain-inspired computational paradigm originated from recurrent neural network suitable for time series processing[60-62]. In the training process of RC systems, only the output weights are modified, while the input and reservoir weights are fixed randomly. Specifically, the time-delay RC system based on a single nonlinear node with delay feedback has been demonstrated in electronic, optoelectronic, and all optical delay systems[63-65]. In recent years, we have also made some attempts to the time-delay RC via photonics approaches[66-70].
An attempt at the time-delay RC systems based on VCSEL is that we proposed a four-channels RC system based on polarization dynamics in mutually coupled VCSELs[67]. As shown in Fig. 20(a), four channels RC were realized in two orthogonal polarization modes of two VCSELs. The outputs obtained from the four channels were combined into one output for post-processing as displayed in Fig. 20(b). The four-channels RC could produce comparable prediction performance but at a faster information processing rate compared with the one-channel RC as shown in Fig. 20(c).
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-20.jpg'" class="figure_img" id="Figure20"/>
Download
Larger image
PowerPoint slide
Figure20.
(Color online) (a) System design of the Four-channels RC based on MDC-VCSELs. (b) The virtual node states matrix for each channel. (c) The NMSE values of Four-channels RC system based on MDC-VCSELs as a function of bias current for Four-channels RC and One-channel RC, respectively. Reprinted with permission from Ref. [67]. ? The Optical Society.
We also have some attempts at the time-delay RC system based on semiconductor nanolaser (SNL), due to the potential of realizing photonic integrated RC system[69, 70]. For example, a high-speed neuromorphic SNL-based RC system under electrical modulation was proposed[69]. The conceptual scheme of the SNL-based RC system is presented in Fig. 21(a). The numerical simulation results are shown in Fig. 21(b), which show that a larger Purcell factor F and larger spontaneous emission coupling factor β could extend the range of high prediction performance of SNL-based RC system.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-21.jpg'" class="figure_img" id="Figure21"/>
Download
Larger image
PowerPoint slide
Figure21.
(Color online) (a) The conceptual scheme of RC based on a semiconductor nanolaser (SNL) with delayed feedback. (b) The NMSE values of SNL-based RC system as a function of the kd for different F and
4.5
Photonic reinforcement learning
Reinforcement learning is also a fundamental learning mechanism inspired by the brain[71]. In reinforcement learning, an agent learns from interaction process with the environment, aims to maximize the benefits via certain learning strategies. Decision-making is a basic component of reinforcement learning, which requires the agent to make decision quickly and accurately in an uncertain and dynamically changing environment[72]. Photonic decision making provides possibilities in applications of ultrafast processing.
The photonic reinforcement learning based on laser chaos with time-delay signature concealment was demonstrated by introducing a phase-modulated Sagnac loop in mutually delay-coupled semiconductor lasers (PMSL-MC)[73], as shown in Fig. 22. The multi-armed bandit problem was solved in parallel with the utilization of dual-channel chaotic signals. The comparison between the PMSL-MC system and conventional mutually-coupled semiconductor lasers system (CSL-MC) further demonstrated that the system with dual-channels chaotic signals can make decision in parallel and converge faster.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-22.jpg'" class="figure_img" id="Figure22"/>
Download
Larger image
PowerPoint slide
Figure22.
(Color online) (a) Experimental setup of a dual-channels chaotic system with a phase-modulated Sagnac loop. (b) Architecture for reinforcement learning based on dual-channels laser chaos. (c) The CDR as a function of cycle for the CSL-MC system and for the PMSL-MC system. (d) The convergence cycle (CC), at which the CDR reaches 0.9, as a function of coupling strength for the CSL-MC system and the PMSL-MC system. (e) The CDR as a function of the number of cycles for dual-channels and one-channel in the PMSL-MC system. (f) The CC as a function of coupling strength for dual-channels and one-channel in the PMSL-MC system. Reprinted with permission from Ref. [73]. ? The Optical Society.
A further parallelized scheme for photonic decision making was experimentally demonstrated in a globally-coupled chaotic semiconductor lasers network as shown in Fig. 23[74]. Triple-channel chaotic signals were applied to solve an 8-armed bandit problem with a parallel architecture given in Fig. 23(b). In Figs. 23(c) and 23(d), the results suggested that the chaotic signals with better time delay signature concealment generally contributed to better decision-making performance. The adaptability of the strategy to environmental change was further demonstrated as in Fig. 23(e). Moreover, such a scheme was also scalable as demonstrated in Fig. 23(f), where both 8-armed and 16-armed bandit problems were solved successfully.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/2/PIC/20110008-23.jpg'" class="figure_img" id="Figure23"/>
Download
Larger image
PowerPoint slide
Figure23.
(Color online) (a) The experimental setup of three globally coupled DFB lasers. (b) A parallel architecture for photonic decision making of 8-armed bandit problem. (c, d) CC and delay concealment as a function of attenuation. (e) The adaptability of the strategy to dynamically changing environment. (f) The scalability to 16-armed problem. Reprinted from Ref. [74]. Copyright (2020) with permission from Chinese Laser Press.
5.
Conclusion and outlook
We have reviewed some representative photonic neural computing in devices, architectures, and algorithms. To further pave the way of photonics neuromorphic computing, there are still some problems that need to be addressed. The optical neuron and synapse are generally designed separately, which results in the different time scales. The optical neuron and synapse with similar time scale have to be further developed to meet up the requirements of on-line training. The design of photonic neural computing system on a chip still requires further exploration. Further attempts have to be made to reduce the area and increase the integration level.
Note, at the present stage, we think it may be more realistic to adopt an ex-situ training approach for the training process of the photonic neural network. Nevertheless, it is promising to build an integrated hardware photonic neural network for realizing the inference process. At present, the optical neuron based on spiking laser could be readily realized with III–V compound semiconductor technology such as indium phosphide (InP) and gallium arsenide (GaAs), while the weight array could be successfully implemented with silicon waveguides or resonators. It is believed that the rapid development of photonic integrated technologies will lead to a bright future for the field of photonic neural computing. For instance, a hybrid III–V and silicon photonics platform may be a candidate to realize an integrated hardware photonic neural network for inference task[75-84], in which spiking lasers array in a bonded InP layer could be interconnected via a silicon layer[13, 78].
Acknowledgements
This work was supported in part by the National Outstanding Youth Science Fund Project of National Natural Science Foundation of China (62022062), by the National Natural Science Foundation of China (61974177, 61674119), by the Fundamental Research Funds for the Central Universities.
