1.
Introduction
A resistive random-access memory (RRAM)-based neural network has been extensively studied as a promising solution to overcome the von Neumann bottleneck faced in conventional artificial intelligence (AI) hardware[1-4]. As inspired by a biological neural network, an artificial neural network consists of synaptic and neuronal devices. In order to improve speed and power efficiency, the RRAM crossbar array, which can significantly accelerate the vector-matrix multiplication, has been developed to implement artificial synapses[5-8]. On the other hand, a neuronal device is needed at the end of each crossbar bit line (BL) to convert the weighted sum current of the analog RRAM synapses into spikes to transmit information to the next layer of neurons. Here integrate-and-fire neurons built with CMOS circuits are typically employed[9]. However, such complex CMOS neurons would occupy a much larger footprint than the BL pitch of the crossbar array, which causes serious column pitch matching problem[10, 11].
Recently, a compact oscillation neuron based on a metal–insulator transition (MIT) threshold switching (TS) device was proposed as a more scalable artificial neuron[12-16]. Compared to the CMOS neuron, an oscillation neuron has the benefits of small size and simple circuit structure, which is appealing for large-scale neuromorphic system applications. However, the on/off ratio of a typical MIT TS device is small (~102), which cannot be used for a large RRAM crossbar array (e.g., 12 × 1 array)[15]. Moreover, the high operation voltage of the MIT TS device may disturb the weights of the RRAM synapses and also increase the power consumption[16]. Alternatively, TS devices based on electrochemical metallization (ECM) filaments have attracted considerable attention due to their simple structure, large on/off ratio, and low operation voltage[17-23]. However, the uniformity of typical ECM TS devices is relatively poor, which may affect the accuracy of artificial neural networks. More recently, we have developed a high-uniformity HfO2-based TS device with patterned Ag nanodots (NDs) as the high-performance selector, which shows low leakage current (< 1 pA), high on/off ratio (> 108), and high endurance (> 108 cycles)[24].
In this work, we further implement an oscillation neuron using the HfO2/Ag NDs TS device. This neuron exhibits self-oscillation behavior at low applied voltage (1 V), where the oscillation frequency increases with the applied voltage and decreases with the load resistance. In addition, it can work with a large RRAM crossbar array (> 128 × 128) owing to the high on/off ratio (> 108) of Ag NDs TS device. Moreover, in the neural network simulation, a high recognition accuracy (loss < 1%) can be achieved by using this oscillation neuron because of its high uniformity.
2.
Results and discussion
Fig. 1(a) illustrates the schematic diagram of a simple artificial neural network. When the input voltages are applied to the crossbar synaptic array, the weighted sum currents are integrated by the neurons at the end of each column (BL) and trigger output spike firing when they reach the neuron thresholds. The circuit implementation of the oscillation neuron based on the TS device is shown in Fig. 1(b). The load resistance (RL) represents the RRAM synaptic weight connected in series with the TS device. Also, CL is the load capacitance including parallel capacitance and parasitic capacitance at the neuron node. Initially, the TS device is in the off state (Roff). When applying an input voltage pulse (Vin), the voltage mainly drops on the TS device since Roff > RL, and CL starts to charge. When Vin is larger than the threshold voltage of the TS device Vth, it turns to the on state (Ron), and then CL starts to discharge since the voltage drop on the TS device is reduced (Ron < RL). Once the voltage drop on the TS device is below the hold voltage of TS device Vhold, it switches back to Roff, ready for the next firing event. In this way, the TS device switches on and off between Ron and Roff, and the neuron outputs oscillation signal. If RL is chosen to satisfy Roff > > RL > > Ron, the ideal oscillation frequency f can be described as[13]:
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/6/PIC/21010010-1.jpg'" class="figure_img" id="Figure1"/>
Download
Larger image
PowerPoint slide
Figure1.
(Color online) (a) Schematic diagram of a typical artificial neural network. (b) Circuit implementation of the oscillation neuron with a TS device.
$$f=1/left[{R}_{ m L}{C}_{ m L}times { m{log}}left(frac{{V}_{ m{hold}}-{V}_{ m{in}}}{{V}_{ m{th}}-{V}_{ m{in}}} ight) ight].$$  | (1) |
Eq. (1) shows a one-to-one correspondence between f and RL at a certain CL and Vin. Therefore, the oscillation frequency can be used to represent the weight of the RRAM synapse accurately.
In this study, we demonstrate the oscillation neuron using the Ag NDs TS device based on ECM filaments. The devices were fabricated with a cell size of 10 × 10 μm2. The bottom electrode was patterned by photolithography and deposited by sputtering of 5 nm Ti and 50 nm Pt. An 8 nm thick HfO2 dielectric was deposited by atomic layer deposition (ALD) at 250 °C. The Ag NDs with diameters of 50 nm were patterned by e-beam lithography (EBL) and deposited by sputtering. A 40 nm-thick Pt was deposited as the top electrode. The transmission electron microscope (TEM) image of the Ag NDs TS device is shown in Fig. 2(a). Fig. 2(b) exhibits the schematic illustration of the threshold switching process in the device. In the initial state, there is no conductive filament in the dielectric layer, and the device is in the high-resistance state (HRS). When applying a voltage (Vappl) larger than Vth, the electric field is locally enhanced in the areas with Ag NDs, so the Ag atoms are easier to be ionized as the ion source (Ag → Ag+ + e?) for diffusing toward the bottom electrode, which leads to the formation of metallic filaments and turns the device to a low-resistance state (LRS). In this device, the Ag filaments are thin and unstable due to the small amount of Ag ions. Spontaneous rupture of the filaments occurs immediately when Vappl goes below Vhold, which turns the device back to HRS. In this process, the Ag atoms form clusters on the trace of filaments. Owing to the highly ordered Ag NDs, Ag filaments tend to form at the same positions and the formed filaments would have similar morphology, in different operation cycles or different devices. Fig. 2(c) shows the typical current–voltage (I–V) curve of the Ag NDs TS device under voltage sweeps between 0 and 1 V. This device exhibits a low leakage current less than 1 pA and large selectivity over 108. The Vth and Vhold of Ag NDs TS device are 0.6 and 0.2 V, respectively, which are carefully tuned to work with the RRAM synapse. In order to evaluate the uniformity of the Ag NDs TS device, the distributions of Vth and Vhold are analyzed, and the cumulative probability of Vth and Vhold is shown in Fig. 2(d). The coefficient of variation is defined as CV = σ/μ to evaluate the variation, where μ and σ are the mean and standard deviation, respectively. This Ag NDs TS device exhibits excellent uniformity (CV < 10%) compared to other TS devices based on ECM filaments[24]. The endurance test of the Ag NDs TS device is shown in Fig. 2(e). In this measurement, the device is repeatedly turned on with SET pulses of Vset = 1 V and tset = 10 μs, and then relaxed to the off state, which is read with a small pulse of Vread = 0.1 V and tread = 10 μs. The on-state current (Ion) is limited to 10 μA. It is found that the Ag NDs TS device exhibits a high endurance of over 108 cycles. To implement an oscillation neuron, we connect the Ag NDs TS device in parallel with a load capacitance (CL) and then in series with a load resistor (RL) following the circuit configuration in Fig. 1(b). Fig. 2(f) shows the measured oscillation waveform of the neuron when RL = 50 kΩ, CL = 750 pF and Vin = 1 V. The test result shows that this oscillation neuron can output a continuous oscillation signal, when Vin, RL and CL are fixed. In addition, the Ag NDs TS oscillation neuron shows a certain time delay before its stable oscillation, which leads to a higher Vout in the first peak. It is owing to the turn-on delay time of the TS device[24]. The higher first peak may have an adverse effect on the synaptic weight precision by distorting the oscillation waveform, which can be minimized by further improving the TS device switching speed.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/6/PIC/21010010-2.jpg'" class="figure_img" id="Figure2"/>
Download
Larger image
PowerPoint slide
Figure2.
(Color online) (a) TEM image of the Ag NDs TS device. (b) Schematic illustration of the threshold switching process in the device. (c) Typical current–voltage (I–V) curves for the Ag NDs TS device. (d) Cumulative probability of Vth and Vhold distributions for the Ag NDs TS device. (e) Endurance test of the Ag NDs TS device with over 108 cycles. (f) Measured oscillation waveform of the oscillation neuron.
More systematic studies on the oscillation neuron characteristics are performed as shown in Fig. 3. Fig. 3(a) shows the oscillation waveforms at different input pulse voltages when RL = 50 kΩ and CL = 750 pF. The oscillation frequencies are 31.6, 40.2, 46.9 and 51.4 kHz for Vin = 1, 1.2, 1.4, and 1.6 V, respectively. It is found that the oscillation frequency increases as a function of the pulse amplitude of Vin as described in Eq. (1), and shows good consistency with the simulation results, as shown in Fig. 3(b). Similarly, Fig. 3(c) shows the oscillation waveforms of the Ag NDs TS devices with different RL values (i.e., different synaptic weights) when Vin = 1.2 V and CL = 750 pF. The oscillation frequencies are 40.2, 30.5, 26.1 and 16.8 kHz for RL = 50, 80, 100 and 150 kΩ, respectively. The oscillation frequency decreases as RL increases, as shown in Fig. 3(d). The test results indicate that this Ag NDs TS oscillation neuron exhibits self-oscillation behavior, and the output oscillation frequency can be used to sense the weight of the RRAM synapse accurately when the neuron is connected to the crossbar array.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/6/PIC/21010010-3.jpg'" class="figure_img" id="Figure3"/>
Download
Larger image
PowerPoint slide
Figure3.
(Color online) (a) Oscillation waveforms of the oscillation neuron with different Vin when RL = 50 kΩ, CL = 750 pF. (b) The oscillation frequency as a function of Vin. (c) Oscillation waveforms of the oscillation neuron with different RL when Vin = 1.2 V, CL = 750 pF. (d) The oscillation frequency as a function of RL.
Based on experimentally derived device data, a model of the RRAM crossbar array is developed to evaluate synaptic weights of different array sizes. Then SIPCE simulation is used to investigate the relationship between the oscillation frequency and the RRAM crossbar array size. In order to simplify the simulation process, the resistance RRAM device is fixed at an intermediate resistance state (RL = 500 kΩ), and Vin is applied to all the word lines (WLs) in parallel. The results are shown in Fig. 4. If the on/off ratio of the TS device is less than 100, only a limited range of the weighted sum could meet the criterion for oscillation (Roff > RL > Ron). It means that the oscillation neuron can only be used for small crossbar arrays (< 16 × 16). With the increase of on/off ratio, the oscillation neuron can work with a wider range of load resistance. Therefore, the weighted sum in a larger RRAM crossbar array can be successfully identified by distinguishable oscillation frequency. As in the case of our present Ag NDs TS device with on/off ratio > 108, it can work with a large RRAM crossbar array of size > 128 × 128.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/6/PIC/21010010-4.jpg'" class="figure_img" id="Figure4"/>
Download
Larger image
PowerPoint slide
Figure4.
(Color online) The oscillation frequency as a function of the RRAM crossbar array size under different on/off ratios of the TS device.
In order to investigate the impact of TS device variation on the oscillation neuron, the oscillation frequency distribution with different CV under the same experimental conditions is shown in Fig. 5(a). The uniformity of the TS device deteriorates with the increase of CV, which leads to a large variation of the oscillation frequency under the same load condition. Fig. 5(b) shows the oscillation frequency distribution of different RL with different TS devices. Compared with the Ag thin film device (CV ~ 30%)[23], the output frequency distribution of the Ag NDs TS neuron (CV ~ 7%) is more concentrated. Therefore, the oscillation neuron based on the Ag NDs TS device can achieve higher accuracy for neuromorphic computing.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/6/PIC/21010010-5.jpg'" class="figure_img" id="Figure5"/>
Download
Larger image
PowerPoint slide
Figure5.
(Color online) (a) The oscillation frequency distribution under different CV. (b) The oscillation frequency distribution of different RL when CV = 7% (top panel) and CV = 30% (bottom panel).
In order to further analyze the impact of the TS device’s uniformity on the artificial neural network, a multi-layer perceptron (MLP) of 784 × 200 × 10 is simulated to classify the handwritten digits in the Modified National Institute of Standards and Technology (MNIST) dataset, as shown in Fig. 6(a)[25]. The simulation results displayed in Fig. 6(b) indicate that the increase in CV leads to a dramatic degradation of the recognition accuracy, especially when CV > 15%. Therefore, the oscillation neuron based on the high uniformity Ag NDs TS device developed in this work is beneficial to reduce the network accuracy loss.
onerror="this.onerror=null;this.src='http://www.jos.ac.cn/fileBDTXB/journal/article/jos/2021/6/PIC/21010010-6.jpg'" class="figure_img" id="Figure6"/>
Download
Larger image
PowerPoint slide
Figure6.
(Color online) (a) The structure of MLP neural network. (b) Simulation results of the MNIST recognition accuracy loss as a function of the variability of the TS device.
3.
Conclusion
In conclusion, we have demonstrated a reliable oscillation neuron using the low-variability Ag NDs TS device for high-performance neuromorphic computing. This neuron exhibits self-oscillation behavior at low applied voltages down to 1 V. A systematic study on the oscillation characteristics reveals that the oscillation frequency increases with the applied voltage and also the synaptic conductance connected as the load resistor. The high uniformity and large on/off ratio of the Ag NDs TS device enable the oscillation neuron to reduce the neural network accuracy loss (< 1%) and make it applicable to a large-scale RRAM crossbar array (> 128 × 128). The developed oscillation neuron hence has great potential for future neuromorphic system applications.
Acknowledgements
This work was supported in part by China Key Research and Development Program (2016YFA0201800), and the National Natural Science Foundation of China (91964104, 61974081).
