EMINENT (Test and Reliability of Emerging Memory-based Spiking Neural Networks)

The research hypothesis of the EMINENT project: the strong restrictions on the size of embedded Spiking Neural Network architectures (limited silicon area and interconnectivity ability) require minimization of the network redundancy which in turn reduces its the intrinsic fault tolerance. Therefore, there is an acute need to evaluate the reliability and perform manufacturing test of the neuromorphic hardware architectures to guarantee their correct operation and robustness.

The EMINENT project is concerned with the following research areas:

The emerging memory technologies favor increasing system complexity and performance, opening the scientific community to great improvements in state-of-the-art computing (such as high performance computing or approximate computing) but also to new applications and computation paradigms (such as in-memory computing or neuromorphic computing) which had been unfeasible a few years back due to technological limitations. This project mainly focusses on the use of memristors or spintronic devices as artificial synapses for bio-inspired computing architectures. In this context, they have double functionality: memory (to save the values of the synaptic weights) and computation (to facilitate the on-line learning process, i.e., update synaptic weights).

The Spiking Neural Networks (SNN) are widely studied nowadays due to the high level of realism they bring to neural simulation, their energy efficiency and their ability for on-line learning. The related bio-inspired learning rule is known as STDP (Spike Based Dependent Plasticity) and is applied on each synapse independently of the global state of the network. In return, the synapse must be doted of computation capabilities. A hardware implementation of an SNN requites architectural co-localization of the processing and memory (non-Von Neumann architecture). The circuits solutions used to implement silicon neurons are application depended, but the vast majority are built with a temporal integration block, a spike generation block, a refractory period mechanism, and a spike adaptation block. Synapses are required to exhibit plasticity (i.e., modulation in their efficacy) and to support online learning algorithms, that manifest in changes in their strengths. Emerging memory devices can be used as synaptic elements thanks to their tunable conductivity, compatibility with advanced CMOS fabrication process, low power consumption, non-volatility and scalability. The synaptic conductance modulation can be emulated using:

Hardware dependability describes the ability of a system or component to function under stated conditions for a specified period of time. Dependability issues in hardware come from the manufacturing process, or operation and environmental conditions. Manufacturing induced variability, defects, stochastic effects, and aging degradation can cause important variations of the electrical characteristics of fabricated devices which in turn can lead to device failure. Manufacturing test together with design-for-reliability solutions assure the quality and reliability of integrated circuits.

The EMINENT project focuses on a fully-connected SNN, that learns using the Spike Timing Dependent Plasticity (STDP) method with lateral inhibition, with integrate-and-fire neuron and resistive synapses. Both analog and binary approach for synaptic conductance modulation will be used. In order to achieve the project’s goal (robust and efficient hardware implementation of SNNs) one has to jointly-consider the characteristics of the SNN itself (connectivity, neuronal activation function, learning rule and synaptic update), the characteristic of the devices used to implement it (CMOS ON/OFF current and threshold voltage, conductivity modulation and current-compliance of the synaptic devices, etc.) and the environment in which the circuit will be deployed.

The reliability estimation campaign will be conducted in a bottom-up approach. At gate level, we will evaluate the effect of aging phenomena, noise injection, and severe operation and environmental conditions. This analysis will culminate in a fault modeling campaign. These faults will be injected at system level, and the robustness, failure rate and mean time to failure (MTTF) of the SNN will be evaluated.

A comprehensive post-fabrication test campaign will be conducted. Both functional and structural test strategies are targeted. We will follow the conventional approach used today to test integrated circuits (using test input vectors able to target all possible faults). SNN architectures resort to the combination of devices with both deterministic and stochastic behaviors and they include both digital and analog elements. A test strategy suitable for SNNs should be able to test the correct operation of both neurons and synapses, it should be economical (in area, power and performance overhead), it should not depend on the training data nor on the context the network would be used. To devise efficient testing strategies will get our inspiration from techniques like March Tests, Iddq (measuring the supply current in the quiescent state), Analog-to-Digital Converter (ADC) testing, true random number generator (TRNG) testing. In parallel we will design test access mechanisms which ideally will allow for (i) independent testing of the synaptic array and neuron layer for structural integrity, and (ii) jointly testing the SNN for functional correctness. Moreover, design-for-test (DfT) and built-in-self-test (BIST) techniques will be devised.

We will provide solutions for architecture dependability improvement which can be based on robust circuit design, defect-aware design, hardware redundancy, design diversity, dynamic reconfiguration, dynamic adaptive signal control, etc.

The EMINENT project focuses on a fully-connected SNN, that learns using the Spike Timing Dependent Plasticity (STDP) method with lateral inhibition, with integrate-and-fire neuron and resistive synapses (see Fig. 1). Both analog and binary approach for synaptic conductance modulation will be used. Fig. 1a) sketches the connectivity between two consecutive layers of such a network. Here N represents a spiking neuron, S represents a synapse and test wrapper is the circuitry not-pertaining to the SNN but necessary to perform the manufacturing test, to be designed in the EMINENT project framework. The synapse and its control circuit are illustrated in Fig. 1b). Several works propose using the resistive-based synaptic devices in a crossbar array (without access device) to assure minimum area footprint. However, such implementation suffers from large leakage currents in the half accessed cells (gravely detrimental to the learning process) and from inefficient forming process (when RRAM devices are used for synapse) and un-controlled current compliance (can cause synapse deterioration or inefficiency). For this reason, in the presented design, an access transistor is used for the access to each synapse. This transistor should isolate the synapse from the rest of the network when there is no activity on either of its connected neurons, and it should allow the passage signal coming from either of the neurons. This behavior is guaranteed by OR-ing enable signals generated by the neurons.