Neuromorphic Network-on-Chip Architecture for Spiking Neural Networks

Team

Supervisors

Table of content

  1. Abstract
  2. Related works
  3. Methodology
  4. Experiment Setup and Implementation
  5. Results and Analysis
  6. Conclusion
  7. Publications
  8. Links

Abstract

The present works of AI show interest in trying to replicate some aspect of brain function. This interest stems from the fact that the brain remains functionally superior to any machine built to date despite consuming power equivalent to that of a compact fluorescent light bulb.

Spiking Neural Network (SNN), a type of artificial neural network, draws inspiration from the brain and uses spikes as the basis for communication in an event- driven manner. Spiking hardware, or neuromorphic hardware are used to implement spike based algorithms to realize their efficiency in computing and ability of doing so with low power, compared to non- spiking hardware. Neuromorphic architecture focuses on developing neuron models inspired by biological neurons and creating hardware implementations of artificial neurons.

Our research presents the design of a scalable and configurable Network-on-Chip (NoC) based on the RISC-V instruction set architecture (ISA) which allows for hardware- level processing of spiking neural networks, and the implementation of the design on a FPGA. RISC-V was chosen as the base ISA since it is not only highly practical and popular, but also completely open source and amenable to custom extensions.

The main goal of this research is to optimize the RISC-V processing node network for low power by integrating custom accelerator hardware units.

A. Spiking Neural Networks

Spiking neural networks fall in the category of artificial neural networks. Several layers of neurons connected by synapses make up an ANN. SNNs follow more complex design rules and represent a newer variant of ANNs, considered to be a third-generation neural network. It resembles the activities of the brain more closely by incorporating time. SNNs transmit spikes as events throughout the neural network. A spike is considered a rapid change of voltage that occurs over a short period of time. When a synapse receives an action potential or a spike from one of its pre-synaptic neurons, it will transmit the spike to its post synaptic neurons. This transmission leads to an update of the postsynaptic neuron’s membrane potential which is influenced by factors such as the weight of the synapses and other neuron parameters.

B. Neuromorphic Architecture Implementations

Neuromorphic research is still highly exploratory, with several proposed implementations aimed at simulating and accelerating SNNs. Specific hardware has been developed to accurately mimic the behaviour of individual neurons and support collections of such neurons to enable accurate and efficient models of SNNs.

Intel’s advanced neuromorphic chip Loihi 1 adopts a variation of the CUBA leaky-integrate-and-fire model. It features a manycore mesh comprising of 128 neuromorphic cores. An asynchronous network-on-chip (NoC) transports all communication between cores as packetized messages. A neuromorphic core implements 1024 spiking neural units. All these units including their fan-in and fan-out connectivity share configuration and state variables in ten erchitectural memories within the neuromorphic core. Such a design allows for a single event to be expanded into a large number of specialised dependent events that can operate in parallel to reduce the neuron update and spike resolution latency.

The OpenSpike project by Modaresi, F. et al. and research by Zhang, J. et al. demonstrate the utilisation of Application-Specific Integrated Circuits (ASICs) in developing neuromorphic architectures. It adopts the leaky integrate-andfire neuron model at the hardware level and uses a timemultiplexed accelerator design. It has also a network implementation for interconnection between hardware neurons. On the other hand, SpiNNaker developed by the Advanced Processor Technologies Research Group (APT) at the Department of Computer Science, University of Manchester focuses more on being large scale and able to simulate the behaviour of aggregates of up to a billion neurons in real time. It utilizes a network of nodes where each node consists of 18 ARM968 cores plus a 128Mbyte off-die Synchronous Dynamic Random Access Memory (SDRAM).

Methodology

Sample Image

The proposed design consists of RISC-V processing nodes specialised to perform operations required to implement spiking networks. The nodes are interconnected using a network interface attached to each processing node. A routing framework is used to negotiate communication between the nodes. Each processing node is responsible for managing one or more logical neurons. The interconnectivity between nodes enables communication between logical neurons that span across multiple nodes. The architecture adopts an event-driven messaging mechanism to effectively emulate the activity of the spiking neurons. The motivation for using a RISC-V processing node network lies in the ability to maintain configurability and programmability at the hardware level. This flexibility will be leveraged to support various spiking neuron model architectures.

Work is underway to develop a RISC-V based system integrated with an accelerator as shown below. The accelerator will perform the intensive task of the continous calculations of potential decaying and along with spike related calculations.