Spiking neural network

Also known as: SNN

On this page 7

A spiking neural network (SNN) is a type of artificial neural network in which neurons exchange information through discrete, timed electrical events called spikes rather than continuous activation values. Each neuron accumulates input over time in an internal membrane potential and emits a spike only when that potential crosses a firing threshold, which makes computation sparse and event-driven.

Among neural networks, SNNs are often called the third generation of models, after the perceptron and conventional networks with continuous activation functions. They are closer to biological neurons than standard deep learning models, and they are the native workload of neuromorphic hardware, where their sparse activity can reduce energy consumption substantially for suitable tasks.

How spiking neurons process information

A spiking neuron integrates incoming spikes over time and fires when its internal state exceeds a threshold. The most widely used abstraction is the leaky integrate-and-fire (LIF) neuron: each incoming spike adds a weighted contribution to the membrane potential, the potential leaks back toward a resting value with a time constant, and after firing the neuron resets and may enter a short refractory period.

Because the state evolves in time, an SNN is inherently a dynamical system with memory, comparable in some ways to a recurrent neural network. Simulations usually advance in discrete time steps of roughly 0.1 to 1 millisecond, while event-driven simulators and neuromorphic chips update a neuron only when a spike actually arrives.

The following table summarizes the main parameters of a LIF neuron and what each one controls.

ParameterTypical roleEffect when increased
Membrane time constantSets how fast the potential leaksLonger memory of past inputs
Firing thresholdPotential level that triggers a spikeFewer spikes, sparser activity
Reset potentialValue after a spike is emittedHigher reset gives faster refiring
Refractory periodTime a neuron cannot fire againCaps the maximum firing rate
Synaptic weightStrength of each incoming spikeStronger influence on the target neuron
Key parameters of a leaky integrate-and-fire neuron

Neuron models and spike coding schemes

SNN designers choose between neuron models that trade biological realism against computational cost. Simple models suit large networks and hardware, while detailed models serve neuroscience research.

ModelRealismTypical use
Integrate-and-fireLowTheory and very large simulations
Leaky integrate-and-fireModerateMachine learning and neuromorphic chips
Izhikevich modelHigh dynamics, low costReproducing many firing patterns
Hodgkin-Huxley modelVery highBiophysical neuroscience studies
Common spiking neuron models

Information must also be encoded into spikes and decoded from them. The choice of spike coding strongly affects latency, accuracy and energy, as the next table shows.

Coding schemeHow information is carriedTrade-off
Rate codingNumber of spikes in a time windowRobust but needs many time steps
Temporal codingExact timing of individual spikesFast and sparse but noise-sensitive
Time-to-first-spikeLatency of the earliest spikeVery few spikes, harder training
Population codingPattern across a group of neuronsRich representation, more neurons
Delta or event codingSpikes only on input changeIdeal for event cameras and sensors
Spike coding schemes compared

Training spiking neural networks

Training an SNN is difficult because the spike function is a step, and its derivative is zero almost everywhere, so standard backpropagation cannot pass useful gradients through it. Three main families of methods address this problem.

  • Surrogate gradient learning replaces the step derivative with a smooth approximation during the backward pass and trains the network with backpropagation through time. It is the dominant method for task accuracy today.
  • ANN-to-SNN conversion trains a conventional network with ReLU activations and maps its activations to firing rates. It reaches high accuracy but usually needs many time steps, which erodes latency and energy gains.
  • Spike-timing-dependent plasticity (STDP) is a local, biologically inspired rule that strengthens a synapse when the input spike precedes the output spike. It supports unsupervised and on-chip learning but lags on complex benchmarks.

Surrogate gradient methods have narrowed the accuracy gap considerably on image and audio benchmarks, yet SNNs still generally trail conventional deep networks on large-scale tasks. Training is also slower, because gradients must be unrolled across every simulated time step.

Spiking networks compared with conventional neural networks

The essential difference is that conventional networks compute dense continuous activations in every layer at every step, while SNNs compute sparse binary events over time. This changes the hardware, the energy profile and the kinds of data that suit each model.

AspectSpiking neural networkConventional neural network
Signal typeBinary spikes with timingContinuous real values
ComputationEvent-driven, sparse accumulateDense multiply-accumulate
Time handlingBuilt into neuron dynamicsAdded through recurrence or attention
Training maturityActive research, fewer toolsMature frameworks and practices
Best hardwareNeuromorphic processorsGPUs and tensor accelerators
Energy profileLow when activity is sparseHigh but predictable throughput
SNN and conventional artificial neural network

The energy benefit is real only under specific conditions. On a GPU, an SNN simulation is often slower and more power-hungry than an equivalent conventional network, because GPUs are optimized for dense matrix operations. Gains appear on neuromorphic chips such as Intel Loihi or IBM TrueNorth, and when firing rates stay low.

Applications and practical limitations

SNNs are most useful where power budgets are tight and inputs arrive as events. Typical applications include keyword spotting on battery devices, gesture recognition from event-based cameras, robotic control loops, processing of biomedical signals and brain-computer interfaces.

Several limitations restrain wider adoption. Neuromorphic hardware remains scarce, software tools are less mature than mainstream deep learning frameworks, and benchmark comparisons often disagree because energy measurements depend on chip, firing rate and number of time steps.

  • Assuming automatic energy savings: savings disappear when firing rates are high or the network runs on a GPU.
  • Using too many time steps: latency and power grow linearly with simulation length.
  • Ignoring input encoding: a poor coding scheme can waste most of the benefit of sparsity.
  • Comparing unequal baselines: fair evaluation measures accuracy, latency and energy on the same task.

The idea in brief

Most AI models work like a room where everyone talks all the time. A spiking neural network works more like a room where people speak only when they have something new to say. Each artificial neuron quietly collects signals and sends a short pulse only when enough has built up.

Because most neurons stay silent most of the time, the network can do useful work while using far less electricity, especially on special chips designed to mimic the brain.

A practical scenario

An engineering team designs a wearable device that must recognize a spoken wake word while running for months on a small battery. A conventional network running continuously would drain the battery too quickly, so the team builds a compact spiking neural network with a few hundred thousand synapses.

Audio is converted into spikes with a delta coding scheme, so silence produces almost no activity. The network is trained offline with surrogate gradients and then deployed on a low-power neuromorphic processor that updates neurons only when spikes arrive.

In field tests, the detector keeps recognition accuracy close to that of a conventional model while drawing power in the sub-milliwatt range during quiet periods. The device wakes its main processor only when the keyword is detected, which extends battery life considerably.