norse.torch
Contents
norse.torch¶
Building blocks for spiking neural networks based on PyTorch.
norse.torch
Containers¶
Lift applies a given torch.nn.Module over 

A sequential model that works like PyTorch's 

A regularisation cell that accumulates some state (for instance number of spikes) for each forward step, which can later be applied to a loss term. 
Encoding¶
Encodes input currents as fixed (constant) voltage currents, and simulates the spikes that occur during a number of timesteps/iterations (seq_length). 

Encodes a tensor of input values, which are assumed to be in the range [0,1] into a tensor of one dimension higher of binary values, which represent input spikes. 

Encodes a tensor of input values, which are assumed to be in the range [0,1] into a tensor of binary values, which represent input spikes. 

Encodes a set of input values into population codes, such that each singular input value is represented by a list of numbers (typically calculated by a radial basis kernel), whose length is equal to the out_features. 

Encodes a tensor of input values, which are assumed to be in the range [1,1] into a tensor of one dimension higher of values in {1,0,1}, which represent signed input spikes. 

For all neurons, remove all but the first spike. 

Encodes an input value by the time the first spike occurs. 
Convolutions¶
Implements a 2dconvolution applied pointwise in time. 
Neuron models¶
Integrateandfire¶
Simple integrators that sums up incoming signals until a threshold.
State of a feed forward integrateandfire neuron 

Parametrization of an integrateandfire neuron 

Izhikevich¶
State of a Izhikevich neuron 

Spiking behavior of a Izhikevich neuron 

A neuron layer that wraps a 

Module that computes a single Izhikevich neuronmodel without recurrence and without time. 

A neuron layer that wraps a 

Module that computes a single eulerintegration step of an Izhikevich neuronmodel with recurrence but without time. 
Leaky integrator¶
Leaky integrators describe a leaky neuron membrane that integrates incoming currents over time, but never spikes. In other words, the neuron adds up incoming input current, while leaking out some of it in every timestep.
The first equation describes how the membrane voltage (\(v\), across the membrane) changes over time. A constant amount of current is leaked out every timestep (\(v_{\text{leak}}\)), while the current (\(i\)) is added.
The second equation describes how the current flowing into the neuron changes in every timestep.
Notice that both equations are parameterized by the time constant \(\tau\). This constant controls how fast the changes in voltage and current occurs. A large time constant means a small change. In Norse, we call this parameter the inverse to avoid having to recalculate the inverse (\(\tau_{\text{mem_inv}}\) and \(\tau_{\text{syn_inv}}\) respectively). So, for Norse a large inverse time constant means rapid changes while a small inverse time constant means slow changes.
Recall that voltage is the difference in charge between two points (in this case the neuron membrane) and current is the rate of change or the amount of current being added/subtracted at each timestep.
More information can be found on Wikipedia.
State of a leakyintegrator 

Parameters of a leaky integrator 

A neuron layer that wraps a leakyintegrator 

Cell for a leakyintegrator without recurrence. 

Cell for a leakyintegrator with an additional linear weighting. 
Leaky integrateandfire (LIF)¶
A popular neuron model that combines a norse.torch.functional.leaky_integrator
with
spike thresholds to produce events (spikes).
The model describes the change in a neuron membrane voltage (\(v\))
and inflow current (\(i\)).
See the leaky_integrator
module for more information.
The F in LIF stands for the thresholded “firing” events that occur if the neuron voltage increases over a certain point or threshold (\(v_{\text{th}}\)).
In regular artificial neural networks, this is referred to as the activation
function. The behaviour can be controlled by setting the method
field in
the neuron parameters, but will default to the superspike
synthetic
gradient approach that uses the heaviside
step function:
More information can be found on Wikipedia or in the book *Neuron Dynamics* by W. Gerstner et al., freely available online.
Parametrization of a LIF neuron 

State of a LIF neuron 

A neuron layer that wraps a 

Module that computes a single eulerintegration step of a leaky integrateandfire (LIF) neuronmodel without recurrence and without time. 

A neuron layer that wraps a 

Module that computes a single eulerintegration step of a leaky integrateandfire (LIF) neuronmodel with recurrence but without time. 
LIF, box model¶
A simplified version of the popular leaky integrateandfire neuron model that combines a norse.torch.functional.leaky_integrator
with spike thresholds to produce events (spikes).
Compared to the norse.torch.functional.lif
modules, this model leaves out the current term, making it computationally simpler but impossible to implement in physical systems because currents cannot “jump” in nature.
It is these sudden current jumps that gives the model its name, because the shift in current is instantaneous and can be drawn as “current boxes”.
State of a feed forward LIF neuron 

Parametrization of a boxed LIF neuron 

Computes a single eulerintegration step for a lif neuronmodel without current terms. 
LIF, conductance based¶
Module that computes a single eulerintegration step of a conductance based LIF neuronmodel. 
LIF, adaptive exponential¶
A neuron layer that wraps a recurrent LIFAdExCell in time such that the layer keeps track of temporal sequences of spikes. After application, the layer returns a tuple containing (spikes from all timesteps, state from the last timestep). 

Computes a single eulerintegration step of a feedforward exponential LIF neuronmodel without recurrence, adapted from http://www.scholarpedia.org/article/Adaptive_exponential_integrateandfire_model. 

A neuron layer that wraps a recurrent LIFAdExRecurrentCell in time (with recurrence) such that the layer keeps track of temporal sequences of spikes. After application, the layer returns a tuple containing (spikes from all timesteps, state from the last timestep). 

Computes a single of eulerintegration step of a recurrent adaptive exponential LIF neuronmodel with recurrence, adapted from http://www.scholarpedia.org/article/Adaptive_exponential_integrateandfire_model. 
LIF, exponential¶
A neuron layer that wraps a 

Computes a single eulerintegration step of a recurrent exponential LIF neuronmodel (without recurrence) adapted from https://neuronaldynamics.epfl.ch/online/Ch5.S2.html. 

A neuron layer that wraps a 

Computes a single eulerintegration step of a recurrent exponential LIFEx neuronmodel (with recurrence) adapted from https://neuronaldynamics.epfl.ch/online/Ch5.S2.html. 
LIF, multicompartmental¶
Computes a single eulerintegration step of a LIF multicompartment neuronmodel. 
LIF, refractory¶
Module that computes a single eulerintegration step of a LIF neuronmodel with absolute refractory period without recurrence. 

Module that computes a single eulerintegration step of a LIF neuronmodel with absolute refractory period. 
Long shortterm memory (LSNN)¶
A Long shortterm memory neuron module without recurrence adapted from https://arxiv.org/abs/1803.09574 

Euler integration cell for LIF Neuron with threshold adaptation without recurrence. 

A Long shortterm memory neuron module wit recurrence adapted from https://arxiv.org/abs/1803.09574 

Module that computes a single eulerintegration step of a LSNN neuronmodel with recurrence. 