norse.torch.functional

Encoding

constant_current_lif_encode	Encodes input currents as fixed (constant) voltage currents, and simulates the spikes that occur during a number of timesteps/iterations (seq_length).
gaussian_rbf	A gaussian radial basis kernel that calculates the radial basis given a distance value (distance between \(x\) and a data value \(x'\), or \(\|\mathbf{x} - \mathbf{x'}\|^2\) below).
euclidean_distance	Simple euclidean distance metric.
population_encode	Encodes a set of input values into population codes, such that each singular input value is represented by a list of numbers (typically calculated by a radial basis kernel), whose length is equal to the out_features.
poisson_encode	Encodes a tensor of input values, which are assumed to be in the range [0,1] into a tensor of one dimension higher of binary values, which represent input spikes.
poisson_encode_step	Encodes a tensor of input values, which are assumed to be in the range [0,1] into a tensor of binary values, which represent input spikes.
signed_poisson_encode	Encodes a tensor of input values, which are assumed to be in the range [-1,1] into a tensor of one dimension higher of binary values, which represent input spikes.
signed_poisson_encode_step	Creates a poisson distributed signed spike vector, when
spike_latency_lif_encode	Encodes an input value by the time the first spike occurs.
spike_latency_encode	For all neurons, remove all but the first spike.
lif_current_encoder	Computes a single euler-integration step of a leaky integrator.
lif_adex_current_encoder	Computes a single euler-integration step of an adaptive exponential LIF neuron-model adapted from http://www.scholarpedia.org/article/Adaptive_exponential_integrate-and-fire_model.
lif_ex_current_encoder	Computes a single euler-integration step of a leaky integrator adapted from https://neuronaldynamics.epfl.ch/online/Ch5.S2.html.

Logical

logical_and	Computes a logical and provided x and y are bitvectors.
logical_xor	Computes a logical xor provided x and y are bitvectors.
logical_or	Computes a logical or provided x and y are bitvectors.
muller_c	Computes the muller-c element next state provided x_1 and x_2 are bitvectors and y_prev is the previous state.
posedge_detector	Determines whether a transition from 0 to 1 has occured providing that z and z_prev are bitvectors

Regularization

regularize_step	Takes one step for a regularizer that aggregates some information (based on the spike_accumulator function), which is pushed forward and returned for future inclusion in an error term.
spike_accumulator	A spike accumulator that aggregates spikes and returns the total sum as an integer.
voltage_accumulator	A spike accumulator that aggregates membrane potentials over time.

Threshold functions

heaviside	A heaviside step function that truncates numbers <= 0 to 0 and everything else to 1.
heavi_erfc_fn
heavi_tanh_fn
logistic_fn
heavi_circ_fn
circ_dist_fn
triangle_fn
super_fn

Temporal operations

lift	Creates a lifted version of the given activation function which applies the activation function in the temporal domain.

Neuron models

Integrate-and-fire (IAF)

IAFParameters	Parametrization of an integrate-and-fire neuron
IAFFeedForwardState	State of a feed forward integrate-and-fire neuron
iaf_feed_forward_step	Feedforward step of an integrate-and-fire neuron, computing a single step

Izhikevich

IzhikevichParameters	Parametrization of av Izhikevich neuron
IzhikevichSpikingBehavior	Spiking behavior of a Izhikevich neuron
tonic_spiking	Spiking behavior of a Izhikevich neuron
tonic_bursting	Spiking behavior of a Izhikevich neuron
phasic_spiking	Spiking behavior of a Izhikevich neuron
phasic_bursting	Spiking behavior of a Izhikevich neuron
mixed_mode	Spiking behavior of a Izhikevich neuron
spike_frequency_adaptation	Spiking behavior of a Izhikevich neuron
class_1_exc	Spiking behavior of a Izhikevich neuron
class_2_exc	Spiking behavior of a Izhikevich neuron
spike_latency	Spiking behavior of a Izhikevich neuron
subthreshold_oscillation	Spiking behavior of a Izhikevich neuron
resonator	Spiking behavior of a Izhikevich neuron
izhikevich_feed_forward_step

Leaky integrator

Leaky integrators describe a leaky neuron membrane that integrates incoming currents over time, but never spikes. In other words, the neuron adds up incoming input current, while leaking out some of it in every timestep.

\[\begin{split}\begin{align*} \dot{v} &= 1/\tau_{\text{mem}} (v_{\text{leak}} - v + i) \\ \dot{i} &= -1/\tau_{\text{syn}} i \end{align*}\end{split}\]

The first equation describes how the membrane voltage (\(v\), across the membrane) changes over time. A constant amount of current is leaked out every timestep (\(v_{\text{leak}}\)), while the current (\(i\)) is added.

The second equation describes how the current flowing into the neuron changes in every timestep.

Notice that both equations are parameterized by the time constant \(\tau\). This constant controls how fast the changes in voltage and current occurs. A large time constant means a small change. In Norse, we call this parameter the inverse to avoid having to recalculate the inverse (\(\tau_{\text{mem_inv}}\) and \(\tau_{\text{syn_inv}}\) respectively). So, for Norse a large inverse time constant means rapid changes while a small inverse time constant means slow changes.

Recall that voltage is the difference in charge between two points (in this case the neuron membrane) and current is the rate of change or the amount of current being added/subtracted at each timestep.

More information can be found on Wikipedia.

LIParameters	Parameters of a leaky integrator
LIState	State of a leaky-integrator
li_feed_forward_step

Leaky integrate-and-fire (LIF)

A popular neuron model that combines a norse.torch.functional.leaky_integrator with spike thresholds to produce events (spikes).

The model describes the change in a neuron membrane voltage (\(v\)) and inflow current (\(i\)). See the leaky_integrator module for more information.

\[\begin{split}\begin{align*} \dot{v} &= 1/\tau_{\text{mem}} (v_{\text{leak}} - v + i) \\ \dot{i} &= 1/\tau_{\text{syn}} i \end{align*}\end{split}\]

The F in LIF stands for the thresholded “firing” events that occur if the neuron voltage increases over a certain point or threshold (\(v_{\text{th}}\)).

\[z = \Theta(v - v_{\text{th}})\]

In regular artificial neural networks, this is referred to as the activation function. The behaviour can be controlled by setting the method field in the neuron parameters, but will default to the superspike synthetic gradient approach that uses the heaviside step function:

\[\begin{split}H[n]=\begin{cases} 0, & n <= 0 \\ 1, & n \gt 0 \end{cases}\end{split}\]

More information can be found on Wikipedia or in the book *Neuron Dynamics* by W. Gerstner et al., freely available online.

LIFParameters	Parametrization of a LIF neuron
LIFFeedForwardState	State of a feed forward LIF neuron
lif_feed_forward_step	Computes a single euler-integration step for a lif neuron-model.
lif_feed_forward_adjoint_step	Implementes a single euler forward and adjoint backward step of a leaky integrate and fire neuron with current based exponential synapses.
lif_feed_forward_adjoint_step_sparse	Implementes a single euler forward and adjoint backward step of a leaky integrate and fire neuron with current based exponential synapses.

LIF, box model

A simplified version of the popular leaky integrate-and-fire neuron model that combines a norse.torch.functional.leaky_integrator with spike thresholds to produce events (spikes). Compared to the norse.torch.functional.lif modules, this model leaves out the current term, making it computationally simpler but impossible to implement in physical systems because currents cannot “jump” in nature. It is these sudden current jumps that gives the model its name, because the shift in current is instantaneous and can be drawn as “current boxes”.

LIFBoxFeedForwardState	State of a feed forward LIF neuron
LIFBoxParameters	Parametrization of a boxed LIF neuron
lif_box_feed_forward_step	Computes a single euler-integration step for a lif neuron-model without current terms.

LIF, conductance based

CobaLIFParameters	Parameters of conductance based LIF neuron.
CobaLIFFeedForwardState	State of a conductance based feed forward LIF neuron.
coba_lif_feed_forward_step	Euler integration step for a conductance based LIF neuron.

LIF, adaptive exponential

LIFAdExParameters	Parametrization of an Adaptive Exponential Leaky Integrate and Fire neuron
LIFAdExFeedForwardState	State of a feed forward LIFAdEx neuron
lif_adex_feed_forward_step	Computes a single euler-integration step of an adaptive exponential LIF neuron-model adapted from http://www.scholarpedia.org/article/Adaptive_exponential_integrate-and-fire_model.
lif_adex_current_encoder	Computes a single euler-integration step of an adaptive exponential LIF neuron-model adapted from http://www.scholarpedia.org/article/Adaptive_exponential_integrate-and-fire_model.

LIF, exponential

LIFExParameters	Parametrization of an Exponential Leaky Integrate and Fire neuron
LIFExFeedForwardState	State of a feed forward LIFEx neuron
lif_ex_feed_forward_step	Computes a single euler-integration step of an exponential LIF neuron-model adapted from https://neuronaldynamics.epfl.ch/online/Ch5.S2.html.
lif_ex_current_encoder	Computes a single euler-integration step of a leaky integrator adapted from https://neuronaldynamics.epfl.ch/online/Ch5.S2.html.

LIF, multicompartmental (MC)

lif_mc_feed_forward_step	Computes a single euler-integration feed forward step of a LIF multi-compartment neuron-model.
lif_mc_refrac_feed_forward_step

LIF, refractory

LIFRefracParameters	Parameters of a LIF neuron with absolute refractory period.
LIFRefracFeedForwardState	State of a feed forward LIF neuron with absolute refractory period.
lif_refrac_feed_forward_step	Computes a single euler-integration step of a feed forward
lif_refrac_feed_forward_adjoint_step	Implementes a single euler forward and adjoint backward step of a leaky integrate and fire neuron with current based exponential synapses and a refractory period.

Long short-term memory (LSNN)

LSNNParameters	Parameters of an LSNN neuron
LSNNFeedForwardState	Integration state kept for a lsnn module
lsnn_feed_forward_step	Euler integration step for LIF Neuron with threshold adaptation.
lsnn_feed_forward_adjoint_step	Implementes a single euler forward and adjoint backward step of a lif neuron with adaptive threshhold and current based exponential synapses.

Plasticity models

Spike-time dependent plasticity (STDP)

STDPSensorParameters	Parameters of an STDP sensor as it is used for event driven plasticity rules.
STDPSensorState	State of an event driven STDP sensor.
stdp_sensor_step	Event driven STDP rule.

Tsodyks-Markram timing-dependent plasticity (TDP)

TsodyksMakramParameters	Parameters of the Tsodyks-Makram Model
TsodyksMakramState	State of the Tsodyks-Makram Model, note that we are tracking the input current state separately.
stp_step	Euler integration step for Tsodyks Makram model of STP.