Hartung-Gorre Verlag
Inh.: Dr.
Renate Gorre D-78465
Konstanz Fon: +49 (0)7533 97227 Fax: +49 (0)7533 97228 www.hartung-gorre.de
|
S
|
Series in Signal and Information Processing, Vol. 35
edited by Hans-Andrea Loeliger
Patrick Murer
A New Perspective on Memorization in
Recurrent Networks of Spiking Neurons
1st Edition 2022. XVIII, 206 pages, € 64,00.
ISBN 978-3-86628-758-7
Abstract
This thesis
studies the capability of spiking recurrent neural network models to memorize
dynamical pulse patterns (or firing signals).
In the first
part, discrete-time firing signals (or firing sequences) are considered. A
recurrent network model, consisting of neurons with bounded disturbance, is
introduced to analyze (simple) local learning. Two modes of
learning/memorization are considered: The first mode is strictly online, with a
single pass through the data, while the second mode uses multiple passes
through the data. In both modes, the learning is strictly local (quasi-Hebbian): At any given time step, only the weights between
the neurons firing (or supposed to be firing) at the previous time step and
those firing (or supposed to be firing) at the present time step are modified.
The main result is an upper bound on the probability that the single-pass
memorization is not perfect. It follows that the memorization capacity in this
mode asymptotically scales like that of the classical Hopfield model (which, in
contrast, memorizes static patterns). However, multiple-rounds memorization is
shown to achieve a higher capacity with an asymptotically nonvanishing
number of bits per connection/synapse. These mathematical findings may be
helpful for understanding the functionality of short-term memory and long-term
memory in neuroscience.
In the second
part, firing signals in continuous-time are studied. It is shown how firing
signals, containing firings only on a regular time grid, can be (robustly)
memorized with a recurrent network model. In principle, the corresponding
weights are obtained by supervised (quasi-Hebbian)
multi-pass learning. The asymptotic memorization capacity is a nonvanishing number measured in bits per connection/synapse
as its discrete-time analogon. Furthermore, the
timing robustness of the memorized firing signals is investigated for different
disturbance models.
The regime of
disturbances, where the relative occurrence-time of the firings is preserved
over a long time span, is elaborated for the various disturbance models. The
proposed models have the potential for energy efficient self-timed neuromorphic
hardware implementations.
Keywords: Spiking recurrent neural network; dynamical pulse
pattern; associative memory; Hopfield model; quasi-Hebbian
learning; online learning; multi-pass memorization; asymptotic memorization
capacity; timing robustness.
Series / Reihe "Series in
Signal and Information Processing" im Hartung-Gorre Verlag
Direkt bestellen bei / to order directly from