W. Maass. Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304, 1997.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....the timing of these pulses has only recently been recognized as a possible means of neuronal information coding. As the biological evidence has been mounting, e.g. 1] it has been shown theoretically that coding with the timing of single spikes allows for powerful neuronal information processing [2]. Furthermore, it has been argued that coordinated spike timing could be instrumental in solving dynamic combinatorial problems [3] Since time coding utilizes only a single spike to transfer information, as apposed to hundreds in firing rate coding, the paradigm could also potentially be ....

W. Maass, "Fast sigmoidal networks via spiking neurons," Neural Comp., vol. 9, no. 2, pp. 279--304, 1997.

....spiking neurons and learning occurs only after the network has reached a stable state of firing. A mechanism of competitive learning and self organization in networks of spiking neurons has also been presented in [Ruf and Schmitt, 1998] The work is based on a spiking neuron model presented in [Maass, 1997a] and implements a mechanism of encoding the signal in the precise timing of the spikes. Experimentally, the model was shown to exhibit the same characteristic behaviour as the standard topology preserving SOM. The learning algorithm is built under the simplifying assumptions of a linear rising ....

Maass, W. (1997a). Fast sigmoidal networks via spiking neurons. Neural Computation, 9(2):279--304.

....probability of correctness. Furthermore, for any deterministic nite automaton one can construct a network of noisy spiking neurons that simulates its computations of any given length with arbitrarily high probability of correctness [70] Similar results have been achieved for analog computations [73], 77] 78] e.g. the noisy spiking networks with temporal encoding of analog values can reliably simulate the feedforward perceptron networks with real inputs and outputs [73] IV. Conclusion and Open Problems We have presented a systematic survey of the known computational properties of ....

.... length with arbitrarily high probability of correctness [70] Similar results have been achieved for analog computations [73] 77] 78] e.g. the noisy spiking networks with temporal encoding of analog values can reliably simulate the feedforward perceptron networks with real inputs and outputs [73]. IV. Conclusion and Open Problems We have presented a systematic survey of the known computational properties of neural network models. While our understanding of these issues has increased dramatically over the past 10 15 years, several intriguing open questions still remain. Among these ....

W. Maass, Fast sigmoidal networks via spiking neurons, Neural Computation 9 (2) (1997) 279-304.

....probability of correctness. Furthermore, for any deterministic nite automaton one can construct a network of noisy spiking neurons that simulates its computations of any given length with arbitrarily high probability of correctness [68] Similar results have been achieved for analog computations [71, 75, 76], e.g. the noisy spiking networks with temporal encoding of analog values can reliably simulate the feedforward perceptron networks with real inputs and outputs [71] 21 ....

.... length with arbitrarily high probability of correctness [68] Similar results have been achieved for analog computations [71, 75, 76] e.g. the noisy spiking networks with temporal encoding of analog values can reliably simulate the feedforward perceptron networks with real inputs and outputs [71]. 21 ....

W. Maass, Fast sigmoidal networks via spiking neurons, Neural Computation 9 (2) (1997) 279-304.

.... widely used in silicon implementations of networks of spiking neurons such as described by Murray and Tarassenko [13] Theoretical investigations concerning the computational power of networks of a neuron type that employs more complex functions as postsynaptic potentials can be found in Maass [5, 6, 7]. A spiking neuron may be viewed as a digital or analog computational element, depending on the type of temporal coding that is used. In the following we restrict our analysis to the coding of binary values. For this binary coding we assume that input neuron a i fires at time 0 if it encodes a 1, ....

W. Maass, Fast sigmoidal networks via spiking neurons, Neural Computation 9 (1997) 279--304.

....or sigmoidal gates with respect to time and network complexity [4] The issue of learning was raised in [7] where it was shown that supervised learning is possible in such SNNs. In this paper we investigate unsupervised learning processes in SNNs. On the basis of a construction introduced in [8] we show how competitive learning can be performed by SNNs using temporal coding. We extend this idea to a learning mechanism that is closely related to one of the most successful paradigms of unsupervised learning: the self organizing map (SOM) by Kohonen [9] Topology preserving maps have been ....

....can be achieved. In Section III we propose the mechanism for unsupervised learning in networks of spiking neurons. The simulation results are summarized in Section IV. An extended abstract of this article was presented in [15] II. Computing with Spiking Neurons Recently Maass has shown in [8] how leaky integrate and fire neurons can compute weighted sums in temporal coding, where the firing time of a neuron encodes a value in the sense that an early firing of the neuron represents a large value. More precisely, one considers a neuron v which receives excitatory input from m neurons u ....

[Article contains additional citation context not shown here]

W. Maass, "Fast sigmoidal networks via spiking neurons," Neural Computation, vol. 9, pp. 279--304, 1997.

....[6] This might indicate that for a neuron, which receives stimuli through several of its synapses, the onset of those stimuli encodes the relevant information. In a theoretical work, Maass has recently shown how one can compute functions with leaky integrate and fire neurons in temporal coding [4] assuming a linear initial segment of the excitatory postsynaptic potentials (EPSP s) This computation is based on the idea that one can make a neuron fire such that its firing time represents the weighted sum of its analogue valued inputs, given by the firing times of its input neurons. In ....

....gives rise to an easy possibility of computing arbitrary bounded continuous functions within temporal coding. 2 Computing linear functions Assuming that the initial segments of the EPSP s are linear, leaky integrate andfire neurons can compute linear functions in the following way, as shown in [4]: We consider a neuron v, which receives excitatory input from neurons u 1 ; un . The corresponding weights w i and input values s i 2 [0; T in ] 1 i n are given, with T in being a sufficiently small constant. The goal is to make v fire at a time which is determined by P w i s i . We ....

[Article contains additional citation context not shown here]

Maass, W.: Fast sigmoidal networks via spiking neurons. To appear in Neural Computation.

....coding scheme, have recently been shown to be computationally more powerful than networks consisting of threshold or sigmoidal gates with respect to time and network complexity [9] In this paper we investigate unsupervised learning processes in SNNs. On the basis of a construction introduced in [8] we show how competitive learning can be performed by SNNs using temporal coding. We extend this idea to a learning mechanism that is closely related to one of the most successful paradigms of unsupervised learning: the self organizing map (SOM) by Kohonen [6] Topology preserving maps have been ....

....is organized as follows: In Section 2 we introduce the formal model of a spiking neuron. In Section 3 we propose the mechanism for unsupervised learning in networks of these model neurons. The simulation results are presented in Section 4. 2 The Spiking Neuron Model Recently Maass has shown in [8] how leaky integrate and fire neurons can compute weighted sums in temporal coding, where the firing time of a neuron encodes a value in the sense that an early firing of the neuron represents a large value. More precisely, one considers a neuron v which receives excitatory input from m neurons u ....

[Article contains additional citation context not shown here]

Maass, W.: Fast sigmoidal networks via spiking neurons. Neural Computation 9 (1997) 279--304.

.... for the postsynaptic neurons [Abbott et al. 1997] One approach for explaining the possibility of fast analog computation relies on the assumption that the relevant analog variables are encoded in small temporal differences among the firing times of neurons [Thorpe et al. 1996, Hopfield, 1995, Maass, 1997] These models are able to explain the possibility of fast analog compu tation in networks where neuronal firing and synaptic transmission is highly reliable, or where the average firing times of pools of neurons encode analog variables on a time scale of a few ms s. Although some evidence for ....

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279-304.

....formal weights Oij defined by Oij . A Oij are automatically normalized: by the definition of A they satisfy Cqj 1. Such automatic weight normalization may be desirable in jE P4 some situations [Haefliger et al. 1997] One can circumvent it by employing an auxiliary input neuron (see [Maass, 1997a] In this way one can compute an arbitrary given weighted sum ci; x; in temporal cod jeT ing by a spiking neuron. Note that in this construction the analog output ci; x; is encoded in exactly the same way as the analog inputs x; 2.4.4 Universal Approximation of Continuous Functions with ....

....a first layer of sigmoidal gates) can be used as inputs for another layer of spiking neurons, simulating another layer of sigmoidal gates. Hence on the basis of the assumption that the initial segments of response functions ei; s) are linear one can show with a rigorous mathematical proof [Maass, 1997a] Theorem 2.3 Any feedforward or recurrent analog neural net (for example any multilayer perceptron) consisting of s sigmoidal neurons that employ the gain function sat, can be simulated arbitrarily closely by a network of s spiking neurons (where is a small constant) with analog inputs and ....

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279-304.

....of neuron i, and it does so even more if the intermediate synaptic weight w ij has a large value. This can be used to compute a weighted sum in the temporal domain, and yields in combination with a straightforward implementation of a nonlinear activation function via spiking neurons: Theorem 3. 2 [Maass, 1997a] Any continuous function F : 0; 1] n [0; 1] m can be approximated arbitrarily closely by a network of spiking neurons with inputs and outputs encoded by temporal delays (relative to some reference time) Thorpe and his coworkers have proposed to view the order of the firing times t j of ....

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304.

....the computation of an arbitrary given Hopfield network H. As a model for a spiking neuron we take the common model of a leaky integrate and fire neuron with noise, respectively the somewhat more general spike response model of [Gerstner and van Hemmen, 1992] For details on the model see [Maass, 1997]. The only specific assumption that is needed for the construction of SH in Theorem 2.1 is that both the beginning of the rising part of an EPSP and the beginning of the descending part of an IPSP can be described by a linear function. Theorem 2.1 Let H be an arbitrary given Hopfield net with ....

....in SH , we need to make sure that each spiking neuron v i fires at time t i (k 1) k 1) T Gamma c x i (k 1) with j x i (k 1) Gamma x i (k 1)j as small as possible. We achieve this by exploiting in SH a mechanism for computing a weighted sum in temporal coding which is described in [Maass, 1997]. This simulation is based on the observation that in the presence of some other excitation which moves the membrane potential close to the firing threshold, individual EPSP s respectively IPSP s (or volleys of synchronized PSP s) are able to shift the firing time of a neuron. This mechanism is ....

[Article contains additional citation context not shown here]

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304.

.... for the postsynaptic neurons Abbott et al. 1997) One approach for explaining the possibility of fast analog computation relies on the assumption that the relevant analog variables are encoded in small temporal differences among the firing times of neurons Thorpe et al. 1996) Hopfield (1995) Maass (1997). These models are able to explain the possibility of fast analog computation in networks where neuronal firing and synaptic transmission is highly reliable, or where the average firing times of pools of neurons encode analog variables on a time scale of a few ms s. Although some evidence for such ....

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304.

....among neurobiologists whether the sign of a synaptic efficacy w i can change in the course of a learning process. This issue will not be relevant for the results of this paper. The model is a simplified version of a leaky integrate and fire neuron. In contrast to more complex models (see e.g. [5, 10, 15]) it models a pulse as a step function, rather than a continuous function of a similar shape. Pulses of this shape are actually very common in silicon implementations of networks of spiking neurons [14] A spiking neuron may be viewed as a digital or analog computational element, depending on the ....

Maass, W. (1997) Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304.

.... V . Experiments have shown that in vitro biological neurons fire with slightly varying delays in response to repetitions of the same current injection. Only under certain conditions neurons are known to fire in a more reliable manner ( 35] Therefore one also considers noisy spiking neurons ([24], 27] where the difference P v (t) Gamma Theta v (t Gamma t 0 ) just governs the probability that neuron v fires at time t. The choice of the exact firing times is left up to some unknown stochastic processes, and it may for example occur that v does not fire in a time interval I during ....

....where one has been able to record spike trains from many neurons in parallel. However this coding scheme has captured the attention of many researchers because it is very simple and because it is one of very few coding methods that might theoretically be used for very fast neural computation [24], 51] We will discuss computation and learning with this neural code in section 4. There also exists substantial evidence that on a larger time scale statistical correlations between firing times of different neurons encode relevant information ( correlation coding ) see e.g. 10] 21] 40] ....

[Article contains additional citation context not shown here]

W. Maass, "Fast sigmoidal networks via spiking neurons", Neural Computation, vol. 9, pp 279--304, 1997.

.... initial segments in their response functions can perform with a small number of spikes basic operations on analog variables in temporal coding such as addition, subtraction, and multiplication with a constant (see [Maass 96a] Also noisy versions of such networks turn out to be quite powerful ( Maass 97a] In this article we focus on SNNs whose response functions are described by piecewise constant functions (i.e. step functions) This is certainly the simplest type of response functions from a mathematical point of view. In addition, such functions approximate quite well the shape of pulses ....

....results regarding the computational power of models with piecewise constant response functions hold even if there is no noise in the system. This should be contrasted with the positive results for models with piecewise linear response functions that hold even in the presence of noise (see e.g. Maass 97a] Another important component of the common model for a biological neuron is its threshold function. Whereas a threshold gate has a static threshold, the firing threshold of a biological neuron varies over time in dependency of its recent firing history (hence we refer to it as a threshold ....

[Article contains additional citation context not shown here]

W. Maass (1997) Fast sigmoidal networks via spiking neurons. Neural Computation, vol. 9, 279--304.

....among neurobiologists whether the sign of a synaptic efficacy w i can change in the course of a learning process. This issue will not be relevant for the results of this article. The model is a simple version of a leaky integrate andfire neuron. In contrast to more complex models (see e.g. [19, 7, 13]) it models a pulse as a step function, rather than a continuous function of a similar shape. Pulses of this shape are actually very common in silicon implementations of networks of spiking neurons [17] A spiking neuron of this type was called a spiking neuron of type A in [14] In this article ....

W. Maass. Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304, 1997.

.... Gamma t j ) at the soma of neuron i. c) State variable u i (t) and resulting firing time t i of neuron i. P j2 Gamma i ff ij = 1. Such automatic weight normalization may be desirable in some situations [Haefliger et al. 1997] One can circumvent it by employing an auxiliary input neuron (see [Maass, 1997a] In this way one can compute an arbitrary given weighted sum P j2T i ff ij Delta x j in temporal coding by a spiking neuron. Note that in this construction the analog output P j2 Gamma i ff ij Delta x j is encoded in exactly the same way as the analog inputs x j . xii Wolfgang Maass ....

....a first layer of sigmoidal gates) can be used as inputs for another layer of spiking neurons, simulating another layer of sigmoidal gates. Hence on the basis of the assumption that the initial segments of response functions ffl ij (s) are linear one can show with a rigorous mathematical proof [Maass, 1997a] Theorem 1.3 Any feedforward or recurrent analog neural net (for example any multilayer perceptron) consisting of s sigmoidal neurons that employ the gain function sat, can be simulated arbitrarily closely by a network of s c spiking neurons (where c is a small constant) with analog inputs ....

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304.

No context found.

W. Maass. Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304, 1997.

No context found.

Maass, W. (1997). Fast sigmoidal networks via spiking neurons. Neural Computation, 9, 279--304.

No context found.

Wolfgang Maass. Fast sigmoidal networks via spiking neurons, neural computation. Neural Computation, 3:105-119, 1997.

No context found.

W. Maass, Fast sigmoidal networks via spiking neurons, Neural Computation 9 (1997), 279--304.

No context found.

Wolfgang Maass. Fast sigmoidal networks via spiking neurons, neural computation. Neural Computation, 3:105--119, 1997.

No context found.

W. Maass. Fast sigmoidal networks via spiking neurons. Neural Computation, 9:279--304, 1997.

No context found.

Maass, W. (1997). Fast sigmoidal net works via spiking neurons. Neural Comput, 9(2), 279-304.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC