#### DMCA

## Incremental Slow Feature Analysis: Adaptive and Episodic Learning from High-Dimensional Input Streams (2011)

Citations: | 9 - 5 self |

### Citations

4699 |
Self-organizing maps
- Kohonen
- 1997
(Show Context)
Citation Context ...tion provides an ordered set of separate features, which are applied to the original input signal. known unsupervised feature extractors (Abut, 1990; Jolliffe, 1986; Comon, 1994; Lee and Seung, 1999; =-=Kohonen, 2001-=-; Hinton, 2002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Rybicka et al., 2001; Jenkins and Mata... |

3400 |
Principal Component Analysis
- Jolliffe
- 2002
(Show Context)
Citation Context ...ges slowly within the current batch. IncSFA’s plasticity, however, makes it lose sensitivity to such events over time. • Covariance-free. BSFA techniques rely upon batch Principal Component Analysis (=-=Jolliffe, 1986-=-) (PCA), which requires the data’s covariance matrix. Estimating, storing and/or updating covariance matrices can be expensive for high-dimensional data and impractical for open-ended learning. IncSFA... |

1847 |
Independent component analysis, a new concept
- Comon
- 1994
(Show Context)
Citation Context ...s and performing an eigendecomposition provides an ordered set of separate features, which are applied to the original input signal. known unsupervised feature extractors (Abut, 1990; Jolliffe, 1986; =-=Comon, 1994-=-; Lee and Seung, 1999; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Ry... |

1663 |
Learning the parts of objects by nonnegative matrix factorization
- Lee, Seung
- 1999
(Show Context)
Citation Context ...ing an eigendecomposition provides an ordered set of separate features, which are applied to the original input signal. known unsupervised feature extractors (Abut, 1990; Jolliffe, 1986; Comon, 1994; =-=Lee and Seung, 1999-=-; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Rybicka et al., 2001; J... |

1293 |
Probability, random variables and stochastic processes
- Papoulis, Pillai
- 2002
(Show Context)
Citation Context ... true feature. In practice, it may be advisable to add a small amount of noise to the MCA update. But we did not find this to be necessary. As for CCIPCA: If the standard conditions on learning rate (=-=Papoulis et al., 1965-=-) (including convergence at zero), the first stage components will converge to the true PCs, leading to a “nearly-correct” whitening matrix in reasonable time. So, if x is stationary, the slow feature... |

850 | Training products of experts by minimizing contrastive divergence
- Hinton
(Show Context)
Citation Context ...n ordered set of separate features, which are applied to the original input signal. known unsupervised feature extractors (Abut, 1990; Jolliffe, 1986; Comon, 1994; Lee and Seung, 1999; Kohonen, 2001; =-=Hinton, 2002-=-), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Rybicka et al., 2001; Jenkins and Matarić, 2004; Lee... |

719 | The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely moving rat. - O’Keefe, Dostrovsky - 1971 |

567 |
A simplified neuron model as a principal component
- OJA
- 1982
(Show Context)
Citation Context ... CCIPCA. They all build on the work of Amari (1977) and Oja (1982), who showed that a linear neural unit using Hebbian updating could compute the first principal component of a data set (Amari, 1977; =-=Oja, 1982-=-) 1 . However, SGA (1985) builds upon Oja’s earlier work, GHA (1989) builds upon SGA, and CCIPCA (2003) builds upon GHA. 1Much earlier work of a non-neural network flavor had shown how the first PC, i... |

560 |
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
- Dayan, Abbott
- 2001
(Show Context)
Citation Context ... of biologically plausible Hebbian and anti-Hebbian updating. 3.1 Hebbian Updating in CCIPCA Hebbian updates of synaptic strengths of some neuron make it more sensitive to expected input activations (=-=Dayan and Abbott, 2001-=-): v ← v + η g(v, u) u, (17) where u represents pre-synaptic (input) activity, and g post-synaptic activity (a function of similarity between synaptic weights v and input potentials u). The basic Eq. ... |

465 |
Advanced Engineering Mathematics
- Kreyszig
- 1999
(Show Context)
Citation Context ...7 SGA use Gram-Schmidt Orthonormalization (GSO) to incrementally find the subspace of all principal components, but there is no guarantee of finding the components themselves. Sanger used Kreyszig’s (=-=Kreyszig, 1988-=-) (1988) (faster/more effective) residual vector method for computing multiple components. His provably converging GHA used the residual method for simultaneous computation of all components. CCIPCA (... |

409 | Connectionist learning procedures
- Hinton
- 1989
(Show Context)
Citation Context ...ning an informative but slowly-changing feature response over time. The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (=-=Hinton, 1989-=-; Földiák, 1991; Mitchison, 1991; Schmidhuber, 1992a; Bergstra and Bengio, 2009). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees ... |

312 | Learning invariance from transformation sequences
- Foldiak
- 1991
(Show Context)
Citation Context ...ative but slowly-changing feature response over time. The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (Hinton, 1989; =-=Földiák, 1991-=-; Mitchison, 1991; Schmidhuber, 1992a; Bergstra and Bengio, 2009). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees that its soluti... |

291 | Optimal unsupervised learning in a single-layer linear feedforward neural network
- Sanger
- 1989
(Show Context)
Citation Context ...networks that perform incremental PCA and MCA. Well-known incremental PCA algorithms are Oja and Karhunen’s Stochastic Gradient Ascent (SGA) (Oja, 1985), Sanger’s Generalized Hebbian Algorithm (GHA) (=-=Sanger, 1989-=-), and CCIPCA. They all build on the work of Amari (1977) and Oja (1982), who showed that a linear neural unit using Hebbian updating could compute the first principal component of a data set (Amari, ... |

253 | How does a brain build a cognitive code
- Grossberg
- 1980
(Show Context)
Citation Context ... learning, convergence is not desired. Yet by using a learning rate that is always nonzero, the stability of the algorithm is reduced. This corresponds to the well-known stability-plasticity dilemma (=-=Grossberg, 1980-=-).Technical Report No. IDSIA-07-11 14 4.2 Setting Learning Rates In CCIPCA, if η = 1 t , Eq. 10 will be the most efficient estimator4 of the principal component. But a learning rate of 1/t is spatiot... |

240 | Slow feature analysis: unsupervised learning of invariances. Neural Comput 14, 715-70. Chapter 3 Blanz V, Vetter T
- Wiskott, Sejnowski
- 2002
(Show Context)
Citation Context ...uch as high-dimensional video) by informative slow features representing meaningful abstract environmental properties. It can handle cases where batch SFA fails. 1 Introduction Slow feature analysis (=-=Wiskott and Sejnowski, 2002-=-; Wiskott et al., 2011)(SFA) is an unsupervised learning technique that extracts features from an input stream with the objective of maintaining an informative but slowly-changing feature response ove... |

192 |
Microstructure of a spatial map in the entorhinal cortex.
- Hafting, Fyhn, et al.
- 2005
(Show Context)
Citation Context ...IA-07-11 2 view cells from high-dimensional visual input (Franzius et al., 2007) — such representations also exist in biological agents (O’Keefe and Dostrovsky, 1971; Taube et al., 1990; Rolls, 1999; =-=Hafting et al., 2005-=-). There are limitations to existing SFA implementations due to their batch processing nature, which becomes especially apparent when attempting to apply it in somewhat uncontrolled environments. To o... |

189 | Headdirection cells recorded from the postsubiculum in freely moving rats: 1. Description and quantitative analysis.
- Taub, Muller, et al.
- 1990
(Show Context)
Citation Context ... spatialTechnical Report No. IDSIA-07-11 2 view cells from high-dimensional visual input (Franzius et al., 2007) — such representations also exist in biological agents (O’Keefe and Dostrovsky, 1971; =-=Taube et al., 1990-=-; Rolls, 1999; Hafting et al., 2005). There are limitations to existing SFA implementations due to their batch processing nature, which becomes especially apparent when attempting to apply it in somew... |

154 |
Principal components minor components, and linear neural networks
- Oja
- 1992
(Show Context)
Citation Context ... for low-dimensional derivative signals ˙z, CCIPCA cannot replace PCA #2. It will be unstable, since the slow features correspond to the least significant components. Minor Components Analysis (MCA) (=-=Oja, 1992-=-) incrementally extracts the principal components with the smallest eigenvalues. We use Peng’s low complexity updating rule (Peng et al., 2007). Peng proved its convergence even for constant learning ... |

127 |
Redundancy reduction revisited
- Barlow
- 2001
(Show Context)
Citation Context ...vant information such as quickly changing noise assumed to be useless. The compact relevant data encodings reduce the search space for downstream goal-directed learning procedures (Schmidhuber, 1999; =-=Barlow, 2001-=-). As an example, consider a high-dimensional dynamical system: a mobile robot sensing with an onboard camera, where each pixel is considered a separate observation component. SFA will use the video s... |

125 |
On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix.
- Oja
- 1985
(Show Context)
Citation Context ...these choices, we briefly review the literature on neural networks that perform incremental PCA and MCA. Well-known incremental PCA algorithms are Oja and Karhunen’s Stochastic Gradient Ascent (SGA) (=-=Oja, 1985-=-), Sanger’s Generalized Hebbian Algorithm (GHA) (Sanger, 1989), and CCIPCA. They all build on the work of Amari (1977) and Oja (1982), who showed that a linear neural unit using Hebbian updating could... |

118 |
Unsupervised feature learning for audio classification using convolutional deep belief networks,” in
- Lee, Largman, et al.
- 2009
(Show Context)
Citation Context ...002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Rybicka et al., 2001; Jenkins and Matarić, 2004; =-=Lee et al., 2010-=-; Gisslen et al., 2011), thus assuming that the state of the system itself can depend on past information. 2.2 SFA: Formulation SFA’s optimization problem (Wiskott and Sejnowski, 2002; Franzius et al.... |

87 | A spatio-temporal extension to isomap nonlinear dimension reduction,” in
- Jenkins, Mataric
- 2004
(Show Context)
Citation Context ...9; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Rybicka et al., 2001; =-=Jenkins and Matarić, 2004-=-; Lee et al., 2010; Gisslen et al., 2011), thus assuming that the state of the system itself can depend on past information. 2.2 SFA: Formulation SFA’s optimization problem (Wiskott and Sejnowski, 200... |

83 | Candid covariance-free incremental principal component analysis,” Pattern Analysis and Machine Intelligence,
- Weng, Zhang, et al.
- 2003
(Show Context)
Citation Context ...s, and θ contains algorithm memory and parameters, which we will discuss later. To replace PCA #1, IncSFA needs to do online whitening of input x. We use Candid Covariance-Free Incremental (CCI) PCA (=-=Weng et al., 2003-=-). CCIPCA incrementally updates both the eigenvectors and eigenvalues necessary for whitening, and does not keep an estimate of the covariance matrix. CCIPCA is also used to reduce dimensionality. Exc... |

80 | Learning complex, extended sequences using the principle of history compression.
- Schmidhuber
- 1992
(Show Context)
Citation Context ...e response over time. The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (Hinton, 1989; Földiák, 1991; Mitchison, 1991; =-=Schmidhuber, 1992-=-a; Bergstra and Bengio, 2009). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees that its solution methods reliably converge to the ... |

75 | Formal Theory of Creativity, Fun, and Intrinsic Motivation
- Schmidhuber
- 1990
(Show Context)
Citation Context ...s actively create data exhibiting novel but learnable regularities measured byTechnical Report No. IDSIA-07-11 28 improvements of emerging slow features, in line with the formal theory of curiosity (=-=Schmidhuber, 2010-=-). Acknowledgments The experimental paradigm used for the distance-encoding high-dimensional video experiment was first developed by the first author under the supervision of Mathias Franzius, at the ... |

58 | Learning factorial codes by predictability minimization
- Schmidhuber
- 1992
(Show Context)
Citation Context ...e response over time. The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (Hinton, 1989; Földiák, 1991; Mitchison, 1991; =-=Schmidhuber, 1992-=-a; Bergstra and Bengio, 2009). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees that its solution methods reliably converge to the ... |

48 | Slowness and sparseness lead to place, head-direction, and spatial-view cells,” PLoS
- Franzius, Sprekeler, et al.
(Show Context)
Citation Context ...ein et al., 2010; Kompella et al., 2011b), and learning of place-cells, head-direction cells, grid-cells, and spatialTechnical Report No. IDSIA-07-11 2 view cells from high-dimensional visual input (=-=Franzius et al., 2007-=-) — such representations also exist in biological agents (O’Keefe and Dostrovsky, 1971; Taube et al., 1990; Rolls, 1999; Hafting et al., 2005). There are limitations to existing SFA implementations du... |

38 |
Neural theory of association and concept-formation
- Amari
- 1977
(Show Context)
Citation Context ...r, 1989), and CCIPCA. They all build on the work of Amari (1977) and Oja (1982), who showed that a linear neural unit using Hebbian updating could compute the first principal component of a data set (=-=Amari, 1977-=-; Oja, 1982) 1 . However, SGA (1985) builds upon Oja’s earlier work, GHA (1989) builds upon SGA, and CCIPCA (2003) builds upon GHA. 1Much earlier work of a non-neural network flavor had shown how the ... |

37 | Spatial view cells and the representation of place in the primate hippocampus
- Rolls
- 1999
(Show Context)
Citation Context ...eport No. IDSIA-07-11 2 view cells from high-dimensional visual input (Franzius et al., 2007) — such representations also exist in biological agents (O’Keefe and Dostrovsky, 1971; Taube et al., 1990; =-=Rolls, 1999-=-; Hafting et al., 2005). There are limitations to existing SFA implementations due to their batch processing nature, which becomes especially apparent when attempting to apply it in somewhat uncontrol... |

37 | An open-source simulator for cognitive robotics research: The prototype of the icub humanoid robot simulator - Tikhanoff, Cangelosi, et al. - 2008 |

36 |
Removing time variation with the anti-Hebbian differential synapse.
- Mitchison
- 1991
(Show Context)
Citation Context ...y-changing feature response over time. The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (Hinton, 1989; Földiák, 1991; =-=Mitchison, 1991-=-; Schmidhuber, 1992a; Bergstra and Bengio, 2009). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees that its solution methods reliab... |

35 |
Modified Hebbian Learning for Curve and Surface Fitting,
- Xu, Oja, et al.
- 1992
(Show Context)
Citation Context ...eng, 2001) to both eigenvectors and eigenvalues, necessary since whitening requires both. Due to its candidness, potentially difficult learning rate “hand-tuning” is minimized. As for MCA: Xu et al. (=-=Xu et al., 1992-=-) were the first to show that a linear neural unit equipped with anti-Hebbian learning could extract minor components. Oja modified SGA’s updating method to an anti-Hebbian variant (Oja, 1992), and sh... |

29 | Learning unambiguous reduced sequence descriptions
- Schmidhuber
- 1992
(Show Context)
Citation Context ...e response over time. The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (Hinton, 1989; Földiák, 1991; Mitchison, 1991; =-=Schmidhuber, 1992-=-a; Bergstra and Bengio, 2009). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees that its solution methods reliably converge to the ... |

26 |
Optimal in-place learning and the lobe component analysis,” in
- Weng, Zhang
(Show Context)
Citation Context ...ith t, eventually converging to 1/r. Unlike with Peng’s MCA, there is no convergence proof for CCIPCA and this type of learning. Instead, plasticity introduces an expected error that will not vanish (=-=Weng and Zhang, 2006-=-). To see this, note that any component estimate is a weighted sum of all the inputs: where ∑t τ=1 ρ(t) = 1. Then, t∑ v(t) = ρ(t)u(t), (32) τ=1 E‖v(t) − v ∗ ‖ 2 = t∑ ρ 2 (t)E‖u‖ 2 = T∑ τ=1 t=1 ρ 2 (t)... |

23 |
A unified algorithm for principal and minor components extraction. Neural Networks
- Chen, Amari, et al.
- 1998
(Show Context)
Citation Context ...s updating method to an anti-Hebbian variant (Oja, 1992), and showed how it could converge to the MC subspace. Studying the nature of the duality between PC and MC subspaces (Wang and Karhunen, 1996; =-=Chen et al., 1998-=-), Chen, Amari and Lin (Chen et al., 2001) (2001) introduced the sequential addition technique, enabling linear networks to efficiently extract multiple MCs simultaneously. Building upon previous MCA ... |

15 |
Method of stochastic approximation in the determination of the largest eigenvalue of the mathematical expectation of random matrices,” Automat
- Krasulina
- 1970
(Show Context)
Citation Context ..., GHA (1989) builds upon SGA, and CCIPCA (2003) builds upon GHA. 1Much earlier work of a non-neural network flavor had shown how the first PC, including the eigenvalue could be learned incrementally (=-=Krasulina, 1970-=-).Technical Report No. IDSIA-07-11 7 SGA use Gram-Schmidt Orthonormalization (GSO) to incrementally find the subspace of all principal components, but there is no guarantee of finding the components ... |

15 | Reinforcement learning on slow features of high-dimensional input streams
- Legenstein, Wilbert, et al.
- 2010
(Show Context)
Citation Context ...ccess in problems such as extraction of driving forces of a dynamical system (Wiskott, 2003), nonlinear blind source separation (Sprekeler et al., 2010), as a preprocessor for reinforcement learning (=-=Legenstein et al., 2010-=-; Kompella et al., 2011b), and learning of place-cells, head-direction cells, grid-cells, and spatialTechnical Report No. IDSIA-07-11 2 view cells from high-dimensional visual input (Franzius et al.,... |

15 | Estimating driving forces of nonstationary time series with slow feature analysis. arXiv.org e-Print archive,
- Wiskott
- 2003
(Show Context)
Citation Context ...solution methods reliably converge to the best solution, given its constraints (no local minima problem). SFA has shown success in problems such as extraction of driving forces of a dynamical system (=-=Wiskott, 2003-=-), nonlinear blind source separation (Sprekeler et al., 2010), as a preprocessor for reinforcement learning (Legenstein et al., 2010; Kompella et al., 2011b), and learning of place-cells, head-directi... |

14 | Modular toolkit for data processing (mdp): a python data processing framework - Zito, Wilbert, et al. - 2008 |

12 |
A unified neural bigradient algorithm for robust pca and mca. International journal of neural systems
- Wang, Karhunen
- 1996
(Show Context)
Citation Context ...onents. Oja modified SGA’s updating method to an anti-Hebbian variant (Oja, 1992), and showed how it could converge to the MC subspace. Studying the nature of the duality between PC and MC subspaces (=-=Wang and Karhunen, 1996-=-; Chen et al., 1998), Chen, Amari and Lin (Chen et al., 2001) (2001) introduced the sequential addition technique, enabling linear networks to efficiently extract multiple MCs simultaneously. Building... |

9 | Decorrelated Features for Pretraining Complex Cell-like Networks,”
- Bergstra, Bengio
- 2009
(Show Context)
Citation Context .... The idea of using temporal stability as an objective in learning systems has motivated some other unsupervised learning techniques (Hinton, 1989; Földiák, 1991; Mitchison, 1991; Schmidhuber, 1992a; =-=Bergstra and Bengio, 2009-=-). SFA is distinguished by its formulation of the feature extraction problem as an eigensystem problem, which guarantees that its solution methods reliably converge to the best solution, given its con... |

9 | Sequential constant size compressors for reinforcement learning
- Gisslen, Luciw, et al.
- 2011
(Show Context)
Citation Context ... dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; Klapper-Rybicka et al., 2001; Jenkins and Matarić, 2004; Lee et al., 2010; =-=Gisslen et al., 2011-=-), thus assuming that the state of the system itself can depend on past information. 2.2 SFA: Formulation SFA’s optimization problem (Wiskott and Sejnowski, 2002; Franzius et al., 2007) is formally wr... |

8 |
Sequential Extraction of Minor Components
- Chen, Amari, et al.
- 2001
(Show Context)
Citation Context ...t eigenvalues. We use Peng’s low complexity updating rule (Peng et al., 2007). Peng proved its convergence even for constant learning rates—good for open-ended learning. MCA with sequential addition (=-=Chen et al., 2001-=-; Peng and Yi, 2006) will extract multiple slow features in parallel. 3.1 Neural Updating for PC and MC Extraction CCIPCA and Peng’s MCA are the most appropriate incremental PCA and MCA algorithms for... |

7 |
Algorithms for accelerated convergence of adaptive pca
- Chatterjee, Kang, et al.
- 2000
(Show Context)
Citation Context ...t2 = 200, c = 4, r = 5000. The MCA learning rate is η = 0.01. Results of IncSFA are shown in Fig. 5, demonstrating successful adaptation. To measure convergence accuracy, we use the direction cosine (=-=Chatterjee et al., 2000-=-) between the estimated feature w(t) and true (unit length) feature w∗ , DirectionCosine(t) = |wT (t) · w∗ | ‖wT (t)‖ · ‖w∗ , (49) ‖ The direction cosine equals one when the directions align (the feat... |

7 | Incremental slow feature analysis
- Kompella, Luciw, et al.
- 2011
(Show Context)
Citation Context ... extraction of driving forces of a dynamical system (Wiskott, 2003), nonlinear blind source separation (Sprekeler et al., 2010), as a preprocessor for reinforcement learning (Legenstein et al., 2010; =-=Kompella et al., 2011-=-b), and learning of place-cells, head-direction cells, grid-cells, and spatialTechnical Report No. IDSIA-07-11 2 view cells from high-dimensional visual input (Franzius et al., 2007) — such represent... |

7 |
Comparison of two unsupervised neural network models for redundancy reduction
- Lindstädt
- 1993
(Show Context)
Citation Context ...Jolliffe, 1986; Comon, 1994; Lee and Seung, 1999; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; =-=Lindstädt, 1993-=-; Klapper-Rybicka et al., 2001; Jenkins and Matarić, 2004; Lee et al., 2010; Gisslen et al., 2011), thus assuming that the state of the system itself can depend on past information. 2.2 SFA: Formulati... |

7 | Neural predictors for detecting and removing redundant information
- Schmidhuber
- 1999
(Show Context)
Citation Context ...tting rid of irrelevant information such as quickly changing noise assumed to be useless. The compact relevant data encodings reduce the search space for downstream goal-directed learning procedures (=-=Schmidhuber, 1999-=-; Barlow, 2001). As an example, consider a high-dimensional dynamical system: a mobile robot sensing with an onboard camera, where each pixel is considered a separate observation component. SFA will u... |

6 | Autoincsfa and vision-based developmental learning for humanoid robots
- Kompella, Pape, et al.
- 2011
(Show Context)
Citation Context ... extraction of driving forces of a dynamical system (Wiskott, 2003), nonlinear blind source separation (Sprekeler et al., 2010), as a preprocessor for reinforcement learning (Legenstein et al., 2010; =-=Kompella et al., 2011-=-b), and learning of place-cells, head-direction cells, grid-cells, and spatialTechnical Report No. IDSIA-07-11 2 view cells from high-dimensional visual input (Franzius et al., 2007) — such represent... |

6 |
Convergence analysis of a simple minor component analysis algorithm. Neural Netw. 20, 842–850. doi: 10.1016/j.neunet.2007.07.001
- Peng, Yi, et al.
- 2007
(Show Context)
Citation Context ...he least significant components. Minor Components Analysis (MCA) (Oja, 1992) incrementally extracts the principal components with the smallest eigenvalues. We use Peng’s low complexity updating rule (=-=Peng et al., 2007-=-). Peng proved its convergence even for constant learning rates—good for open-ended learning. MCA with sequential addition (Chen et al., 2001; Peng and Yi, 2006) will extract multiple slow features in... |

6 |
Slow Feature Analysis
- Wiskott, Sejnowski
- 2002
(Show Context)
Citation Context ...) by informative slow features representing meaningful abstract environmental properties. It can handle cases where batch SFA fails. 1 Introduction Slow feature analysis (Wiskott and Sejnowski, 2002; =-=Wiskott et al., 2011-=-)(SFA) is an unsupervised learning technique that extracts features from an input stream with the objective of maintaining an informative but slowly-changing feature response over time. The idea of us... |

5 | Unsupervised learning in LSTM recurrent neural networks
- Klapper-Rybicka, Schraudolph, et al.
- 2001
(Show Context)
Citation Context ...omon, 1994; Lee and Seung, 1999; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and 2. Other UL systems that both learn and apply features to sequences (Schmidhuber, 1992a,c,b; Lindstädt, 1993; =-=Klapper-Rybicka et al., 2001-=-; Jenkins and Matarić, 2004; Lee et al., 2010; Gisslen et al., 2011), thus assuming that the state of the system itself can depend on past information. 2.2 SFA: Formulation SFA’s optimization problem ... |

4 | An extension of slow feature analysis for nonlinear blind source separation
- Sprekeler, Zito, et al.
- 2010
(Show Context)
Citation Context ...tion, given its constraints (no local minima problem). SFA has shown success in problems such as extraction of driving forces of a dynamical system (Wiskott, 2003), nonlinear blind source separation (=-=Sprekeler et al., 2010-=-), as a preprocessor for reinforcement learning (Legenstein et al., 2010; Kompella et al., 2011b), and learning of place-cells, head-direction cells, grid-cells, and spatialTechnical Report No. IDSIA... |

2 | A new algorithm for sequential minor component analysis
- Peng, Yi
- 2006
(Show Context)
Citation Context ...se Peng’s low complexity updating rule (Peng et al., 2007). Peng proved its convergence even for constant learning rates—good for open-ended learning. MCA with sequential addition (Chen et al., 2001; =-=Peng and Yi, 2006-=-) will extract multiple slow features in parallel. 3.1 Neural Updating for PC and MC Extraction CCIPCA and Peng’s MCA are the most appropriate incremental PCA and MCA algorithms for IncSFA. To justify... |