E. Oja, "Principal components, minor components, and linear neural networks, " Neural Networks, vol. 5, pp. 927--935, 1992.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... (4) for the linear case, where i = WTy = WTWx, it is sufficient to find the PCA subspace only, i.e. the subspace spanned by the principal components, rather than the set of principal components themselves [17] A well known algorithm for this is the the Oja and Karhunen Subspace network [18] [19], also introduced by Williams as his Symmetric Error Correction (SEC) network [20] This uses the update rule W(t q l) W(t) rky(x T i T) 5) where rh is a small scalar update factor. Algorithm (5) can be proved to converge to an orthonormal forward weight matrix that miniraises the mean ....

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927 935, 1992.

....achieve the solution where T = I, one must have some additional knowledge of the different symbols, e.g. known pilot signals. This is a common problem in all the ICA algorithms. We compare the algorithm (24) to two simpler algorithms. The second, simpler algorithm is an Oja s Multiunit batch rule [9]: W t 1 = sign[ RW t (W T t RW t ) Gamma1 ] 25) The difference of (25) to (24) is that W T W has been dropped out. The third and simplest algorithm is a Hopfield type network applied to the matrix W W t 1 = sign( RW t ) 26) However, here we do not set the diagonal elements of R ....

E. Oja, "Principal Components, Minor Components, and Linear Neural Networks", Neural Networks, vol. 5, 1992, pp. 927-936.

....c(2) c(M) is called the PCA subspace. In the PCA neural networks, the weight vectors w(j) j=1,2, M are required to converge to c(j) Without loss of generality, standard PCA, such as those implemented with Oja s and Sanger s rules, can be represented as a linear approximation problem [6]. The linear approximator of the data x(n) in terms of the set of weight vectors w(j) j=1,2, M, is [5] where y(j) is the jth output, and T denotes the transpose operator. Assume that the weight vectors span the PCA subspace of the input data x, then, the optimal approximation where W= w(1) ....

Oja, E., "Principal components, minor components, and linear neural networks," Neural Networks, Vol. 5, 1992.

....orders the components by energy, but the method is not an on line algorithm[10] We propose, in this paper, an orthogonal transform using temporal PCA learning to implement an on line and efficient LMS algorithm. It is well known that the PCA can be implemented on line by Oja or Sanger s rules [7], 9] Furthermore, data reduction can be achieved with PCA transform since the outputs of the PCA network are ordered based on the eigenvalues of the input signal correlation matrix. In order to track a time varying signal and also speed up the convergence rate of the original PCA algorithm, ....

Oja, E., "Principal components, minor components, and linear neural networks," Neural Networks 5, 927-936, 1992.

....the largest eigenvalues and the respective eigenvectors of the covariance matrix of the input data. PCA is used in many applications because of its optimality properties in data compression and information representation [10] It is now well known that PCA can be realized neurally in various ways [3, 16]. Currently, there is a growing interest among neural network researchers in unsupervised learning beyond PCA, called often nonlinear PCA. Such methods take into account higher order statistics, and are often more competitive than standard PCA when realized neurally [9, 10] Nonlinear PCA type ....

....and E = c(1) c(M ) where (i) is the ith largest eigenvalue of the data covariance matrix C xx = Efx k x T k g, and c(i) is the respective principal eigenvector. PCA whitening is easy to do using standard software. Alternatively one can estimate the principal eigenvectors neurally [3, 16]. Assuming that the ith weight vector w k (i) of a PCA network at step k is a roughly normalized estimate of c(i) the respective eigenvalue (i) can be estimated adaptively using the simple algorithm v y x T W V Fig. 1: The two layer network structure used in source separation. 14] k 1 ....

E. Oja, "Principal Components, Minor Components, and Linear Neural Networks". Neural Networks, Vol. 5, 1992, pp. 927-935.

.... and jjwjj 2 = P n i=1 w 2 i ) 1=2 is the L 2 norm, 1) 2) becomes principal component analysis (PCA) or minor component analysis (MCA) a task for which many algorithms have been developed [Krasulina 1970 ,Oja 1982, Fuhrmann and Liu 1984, Oja and Karhunen 1985, Comon and Golub 1990, Oja 1992, Cichocki and Unbehauen 1993, Wang and Karhunen 1996, Diamantaras and Kung 1996, Chen 1997, Luo et al. 1997, Solo and Kong 1998] Similar algorithms are also useful as methods for bias removal in parameter estimation problems when noises are present [Owsley 1978, Thompson 1979, Reddy et al. 1982, ....

....gradient methods and propose simple stabilization methods for them. In particular, we provide a self normalized algorithm for the minor component analysis task that avoids using periodic renormalization, additional penalty terms, and costly divides or square roots [Fuhrmann and Liu 1984, Oja 1992, Wang and Karhunen 1996, Luo et al. 1997, Owsley 1978, Thompson 1979] The organization of this report is as follows. In the next section, several iterative gradient based methods for solving (2) are derived and discussed, and their general relationships are illustrated via graphical examples. ....

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927-935, Nov.-Dec. 1992

....As discussed in [7] the term xy T in 14 is a typical anti Hebbian term. The term Gammaxy T D(y) has the twofold effect of stabilizing the algorithm and incorporating high order statistics to reach output independence 1 . It constitutes an improved version of the adaptive Oja s algorithm [8] for principal component analysis. Finally, the term Theta W T Gamma1 precludes convergence towards a singular matrix W. 4. ANALYSIS OF STATIONARY POINTS The analysis of the stationary points in (11) is done for the case of N = 2. First, we find the matrices W for which r W OE(W) 0. ....

E. Oja, "Principal Components, Minor Components, and Linear Neural Networks", Neural Networks, vol. 5, pp. 927-935, 1992.

....be represented by much less data bits than training patterns include. Data compression paradigm gives us a constraint which leads to a different than template matching solution. This will be shown in section II. Another solution for an autoassociative system can be based on a local subspace method [9, 12]. The method takes a training subset from each class C i ae R N ; and next it applies to it the Principal Component Analysis [4] PCA produces K dimensional subspace of R N with an orthonormal basis in columns of the matrix W i : The basis is centered on the centroid c i of the training set. ....

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, 5, pp.927-935, 1992.

....and show how a local implementation can be obtained, with a neural interpretation. The connection between neural matrix adaptation laws, and signal processing algorithms is not new. Principal component analysis is a well established topic in neural network literature. A recent overview is given in [11]. The algorithms are formulated in discrete time, and continuous time equations play a role in the convergence analysis for stochastic input signals. 9] focuses on principal component analysis, in the context of uniform local implementability. Finally, our work shows a strong relation with some ....

.... famous learning law, originally formulated for biological neural networks [12] An example of a law using contributions of type I only, is A = xx T A Gamma AA T xx T A = x(A T x) T Gamma (AA T x) A T x) T : 3) This is a continuous time version of Oja s Subspace Algorithm (SA) [11]. The input signal x is supplied to the first layer, and multiplied by A T to obtain A T x in the second layer and further multiplied by A to obtain AA T x in the first layer. Contributions of type II are inspired by two other discrete time neural weight updating laws known as the ....

[Article contains additional citation context not shown here]

E. Oja, "Principal components, minor components, and linear neural networks", Neural Networks, Vol. 5, pp. 927-935, 1992.

....Rakentajanaukio 2 C, FIN 02150 Espoo, Finland Email: Juha.Karhunen hut.fi, Erkki.Oja hut.fi Introduction Principal Component Analysis (PCA) is a widely used technique in signal processing. It is now well known how it can be realized in different ways using neural networks; for examples, cf. [4, 9, 21]. Recently, there has been an increasing interest in extending the unsupervised Hebbian learning rules used in PCA to nonlinear Hebbian learning: such techniques are often called nonlinear PCA methods. The main reason for this interest is that even though PCA is optimal for example in ....

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.

....vectors. Typically Hebbian type learning rules are used based on the one unit learning algorithm originally proposed by one of the authors in [24] Many different versions and extensions of this basic algorithm have been proposed during the recent years; for reviews and introductions, see e.g. [29, 6, 13]. PCA networks are useful in signal characterization, optimal feature extraction, and data compression. PCA is also the basis of subspace classifiers that have been used e.g. in speech and texture classification [27] 1 In the field of neural networks, there has been growing interest in ....

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.

....analyzed in depth both by mathematical analysis and by simulations on the learning rule. Acknowledgement. The author wishes to thank Dr. Juha Karhunen and Mr. Aapo Hyvrinen for helpful comments on the preliminary version of the manuscript. 1 Introduction The PCA neuron [12] and the PCA network [15, 17] were introduced by the author with the purpose of demonstrating how the well known Principal Component Analysis expansion can be realized in a simple parallel layer of adaptive elements. These artificial neurons were linear because PCA is an inherently linear technique. For one linear neuron, the ....

....is: W k 1 = W k k [x k Gamma W k y k ]y T k (2) where now W k = w k (1) w k (2) w k (m) is the weight matrix, whose columns are the individual neuron weight vectors w k (i) and y k = W T k x k is the output vector of m elements. This algorithm and some extensions were analyzed in [17]. Now the weight vectors w k (i) become orthonormal and tend to a basis of the m dimensional dominant eigenvector subspace of the input correlation matrix; however, usually the individual weight vectors do not tend to the eigenvectors. It was obvious from the start that exactly the same learning ....

[Article contains additional citation context not shown here]

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, 5, 927 - 935 (1992).

No context found.

E. Oja, "Principal components, minor components, and linear neural networks, " Neural Networks, vol. 5, pp. 927--935, 1992.

No context found.

E. Oja, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.

No context found.

E. Oja, "Principal Components, Minor Components, and Linear Neural Networks", Neural Networks 5 (1992) 927-935.

No context found.

E. Oja, "Principal Components, Minor Components, and Linear Neural Networks", Neural Networks 5 (1992) 927-935.

No context found.

E. Oja, "Principal Components, Minor Components, and Linear Neural Networks", Neural Networks 5 (1992) 927-935.

No context found.

E. Oja, "Principal components, minor components, and linear neural networks, " Neural Networks, vol. 5, pp. 927--935, 1992.

No context found.

E. Oja, "Principal Components, minor components, and linear neural networks", Neural Networks, vol. 5, pp. 927--935, 1982.

No context found.

E. Oja, "Principal components, minor components and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC