Independent Component Analysis (ICA) is a recently developed technique that in many cases characterizes the data in a natural way. The main application area of the linear ICA model is blind source separation. Here, unknown source signals are estimated from their unknown linear mixtures using the strong assumption that the sources are mutually independent. In practice, separation can be achieved by using suitable higher-order statistics or nonlinearities. Various neural approaches have recently been proposed for blind source separation and ICA. In this paper, these approaches and the respective learning algorithms are briefly reviewed, and some extensions of the basic ICA model are discussed. 1. Introduction A recent trend in neural network research is to study various forms of unsupervised learning beyond standard Principal Component Analysis (PCA). Such techniques are often called nonlinear PCA methods. They can be developed from various starting points, usually leading to different ...

this paper, we introduce a neural network that can be used for both source separation and the estimation of the basis vectors of ICA. The remainder of the paper is organized as follows. The next section presents the necessary background on ICA and source separation. In the third section, we introduce and justify the basic neural network learning algorithms for signal separation. The fourth section provides mathematical analysis justifying the separation ability of the nonlinear PCA type learning algorithm. The fifth section then introduces the ICA neural network, a three-layer network whose layers perform input data whitening, separation, and ICA basis vector estimation, respectively. In the sixth section, we present experimental results. In the last section, the conclusions of this study are presented, and some possibilities for extending the data model are outlined.

In this paper, we study the application of some nonlinear neural PCA type approaches to the separation of independent source signals from their linear mixture. This problem is important in signal processing and communications, and it cannot be solved using standard PCA. Using prewhitening and appropriate choice of nonlinearities, several algorithms proposed by us yield good separation results for sub-Gaussian (or super-Gaussian) source signals. We discuss the related problem of estimating the basis vectors in Independent Component Analysis briefly, too. 1. Introduction Principal Component Analysis (PCA) is a standard statistical technique which is defined in terms of the largest eigenvalues and the respective eigenvectors of the covariance matrix of the input data. PCA is used in many applications because of its optimality properties in data compression and information representation [10]. It is now well-known that PCA can be realized neurally in various ways [3, 16]. Currently...

It has been verified experimentally that nonlinear versions of the PCA network learning rules for the weights of a neural layer produce neurons that have signal separation capabilities. One of the learning rules earlier proposed by the author is studied here mathematically to analyze why and how the algorithm works in this application. It is shown that for input vectors whose density is symmetrical around the origin and has equal variances for each element, the weight matrix obtained as the asymptotic solution of the nonlinear PCA learning rule is in some cases a rotation of the input vector to statistically independent directions. This explains why it can be used for image and speech signal separation. Sufficient conditions are formulated, depending on the nonlinear neuron activation function and on the probability densities of the original signal components. It is shown that a sigmoidal nonlinearity as the activation function is feasible for flat sub-Gaussian densities of the origina...

Nonlinear extensions of one-unit and multi-unit Principal Component Analysis (PCA) neural networks, introduced earlier by the authors, are reviewed. The networks and their nonlinear Hebbian learning rules are related to other signal expansions like Projection Pursuit (PP) and Independent Component Analysis (ICA). Separation results for mixtures of real world signals and images are given. 1 Introduction Principal Component Analysis (PCA) is a widely used technique in data analysis. Mathematically, it is defined as follows: let C = Efxx T g be the covariance matrix of L-dimensional zero mean input data vectors x. The ith principal component of x is x T c(i), where c(i) is the normalized eigenvector of C corresponding to the ith largest eigenvalue (i). The subspace spanned by the principal eigenvectors c(1); : : : ; c(M) (M ! L) is called the PCA subspace (of dimensionality M ). PCA is used in many applications because of its optimality properties in data compression and inform...