Abstract

It has been verified experimentally that nonlinear versions of the PCA network learning rules for the weights of a neural layer produce neurons that have signal separation capabilities. One of the learning rules earlier proposed by the author is studied here mathematically to analyze why and how the algorithm works in this application. It is shown that for input vectors whose density is symmetrical around the origin and has equal variances for each element, the weight matrix obtained as the asymptotic solution of the nonlinear PCA learning rule is in some cases a rotation of the input vector to statistically independent directions. This explains why it can be used for image and speech signal separation. Sufficient conditions are formulated, depending on the nonlinear neuron activation function and on the probability densities of the original signal components. It is shown that a sigmoidal nonlinearity as the activation function is feasible for flat sub-Gaussian densities of the origina...

Citations