Abstract

Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. In this paper, we use a combination of two different approaches for linear ICA: Comon's information-theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast (objective) functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple fixed-point algorithms for practical optimizatio...

Citations

6515	Elements of Information Theory - Cover, Thomas - 1991
1034	Independent component analysis, a new concept - Comon - 1994
830	An information-maximization approach to blind separation and blind deconvolution - Bell, Sejnowski - 1995
520	Optimization by Vector Space Methods - Luenberger - 1969
381	The ’independent components’ of natural scenes are edge filters - Bell, Sejnowski - 1997
368	Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture - Jutten, Herault - 1991
312	A fast fixed-point algorithm for independent component analysis - Hyvärinen, Oja - 1997
243	Possible principles underlying the transformation of sensory messages - Barlow - 1961
219	Independent component filters of natural images compared with simple cells in primary visual cortex - Hateren, Schaaf - 1998
206	Exploratory projection pursuit - Friedman - 1987
189	Projection pursuit - HUBER - 1985
185	A projection pursuit algorithm for exploratory data analysis - FRIEDMAN, TUKEY - 1974
108	A class of neural networks for independent component analysis - Karhunan, Oja, et al. - 1997
107	Separation of a mixture of independent sources through a maximum likelihood approach - Pham, Garrat, et al. - 1992
105	Adaptive blind separation of independent sources: a deflation approach. Signal Proc 45: 59–83 - Delfosse, Loubaton - 1995
98	Modern Factor Analysis - Harman - 1976
94	Principal components, minor components, and linear neural networks - Oja - 1992
90	Independent component analysis—a new concept - Comon - 1994
72	A new learning algorithm for blind source separation - Amari, Cichocki, et al. - 1996
71	What is Projection Pursuit - Jones, Sibson - 1987
65	Robust Statistics - Hampel, Ronchetti, et al. - 1986
63	Natural image statistics and efficient coding - Olshausen, Field - 1996
61	New approximations of differential entropy for independent component analysis and projection pursuit - Hyvärinen - 1998
50	A fast xed-point algorithm for independent component analysis - Hyvarinen, Oja - 1997
48	Independent component analysis for identification of artifacts in magnetoencephalographic recordings - Vigario, Jousmaki, et al. - 1998
47	The nonlinear PCA learning rule in independent component analysis - Oja - 1997
46	Independent component analysis by general nonlinear Hebbian-like learning rules - Hyvärinen, Oja - 1998
44	Emergence of simple-cell receptive eld properties by learning a sparse code for natural images - Olshausen, Field - 1996
43	Laheld."Equivariant adaptive source separation - Cardoso, H - 1996
35	The independent components of natural scenes are edge lters - Bell, Sejnowski - 1997
29	Extraction of ocular artifacts from EEG using independent component analysis - Vig'ario, N - 1997
21	The fixed-point algorithm and maximum likelihood estimation for independent component analysis - Hyvärinen - 1999
18	A family of ��xed-point algorithms for independent component analysis - Hyvrinen - 1997
16	Independent component analysis by general nonlinear Hebbian-like learning rules - Hyvrinen, Oja - 1998
14	One-unit contrast functions for independent component analysis: a statistical analysis - Hyvärinen - 1997
11	A fast algorithm for estimating overcomplete ICA bases for image windows - Hyvärinen, Cristescu, et al. - 1999
11	Image feature extraction by sparse coding and independent component analysis - Hyvarinen, Oja, et al. - 1998
11	Independent component lters of natural images compared with simple cells in primary visual cortex - Hateren, Schaaf - 1998
10	An Experimental Comparison of Neural ICA Algorithms. pages 651–656 - Giannakapoulos, Karhunen, et al. - 1998
7	Independent component analysis in wave decomposition of auditory evoked fields - Vigário, Särelä, et al. - 1998
6	Neural Networks for Signal Processing and Optimization - Cichocki, Unbehauen - 1994
6	New approximations of dioeerential entropy for independent component analysis and projection pursuit - Hyvrinen - 1998
6	Independent component analysis for identication of artifacts in magnetoencephalographic recordings - Vigrio, Jousmki, et al. - 1998
5	The xed-point algorithm and maximum likelihood estimation for independent component analysis - Hyvrinen - 1999
4	One-unit contrast functions for independent component analysis: A statistical analysis - Hyvrinen - 1997
4	A fast algorithm for estimating overcomplete ICA bases for image windows - Hyvrinen, Cristescu, et al. - 1999
4	Extraction of ocular artifacts from EEG using independent component analysis - Vigrio - 1997
2	Fast independent component analysis with noisy data using gaussian moments - Hyvärinen - 1999
2	Fast independent component analysis with noisy data using gaussian moments - Hyvrinen - 1999
2	Independent component analysis in wave decomposition of auditory evoked ��elds - Vigrio, Srel, et al. - 1998