Results 1  10
of
44
Beyond independent components: trees and clusters
 Journal of Machine Learning Research
, 2003
"... We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a treestructured graphical model. This treedependent component analysi ..."
Abstract

Cited by 56 (0 self)
 Add to MetaCart
We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a treestructured graphical model. This treedependent component analysis (TCA) provides a tractable and flexible approach to weakening the assumption of independence in ICA. In particular, TCA allows the underlying graph to have multiple connected components, and thus the method is able to find “clusters ” of components such that components are dependent within a cluster and independent between clusters. Finally, we make use of a notion of graphical models for time series due to Brillinger (1996) to extend these ideas to the temporal setting. In particular, we are able to fit models that incorporate treestructured dependencies among multiple time series.
Natural gradient multichannel blind deconvolution and source separation using causal fir filters
 in Proc. IEEE ICASSP, May 2004
"... Practical gradientbased adaptive algorithms for multichannel blind deconvolution and convolutive blind source separation typically employ FIR filters for the separation system. Inadequate use of signal truncation within these algorithms can introduce steadystate biases into their converged solution ..."
Abstract

Cited by 33 (4 self)
 Add to MetaCart
(Show Context)
Practical gradientbased adaptive algorithms for multichannel blind deconvolution and convolutive blind source separation typically employ FIR filters for the separation system. Inadequate use of signal truncation within these algorithms can introduce steadystate biases into their converged solutions that lead to degraded separation and deconvolution performances. In this paper, we derive a natural gradient multichannel blind deconvolutionand source separation algorithm that mitigates these effects for estimating causal FIR solutions to these tasks. Numerical experiments verify the robust convergence performance of the new method both in multichannel blind deconvolution tasks for i.i.d. sources and in convolutive BSS tasks for acoustic sources, even for extremelyshort separation filters. 1.
Fas algorithm for estimating mutual information, entropies ans score functions
 in Proceedings of ICA2003
, 2003
"... This papers proposes a fast algorithm for estimating the mutual information, difference score function, conditional score and conditional entropy, in possibly high dimensional space. The idea is to discretise the integral so that the density needs only be estimated over a regular grid, which can be ..."
Abstract

Cited by 23 (0 self)
 Add to MetaCart
(Show Context)
This papers proposes a fast algorithm for estimating the mutual information, difference score function, conditional score and conditional entropy, in possibly high dimensional space. The idea is to discretise the integral so that the density needs only be estimated over a regular grid, which can be done with little cost through the use of a cardinal spline kernel estimator. Score functions are then obtained as gradient of the entropy. An example of application to the blind separation of postnonlinear mixture is given. 1.
A Bayesian Approach To Source Separation
 in Proceedings of The Nineteenth International Conference on Maximum Entropy and Bayesian Methods
, 1999
"... Source separation is one of the signal processing's main emerging domain. Many techniques such as maximum likelihood (ML), Infomax, cumulant matching, estimating function, etc. have been used to address this difficult problem. Unfortunately, up to now, many of these methods could not account co ..."
Abstract

Cited by 22 (5 self)
 Add to MetaCart
(Show Context)
Source separation is one of the signal processing's main emerging domain. Many techniques such as maximum likelihood (ML), Infomax, cumulant matching, estimating function, etc. have been used to address this difficult problem. Unfortunately, up to now, many of these methods could not account completely for noise on the data, for different number of sources and sensors, for lack of spatial independence and for time correlation of the sources. Recently, the Bayesian approach has been used to push farther these limitations of the conventional methods. This paper proposes a unifying approach to source separation based on the Bayesian estimation. We first show that this approach gives the possibility to explain easily the major known techniques in sources separation as special cases. Then we propose new methods based on maximum a posteriori (MAP) estimation, either to estimate directly the sources, or the mixing matrices or even both. Key words: Sources separation, Bayesian estimation 1.
Fast Algorithms for Mutual Information Based Independent Component Analysis
, 2002
"... This paper provides fast algorithms to perform independent component analysis based on the mutual information criterion. The main ingredient is the binning technique and the use of cardinal splines, which allows the fast computation of the density estimator over a regular grid. Using a discretized ..."
Abstract

Cited by 22 (6 self)
 Add to MetaCart
This paper provides fast algorithms to perform independent component analysis based on the mutual information criterion. The main ingredient is the binning technique and the use of cardinal splines, which allows the fast computation of the density estimator over a regular grid. Using a discretized form of the entropy, the criterion can be evaluated quickly together with its gradient, which can be expressed in terms of the score functions. Both offline and online separation algorithms have been developed. Our density, entropy and score estimators also have their own interest.
Multichannel blind deconvolution of nonminimum phase systems using information backpropagation
 in: Proceedings of the Fifth International Conference on Neural Information Processing (ICONIP’99
, 1999
"... Abstract—In this paper, we present a new filter decomposition method for multichannel blind deconvolution of nonminimumphase systems. With this approach, we decompose a doubly finite impulse response filter into a cascade form of two filters: a causal finite impulse response (FIR) filter and an an ..."
Abstract

Cited by 21 (13 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we present a new filter decomposition method for multichannel blind deconvolution of nonminimumphase systems. With this approach, we decompose a doubly finite impulse response filter into a cascade form of two filters: a causal finite impulse response (FIR) filter and an anticausal FIR filter. After introducing a Lie group to the manifold of FIR filters, we discuss geometric properties of the FIR filter manifold. Using the nonholonomic transform, we derive the natural gradient on the FIR manifold. By simplifying the mutual information rate, we present a very simple cost function for blind deconvolution of nonminimumphase systems. Subsequently, the natural gradient algorithms are developed both for the causal FIR filter and for the anticausal FIR filter. Simulations are presented to illustrate the validity and favorable learning performance of the proposed algorithms. Index Terms—Blind deconvolution, independent component analysis, natural gradient, nonmimimumphase systems. I.
The kernel mutual information
 In IEEE ICASSP
, 2003
"... We introduce a new contrast function, the kemel mutual information (KMIj, to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estima ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
(Show Context)
We introduce a new contrast function, the kemel mutual information (KMIj, to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate of the mutual information between a discretised approximation of the continuous random variables. We show that Bach and Jordan’s kernel generalised variance (KGV) is also an upper bound on the same kernel density estimate, but is looser. Finally, we suggest that the addition of a regularising term in the KGV causes it to approach the KMI, which motivates the introduction of this regularisation. 1.
Source Separation: From Dusk Till Dawn
"... The first part of this paper is concerned by the history of source separation. It include our comments and those of a few other researchers on the development of this new research field. The second part is focused on recent developments of the separation in nonlinear mixtures. ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
The first part of this paper is concerned by the history of source separation. It include our comments and those of a few other researchers on the development of this new research field. The second part is focused on recent developments of the separation in nonlinear mixtures.
Finding clusters in independent component analysis
 IN: 4TH INTL. SYMP. ON INDEPENDENT COMPONENT ANALYSIS AND SIGNAL SEPARATION (ICA2003
, 2003
"... We present a class of algorithms that find clusters in independent component analysis: the data are linearly transformed so that the resulting components can be grouped into clusters, such that components are dependent within clusters and independent between clusters. In order to find such clusters, ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
We present a class of algorithms that find clusters in independent component analysis: the data are linearly transformed so that the resulting components can be grouped into clusters, such that components are dependent within clusters and independent between clusters. In order to find such clusters, we look for a transform that fits the estimated sources to a foreststructured graphical model. In the nonGaussian, temporally independent case, the optimal transform is found by minimizing a contrast function based on mutual information that directly extends the contrast function used for classical ICA. We also derive a contrast function in the Gaussian stationary case that is based on spectral densities and generalizes the contrast function of Pham [22] to richer classes of dependency.
Source Separation Using Information Measures in the Time and Frequency Domains
, 1999
"... First and foremost I wish to thank my advisor, Dr. José Principe, for being my advisor and for his inspiration and support through my Ph. D. study. Without his thoughtprovoking guidance and neverending encouragement, this dissertation would not have been possible. I also wish to thank the members ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
(Show Context)
First and foremost I wish to thank my advisor, Dr. José Principe, for being my advisor and for his inspiration and support through my Ph. D. study. Without his thoughtprovoking guidance and neverending encouragement, this dissertation would not have been possible. I also wish to thank the members of my committee, Dr. Fredrick Taylor, Dr. John Harris, Dr. William Edmonson, and Dr. Howard Rothman, for their invaluable time and interest in serving on my supervisory committee, as well as their insightful comments which improved the quality of this dissertation. Special thanks go out to all the former and current CNEL colleagues. Especially I would like to express my immense gratitude to Dr. Dongxin Xu, Dr. YuMing Chiang and Dr.tobe ShaoJen Lim for their friendship and their offering stimulating discussions during the course of my Ph. D. research. Last but not least, I wish to thank my parents, brother and sister for their ceaseless love and firm support and for instilling in me a love of learning. I would like to thank my wife, HuiChen, for enduring a seemingly endless ordeal, for sacrificing some of her best years so that I could finally finish this Ph. D. research.