A.J. Bell, T.J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution", Neural Computation, 7(6), 1129--1159 (1995).  Home/Search   Document Details and Download   Summary   Related Articles   Check

....signals when mixed signals only are observed, with little or no knowledge about the source statistics nor their temporal characteristics. In particular, under the hypothesis that the source signals to be separated out are statistically independent, the independent component analysis (ICA) theory [4], 5] 7] may be employed. Over recent years, blind source separation by the independent component analysis has received much attention owing to its potential applications, as in speech recognition systems, telecommunications, fault detection, medical imaging, and other important research fields ....

A.J. Bell and T.J. Sejnowski, "An Information Maximisation Approach to Blind Separation and Blind Deconvolution", Neural Computation, Vol. 7, No. 6, pp. 1129 -- 1159, 1995

....model. The validity of the new learning algorithm are verified by computer simulations. 1 INTRODUCTION The problem of blind signal separation arises in many areas such as speech recognition, data communication, sensor signal processing, and medical science. Several neural network algorithms [3, 5, 7] have been proposed for solving this problem. The performance of these algorithms is usually affected by the selection of the activation functions for the formal neurons in the networks. However, all activation Lab. for Information Representation, FRP, RIKEN, Wako shi, Saitama, JAPAN functions ....

.... W : 18) Applying the previous approximation of the gradient to (18) we obtain the following algorithm: gW (19) which has the same equivariant property as the algorithms developed in [4, 5] Although the on line learning algorithms (16) and (19) look similar to those in [3, 7] and [5] respectively, the selection of the activation function in this paper is rational, not ad hoc. The activation function (14) is determined by the ICA. It is a non monotonic activation function different from those used in [3, 5, 7] There is a simple way to justify the stability of the ....

[Article contains additional citation context not shown here]

A. J. Bell and T. J. Sejnowski. An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7:1129--1159, 1995.

....are the method of choice due to their probabilistic treatment of acoustic coefficients and the Markov assumptions necessary for time varying signals. Even auditory scene analysis and sound texture modeling has been cast into a probabilistic learning framework with independent component analysis [14] [15] Word distributions have also been modeled using bigrams, trigrams or Markov models and topic modeling often uses multinomial distributions. A topic spotting system is shown in Figure 1.2(c) which tracks the conversation of multiple speakers and displays related material for the users to ....

....latent variables constrain that Gaussian to a single mean value which, when sampled, will generate a spherical Gaussian over the data. There are indeed many shortcomings with linear PCA as a representation which have encouraged variants such as Bayesian PCA [21] independent components analysis [14], auto associator networks [20] and nonlinear embedding [183] However, in applying these techniques to images, audio and time series, an important piece of knowledge is overlooked. All the aforementioned approaches assume the data they deal with can be rasterized directly into a vector form in a ....

A.J. Bell and T.J. Sejnowski. An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129--1159, 1995.

....separation technique based on the independent component analysis (ICA) may be envisaged in order to blindly recover the source signals and, as an useful by product, to locate the sources themselves. In fact, the classical ICA techniques aim at recovering the source signals from their mixtures [6,7,8,9,10,11,12,13,14,15], while only recently the attention has been turned to the physical meaning of the mixing operators, which may reveal important information about the geometry of the sources and the signal propagation models. These studies are also related to neurological electromagnetic source localization by EEG ....

....The theory here presented has been applied with success to a practical problem dealing with amplitude modulated radio transmissions, and the obtained results are meant as the first step of a feasibility study. 2. Overview of Independent Component Analysis The Independent Component Analysis (ICA) [6,7,8,9,10,11,12,13,14,15,29] is a wellestablished statistical signal processing technique that aims at decomposing a set of multivariate signals into a base of signals as statistically independent as possible, with the minimal loss of information content. The main two recognized purposes of ICA are: Data representation ....

A.J. Bell and T.J. Sejnowski, An information maximisation approach to blind separation and blind deconvolution, Neural Computation, Vol. 7, No. 6, pp. 1129 - 1159, 1995

....form of BSS, mixtures are assumed to be linear instantaneous mixtures of sources. For this simple data model, it is known that independent component analysis (ICA) performs the separation task [13] Since Jutten and Herault s first solution to ICA [17] a variety of methods havebeendeveloped [13, 12, 1, 3, 11, 4, 16,9 , 14, 21]. One important application of BSS might be cocktail party problem, the goal of which is to separate out the contribution of each speaker or acoustic source, given signals obtained by an array of microphones. In order to tackle the cocktail party problem, the multipath effect and reverberation ....

A. Bell and T. Sejnowski. An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7:1129, 19,12

....to derive the contrast function (12) is very popular among the neural network community. Denote g i ( the distribution function g i (s) # q i (t)dt [0, 1] 1 so that g # i = q i and denote g(s) g 1 (s 1 ) g n (s n ) An interpretation of the infomax principle (see[9], 55] and references therein) suggests the contrast function # IM [y] # H[g(y) 14) where H[ denotes the Shannon entropy (for a random vector u with density p(u) this is H[u] p(u) log p(u)du with the convention 0 log 0 = 0) This idea can be understood as follows: on one hand, g(s) ....

A. J. Bell and T. J. Sejnowski. An information-maximisation approach to blind separation and blind deconvolution. Neural computation, 7(6):1004--1034, 1995.

....the component wise non linear function: y) 1 (y 1 ) n (y n ) 7) Proceedings of NNSP 98. Cambridge, UK. and I denotes the identity matrix. Therefore the maximum likelihood estimate AML of A is a solution of the equation G r ( ML x(t) 0 (8) see [5] 2] and [3] for a similar estimating function derived from the infomax principle) Thus, algorithm (2) with G = G r implicitly is an on line likelihood maximizer by (relative) stochastic gradient ascent [2] Another approach is the so called orthogonal approach . It consists in enforcing the decorrelation ....

A. J. Bell and T. J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution," Neural computation, vol. 7, no. 6, pp. 1004--1034, 1995.

.... to blind source separation studies, unsupervised learning rules based on information theoretic principles were proposed by [13] These learning rules are based on the principle of redundancy reduction as a coding strategy for neurons of the perceptual sys tem [14] More recently, 15] and [16] introduced a surprisingly simple blind source separation algorithm for a non linear feed forward network from an information maximization viewpoint. This algorithm was subsequently improved, extended and modified [17] 18] and its relation to max imum likelihood estimation and redundancy ....

....consistent temporal dynamic, which is characterised by a time course contained in the associated column of the square mixing matrix A. In [7] the sources are estimated by iteratively optimising an unmixing matrix W A so that WX contains mutually independent rows, using the infomax algorithm [16]. The ICA model above, though being a simple linear regression model, differs from the standard GLM as used in neuroimaging in two essential aspects: firstly, the mixing is assumed to be square, i.e. the signal is not constrained to be contained within a lower dimensional signal sub space. ....

A. Bell and T Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution;' Neural Computation, vol. 7, no. 6, pp. 112%1159, 1995.

....contrast function (12) is very popular among the neural network community. Denote g i ( Delta) the distribution function g i (s) Z s q i (t)dt 2 [0; 1] 1 i n (13) so that g i = q i and denote g(s) g 1 (s 1 ) gn (s n ) An interpretation of the infomax principle (see[9], 55] and references therein) suggests the contrast function OE IM [y] GammaH[g(y) 14) where H[ Delta] denotes the Shannon entropy (for a random vector u with density p(u) this is H[u] p(u) log p(u)du with the convention 0 log 0 = 0) This idea can be understood as follows: on one ....

A. J. Bell and T. J. Sejnowski. An informationmaximisation approach to blind separation and blind deconvolution. Neural computation, 7(6):1004--1034, 1995.

....Bitstream Psychoacoustic Model Filter Bank Fig. 1. Basic perceptual audio coder architecture. 2.2. Independent Component Analysis Independent Component Analysis (ICA) is a recently developed statistical tool for extracting statistically independent components from a random vector [5]. ICA can be used to solve the classical cocktail party problem in which sensors record a mixture of people speaking simultaneously. ICA is used to recover the original speaker signals from mixtures. Other applications of ICA are audio analysis, natural images analysis, financial data, medical ....

A. J. Bell and T. J. Sejnowsky, "An information maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, no. 6, pp. 1129--1159, 1995.

....ICA might be a better tool in removing this information redundancy between the different subbands. 3. INDEPENDENT COMPONENT ANALYSIS In this work Independent Component Analysis (ICA) is applied for extracting an efficient signal representation in terms of statistically independent components [6]. Let x = x 1 , x 2 . x n ) be the observed data vector. ICA s goal is to find the matrix A such that: x = As (3) 0.05 0 0.05 0.1 0.15 10 15 20 25 30 5 10 15 20 25 30 0.1 0.05 0 0.05 0.1 0.15 10 15 20 25 30 5 10 15 20 25 30 Fig. 2. Mutual Information between ....

A. J. Bell and T. J. Sejnowsky, "An information maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, no. 6, pp. 1129--1159, 1995.

....original source # (unfiltered) plus delayed filtered versions of the other sources. In the real world, this assumption does, of course, not hold, since all source signals are filtered (room impulse response) There are methods that take this into account, and try to equalize each separated signal, [3, 9, 5, 7]. However, these methods are computationally much more expensive. There is also a robustness issue, in that the transfer functions (room impulse responses) are modeled as FIR filters that are not necessarily minimum phase, and accordingly the inverse IIR filter FIR filter approximation may not be ....

Bell, A.J. and Sejnowski, T.J., "An informationmaximisation approach to blind separation and blind deconvolution", Neural Computations 7, pp. 1004-- 1034, 1995.

....needed for applying ICA to group data: separability, stationarity, and inference. Our results further demonstrate the utility of using such a method for making group inferences on fMRI data using ICA. 1. TRODUCTION Independent Component Analysis (ICA) is being increasingly applied to fMRI data [1,2]. ICA as applied to fMRI data can be used to separate either spatially [3] or temporally [4] independent sources and works well in both situations when appropriate assumptions are met [5,6] We have recently developed an approach for performing an ICA analysis on a group of subjects [7,8] This ....

....data set was entered into an AIC MDL estimation to determine the 158 number of sources existing in the group data. The aggregate data were then reduced to this dimension using PCA, followed by an independent component estimation using an algorithm which attempts to minimize mutual information [1]. Time courses and spatial maps were then reconstructed for each subject and the spatial maps were thresholded at p 0.001 (t =4.5, df =8) 4. RESULTS Results from simulation 1 are presented in Figure 5. ICA spatial maps generated from individual subjects were very similar to spatial maps ....

A.J.Bell and T.J. Sejnowski, An Information Maximisation Approach to Blind Separation and Blind Deconvolution Neural Computation, vol. 7, pp. 1129- 1159, 1995.

....a physical meaning. An application to data from an MEG experiment underlines the usefulness of our approach. 1. INTRODUCTION Blind source separation (BSS) techniques have found widespread use in various application domains, e.g. acoustics, telecommunication or biomedical signal processing. [1, 2, 3, 4, 5, 6, 7, 8]) BSS is a statistical technique to reveal unknown source signals als 1 when only mixtures of them can be observed. For a linear mixture model, each of the observed signals is assumed to be generated by 158 8192 22116 In the following we will work in ....

A. J. Bell and T. J. Sejnowski, "An information maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129--1159, 1995.

....problem . The model has the same number of hidden and visible units, and the energy of the model is defined as 1 3 4 : 8 (11) This model is strictly equivalent to the noiseless ICA model with sigmoidal outputs used by Bell and Sejnowski [9]. Below we will give a very brief description of the different MCMC methods, but refer to [10] for more details. Algorithm HMC uses 8 step of hybrid Monte Carlo simulation to sample from . This involves sampling a momentum variable from a standard normal distribution, followed by ....

A. J. Bell and T. J. Sejnowski, "An information maximisation approach to blind separation and blind deconvolution, " Neural Computation, vol. 7, pp. 1129-- 1159, 1995.

....achieves blind separation of two convolutive mixed sources in real time. 1. INTRODUCTION Recently, independent component analysis (ICA) has gained great importance in the field of blind source separation. Many algorithms are available for blind separation of instantaneously mixed signals, e.g. [1, 2, 3]. Applications have been reported e.g. in [4, 5] In the instantaneous case the mixing process can be expressed in terms of weighted additions. This is not true for microphone recordings, where the mixing process yields convolutions with the room impulse responses between the sources and the ....

A. J. Bell and T. J. Sejnowski. An informationmaximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1004--1034, 1995.

....the output signals as statistically independent as possible by evaluating higher order statistics. The idea of ICA was first expressed by Jutten and Herault [2] while the term ICA was first coined by Comon in [3] However the field became popular only with the seminal paper by Bell and Sejnowski [4] who elaborated upon the Infomax principle first advocated by Linsker [5] 6] Classically, linear BSS has been treated most thoroughly [7] 8] where the mixing function of the source signals corresponds to a linear function (matrix) Two principles became popular: Minimal Mutual Information ....

....the mutual information of the output as a contrast function because minimizing the mutual information (MMI) induces statistical independence of the output. This has to be compared with Bell and Sejnowski s suggestion to maximize the entropy (ME) of the output. But ME does not always induce MMI ([4], section 4 and [9] and therefore statistical independence. ME performs best when the non linear demixing function in the ME algorithm matches with the cumulative distribution of the given source. However, nowadays many algorithms are based on the ME contrast function. This raises the question ....

A. J. Bell and T. J. Sejnowski, "An informationmaximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129--1159, 1995.

....output signals as statistically independent as possible by evaluating higher order statistics. The idea of ICA was first expressed by Jutten and Herault [3] 4] while the term ICA was later coined by Comon in [5] However the field became popular only with the seminal paper by Bell and Sejnowski [6] who elaborated upon the Infomax principle first advocated by Linsker [7] 8] Recently, geometric ICA algorithms have received further attention due to their relative ease of implementation [1] 9] They have been applied successfully to the analysis of real world biomedical data [10] 11] and ....

.... CIHKJLFM9 . In the nonlinear case, where is any function L7 = little is known because, without further restrictions, the problem is generally ill posed. But in the linear case described above, many different algorithms have been proposed with the Bell Sejnowski maximum entropy algorithm [6] being the most popular and also most widely studied among them. In this paper we consider a geometric approach to the source separation problem. As we need a certain uniqueness of the solution, we want at most one of the source variables ON QP N F N K B75=R denotes the projection on ....

A. J. Bell and T. J. Sejnowski, "An informationmaximisation approach to blind separation and blind deconvolution, " Neural Computation, vol. 7, pp. 1129--1159, 1995.

....derived. Experimental results show that the performances of the ICA based algorithm are much better than those of the popular LMS algorithm. 2. LEARNING RULE ICA was proposed to recover independent sources from given sensor signals in which the sources have been mixed through unknown channels [3][4] Bell and Sejnowski proposed to learn the unmixing matrix by minimizing the mutual information among components of ( where is a nonlinear function approximating the cumulative den394 sity function (cdf) of the sources and denotes recovered sources [3] They showed that for ....

.... through unknown channels [3] 4] Bell and Sejnowski proposed to learn the unmixing matrix by minimizing the mutual information among components of ( where is a nonlinear function approximating the cumulative den394 sity function (cdf) of the sources and denotes recovered sources [3]. They showed that for super Gaussian signals minimizing the mutual information between components of is equal to maximizing the entropy of . Lee et al. and Torrkola have addressed blind separation of convolved sources [5] 6] Learning rules of adaptive filter coefficients in the noise ....

Anthony J. Bell and Terrence J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129--1159, 1995.

....minimizes the potential energy function. 2. APPLICATION TO INDEPENDENT COMPONENT ANALYSIS The independent component analysis (ICA) aims at extracting independent signals from their linear mixtures or to extract independent features (as latent variables) from signals having complex structure [4, 9, 12, 19, 20, 17]. A way to define the independent components is to employ the maximum or minimum kurtosis principle: Under some conditions, the output of a linear neuron with multiple inputs described by , # ) contains an independent component of the input if the weight vector maximizes ....

A.J. BELL AND T.J. SEJNOWSKI, An information maximisation approach to blind separation and blind deconvolution, Neural Computation, Vol.7, No. 6, pp. 1129 -- 1159, 1996

....a signal in ICA basis domain by ICABFT given by Y(m) A oo [y m (0) y m (1) y m (M 1) 3) where A oo is a frequency ordered and orthogonalized version of the matrix A, columns of which are ICA basis functions. ICA basis functions can be obtained by various algorithms [6] [7], 8] with the clean speech data pre processed as described above. After estimating the ICA basis function matrix A, we ordered the basis functions by the location of their power spectral densities, resulting in a frequencyordered basis function matrix, A o . The term frequencyordered means ....

Anthony J. Bell and Terrence J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129--1159, 1995.

No context found.

A.J. Bell, T.J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution", Neural Computation, 7(6), 1129--1159 (1995).

No context found.

A. J. Bell and T.J. Sejnowski. An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7:1129--1159, 1995.

No context found.

A. J. Bell and T. J. Sejnowski, "An informationmaximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129--1159, 1995.

No context found.

A. J. Bell and T.J. Sejnowski. An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7:1129--1159, 1995. 19

No context found.

A. Bell, T. Sejnowski, An information-maximisation approach to blind separation and blind deconvolution, Neural Computation 7 (1995) 1129--1159.

No context found.

A. J. Bell and T. J. Sejnowski. An information maximisation approach to blind separation and blind devonvolution. Neural Cmputation, 7:1129-- 1159, 1995.

No context found.

Bell A.J. and Sejnowski T.J. An information maximisation approach to blind separation and blind deconvolution, Neural Computation, Vol.7, No. 6, pp. 1129--1159, 1996.

No context found.

A. J. Bell and T. J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution," Neural Computation, 7 (1995), pp. 1129--1159.

No context found.

A.J. Bell, T.J. Sejnowski (1995). An information maximisation approach to blind separation and blind deconvolution. Neural Computation 7, 1129-1159.

No context found.

A.J. Bell, T.J. Sejnowski (1995). An information maximisation approach to blind separation and blind deconvolution. Neural Computation 7, 1129-1159.

No context found.

A. J. Bell and T. J. Sejnowski. An information-maximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129--1159, 1995.

No context found.

A.J. Bell, T.J. Sejnowski (1995). An information maximisation approach to blind separation and blind deconvolution. Neural Computation 7, 1129-1159.

No context found.

A. J. Bell and T. J. Sejnowski, "An information-maximisation approach to blind separation and blind deconvolution," Neural Computation, 7 (1995), pp. 1129--1159.

No context found.

Anthony J. Bell and Terrence J. Sejnowski, "An informationmaximisation approach to blind separation and blind deconvolution, " Neural Computation, vol. 7, pp. 1129--1159, 1995.

No context found.

A. Bell and T. Sejnowski. An Information Maximisation Approach to Blind Separation and Blind Deconvolution. Neural Computation, 7(6):1129--1159, 1995.

No context found.

Anthony J. Bell and Terrence J. Sejnowski, "An information -maximisation approach to blind separation and blind deconvolution," Neural Computation, vol. 7, pp. 1129--1159, 1995.

No context found.

A. J. Bell and T. J. Sejnowski, "An information maximisation approach to blind separation and blind deconvolution," Neural Computation , Vol. 7, No. 6, pp. 1129-1159 (1995).

No context found.

Anthony J. Bell and Terry J. Sejnowski. An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129-- 1159, 1995.

No context found.

Bell, A. J. and Sejnowski, T. J. (1995). An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129--1159. ftp://ftp.cnl.salk.edu/pub/tony/bell.blind.ps.Z. CiteSeer: http://citeseer.nj.nec.com/bell95informationmaximization.html.

No context found.

A.J. Bell and T.J. Sejnowski, An Information Maximisation Approach to Blind Separation and Blind Deconvolution Neural Comput., vol. 7, pp. 1129-1159, 1995.

No context found.

A.J. Bell and T.J. Sejnowski. An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129--1159, 1995. 16

No context found.

A. J. Bell and T. J. Sejnowski. An informationmaximisation approach to blind separation and blind deconvolution. Neural computation,7(6)) 1995.

No context found.

A. Bell and T. Sejnowski. An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7:1129--1159, 1995.

No context found.

Anthony J Bell & Terrance J Sejnowski, An information-Maximisation approach to blind separation and blind deconvolution, Neural Computation, 7,6, 1004-1034 (1995)

No context found.

T. Bell and T. Sejnowski, \An Information-maximisation approach to blind separation and blind deconvolution", Neural Computation, Vol. 7, pp. 1004-1034, 1995.

No context found.

A.J. Bell and T.J. Sejnowski. An information maximisation approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129--1159, 1995.

No context found.

Bell A.J., Sejnowski T.J.: An information maximisation approach to blind separation and blind deconvolution, Neural Computation, 7(6), pp.1129-1159, 1995.

No context found.

A.J. Bell and T.J. Sejnowski, An information maximisation approach to blind separation and blind deconvolution, Neural Computation, Vol. 7, No. 6, pp. 1129 -- 1159, 1995

No context found.

A. J. Bell and T. J. Sejnowski 1996, "An information maximisation approach to blind separation and blind deconvolution," Neural Computation 7(6), 1129--1159.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC