Results 1  10
of
36
A Survey of Medical Image Registration
, 1998
"... The purpose of this chapter is to present a survey of recent publications concerning medical image registration techniques. These publications will be classified according to a model based on nine salient criteria, the main dichotomy of which is extrinsic versus intrinsic methods The statistics of t ..."
Abstract

Cited by 548 (5 self)
 Add to MetaCart
The purpose of this chapter is to present a survey of recent publications concerning medical image registration techniques. These publications will be classified according to a model based on nine salient criteria, the main dichotomy of which is extrinsic versus intrinsic methods The statistics of the classification show definite trends in the evolving registration techniques, which will be discussed. At this moment, the bulk of interesting intrinsic methods is either based on segmented points or surfaces, or on techniques endeavoring to use the full information content of the images involved.
Learning Joint Statistical Models for AudioVisual Fusion and Segregation
, 2001
"... People can understand complex auditory and visual information, often using one to disambiguate the other. Automated analysis, even at a lowlevel, faces severe challenges, including the lack of accurate statistical models for the signals, and their highdimensionality and varied sampling rates. Previ ..."
Abstract

Cited by 75 (2 self)
 Add to MetaCart
(Show Context)
People can understand complex auditory and visual information, often using one to disambiguate the other. Automated analysis, even at a lowlevel, faces severe challenges, including the lack of accurate statistical models for the signals, and their highdimensionality and varied sampling rates. Previous approaches [6] assumed simple parametric models for the joint distribution which, while tractable, cannot capture the complex signal relationships. We learn the joint distribution of the visual and auditory signals using a nonparametric approach. First, we project the data into a maximally informative, lowdimensional subspace, suitable for density estimation. We then model the complicated stochastic relationships between the signals using a nonparametric density estimator. These learned densities allow processing across signal modalities. We demonstrate, on synthetic and real signals, localization in video of the face that is speaking in audio, and, conversely, audio enhan...
Generalized information potential criterion for adaptive system training
 IEEE Trans. Neural Networks
, 2002
"... Abstract—We have recently proposed the quadratic Renyi’s error entropy as an alternative cost function for supervised adaptive system training. An entropy criterion instructs the minimization of the average information content of the error signal rather than merely trying to minimize its energy. In ..."
Abstract

Cited by 58 (28 self)
 Add to MetaCart
(Show Context)
Abstract—We have recently proposed the quadratic Renyi’s error entropy as an alternative cost function for supervised adaptive system training. An entropy criterion instructs the minimization of the average information content of the error signal rather than merely trying to minimize its energy. In this paper, we propose a generalization of the error entropy criterion that enables the use of any order of Renyi’s entropy and any suitable kernel function in density estimation. It is shown that the proposed entropy estimator preserves the global minimum of actual entropy. The equivalence between global optimization by convolution smoothing and the convolution by the kernel in Parzen windowing is also discussed. Simulation results are presented for timeseries prediction and classification where experimental demonstration of all the theoretical concepts is presented. Index Terms—Minimum error entropy, Parzen windowing, Renyi’s entropy, supervised training.
Causality detection based on informationtheoretic approaches in time series analysis
, 2007
"... ..."
A Methodology for Information Theoretic Feature Extraction
 in World Congress on Computational Intelligence
, 1998
"... We discuss an unsupervised feature extraction method which is driven by an information theoretic based criterion: mutual information. While information theoretic signal processing has been examined by many authors the method presented here is more closely related to the approaches of Linsker (1988,1 ..."
Abstract

Cited by 50 (9 self)
 Add to MetaCart
We discuss an unsupervised feature extraction method which is driven by an information theoretic based criterion: mutual information. While information theoretic signal processing has been examined by many authors the method presented here is more closely related to the approaches of Linsker (1988,1990), Bell and Sejnowski (1995), and Viola et al (1996). The method we discuss differs from previous work in several aspects. It is extensible to a feedforward multilayer perceptron with an arbitrary number of layers. No assumptions are made about the underlying PDF of the input space. It exploits a property of entropy coupled with a saturating nonlinearity resulting in a method for entropy manipulation with computational complexity proportional to the number of data samples squared. This represents a significant computational savings over previous methods (Viola et al, 1996). As mutual information is a function of two entropy terms, the method for entropy manipulation can be directly appl...
Learning from examples with Information Theoretic Criteria
 Journal of VLSI Systems, Kluwer
, 1999
"... This paper discusses a framework for learning based on information theoretic criteria. A novel algorithm based on Renyi’s quadratic entropy is used to train, directly from a data set, linear or nonlinear mappers for entropy maximization or minimization. We provide an intriguing analogy between the c ..."
Abstract

Cited by 37 (10 self)
 Add to MetaCart
(Show Context)
This paper discusses a framework for learning based on information theoretic criteria. A novel algorithm based on Renyi’s quadratic entropy is used to train, directly from a data set, linear or nonlinear mappers for entropy maximization or minimization. We provide an intriguing analogy between the computation and an information potential measuring the interactions among the data samples. We also propose two approximations to the KulbackLeibler divergence based on quadratic distances (CauchySchwartz inequality and Euclidean distance). These distances can still be computed using the information potential. We test the newly proposed distances in blind source separation (unsupervised learning) and in feature extraction for classification (supervised learning). In blind source separation our algorithm is capable of separating instantaneously mixed sources, and for classification the performance of our classifier is comparable to the support vector machines (SVMs). 1
General multimodal elastic registration based on mutual information
 Image Processing
, 1998
"... Recent studies indicate that maximizing the mutual information of the joint histogram of two images is an accurate and robust way to rigidly register two mono or multimodal images. Using mutual information for registration directly in a local manner is often not admissible owing to the weakened sta ..."
Abstract

Cited by 33 (1 self)
 Add to MetaCart
Recent studies indicate that maximizing the mutual information of the joint histogram of two images is an accurate and robust way to rigidly register two mono or multimodal images. Using mutual information for registration directly in a local manner is often not admissible owing to the weakened statistical power of the local histogram compared to a global one. We propose to use a global joint histogram based on optimized mutual information combined with a local registration measure to enable local elastic registration.
A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case
 Neural Computation
, 2003
"... Spatiotemporal connectionist networks (STCN's) comprise an important class of neural models that can deal with patterns distributed both in time and space. In this paper, we widen the application domain of the taxonomy for supervised STCN's recently proposed by Kremer (2001) to the unsuper ..."
Abstract

Cited by 27 (1 self)
 Add to MetaCart
Spatiotemporal connectionist networks (STCN's) comprise an important class of neural models that can deal with patterns distributed both in time and space. In this paper, we widen the application domain of the taxonomy for supervised STCN's recently proposed by Kremer (2001) to the unsupervised case. This is possible through a reinterpretation of the state vector as a vector of latent (hidden) variables, as proposed by Meinicke (2000). The goal of this generalized taxonomy is then to provide a nonlinear generative framework for describing unsupervised spatiotemporal networks, making it easier to compare and contrast their representational and operational characteristics. Computational properties, representational issues and learning are also discussed and a number of references to the relevant source publications are provided. It is argued that the proposed approach is simple and more powerful than the previous attempts, from a descriptive and predictive viewpoint. We also discuss the relation of this taxonomy with automata theory and state space modeling, and suggest directions for further work.
OnLine Entropy Manipulation: Stochastic Information Gradient
 IEEE Signal Processing Letters
, 2003
"... Abstract—Entropy has found significant applications in numerous signal processing problems including independent components analysis and blind deconvolution. In general, entropy estimators require ( 2) operations, being the number of samples. For practical online entropy manipulation, it is desirabl ..."
Abstract

Cited by 23 (12 self)
 Add to MetaCart
(Show Context)
Abstract—Entropy has found significant applications in numerous signal processing problems including independent components analysis and blind deconvolution. In general, entropy estimators require ( 2) operations, being the number of samples. For practical online entropy manipulation, it is desirable to determine a stochastic gradient for entropy, which has ( ) complexity. In this letter, we propose a stochastic Shannon’s entropy estimator. We determine the corresponding stochastic gradient and investigate its performance. The proposed stochastic gradient for Shannon’s entropy can be used in online adaptation problems where the optimization of an entropybased cost function is necessary. Index Terms—Shannon’s entropy, stochastic gradient for entropy. I.
InformationTheoretic Learning
, 1999
"... This chapter seeks to extend the ubiquitous meansquare error criterion (MSE) to cost functions that include more information about the training data. Since the learning process ultimately should transfer the information carried in the data samples onto the system's parameters, a natural goal i ..."
Abstract

Cited by 22 (0 self)
 Add to MetaCart
This chapter seeks to extend the ubiquitous meansquare error criterion (MSE) to cost functions that include more information about the training data. Since the learning process ultimately should transfer the information carried in the data samples onto the system's parameters, a natural goal is to find cost functions that directly manipulate information. Hence the name informationtheoretic learning (ITL). In order to be useful, ITL should be independent of the learning machine architecture, and require solely the availability of the data, i.e. it should not require a priori assumptions about the data distributions. The chapter presents our current efforts to develop ITL criteria based on the integration of nonparametric density estimators with Renyi's quadratic entropy definition. As a motivation we start with an application of the MSE to manipulate information using the nonlinear characteristics of the learning machine. This section illustrates the issues faced when we attempt to use...