Results 1  10
of
18
Pixels that sound
 In Proc. Computer Vision and Pattern Recognition
, 2005
"... People and animals fuse auditory and visual information to obtain robust perception. A particular benefit of such crossmodal analysis is the ability to localize visual events associated with sound sources. We aim to achieve this using computervision aided by a single microphone. Past efforts encou ..."
Abstract

Cited by 36 (1 self)
 Add to MetaCart
(Show Context)
People and animals fuse auditory and visual information to obtain robust perception. A particular benefit of such crossmodal analysis is the ability to localize visual events associated with sound sources. We aim to achieve this using computervision aided by a single microphone. Past efforts encountered problems stemming from the huge gap between the dimensions involved and the available data. This has led to solutions suffering from low spatiotemporal resolutions. We present a rigorous analysis of the fundamental problems associated with this task. Then, we present a stable and robust algorithm which overcomes past deficiencies. It grasps dynamic audiovisual events with high spatial resolution, and derives a unique solution. The algorithm effectively detects pixels that are associated with the sound, while filtering out other dynamic pixels. It is based on canonical correlation analysis (CCA), where we remove inherent illposedness by exploiting the typical spatial sparsity of audiovisual events. The algorithm is simple and efficient thanks to its reliance on linear programming and is free of userdefined parameters. To quantitatively assess the performance, we devise a localization criterion. The algorithm capabilities were demonstrated in experiments, where it overcame substantial visual distractions and audio noise. 1
Information bottleneck for gaussian variables
 in Advances in Neural Information Processing Systems 16
, 2003
"... ∗ Both authors contributed equally The problem of extracting the relevant aspects of data was addressed through the information bottleneck (IB) method, by (soft) clustering one variable while preserving information about another relevance variable. An interesting question addressed in the current ..."
Abstract

Cited by 23 (2 self)
 Add to MetaCart
(Show Context)
∗ Both authors contributed equally The problem of extracting the relevant aspects of data was addressed through the information bottleneck (IB) method, by (soft) clustering one variable while preserving information about another relevance variable. An interesting question addressed in the current work is the extension of these ideas to obtain continuous representations (embeddings) that preserve relevant information, rather than discrete clusters. We give a formal definition of the general continuous IB problem and obtain an analytic solution for the optimal representation for the important case of multivariate Gaussian variables. The obtained optimal representation is a noisy linear projection to eigenvectors of the normalized correlation matrix Σ xyΣ −1 x, which is also the basis obtained in Canonical Correlation Analysis. However, in Gaussian IB, the compression tradeoff parameter uniquely determines the dimension, as well as the scale of each eigenvector. This introduces a novel interpretation where solutions of different ranks lie on a continuum parametrized by the compression level. Our analysis also provides analytic expression for the optimal tradeoff the information curve in terms of the eigenvalue spectrum. 1
The kernel mutual information
 In IEEE ICASSP
, 2003
"... We introduce a new contrast function, the kemel mutual information (KMIj, to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estima ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
(Show Context)
We introduce a new contrast function, the kemel mutual information (KMIj, to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate of the mutual information between a discretised approximation of the continuous random variables. We show that Bach and Jordan’s kernel generalised variance (KGV) is also an upper bound on the same kernel density estimate, but is looser. Finally, we suggest that the addition of a regularising term in the KGV causes it to approach the KMI, which motivates the introduction of this regularisation. 1.
Estimating Multiple Depths in Semitransparent Stereo Images
"... A stereo algorithm that can estimate multiple depths in semitransparent images is presented. The algorithm is based on a combination of phase analysis and canonical correlation analysis. The algorithm adapts lters in each local neighbourhood of the image in a way which maximizes the correlation bet ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
A stereo algorithm that can estimate multiple depths in semitransparent images is presented. The algorithm is based on a combination of phase analysis and canonical correlation analysis. The algorithm adapts lters in each local neighbourhood of the image in a way which maximizes the correlation between the ltered images. The adapted lters are then analysed to nd the disparity. This is done by a simple phase analysis of the scalar product of the lters. For images with di erent but constant depths, a simple reconstruction procedure is suggested.
Learning Visual Operators from Examples: A New Paradigm in Image Processing
 ICIAP’99
, 1999
"... This paper presents a general strategy for designing efficient visual operators. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
This paper presents a general strategy for designing efficient visual operators. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise independent. Particularly important concepts in the work are mutual information and canonical correlation. Visual operators learned from examples are presented, e.g. local shift invariant orientation operators and image content invariant disparity operators. Interesting similarities to biological vision functions are observed.
A Unified Approach to PCA, PLS, MLR and CCA
"... This paper presents a novel algorithm for analysis of stochastic processes. The algorithm can be used to nd the required solutions in the cases of principal component analysis (PCA), partial least squares (PLS), canonical correlation analysis (CCA) or multiple linear regression (MLR). The algorithm ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
This paper presents a novel algorithm for analysis of stochastic processes. The algorithm can be used to nd the required solutions in the cases of principal component analysis (PCA), partial least squares (PLS), canonical correlation analysis (CCA) or multiple linear regression (MLR). The algorithm is iterative and sequential in its structure and uses online stochastic approximation to reach an equilibrium point. A quotient between two quadratic forms is used as an energy function and it is shown that the equilibrium points constitute solutions to the generalized eigenproblem.
Finding Efficient Nonlinear Visual Operators using Canonical Correlation Analysis
 In Proceedings of the SSAB Symposium on Image Analysis
, 2000
"... This paper presents a general strategy for designing ecient visual operators. The approach is highly task oriented and what constitutes the relevant information is dened by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise inde ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
This paper presents a general strategy for designing ecient visual operators. The approach is highly task oriented and what constitutes the relevant information is dened by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise independent. Particularly important concepts in the work are mutual information and canonical correlation. Visual operators learned from examples are presented, e.g. local shift invariant orientation operators and image content invariant disparity operators. Interesting similarities to biological vision functions are observed. 1 Introduction The need for a generally applicable method for learning is evident in problems involving vision. The dimensionality of typical inputs often exceed 10 6 eectively ruling out any type of complete analysis. In common practice vision problems are handled by reducing the dimensionality to typically < 10 by throwing away almost all available information in a...
Automated Generation of Representations in Vision
 In Proceedings of the 15th International Conference on Pattern Recognition
, 2000
"... This paper presents a general strategy for automated generation of efficient representations in vision. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of situations that are dependent through the chosen featu ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
This paper presents a general strategy for automated generation of efficient representations in vision. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of situations that are dependent through the chosen feature but are otherwise independent. Particularly important concepts in the work are mutual information and canonical correlation. How visual operators and representations can be generated from examples are presented for a number of features, e.g. local orientation, disparity and motion. Interesting similarities to biological vision functions are observed. The results clearly demonstrates the potential of combining advanced filtering techniques and learning strategies based on canonical correlation analysis (CCA). 1 Introduction For high dimensional data relevant feature extractors are in practice impossible to design by hand due to the overpowering amount off possible signal combinations. It...
An adaptive stereo algorithm based on canonical correlation analysis
 Proceedings of the Second IEEE International Conference on Inteligent Processing Systems
, 1998
"... Abstract – This paper presents a novel algorithm that uses CCA and phase analysis to detect the disparity in stereo images. The algorithm adapts filters in each local neighbourhood of the image in a way which maximizes the correlation between the filtered images. The adapted filters are then analyse ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract – This paper presents a novel algorithm that uses CCA and phase analysis to detect the disparity in stereo images. The algorithm adapts filters in each local neighbourhood of the image in a way which maximizes the correlation between the filtered images. The adapted filters are then analysed to find the disparity. This is done by a simple phase analysis of the scalar product of the filters. The algorithm can even handle cases where the images have different scales. The algorithm can also handle depth discontinuities and give multiple depth estimates for semitransparent images. I.