Results 1 
8 of
8
Pixels that sound
 In Proc. Computer Vision and Pattern Recognition
, 2005
"... People and animals fuse auditory and visual information to obtain robust perception. A particular benefit of such crossmodal analysis is the ability to localize visual events associated with sound sources. We aim to achieve this using computervision aided by a single microphone. Past efforts encou ..."
Abstract

Cited by 36 (1 self)
 Add to MetaCart
(Show Context)
People and animals fuse auditory and visual information to obtain robust perception. A particular benefit of such crossmodal analysis is the ability to localize visual events associated with sound sources. We aim to achieve this using computervision aided by a single microphone. Past efforts encountered problems stemming from the huge gap between the dimensions involved and the available data. This has led to solutions suffering from low spatiotemporal resolutions. We present a rigorous analysis of the fundamental problems associated with this task. Then, we present a stable and robust algorithm which overcomes past deficiencies. It grasps dynamic audiovisual events with high spatial resolution, and derives a unique solution. The algorithm effectively detects pixels that are associated with the sound, while filtering out other dynamic pixels. It is based on canonical correlation analysis (CCA), where we remove inherent illposedness by exploiting the typical spatial sparsity of audiovisual events. The algorithm is simple and efficient thanks to its reliance on linear programming and is free of userdefined parameters. To quantitatively assess the performance, we devise a localization criterion. The algorithm capabilities were demonstrated in experiments, where it overcame substantial visual distractions and audio noise. 1
Deep Canonical Correlation Analysis
"... We introduce Deep Canonical Correlation Analysis (DCCA), a method to learn complex nonlinear transformations of two views of data such that the resulting representations are highly linearly correlated. Parameters of both transformations are jointly learned to maximize the (regularized) total correla ..."
Abstract

Cited by 20 (4 self)
 Add to MetaCart
We introduce Deep Canonical Correlation Analysis (DCCA), a method to learn complex nonlinear transformations of two views of data such that the resulting representations are highly linearly correlated. Parameters of both transformations are jointly learned to maximize the (regularized) total correlation. It can be viewed as a nonlinear extension of the linear method canonical correlation analysis (CCA). It is an alternative to the nonparametric method kernel canonical correlation analysis (KCCA) for learning correlated nonlinear transformations. Unlike KCCA, DCCA does not require an inner product, and has the advantages of a parametric method: training time scales well with data size and the training data need not be referenced when computing the representations of unseen instances. In experiments on two realworld datasets, we find that DCCA learns representations with significantly higher correlation than those learned by CCA and KCCA. We also introduce a novel nonsaturating sigmoid function based on the cube root that may be useful more generally in feedforward neural networks.
Bayesian Canonical Correlation Analysis
 JOURNAL OF MACHINE LEARNING RESEARCH 14 (2013) 9651003
, 2013
"... Canonical correlation analysis (CCA) is a classical method for seeking correlations between two multivariate data sets. During the last ten years, it has received more and more attention in the machine learning community in the form of novel computational formulations and a plethora of applications. ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
Canonical correlation analysis (CCA) is a classical method for seeking correlations between two multivariate data sets. During the last ten years, it has received more and more attention in the machine learning community in the form of novel computational formulations and a plethora of applications. We review recent developments in Bayesian models and inference methods for CCA which are attractive for their potential in hierarchical extensions and for coping with the combination of large dimensionalities and small sample sizes. The existing methods have not been particularly successful in fulfilling the promise yet; we introduce a novel efficient solution that imposes groupwise sparsity to estimate the posterior of an extended model which not only extracts the statistical dependencies (correlations) between data sets but also decomposes the data into shared and data setspecific components. In statistics literature the model is known as interbattery factor analysis (IBFA), for which we now provide a Bayesian treatment.
Dependency detection with similarity constraints
 In Proc. MLSP’09 IEEE International Workshop on Machine Learning for Signal Processing
, 2009
"... ar ..."
(Show Context)
MULTIVIEW ACOUSTIC FEATURE LEARNING USING ARTICULATORY MEASUREMENTS
"... We consider the problem of learning a linear transformation of acoustic feature vectors for phonetic classification, in a setting where articulatory measurements are available at training time. We use the acoustic and articulatory data together in a multiview learning approach, in particular using ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
We consider the problem of learning a linear transformation of acoustic feature vectors for phonetic classification, in a setting where articulatory measurements are available at training time. We use the acoustic and articulatory data together in a multiview learning approach, in particular using canonical correlation analysis to learn linear transformations of the acoustic features that are maximally correlated with the articulatory data. We also investigate simple approaches for combining information shared across the acoustic and articulatory views with information that is private to the acoustic view. We apply these methods to phonetic frame classification on data drawn from the University of Wisconsin Xray Microbeam Database. We find a small but consistent advantage to the multiview approaches combining shared and private information, compared to the baseline acoustic features or unsupervised dimensionality reduction using principal components analysis. Index Terms — Multiview learning, canonical correlation analysis, articulatory measurements, dimensionality reduction, acoustic features 1.
1 DEPENDENCY DETECTION WITH SIMILARITY CONSTRAINTS
"... c©2009 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other w ..."
Abstract
 Add to MetaCart
(Show Context)
c©2009 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Preprint for Lahti et al., MLSP’09 DEPENDENCY DETECTION WITH SIMILARITY CONSTRAINTS
"... Unsupervised twoview learning, or detection of dependencies between two paired data sets, is typically done by some variant of canonical correlation analysis (CCA). CCA searches for a linear projection for each view, such that the correlations between the projections are maximized. The solution is ..."
Abstract
 Add to MetaCart
(Show Context)
Unsupervised twoview learning, or detection of dependencies between two paired data sets, is typically done by some variant of canonical correlation analysis (CCA). CCA searches for a linear projection for each view, such that the correlations between the projections are maximized. The solution is invariant to any linear transformation of either or both of the views; for tasks with small sample size such flexibility implies overfitting, which is even worse for more flexible nonparametric or kernelbased dependency discovery methods. We develop variants which reduce the degrees of freedom by assuming constraints on similarity of the projections in the two views. A particular example is provided by a cancer gene discovery application where chromosomal distance affects the dependencies between gene copy number and activity levels. Similarity constraints are shown to improve detection performance of known cancer genes. 1.