Results 1  10
of
46
Semisupervised dimensionality reduction
 In: Proceedings of the 7th SIAM International Conference on Data Mining
, 2007
"... Dimensionality reduction is among the keys in mining highdimensional data. This paper studies semisupervised dimensionality reduction. In this setting, besides abundant unlabeled examples, domain knowledge in the form of pairwise constraints are available, which specifies whether a pair of instance ..."
Abstract

Cited by 53 (7 self)
 Add to MetaCart
(Show Context)
Dimensionality reduction is among the keys in mining highdimensional data. This paper studies semisupervised dimensionality reduction. In this setting, besides abundant unlabeled examples, domain knowledge in the form of pairwise constraints are available, which specifies whether a pair of instances belong to the same class (mustlink constraints) or different classes (cannotlink constraints). We propose the SSDR algorithm, which can preserve the intrinsic structure of the unlabeled data as well as both the mustlink and cannotlink constraints defined on the labeled examples in the projected lowdimensional space. The SSDR algorithm is efficient and has a closed form solution. Experiments on a broad range of data sets show that SSDR is superior to many established dimensionality reduction methods. 1
Multiview regression via canonical correlation analysis
 In Proc. of Conference on Learning Theory
, 2007
"... Abstract. In the multiview regression problem, we have a regression problem where the input variable (which is a real vector) can be partitioned into two different views, where it is assumed that either view of the input is sufficient to make accurate predictions — this is essentially (a significan ..."
Abstract

Cited by 50 (7 self)
 Add to MetaCart
(Show Context)
Abstract. In the multiview regression problem, we have a regression problem where the input variable (which is a real vector) can be partitioned into two different views, where it is assumed that either view of the input is sufficient to make accurate predictions — this is essentially (a significantly weaker version of) the cotraining assumption for the regression problem. We provide a semisupervised algorithm which first uses unlabeled data to learn a norm (or, equivalently, a kernel) and then uses labeled data in a ridge regression algorithm (with this induced norm) to provide the predictor. The unlabeled data is used via canonical correlation analysis (CCA, which is a closely related to PCA for two random variables) to derive an appropriate norm over functions. We are able to characterize the intrinsic dimensionality of the subsequent ridge regression problem (which uses this norm) by the correlation coefficients provided by CCA in a rather simple expression. Interestingly, the norm used by the ridge regression algorithm is derived from CCA, unlike in standard kernel methods where a special apriori norm is assumed (i.e. a Banach space is assumed). We discuss how this result shows that unlabeled data can decrease the sample complexity. 1
An rkhs for multiview learning and manifold coregularization
 in Proc. of ICML’08, 2008
"... Inspired by cotraining, many multiview semisupervised kernel methods implement the following idea: find a function in each of multiple Reproducing Kernel Hilbert Spaces (RKHSs) such that (a) the chosen functions make similar predictions on unlabeled examples, and (b) the average prediction given ..."
Abstract

Cited by 45 (4 self)
 Add to MetaCart
(Show Context)
Inspired by cotraining, many multiview semisupervised kernel methods implement the following idea: find a function in each of multiple Reproducing Kernel Hilbert Spaces (RKHSs) such that (a) the chosen functions make similar predictions on unlabeled examples, and (b) the average prediction given by the chosen functions performs well on labeled examples. In this paper, we construct a single RKHS with a datadependent “coregularization ” norm that reduces these approaches to standard supervised learning. The reproducing kernel for this RKHS can be explicitly derived and plugged into any kernel method, greatly extending the theoretical and algorithmic scope of coregularization. In particular, with this development, the Rademacher complexity bound for coregularization given in (Rosenberg & Bartlett, 2007) follows easily from wellknown results. Furthermore, more refined bounds given by localized Rademacher complexity can also be easily applied. We propose a coregularization based algorithmic alternative to manifold regularization (Belkin et al., 2006; Sindhwani et al., 2005a) that leads to major empirical improvements on semisupervised tasks. Unlike the recently proposed transductive approach of (Yu et al., 2008), our RKHS formulation is truly semisupervised and naturally extends to unseen test data.
Semisupervised regression with cotraining style algorithms
, 2007
"... The traditional setting of supervised learning requires a large amount of labeled training examples in order to achieve good generalization. However, in many practical applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, semisup ..."
Abstract

Cited by 43 (8 self)
 Add to MetaCart
The traditional setting of supervised learning requires a large amount of labeled training examples in order to achieve good generalization. However, in many practical applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, semisupervised learning has attracted much attention. Previous research on semisupervised learning mainly focuses on semisupervised classification. Although regression is almost as important as classification, semisupervised regression is largely understudied. In particular, although cotraining is a main paradigm in semisupervised learning, few works has been devoted to cotraining style semisupervised regression algorithms. In this paper, a cotraining style semisupervised regression algorithm, i.e. COREG, is proposed. This algorithm uses two regressors each labels the unlabeled data for the other regressor, where the confidence in labeling an unlabeled example is estimated through the amount of reduction in mean square error over the labeled neighborhood of that example. Analysis and experiments show that COREG can effectively exploit unlabeled data to improve regression estimates.
Linear Algorithms for Online Multitask Classification
"... We design and analyze interacting online algorithms for multitask classification that perform better than independent learners whenever the tasks are related in a certain sense. We formalize task relatedness in different ways, and derive formal guarantees on the performance advantage provided by int ..."
Abstract

Cited by 33 (4 self)
 Add to MetaCart
(Show Context)
We design and analyze interacting online algorithms for multitask classification that perform better than independent learners whenever the tasks are related in a certain sense. We formalize task relatedness in different ways, and derive formal guarantees on the performance advantage provided by interaction. Our online analysis gives new stimulating insights into previously known coregularization techniques, such as the multitask kernels and the margin correlation analysis for multiview learning. In the last part we apply our approach to spectral coregularization: we introduce a natural matrix extension of the quasiadditive algorithm for classification and prove bounds depending on certain unitarily invariant norms of the matrix of task coefficients. 1
Bayesian CoTraining
"... We propose a Bayesian undirected graphical model for cotraining, or more generally for semisupervised multiview learning. This makes explicit the previously unstated assumptions of a large class of cotraining type algorithms, and also clarifies the circumstances under which these assumptions fai ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
(Show Context)
We propose a Bayesian undirected graphical model for cotraining, or more generally for semisupervised multiview learning. This makes explicit the previously unstated assumptions of a large class of cotraining type algorithms, and also clarifies the circumstances under which these assumptions fail. Building upon new insights from this model, we propose an improved method for cotraining, which is a novel cotraining kernel for Gaussian process classifiers. The resulting approach is convex and avoids localmaxima problems, unlike some previous multiview learning methods. Furthermore, it can automatically estimate how much each view should be trusted, and thus accommodate noisy or unreliable views. Experiments on toy data and real world data sets illustrate the benefits of this approach. 1
We are not contortionists: Coupled adaptive learning for head and body orientation estimation in surveillance
 in Proc. of the IEEE CVPR
"... In this paper, we deal with the estimation of body and head poses (i.e orientations) in surveillance videos, and we make three main contributions. First, we address this issue as a joint model adaptation problem in a semisupervised framework. Second, we propose to leverage the adaptation on multipl ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
(Show Context)
In this paper, we deal with the estimation of body and head poses (i.e orientations) in surveillance videos, and we make three main contributions. First, we address this issue as a joint model adaptation problem in a semisupervised framework. Second, we propose to leverage the adaptation on multiple information sources (external labeled datasets, weak labels provided by the motion direction, data structure manifold), and in particular, on the coupling at the output level of the head and body classifiers, accounting for the restriction in the configurations that the head and body pose can jointly take. Third, we propose a kernelformulation of this principle that can be efficiently solved using a global optimization scheme. The method is applied to body and head features computed from automatically extracted body and head location tracks. Thorough experiments on several datasets demonstrate the validity of our approach, the benefit of the coupled adaptation, and that the method performs similarly or better than a stateoftheart algorithm. 1.
SemiSupervised Document Retrieval
, 2008
"... This paper proposes a new machine learning method for constructing ranking models in document retrieval. The method, which is referred to as SSRank, aims to use the advantages of both the traditional Information Retrieval (IR) methods and the supervised learning methods for IR proposed recently. The ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
This paper proposes a new machine learning method for constructing ranking models in document retrieval. The method, which is referred to as SSRank, aims to use the advantages of both the traditional Information Retrieval (IR) methods and the supervised learning methods for IR proposed recently. The advantages include the use of limited amount of labeled data and rich model representation. To do so, the method adopts a semisupervised learning framework in ranking model construction. Specifically, given a small number of labeled documents with respect to some queries, the method effectively labels the unlabeled documents for the queries. It then uses all the labeled data to train a machine learning model (in our case, Neural Network). In the data labeling, the method also makes use of a traditional IR model (in our case, BM25). A stopping criterion based on machine learning theory is given for the datalabeling process. Experimental results on three benchmark data sets and one web search data set indicate that SSRank consistently and almost always significantly outperforms the baseline methods (unsupervised and supervised
A Unifying Framework for Vectorvalued Manifold Regularization and Multiview Learning
"... This paper presents a general vectorvalued reproducing kernel Hilbert spaces (RKHS) formulation for the problem of learning an unknown functional dependency between a structured input space and a structured output space, in the SemiSupervised Learning setting. Our formulation includes as special c ..."
Abstract

Cited by 9 (5 self)
 Add to MetaCart
(Show Context)
This paper presents a general vectorvalued reproducing kernel Hilbert spaces (RKHS) formulation for the problem of learning an unknown functional dependency between a structured input space and a structured output space, in the SemiSupervised Learning setting. Our formulation includes as special cases Vectorvalued Manifold Regularization and Multiview Learning, thus provides in particular a unifying framework linking these two important learning approaches. In the case of least square loss function, we provide a closed form solution with an efficient implementation. Numerical experiments on challenging multiclass categorization problems show that our multiview learning formulation achieves results which are comparable with state of the art and are significantly better than singleview learning. 1.
A Kernel for SemiSupervised Learning With MultiView Point
, 2009
"... In semisupervised learning (SSL), we learn a predictive model from a collection of labeled data and a typically much larger collection of unlabeled data. These lecture notes present a framework called multiview point cloud regularization (MVPCR) [5], which unifies and generalizes several semisupe ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
(Show Context)
In semisupervised learning (SSL), we learn a predictive model from a collection of labeled data and a typically much larger collection of unlabeled data. These lecture notes present a framework called multiview point cloud regularization (MVPCR) [5], which unifies and generalizes several semisupervised kernel methods that are based on datadependent regularization in reproducing kernel Hilbert spaces (RKHS). Special cases of MVPCR include coregularized least squares (CoRLS) [7, 3, 6], manifold regularization (MR) [1, 8, 4], and graphbased semisupervised learning. An accompanying theorem shows how to reduce any MVPCR problem to standard supervised learning with a new multiview kernel. Relevance RKHS techniques form the basis of many stateoftheart supervised learning algorithms, such as support vector machines (SVMs), kernel ridge regression, and Gaussian processes. By plugging the new multiview kernel into these, or any other standard kernel method, we can conveniently convert them to semisupervised learning algorithms. Via the reduction of MVPCR to supervised RKHS learning, we can easily derive generalization error bounds using standard results. In particular, we generalize the bound given in [6] for CoRLS. From an experimental