Results 1  10
of
45
SemiSupervised Learning Literature Survey
, 2006
"... We review the literature on semisupervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole
spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semisupervised learning. This document is a chapter ..."
Abstract

Cited by 782 (8 self)
 Add to MetaCart
We review the literature on semisupervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole
spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semisupervised learning. This document is a chapter excerpt from the author’s
doctoral thesis (Zhu, 2005). However the author plans to update the online version frequently to incorporate the latest development in the field. Please obtain the latest
version at http://www.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf
Stochastic relational models for discriminative link prediction
 Advances in Neural Information Processing Systems
, 2007
"... We introduce a Gaussian process (GP) framework, stochastic relational models (SRM), for learning social, physical, and other relational phenomena where interactions between entities are observed. The key idea is to model the stochastic structure of entity relationships (i.e., links) via a tensor int ..."
Abstract

Cited by 49 (18 self)
 Add to MetaCart
(Show Context)
We introduce a Gaussian process (GP) framework, stochastic relational models (SRM), for learning social, physical, and other relational phenomena where interactions between entities are observed. The key idea is to model the stochastic structure of entity relationships (i.e., links) via a tensor interaction of multiple GPs, each defined on one type of entities. These models in fact define a set of nonparametric priors on infinite dimensional tensor matrices, where each element represents a relationship between a tuple of entities. By maximizing the marginalized likelihood, information is exchanged between the participating GPs through the entire relational network, so that the dependency structure of links is messaged to the dependency of entities, reflected by the adapted GP kernels. The framework offers a discriminative approach to link prediction, namely, predicting the existences, strengths, or types of relationships based on the partially observed linkage network as well as the attributes of entities (if given). We discuss properties and variants of SRM and derive an efficient learning algorithm. Very encouraging experimental results are achieved on a toy problem and a usermovie preference link prediction task. In the end we discuss extensions of SRM to general relational learning tasks. 1
Dissimilarity in graphbased semisupervised classification
 Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS
, 2007
"... Label dissimilarity specifies that a pair of examples probably have different class labels. We present a semisupervised classification algorithm that learns from dissimilarity and similarity information on labeled and unlabeled data. Our approach uses a novel graphbased encoding of dissimilarity th ..."
Abstract

Cited by 39 (2 self)
 Add to MetaCart
(Show Context)
Label dissimilarity specifies that a pair of examples probably have different class labels. We present a semisupervised classification algorithm that learns from dissimilarity and similarity information on labeled and unlabeled data. Our approach uses a novel graphbased encoding of dissimilarity that results in a convex problem, and can handle both binary and multiclass classification. Experiments on several tasks are promising. 1
MultiRelational Learning with Gaussian Processes
"... Due to their flexible nonparametric nature, Gaussian process models are very effective at solving hard machine learning problems. While existing Gaussian process models focus on modeling one single relation, we present a generalized GP model, named multirelational Gaussian process model, that is ab ..."
Abstract

Cited by 22 (8 self)
 Add to MetaCart
Due to their flexible nonparametric nature, Gaussian process models are very effective at solving hard machine learning problems. While existing Gaussian process models focus on modeling one single relation, we present a generalized GP model, named multirelational Gaussian process model, that is able to deal with an arbitrary number of relations in a domain of interest. The proposed model is analyzed in the context of bipartite, directed, and undirected univariate relations. Experimental results on realworld datasets show that exploiting the correlations among different entity types and relations can indeed improve prediction performance. 1
Pseudolikelihood EM for WithinNetwork Relational Learning
"... In this work, we study the problem of withinnetwork relational learning and inference, where models are learned on a partially labeled relational dataset and then are applied to predict the classes of unlabeled instance in the same graph. We categorized recent work in statistical relational learnin ..."
Abstract

Cited by 19 (5 self)
 Add to MetaCart
(Show Context)
In this work, we study the problem of withinnetwork relational learning and inference, where models are learned on a partially labeled relational dataset and then are applied to predict the classes of unlabeled instance in the same graph. We categorized recent work in statistical relational learning into three alternative approaches for this setting: disjoint learning with disjoint inference, disjoint learning with collective inference, and collective learning with collective inference. Models from each of these categories has been employed previously in different settings, but to our knowledge there has been no systematic comparison of models from all three categories. In this paper, we develop a novel pseudolikelihood EM method that facilitates more general collective learning and collective inference on partially labeled relational networks. We then compare this method to competing methods from the other categories on both synthetic and realworld data. We show that there is a region of performance, when there is a moderate number of labeled examples, where the pseudolikelihood EM approach achieves significantly higher accuracy. 1
Gaussian process models for link analysis and transfer learning
 in Proc. of European Symp. on Artificial Neural Networks, 2008
"... Abstract This paper aims to model relational data on edges of networks. We describe appropriate Gaussian Processes (GPs) for directed, undirected, and bipartite networks. The interdependencies of edges can be effectively modeled by adapting the GP hyperparameters. The framework suggests an intima ..."
Abstract

Cited by 18 (1 self)
 Add to MetaCart
(Show Context)
Abstract This paper aims to model relational data on edges of networks. We describe appropriate Gaussian Processes (GPs) for directed, undirected, and bipartite networks. The interdependencies of edges can be effectively modeled by adapting the GP hyperparameters. The framework suggests an intimate connection between link prediction and transfer learning, which were traditionally two separate research topics. We develop an efficient learning algorithm that can handle a large number of observations. The experimental results on several realworld data sets verify superior learning capacity.
The hidden life of latent variables: Bayesian learning with mixed graph models
, 2008
"... Directed acyclic graphs (DAGs) have been widely used as a representation of conditional independence in machine learning and statistics. Moreover, hidden or latent variables are often an important component of graphical models. However, DAG models suffer from an important limitation: the family of D ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
Directed acyclic graphs (DAGs) have been widely used as a representation of conditional independence in machine learning and statistics. Moreover, hidden or latent variables are often an important component of graphical models. However, DAG models suffer from an important limitation: the family of DAGs is not closed under marginalization of hidden variables. This means that in general we cannot use a DAG to represent the independencies over a subset of variables in a larger DAG. Directed mixed graphs (DMGs) are a representation that includes DAGs as a special case, and overcomes this limitation. This paper introduces algorithms for performing Bayesian inference in Gaussian and probit DMG models. An important requirement for inference is the characterization of the distribution over parameters of the models. We introduce a new distribution for covariance matrices of Gaussian DMGs. We discuss and illustrate how several Bayesian machine learning tasks can benefit from the principle presented here: the power to model dependencies that are generated from hidden variables, but without necessarily modelling such variables explicitly.
Hidden common cause relations in relational learning
 In NIPS
, 2007
"... When predicting class labels for objects within a relational database, it is often helpful to consider a model for relationships: this allows for information between class labels to be shared and to improve prediction performance. However, there are different ways by which objects can be related wit ..."
Abstract

Cited by 14 (1 self)
 Add to MetaCart
(Show Context)
When predicting class labels for objects within a relational database, it is often helpful to consider a model for relationships: this allows for information between class labels to be shared and to improve prediction performance. However, there are different ways by which objects can be related within a relational database. One traditional way corresponds to a Markov network structure: each existing relation is represented by an undirected edge. This encodes that, conditioned on input features, each object label is independent of other object labels given its neighbors in the graph. However, there is no reason why Markov networks should be the only representation of choice for symmetric dependence structures. Here we discuss the case when relationships are postulated to exist due to hidden common causes. We discuss how the resulting graphical model differs from Markov networks, and how it describes different types of realworld relational processes. A Bayesian nonparametric classification model is built upon this graphical representation and evaluated with several empirical studies. 1
Flexible manifold embedding: A framework for semisupervised and unsupervised dimension reduction
 IEEE Transactions on Image Processing
, 2010
"... Abstract—We propose a unified manifold learning framework for semisupervised and unsupervised dimension reduction by employing a simple but effective linear regression function to map the new data points. For semisupervised dimension reduction, we aim to find the optimal prediction labels for all ..."
Abstract

Cited by 14 (9 self)
 Add to MetaCart
(Show Context)
Abstract—We propose a unified manifold learning framework for semisupervised and unsupervised dimension reduction by employing a simple but effective linear regression function to map the new data points. For semisupervised dimension reduction, we aim to find the optimal prediction labels for all the training samples, the linear regression function ( ) and the regression residue 0 = ( ) simultaneously. Our new objective function integrates two terms related to label fitness and manifold smoothness as well as a flexible penalty term defined on the residue 0. Our SemiSupervised learning framework, referred to as flexible manifold embedding (FME), can effectively utilize label information from labeled data as well as a manifold structure from both labeled and unlabeled data. By modeling the mismatch between ( ) and, we show that FME relaxes the hard linear constraint = ( ) in manifold regularization (MR), making it better cope with the data sampled from a nonlinear manifold. In addition, we propose a simplified version (referred to as FME/U) for unsupervised dimension reduction. We also show that our proposed framework provides a unified view to explain and understand many semisupervised, supervised and unsupervised dimension reduction techniques. Comprehensive experiments on several benchmark databases demonstrate the significant improvement over existing dimension reduction algorithms. Index Terms—Dimension reduction, face recognition, manifold embedding, semisupervised learning. I.
SemiSupervised Collective Classification via Hybrid Label Regularization
"... Many classification problems involve data instances that are interlinked with each other, such as webpages connected by hyperlinks. Techniques for collective classification (CC) often increase accuracy for such data graphs, but usually require a fullylabeled training graph. In contrast, we examine ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
Many classification problems involve data instances that are interlinked with each other, such as webpages connected by hyperlinks. Techniques for collective classification (CC) often increase accuracy for such data graphs, but usually require a fullylabeled training graph. In contrast, we examine how to improve the semisupervised learning of CC models when given only a sparselylabeled graph, a common situation. We first describe how to use novel combinations of classifiers to exploit the different characteristics of the relational features vs. the nonrelational features. We also extend the ideas of label regularization to such hybrid classifiers, enabling them to leverage the unlabeled data to bias the learning process. We find that these techniques, which are efficient and easy to implement, significantly increase accuracy on three real datasets. In addition, our results explain conflicting findings from prior related studies. 1.