Results 1 
9 of
9
SemiSupervised Learning Literature Survey
, 2006
"... We review the literature on semisupervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole
spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semisupervised learning. This document is a chapter ..."
Abstract

Cited by 782 (8 self)
 Add to MetaCart
We review the literature on semisupervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole
spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semisupervised learning. This document is a chapter excerpt from the author’s
doctoral thesis (Zhu, 2005). However the author plans to update the online version frequently to incorporate the latest development in the field. Please obtain the latest
version at http://www.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf
Transductive Rademacher complexity and its applications
 in Proceedings of COLT07, 20th Annual Conference on Learning Theory
, 2007
"... Abstract We develop a technique for deriving datadependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bou ..."
Abstract

Cited by 22 (2 self)
 Add to MetaCart
(Show Context)
Abstract We develop a technique for deriving datadependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bounding technique for Rademacher averages for particular algorithms, in terms of their "unlabeledlabeled" representation. This technique is relevant to many advanced graphbased transductive algorithms and we demonstrate its effectiveness by deriving error bounds to three well known algorithms. Finally, we present a new PACBayesian bound for mixtures of transductive algorithms based on our Rademacher bounds.
Nonparametric SemiSupervised Learning for Network Intrusion Detection: Combining Performance Improvements with Realistic InSitu Training
"... A barrier to the widespread adoption of learningbased network intrusion detection tools is the insitu training requirements for effective discrimination of malicious traffic. Supervised learning techniques necessitate a quantity of labeled examples that is often intractable, and at best costp ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
A barrier to the widespread adoption of learningbased network intrusion detection tools is the insitu training requirements for effective discrimination of malicious traffic. Supervised learning techniques necessitate a quantity of labeled examples that is often intractable, and at best costprohibitive. Recent advances in semisupervised techniques have demonstrated the ability to generalize knowledge based on a significantly smaller number of provided examples. In network intrusion detection, placing reasonable requirements on the number of training examples provides realistic expectations that a learningbased system can be trained in the environment where it will be deployed. This insitu training is necessary to ensure that the assumptions associated with the learning process hold, and thereby support a reasonable belief in the generalization ability of the resulting model. In this paper, we describe the application of a carefully selected nonparametric, semisupervised learning algorithm to the network intrusion problem, and compare the performance to other model types using featurebased data derived from an operational network. We demonstrate dramatic performance improvements over supervised learning and anomaly detection in discriminating real, previously unseen, malicious network traffic while generating an order of magnitude fewer false alerts than any alternative, including a signature IDS tool deployed on the same network.
MultiView Budgeted Learning under Label and Feature Constraints Using LabelGuided GraphBased Regularization
"... Budgeted learning under constraints on both the amount of labeled information and the availability of features at test time pertains to a large number of real world problems. Ideas from multiview learning, semisupervised learning, and even active learning have applicability, but a common framework ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Budgeted learning under constraints on both the amount of labeled information and the availability of features at test time pertains to a large number of real world problems. Ideas from multiview learning, semisupervised learning, and even active learning have applicability, but a common framework whose assumptions fit these problem spaces is nontrivial to construct. We leverage ideas from these fields based on graph regularizers to construct a robust framework for learning from labeled and unlabeled samples in multiple views that are nonindependent and include features that are inaccessible at the time the model would need to be applied. We describe examples of applications that fit this scenario, and we provide experimental results to demonstrate the effectiveness of knowledge carryover from trainingonly views. 1.
Research Statement MariaFlorina Balcan My primary research interests are in Machine Learning, Algorithmic Game Theory, and Algorithms, as well as
"... on the interactions between these areas and other areas of Computer Science and Economics. The common thread of my work is a rigorous study of real world problems, involving both the development of mathematical models and the design of algorithms and tools for their analysis. 1 Brief Overview New Fr ..."
Abstract
 Add to MetaCart
(Show Context)
on the interactions between these areas and other areas of Computer Science and Economics. The common thread of my work is a rigorous study of real world problems, involving both the development of mathematical models and the design of algorithms and tools for their analysis. 1 Brief Overview New Frameworks and Algorithms for Machine Learning I am particularly excited by problems that explore new frontiers of learning, and as a consequence a significant part of my work has focused on providing theoretical models for important new learning paradigms which are not captured by existing theoretical frameworks. Over the years, Machine Learning has grown into a broad discipline that has produced fundamental theories of learning processes, as well as learning algorithms that are routinely used in commercial systems. The primary theoretical advances have been for supervised learning problems [24], where a target function (e.g., a classifier) is estimated using only labeled examples. For example, in spam detection an automatic classifier to label emails as spam or not would be trained using a sample of previous emails labeled by a human user. However, for most contemporary practical problems there is often useful additional information available in form of cheap and plentiful unlabeled data: e.g., unlabeled emails for the spam detection problem. As a consequence, there has recently been substantial interest in using unlabeled data to improve learning. Several different algorithmic approaches have been developed
REQUIREMENTS
"... This thesis would be impossible without a wise guidance of my advisor, Prof. Ran ElYaniv. The path to the results presented in this thesis was long and I thank Ran for never loosing the faith in the final success. In both peaceful and stressful times, Ran constantly supported and navigated me towar ..."
Abstract
 Add to MetaCart
(Show Context)
This thesis would be impossible without a wise guidance of my advisor, Prof. Ran ElYaniv. The path to the results presented in this thesis was long and I thank Ran for never loosing the faith in the final success. In both peaceful and stressful times, Ran constantly supported and navigated me towards stronger results. Many thanks go to my coauthors during the thesis period: Ron
UNIVERSITY OF WISCONSIN–MADISON
"... All Rights ReservedFor my parents, who always taught me to strive for the highest achievements possible. i ii In many realworld learning scenarios, acquiring a large amount of labeled training data is expensive and timeconsuming. Semisupervised learning (SSL) is the machine learning paradigm conc ..."
Abstract
 Add to MetaCart
(Show Context)
All Rights ReservedFor my parents, who always taught me to strive for the highest achievements possible. i ii In many realworld learning scenarios, acquiring a large amount of labeled training data is expensive and timeconsuming. Semisupervised learning (SSL) is the machine learning paradigm concerned with utilizing unlabeled data to try to build better classifiers and regressors. Unlabeled data is a powerful resource, yet SSL can be difficult to apply in practice. The objective of this dissertation is to move the field toward more practical and robust SSL. This is accomplished by several key contributions. First, we introduce the online (and active) semisupervised learning setting, which considers large amounts of mostly unlabeled data arriving constantly over time. An online SSL classifier must be able to make efficient predictions at any moment and update itself in response to labeled and unlabeled data. Previously, almost all SSL assumed a fixed dataset was available before training began, and receiving new data meant retraining a potentially slow model. We present two families of online semisupervised learners that reformulate the popular manifold and cluster assumptions
Spectral Norm Regularization of Orthonormal Representations for
"... Recent literature [1] suggests that embedding a graph on an unit sphere leads to better generalization for graph transduction. However, the choice of optimal embedding and an efficient algorithm to compute the same remains open. In this paper, we show that orthonormal representations, a class of un ..."
Abstract
 Add to MetaCart
(Show Context)
Recent literature [1] suggests that embedding a graph on an unit sphere leads to better generalization for graph transduction. However, the choice of optimal embedding and an efficient algorithm to compute the same remains open. In this paper, we show that orthonormal representations, a class of unitsphere graph embeddings are PAC learnable. Existing PACbased analysis do not apply as the VC dimension of the function class is infinite. We propose an alternative PACbased bound, which do not depend on the VC dimension of the underlying function class, but is related to the famous Lovász ϑ function. The main contribution of the paper is SPORE, a SPectral regularized ORthonormal Embedding for graph transduction, derived from the PAC bound. SPORE is posed as a nonsmooth convex function over an elliptope. These problems are usually solved as semidefinite programs (SDPs) with time complexity O(n6). We present, Infeasible Inexact proximal (IIP): an Inexact proximal method which performs subgradient procedure on an approximate projection, not necessarily feasible. IIP is more scalable than SDP, has an O ( 1√ T) convergence, and is generally applicable whenever a suitable approximate projection is available. We use IIP to compute SPORE where the approximate projection step is computed by FISTA, an accelerated gradient descent procedure. We show that the method has a convergence rate of O ( 1√ T The proposed algorithm easily scales to 1000’s of vertices, while the standard SDP computation does not scale beyond few hundred vertices. Furthermore, the analysis presented here easily extends to the multiple graph setting. 1