Results 1  10
of
12
Statistical and computational tradeoffs in estimation of sparse principal components
, 2014
"... In recent years, Sparse Principal Component Analysis has emerged as an extremely popular dimension reduction technique for highdimensional data. The theoretical challenge, in the simplest case, is to estimate the leading eigenvector of a population covariance matrix under the assumption that this e ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
In recent years, Sparse Principal Component Analysis has emerged as an extremely popular dimension reduction technique for highdimensional data. The theoretical challenge, in the simplest case, is to estimate the leading eigenvector of a population covariance matrix under the assumption that this eigenvector is sparse. An impressive range of estimators have been proposed; some of these are fast to compute, while others are known to achieve the minimax optimal rate over certain Gaussian or subgaussian classes. In this paper we show that, under a widelybelieved assumption from computational complexity theory, there is a fundamental tradeoff between statistical and computational performance in this problem. More precisely, working with new, larger classes satisfying a Restricted Covariance Concentration condition, we show that no randomised polynomial time algorithm can achieve the minimax optimal rate. On the other hand, we also study a (polynomial time) variant of the wellknown semidefinite relaxation estimator, and show that it attains essentially the optimal rate among all randomised polynomial time algorithms.
Statistical Active Learning Algorithms
, 1307
"... We describe a framework for designing efficient active learning algorithms that are tolerant to random classification noise. The framework is based on active learning algorithms that are statistical in the sense that they rely on estimates of expectations of functions of filtered random examples. It ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
We describe a framework for designing efficient active learning algorithms that are tolerant to random classification noise. The framework is based on active learning algorithms that are statistical in the sense that they rely on estimates of expectations of functions of filtered random examples. It builds on the powerful statistical query framework of Kearns [Kea98]. We show that any efficient active statistical learning algorithm can be automatically converted to an efficient active learning algorithm which is tolerant to random classification noise as well as other forms of “uncorrelated ” noise. The complexity of the resulting algorithms has informationtheoretically optimal quadratic dependence on 1/(1−2η), where η is the noise rate. We demonstrate the power of our framework by showing that commonly studied concept classes including thresholds, rectangles, and linear separators can be efficiently actively learned in our framework. These results combined with our generic conversion lead to the first known computationallyefficient algorithms for actively learning some of these concept classes in the presence of random classification noise that provide exponential improvement in the dependence on the error ǫ over their passive counterparts. In addition, we show that our algorithms can be automatically converted to efficient active differentiallyprivate algorithms. This leads to the first differentiallyprivate active learning algorithms with exponential label savings over the passive case. 1
Random Structures and Algorithms
, 2004
"... We provide an introduction to the analysis of random combinatorial structures and some of the associated computational problems. ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
We provide an introduction to the analysis of random combinatorial structures and some of the associated computational problems.
On the Hardness of Signaling
, 2014
"... There has been a recent surge of interest in the role of information in strategic interactions. Much of this work seeks to understand how the realized equilibrium of a game is influenced by uncertainty in the environment and the information available to players in the game. Lurking beneath this lite ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
There has been a recent surge of interest in the role of information in strategic interactions. Much of this work seeks to understand how the realized equilibrium of a game is influenced by uncertainty in the environment and the information available to players in the game. Lurking beneath this literature is a fundamental, yet largely unexplored, algorithmic question: how should a “market maker ” who is privy to additional information, and equipped with a specified objective, inform the players in the game? This is an informational analogue of the mechanism design question, and views the information structure of a game as a mathematical object to be designed, rather than an exogenous variable. We initiate a complexitytheoretic examination of the design of optimal information structures in general Bayesian games, a task often referred to as signaling. We focus on one of the simplest instantiations of the signaling question: Bayesian zerosum games, and a principal who must choose an information structure maximizing the equilibrium payoff of one of the players. In this setting, we show that optimal signaling is computationally intractable, and in some cases hard to approximate, assuming that it is hard to recover a planted clique from an ErdősRényi random graph. This is despite the fact that equilibria in these games are computable in polynomial time, and therefore suggests that the hardness of optimal signaling is a distinct phenomenon from the hardness of equilibrium computation. Necessitated by the nonlocal nature of information structures, enroute to our results we prove an “amplification lemma ” for the planted clique problemwhichmay be of independent interest. Specifically, we show that even if we plant many cliques in an ErdősRényi random graph, so much so that most nodes in the graph are in some planted clique, recovering a constant fraction of the planted cliques is no easier than the traditional planted clique problem. 1
Usable Human Authentication: A Quantitative Treatment
, 2014
"... and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government or any other entity. ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government or any other entity.
Structure learning of antiferromagnetic Ising models
 In NIPS
, 2014
"... In this paper we investigate the computational complexity of learning the graph structure underlying a discrete undirected graphical model from i.i.d. samples. Our first result is an unconditional computational lower bound of (pd/2) for learning general graphical models on p nodes of maximum degree ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
In this paper we investigate the computational complexity of learning the graph structure underlying a discrete undirected graphical model from i.i.d. samples. Our first result is an unconditional computational lower bound of (pd/2) for learning general graphical models on p nodes of maximum degree d, for the class of socalled statistical algorithms recently introduced by Feldman et al. [1]. The construction is related to the notoriously dicult learning parities with noise problem in computational learning theory. Our lower bound suggests that the ÂO(pd+2) runtime required by Bresler, Mossel, and Sly’s [2] exhaustivesearch algorithm cannot be significantly improved without restricting the class of models. Aside from structural assumptions on the graph such as it being a tree, hypertree, treelike, etc., many recent papers on structure learning assume that the model has the correlation decay property. Indeed, focusing on ferromagnetic Ising models, Bento and Montanari [3] showed that all known lowcomplexity algorithms fail to learn simple graphs when the interaction strength exceeds a number related to the correlation decay threshold. Our second set of results gives a class of repelling (antiferromagnetic) models that have the opposite behavior: very strong interaction allows ecient learning in time ÂO(p2). We provide an algorithm whose performance interpolates between ÂO(p2) and ÂO(pd+2) depending on the strength of the repulsion. 1