Results 1 - 10
of
277
Monotone Complexity
, 1990
"... We give a general complexity classification scheme for monotone computation, including monotone space-bounded and Turing machine models not previously considered. We propose monotone complexity classes including mAC i , mNC i , mLOGCFL, mBWBP , mL, mNL, mP , mBPP and mNP . We define a simple ..."
Abstract
-
Cited by 2825 (11 self)
- Add to MetaCart
We give a general complexity classification scheme for monotone computation, including monotone space-bounded and Turing machine models not previously considered. We propose monotone complexity classes including mAC i , mNC i , mLOGCFL, mBWBP , mL, mNL, mP , mBPP and mNP . We define a simple notion of monotone reducibility and exhibit complete problems. This provides a framework for stating existing results and asking new questions. We show that mNL (monotone nondeterministic log-space) is not closed under complementation, in contrast to Immerman's and Szelepcs 'enyi's nonmonotone result [Imm88, Sze87] that NL = co-NL; this is a simple extension of the monotone circuit depth lower bound of Karchmer and Wigderson [KW90] for st-connectivity. We also consider mBWBP (monotone bounded width branching programs) and study the question of whether mBWBP is properly contained in mNC 1 , motivated by Barrington's result [Bar89] that BWBP = NC 1 . Although we cannot answer t...
Bounded-width polynomial-size branching programs recognize exactly those languages
- in NC’, in “Proceedings, 18th ACM STOC
, 1986
"... We show that any language recognized by an NC ’ circuit (fan-in 2, depth O(log n)) can be recognized by a width-5 polynomial-size branching program. As any bounded-width polynomial-size branching program can be simulated by an NC ’ circuit, we have that the class of languages recognized by such prog ..."
Abstract
-
Cited by 270 (14 self)
- Add to MetaCart
(Show Context)
We show that any language recognized by an NC ’ circuit (fan-in 2, depth O(log n)) can be recognized by a width-5 polynomial-size branching program. As any bounded-width polynomial-size branching program can be simulated by an NC ’ circuit, we have that the class of languages recognized by such programs is exactly nonuniform NC’. Further, following
Which Problems Have Strongly Exponential Complexity?
- Journal of Computer and System Sciences
, 1998
"... For several NP-complete problems, there have been a progression of better but still exponential algorithms. In this paper, we address the relative likelihood of sub-exponential algorithms for these problems. We introduce a generalized reduction which we call Sub-Exponential Reduction Family (SERF) t ..."
Abstract
-
Cited by 242 (11 self)
- Add to MetaCart
(Show Context)
For several NP-complete problems, there have been a progression of better but still exponential algorithms. In this paper, we address the relative likelihood of sub-exponential algorithms for these problems. We introduce a generalized reduction which we call Sub-Exponential Reduction Family (SERF) that preserves sub-exponential complexity. We show that CircuitSAT is SERF-complete for all NP-search problems, and that for any fixed k, k-SAT, k-Colorability, k-Set Cover, Independent Set, Clique, Vertex Cover, are SERF--complete for the class SNP of search problems expressible by second order existential formulas whose first order part is universal. In particular, sub-exponential complexity for any one of the above problems implies the same for all others. We also look at the issue of proving strongly exponential lower bounds for AC 0 ; that is, bounds of the form 2 \Omega\Gamma n) . This problem is even open for depth-3 circuits. In fact, such a bound for depth-3 circuits with even l...
Learning Deep Architectures for AI
"... Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as i ..."
Abstract
-
Cited by 183 (30 self)
- Add to MetaCart
Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers or in complicated propositional formulae re-using many sub-formulae. Searching the parameter space of deep architectures is a difficult task, but learning algorithms such as those for Deep Belief Networks have recently been proposed to tackle this problem with notable success, beating the state-of-the-art in certain areas. This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks.
Representation learning: A review and new perspectives.
- of IEEE Conf. Comp. Vision Pattern Recog. (CVPR),
, 2005
"... Abstract-The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can b ..."
Abstract
-
Cited by 173 (4 self)
- Add to MetaCart
(Show Context)
Abstract-The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning.
Every monotone graph property has a sharp threshold
- PROC. AMER. MATH. SOC
, 1996
"... In their seminal work which initiated random graph theory Erdös and Rényi discovered that many graph properties have sharp thresholds as the number of vertices tends to infinity. We prove a conjecture of Linial that every monotone graph property has a sharp threshold. This follows from the followin ..."
Abstract
-
Cited by 170 (16 self)
- Add to MetaCart
(Show Context)
In their seminal work which initiated random graph theory Erdös and Rényi discovered that many graph properties have sharp thresholds as the number of vertices tends to infinity. We prove a conjecture of Linial that every monotone graph property has a sharp threshold. This follows from the following theorem. Let Vn(p) ={0,1} n denote the Hamming space endowed with the probability measure µp defined by µp(ɛ1,ɛ2,...,ɛn) = pk ·(1 − p) n−k,where k = ɛ1+ ɛ2+ ···+ ɛn. Let A be a monotone subset of Vn. We say that A is symmetric if there is a transitive permutation group Γ on {1, 2,...,n} such that A is invariant under Γ. Theorem. For every symmetric monotone A,ifµp(A)>ɛthen µq(A)> 1−ɛ for q = p + c1 log(1/2ɛ) / log n. (c1isan absolute constant.) 1. Graph properties A graph property is a property of graphs which depends only on their isomorphism class. Let P be a monotone graph property; that is, if a graph G satisfies P
Why does unsupervised pre-training help deep learning?
, 2010
"... Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of autoencoder variants with impressive results being obtained in several areas, mostly on vision and language datasets. The best results obtained on supervised learning tasks ..."
Abstract
-
Cited by 155 (20 self)
- Add to MetaCart
Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of autoencoder variants with impressive results being obtained in several areas, mostly on vision and language datasets. The best results obtained on supervised learning tasks often involve an unsupervised learning component, usually in an unsupervised pre-training phase. The main question investigated here is the following: why does unsupervised pre-training work so well? Through extensive experimentation, we explore several possible explanations discussed in the literature including its action as a regularizer (Erhan et al., 2009b) and as an aid to optimization (Bengio et al., 2007). Our results build on the work of Erhan et al. (2009b), showing that unsupervised pre-training appears to play predominantly a regularization role in subsequent supervised training. However our results in an online setting, with a virtually unlimited data stream, point to a somewhat more nuanced interpretation of the roles of optimization and regularization in the unsupervised pre-training effect.
Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion
, 2010
"... ..."
A new approach to the minimum cut problem
- Journal of the ACM
, 1996
"... Abstract. This paper presents a new approach to finding minimum cuts in undirected graphs. The fundamental principle is simple: the edges in a graph’s minimum cut form an extremely small fraction of the graph’s edges. Using this idea, we give a randomized, strongly polynomial algorithm that finds th ..."
Abstract
-
Cited by 128 (9 self)
- Add to MetaCart
(Show Context)
Abstract. This paper presents a new approach to finding minimum cuts in undirected graphs. The fundamental principle is simple: the edges in a graph’s minimum cut form an extremely small fraction of the graph’s edges. Using this idea, we give a randomized, strongly polynomial algorithm that finds the minimum cut in an arbitrarily weighted undirected graph with high probability. The algorithm runs in O(n 2 log 3 n) time, a significant improvement over the previous Õ(mn) time bounds based on maximum flows. It is simple and intuitive and uses no complex data structures. Our algorithm can be parallelized to run in �� � with n 2 processors; this gives the first proof that the minimum cut problem can be solved in ���. The algorithm does more than find a single minimum cut; it finds all of them. With minor modifications, our algorithm solves two other problems of interest. Our algorithm finds all cuts with value within a multiplicative factor of � of the minimum cut’s in expected Õ(n 2 � ) time, or in �� � with n 2 � processors. The problem of finding a minimum multiway cut of a graph into r pieces is solved in expected Õ(n 2(r�1) ) time, or in �� � with n 2(r�1) processors. The “trace ” of the algorithm’s execution on these two problems forms a new compact data structure for representing all small cuts and all multiway cuts in a graph. This data structure can be efficiently transformed into the
