#### DMCA

## Sum-of-Squares Proofs and the Quest toward Optimal Algorithms

Citations: | 5 - 0 self |

### Citations

1798 |
Independent component analysis, a new concept
- Comon
- 1994
(Show Context)
Citation Context ...s guess a sufficiently close approximation to it.) 15If the distribution x consists of m independent random variables then better guarantees can be achieved using Independent Component Analysis (ICA) =-=[Com94]-=-. See [GVX14] for the current state of art in this setting. However we are interested here in the more general case. 18 Boaz Barak and David Steurer Why does the sum-of-squares method work? The analys... |

1286 |
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
- Olshausen, Field
- 1996
(Show Context)
Citation Context ...n distribution of the form {y = Ax}, where A ∈ Rn×m is a matrix and x is a random m-dimensional vector from a distribution over sparse vectors. This problem, initiated by the work Olshausen and Field =-=[OF96]-=- in computational neuroscience, has found a variety of uses in machine learning, computer vision, and image processing (see, e.g. [AAJ+13] and the references therein). The appeal of this problem is th... |

1195 | Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming
- Goemans, Williamson
- 1995
(Show Context)
Citation Context ...which give better guarantees than (1) for graphs where φG is sufficiently small as a function of n. 5Paraphrasing John Wheeler. 4 Boaz Barak and David Steurer the Goemans–Williamson Max Cut algorithm =-=[GW95]-=-, and the Lovász ϑ function [Lov79]. As we’ve seen for the example of Cheeger’s Inequality, in many of those settings this meta-algorithm gives non-trivial approximation guarantees which are the best... |

558 | Global optimization with polynomials and the problems of moments
- Lasserre
- 2001
(Show Context)
Citation Context ... are good sources for some of these topics. The SOS algorithm was developed in slightly different forms by several researchers, including Shor [Sho87], Nesterov [Nes00], Parrilo [Par00], and Lasserre =-=[Las01]-=-. It can be viewed as a strengthening of other “meta-algorithms” proposed by [SA90, LS91] (also known as linear and semi-definite programming hierarchies).7 Our description of the SOS meta algorithm f... |

461 |
On the Shannon capacity of a graph
- Lovász
- 1979
(Show Context)
Citation Context ...1) for graphs where φG is sufficiently small as a function of n. 5Paraphrasing John Wheeler. 4 Boaz Barak and David Steurer the Goemans–Williamson Max Cut algorithm [GW95], and the Lovász ϑ function =-=[Lov79]-=-. As we’ve seen for the example of Cheeger’s Inequality, in many of those settings this meta-algorithm gives non-trivial approximation guarantees which are the best known, but there are no hardness re... |

370 | Structured Semidefinite Programs and Semialgebraic Geometry Methods in Robustness and Optimization
- Parrilo
- 2000
(Show Context)
Citation Context ... the monograph [Lau09] are good sources for some of these topics. The SOS algorithm was developed in slightly different forms by several researchers, including Shor [Sho87], Nesterov [Nes00], Parrilo =-=[Par00]-=-, and Lasserre [Las01]. It can be viewed as a strengthening of other “meta-algorithms” proposed by [SA90, LS91] (also known as linear and semi-definite programming hierarchies).7 Our description of th... |

343 | Cones of matrices and set-functions and 0-1 optimization - Lovász, Schrijver - 1991 |

332 | A lower bound for the smallest eigenvalue of the Laplacian, in “Problems in analysis (Papers dedicated to Salomon Bochner,1969 - Cheeger - 1970 |

311 | Expander flows, geometric embeddings and graph partitioning
- Arora, Rao, et al.
(Show Context)
Citation Context ...some examples where using higher degree proofs does help, some of them suspiciously close in nature to the expansion problem. One such example comes from the beautiful work of Arora, Rao and Vazirani =-=[ARV09]-=- who showed that φG ≤ O( √ log n) · φ(6)G , which is better than the guarantee of Theorem 3.1 for φG 1/ log n. However, this is not known to contradict the SSEH or UGC, which apply to the case when ... |

311 | On the power of unique 2-prover 1-round games
- Khot
- 2002
(Show Context)
Citation Context ...ntees for a problem, it’s very hard to rule out that cleverer algorithms couldn’t get even better guarantees. In 2002, Subhash Khot formulated a conjecture, known as the Unique Games Conjecture (UGC) =-=[Kho02]-=-. A large body of follow up works has shown that this conjecture (whose description is deferred to Section 1.1 below) implies many hardness results that overcome the above challenge and match the best... |

275 | λ1, isoperimetric inequalities for graphs, and superconcentrators - ALON, MILMAN - 1985 |

246 | A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems - Sherali, Adams - 1990 |

170 | The Unique Games conjecture, integrality gap for cut problems and embeddability of negative type metrics into l 1 - Khot, Vishnoi |

162 | Difference equations, isoperimetric inequality and transience of certain random walks - Dodziuk - 1984 |

154 |
Résumé de la théorie métrique des produits tensoriels topologiques
- Grothendieck
- 1953
(Show Context)
Citation Context ...o this algorithm as the UGC meta-algorithm. It can be viewed as a common generalization of several well known algorithms, including those that underlie Cheeger’s Inequality, Grothendieck’s Inequality =-=[Gro53]-=-, 3The adjacency matrix of a graph G is the |V | × |V | matrix A with 0/1 entries such that Au,v = 1 iff {u, v} ∈ E. 4As we will mention later, there are algorithms to approximate φG up to factors dep... |

154 | Some concrete aspects of Hilbert’s 17th Problem, Real algebraic geometry and ordered structures (Baton Rouge
- Reznick
- 1996
(Show Context)
Citation Context ...ee lower bounds for it, that were later rediscovered and expanded upon by [Sch08, Tul09]. All these are motivated by the works in real geometry related to Hilbert’s 17th problem; see Reznick’s survey =-=[Rez00]-=- for more on this research area. One difference between our focus here and much of the other literature on the SOS algorithm is that we are content with proving that the algorithm supplies an approxim... |

150 | Sums of squares, moment matrices and optimization over polynomials
- Laurent
- 2009
(Show Context)
Citation Context ... many applications that have nothing to do with the UGC or even approximation algorithms at large. The volume [BPT13] and Sum-of-squares proofs and the quest toward optimal algorithms 7 the monograph =-=[Lau09]-=- are good sources for some of these topics. The SOS algorithm was developed in slightly different forms by several researchers, including Shor [Sho87], Nesterov [Nes00], Parrilo [Par00], and Lasserre ... |

134 | Optimal Algorithms and Inapproximability Results For Every CSP
- Raghavendra
- 2008
(Show Context)
Citation Context ...own that this conjecture (and related ones) imply that this meta-algorithm is optimal for a vast number of problems, including all those examples above. For example, a beautiful result of Raghavendra =-=[Rag08]-=- showed that for every constraint-satisfaction problem (a large class of problems that includes many problems of interest such as Max k-SAT, k-Coloring, and Max-Cut), the UGC meta-algorithm gives the ... |

129 |
Squared functional systems and optimization problems
- Nesterov
- 2000
(Show Context)
Citation Context ...imal algorithms 7 the monograph [Lau09] are good sources for some of these topics. The SOS algorithm was developed in slightly different forms by several researchers, including Shor [Sho87], Nesterov =-=[Nes00]-=-, Parrilo [Par00], and Lasserre [Las01]. It can be viewed as a strengthening of other “meta-algorithms” proposed by [SA90, LS91] (also known as linear and semi-definite programming hierarchies).7 Our ... |

123 | A Nullstellensatz and a Positivstellensatz in semialgebraic geometry - Stengle - 1974 |

122 | A comparison of the Sherali-Adams, Lovász-Schrijver, and Lasserre relaxations for 0-1 programming.Math
- Laurent
(Show Context)
Citation Context ...ynomials and −1 = S + ∑ Qi · Pi . (2) We say that the polynomials S,Q1, . . . , Qm in the conclusion of the theorem form an SOS proof refuting the system of polynomial equations8 E . Clearly the 7See =-=[Lau03]-=- for a comparison. 8In this survey we restrict attention to polynomial equalities as opposed to inequalities, which turns out to be without loss of generality for our purposes. If we have a system of ... |

114 | Fixed-parameter tractability and completeness ii: On completeness for w[1 - Downey, Fellows - 1995 |

79 | Subexponential algorithms for Unique Games and related problems
- Arora, Barak, et al.
- 2010
(Show Context)
Citation Context ... small set expansion problem of approximating φG(S) for small sets S, we can beat the degree 2 bounds with degree ` = nτ proofs where τ is a parameter tending to zero with the parameter ε of the SSEH =-=[ABS10]-=-. This yields a sub-exponential algorithm for the small-set expansion problem (which can be extended to the Unique Games problem as well) that “barely misses” refuting the SSEH and UGC. We will also s... |

68 | Clique is hard to approximate within n1 - Hastad - 1999 |

54 | Anneaux préordonnés - Krivine - 1964 |

51 |
Über die Zerlegung definiter Funktionen in Quadrate. Abhandlungen aus dem mathematischen Seminar der Univ
- Artin
- 1927
(Show Context)
Citation Context ...swered negatively by Hilbert in 1888, who went on to ask as his 17th problem whether any such polynomial can be written as a sum of squares of rational functions. A positive answer was given by Artin =-=[Art27]-=-, and considerably strengthened by Krivine and Stengle. In particular, the following theorem is a corollary of their results, which captures much of the general case. Theorem 2.1 (Corollary of the Pos... |

43 |
Semidefinite optimization and convex algebraic geometry
- Blekherman, Parrilo, et al.
- 2013
(Show Context)
Citation Context ...he SOS algorithm barely scratches the surface of this fascinating topic, which has a great many applications that have nothing to do with the UGC or even approximation algorithms at large. The volume =-=[BPT13]-=- and Sum-of-squares proofs and the quest toward optimal algorithms 7 the monograph [Lau09] are good sources for some of these topics. The SOS algorithm was developed in slightly different forms by sev... |

41 | Analysis of Boolean Functions - O’Donnell - 2014 |

41 | Graph expansion and the unique games conjecture
- Raghavendra, Steurer
- 2010
(Show Context)
Citation Context ...computational complexity. 1.1. The UGC and SSEH conjectures. Instead of the Unique Games Conjecture, in this survey we focus on a related conjecture known as the Small-Set Expansion Hypothesis (SSEH) =-=[RS10]-=-. The SSEH implies the UGC [RS10], and while there is no known implication in the other direction, there are several results suggesting that these two conjectures are probably equivalent [RS10, RST10,... |

40 | CSP gaps and reductions in the Lasserre hierarchy - Tulsiani - 2009 |

39 | Clique is hard to approximate within n1−ǫ - H̊astad - 1996 |

37 | Exact recovery of sparsely-used dictionaries
- Spielman, Wang, et al.
- 2012
(Show Context)
Citation Context ...oal is to reconstruct the vector x(0). This is a natural problem in its own right, and is also a useful subroutine in various settings; see [DH13]. Demanet and Hand [DH13] gave an algorithm (based on =-=[SWW12]-=-) that recovers x(0) by searching for the vector x in U that maximizes ‖x‖∞/‖x‖1 (which can be done efficiently by n linear programs). It is not hard to show that x(0) has to have less than |n|/√d coo... |

36 | Strong computational lower bounds via parameterized complexity - Chen, Huang, et al. - 2006 |

29 | level Lasserre lower bounds for certain k-CSPs - Schoenebeck - 2008 |

25 | Sum-of-Squares Proofs, and their Applications - Hypercontractivity - 2012 |

24 | Improved inaproximability results for maxclique, chromatic number and approximate graph coloring - Khot - 2001 |

23 | Learning sparsely used overcomplete dictionaries via alternating minimization - Agarwal, Anandkumar, et al. |

22 | Linear lower bound on degrees of Positivstellensatz calculus proofs for the parity
- Grigoriev
(Show Context)
Citation Context ...expectation notation introduced in [BBH+12] instead of Lasserre’s notion of “moment matrices”. The Positivstellensatz/SOS proof system was first studied by Grigoriev and Vorobjov [GV01] and Grigoriev =-=[Gri01]-=- proved some degree lower bounds for it, that were later rediscovered and expanded upon by [Sch08, Tul09]. All these are motivated by the works in real geometry related to Hilbert’s 17th problem; see ... |

21 | Inapproximability results for maximum edge biclique, minimum linear arrangement, and sparsest cut
- Ambühl, Mastrolilli, et al.
(Show Context)
Citation Context ...pendent set a hard one, but what about expansion? Here the situation is more complicated. We know that we can’t efficiently compute φG exactly, and we can’t even get an arbitrarily good approximation =-=[AMS11]-=-, but we actually do have efficient algorithms with nontrivial approximation guarantees for φG. Discrete versions of Cheeger’s inequality [Che70, Dod84, AM85, Alo86] yield such an estimate, namely d−λ... |

16 | New Algorithms for Learning Incoherent and Overcomplete Dictionaries,” arXiv - Arora, Ge, et al. - 2013 |

16 | Towards computing the grothendieck constant
- Raghavendra, Steurer
- 2009
(Show Context)
Citation Context ...e swaths of problems via a single algorithm; we may just need to consider a different algorithm. To summarize, regardless of whether it refutes the UGC or not, understanding the power of the SOS 6See =-=[RS09b]-=- for the precise statement of Grothendieck’s Inequality and this result. Curiously, the UGC implies that Grothendieck’s Inequality yields the best efficient approximation factor for the correlation of... |

15 |
On the unique games conjecture (invited survey
- Khot
- 2010
(Show Context)
Citation Context ...ar14a] for informal overviews of some of these issues. For the reader interested in learning more about the Unique Games Conjecture, there are three excellent surveys on this topic. Khot’s CCC survey =-=[Kho10b]-=- gives a fairly comprehensive overview of the state of knowledge on the UGC circa 2010, while his ICM survey [Kho10a] focuses on some of the techniques and connections that arose in the works around t... |

14 | Approximations for the isoperimetric and spectral profile of graphs and related parameters - Raghavendra, Steurer, et al. - 2010 |

14 | Reductions between expansion problems
- Raghavendra, Steurer, et al.
(Show Context)
Citation Context ...isfy. Similarly, the UGC (or closely related variants) imply there are no efficient algorithms that give a better estimate for the sparsest cut of a graph than the one implied by Cheeger’s Inequality =-=[RST12]-=- and no better efficient estimate for the maximum correlation of a matrix with ±1-valued vectors than the one given by Grothendieck’s Inequality.6 To summarize: If true, the Unique Games Conjecture te... |

13 |
An approach to obtaining global extremums in polynomial mathematical programming problems
- Shor
- 1987
(Show Context)
Citation Context ...e quest toward optimal algorithms 7 the monograph [Lau09] are good sources for some of these topics. The SOS algorithm was developed in slightly different forms by several researchers, including Shor =-=[Sho87]-=-, Nesterov [Nes00], Parrilo [Par00], and Lasserre [Las01]. It can be viewed as a strengthening of other “meta-algorithms” proposed by [SA90, LS91] (also known as linear and semi-definite programming h... |

12 | Sdp integrality gaps with local ℓ1-embeddability - Khot, Saket |

11 | Fast SDP algorithms for constraint satisfaction problems
- Steurer
- 2010
(Show Context)
Citation Context ...be optimal is based on semidefinite programming and it uses this technique in a very particular and quite restricted way. (In many settings, this meta-algorithm can be implemented in near-linear time =-=[Ste10]-=-.) We will refer to this algorithm as the UGC meta-algorithm. It can be viewed as a common generalization of several well known algorithms, including those that underlie Cheeger’s Inequality, Grothend... |

8 |
Complexity of null-and positivstellensatz proofs
- Grigoriev, Vorobjov
- 2001
(Show Context)
Citation Context ...ugh we use the pseudoexpectation notation introduced in [BBH+12] instead of Lasserre’s notion of “moment matrices”. The Positivstellensatz/SOS proof system was first studied by Grigoriev and Vorobjov =-=[GV01]-=- and Grigoriev [Gri01] proved some degree lower bounds for it, that were later rediscovered and expanded upon by [Sch08, Tul09]. All these are motivated by the works in real geometry related to Hilber... |

8 | R.: SDP integrality gaps with local `1-embeddability - Khot, Saket - 2009 |

7 | Inapproximability of NP-complete problems, discrete Fourier analysis, and geometry. To appear
- Khot
- 2010
(Show Context)
Citation Context ...mes Conjecture, there are three excellent surveys on this topic. Khot’s CCC survey [Kho10b] gives a fairly comprehensive overview of the state of knowledge on the UGC circa 2010, while his ICM survey =-=[Kho10a]-=- focuses on some of the techniques and connections that arose in the works around the UGC. Trevisan [Tre12] gives a wonderfully accessible introduction to the UGC, using the Max-Cut problem as a runni... |

4 | Convergence of SDP hierarchies for polynomial optimization on the hypersphere
- Doherty, Wehner
(Show Context)
Citation Context ... the condition implies that d = dimW ≤ O(n1/p), and it is known that approximately bounding a degree-O(1) polynomial on the ddimensional sphere requires an SOS proof of at most O(d) degree (e.g., see =-=[DW12]-=- and the references therein). Combining Corollary 5.3 with Theorem 5.1 implies that for every ε, δ there exists some τ (tending to zero with ε), such that if we want to distinguish between the case th... |

2 |
Fourier pca
- Goyal, Vempala, et al.
- 2014
(Show Context)
Citation Context ...ficiently close approximation to it.) 15If the distribution x consists of m independent random variables then better guarantees can be achieved using Independent Component Analysis (ICA) [Com94]. See =-=[GVX14]-=- for the current state of art in this setting. However we are interested here in the more general case. 18 Boaz Barak and David Steurer Why does the sum-of-squares method work? The analysis of algorit... |

2 | Integrality Gaps for Strong SDP - Raghavendra, Steurer - 2009 |

2 | On Khot’s Unique Games Conjecture
- Trevisan
(Show Context)
Citation Context ...omprehensive overview of the state of knowledge on the UGC circa 2010, while his ICM survey [Kho10a] focuses on some of the techniques and connections that arose in the works around the UGC. Trevisan =-=[Tre12]-=- gives a wonderfully accessible introduction to the UGC, using the Max-Cut problem as a running example to explain in detail the UGC’s connection to semidefinite programming. As a sign of how rapidly ... |

1 | Decomposition of the completer-graph into completer-partitergraphs - Alon - 1986 |

1 | Truth vs - Barak - 2012 |

1 | Fun and Games with Sums of Squares - Barak - 2014 |

1 | Structure vs. Combinatorics in Computational Complexity - Barak - 2014 |

1 |
Dictionary Learning via the Sum-ofSquares Method. Unpublished manuscript
- Barak, Kelner, et al.
- 2014
(Show Context)
Citation Context ...namely less than √ n nonzero entries.15 In contrast, the SOS method can be used to approximately recover the dictionary matrix A as long as x has o(n) nonzero (or more generally, significant) entries =-=[BKS14a]-=-. 4.1. Sparse vector recovery. We say a vector x is µ-sparse if the 0/1 indicator 1supp x of the support of x has norm-squared µ = ‖1supp x‖22. The ratio µ/‖1‖22 is the fraction of non-zero coordinate... |

1 |
Rounding Sum of Squares Relaxations
- Barak, Kelner, et al.
- 2014
(Show Context)
Citation Context ...odistribution {x} satisfying the system {P = ϕ, P1 = 0, . . . , Pm = 0}, we can find some particular x∗ that satisfies P (x∗) ≤ f(ϕ). Our approach to doing so (based on the authors’ paper with Kelner =-=[BKS14b]-=-) can be summarized as follows: Solve the problem pretending that {x} is an actual distribution over solutions, and if all the steps you used have low-degree SOS proofs, the solution still works even ... |

1 | Recovering the Sparsest Element in a Subspace
- Demanet, Hand
- 2013
(Show Context)
Citation Context ... . . , x(d) are independent standard Gaussian vectors. The goal is to reconstruct the vector x(0). This is a natural problem in its own right, and is also a useful subroutine in various settings; see =-=[DH13]-=-. Demanet and Hand [DH13] gave an algorithm (based on [SWW12]) that recovers x(0) by searching for the vector x in U that maximizes ‖x‖∞/‖x‖1 (which can be done efficiently by n linear programs). It i... |