Citation Context

...s, minA F (A), can be done in polynomial time. While general purpose algorithms [8] can be impractical due to their high order, several classes of functions admit faster, specialized algorithms, e.g. =-=[17, 18, 19]-=-. Many important problems can be cast as the minimization of a submodular objective, ranging from image segmentation [20, 12] to clustering [6]. Submodular maximization has also found numerous applica...

Citation Context

... logistic fused-lasso [Kolar et al., 2010], in changepoint detection [Harchaoui and Lévy-Leduc, 2010], in graph-cut based image segmentation [Chambolle and Darbon, 2009], in submodular optimization [=-=Jegelka et al., 2013-=-]; see also the related work in [Vert and Bleakley, 2010]. This broad applicability and importance of anisotropic TV is the key motivation towards developing carefully tuned proximity operators. Isotr...

Citation Context

...ε-approximate path. Contributions. Overall, we make the following contributions: (i) adaptive non-uniform sampling of the training objects; (ii) pairwise and away steps in the blockcoordinate setting; (iii) gap-based criterion for caching the oracle calls; (iv) regularization path for SSVM. The first three contributions are general to BCFW and thus could be applied to other block-separable optimization problems where BCFW could or have been used such as video colocalization (Joulin et al., 2014), multiple sequence alignment (Alayrac et al., 2016, App. B) or structured submodular optimization (Jegelka et al., 2013), among others. This paper is organized as follows. In Section 2, we describe the setup and review the BCFW algorithm. In Section 3, we describe our contributions: adaptive sampling (Section 3.1), pairwise and away steps (Section 3.2), caching (Section 3.3). In Section 4, we explain our algorithm to compute the regularization path. We discuss the related work in the relevant sections of the paper. Section 5 contains the experimental study of the methods. The code and datasets are available at our project webpage.2 2. Background 2.1. Structured Support Vector Machine (SSVM) In structured predic...

Citation Context

... O(n2) general optimization methods based on the semidifferential. Wei et al. [24] combine approximation with pruning to accelerate the greedy algorithm for uniform matroid constrained submodular Appearing in Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS) 2016, Cadiz, Spain. JMLR: W&CP volume 51. Copyright 2016 by the authors. maximization. Mirzasoleiman et al. [20] and Pan et al. [23] use distributed implementation to accelerate existing optimization methods. Other techniques, including stochastic sampling [19] and decomposable assumption [14], are also applied to scale up submodular optimization methods. While in this paper, we focus on the reducibility of submodular functions, a favourable property that can substantially improve the scalability of submodular optimization methods. The reducibility can directly reduce the solution space of the submodular optimization problems, while preserve all the optima in the reduced space, thereby enables us to accelerate the optimization process without incurring performance loss. Recent research shows that for some submodular functions, by evaluating marginal gains, reduction can be applied ...

Citation Context

...uding to regularized optimization [29, 5, 11], to linearly convergent algorithms [17, 10], to stochastic/online versions [23, 12], and to a powerful randomized block-coordinate version [18]. Motivated by [18], we consider FW methods for the following convex optimization problem min x f(x) s.t. x = [x(1), ..., x(n)] ∈M1 × · · · ×Mn :=M (1) where eachMi ⊂ Rmi (1 ≤ i ≤ n) is a compact convex set. Such Cartesian product constraints arise in several problems, notably the structured SVM dual [18], routing problems [19], dual of the group fused-lasso [1, 4], nearest point projection onto convex sets [15], among others. A typical approach to solving (1) is to use a block-coordinate descent method, which involves solving a projection subproblem in every iteration [21, 24, 3]. This can sometimes be very expensive, e.g., submodular minimization [9], or even computationally intractable [7]. Frank-Wolfe (FW) methods often shine in such scenarios because in contrast to (quadratic) projection based techniques at each iteration they only require optimizing a linear objective of the form min x 〈x,∇f(·)〉 s.t. x ∈M. (2) This simpler structure can bestow great computational advantages; moreover, it enable...

Citation Context

...lanar problems, or problems on loopy subgraphs with small tree-width, for which MRF optimization can still be solved efficiently), which can lead to even tighter relaxations (see framebox on page 27) =-=[70]-=-–[75]. GRAPH-CUTS AND MRF OPTIMIZATION For certain MRFs, optimizing their cost is known to be equivalent to solving a polynomial mincut problem [76], [77]. These are exactly all the binary MRFs (|L| =...