Results 1  10
of
10
From MAP to marginals: Variational inference in Bayesian submodular models
 In Neural Information Processing Systems (NIPS
, 2014
"... Submodular optimization has found many applications in machine learning and beyond. We carry out the first systematic investigation of inference in probabilistic models defined through submodular functions, generalizing regular pairwise MRFs and Determinantal Point Processes. In particular, we pres ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Submodular optimization has found many applications in machine learning and beyond. We carry out the first systematic investigation of inference in probabilistic models defined through submodular functions, generalizing regular pairwise MRFs and Determinantal Point Processes. In particular, we present LFIELD, a variational approach to general logsubmodular and logsupermodular distributions based on sub and supergradients. We obtain both lower and upper bounds on the logpartition function, which enables us to compute probability intervals for marginals, conditionals and marginal likelihoods. We also obtain fully factorized approximate posteriors, at the same computational cost as ordinary submodular optimization. Our framework results in convex problems for optimizing over differentials of submodular functions, which we show how to optimally solve. We provide theoretical guarantees of the approximation quality with respect to the curvature of the function. We further establish natural relations between our variational approach and the classical meanfield method. Lastly, we empirically demonstrate the accuracy of our inference scheme on several submodular models. 1
Provable submodular minimization using Wolfe’s algorithm
 In NIPS
, 2014
"... Owing to several applications in large scale learning and vision problems, fast submodular function minimization (SFM) has become a critical problem. Theoretically, unconstrained SFM can be performed in polynomial time [10, 11]. However, these algorithms are typically not practical. In 1976, Wolfe [ ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Owing to several applications in large scale learning and vision problems, fast submodular function minimization (SFM) has become a critical problem. Theoretically, unconstrained SFM can be performed in polynomial time [10, 11]. However, these algorithms are typically not practical. In 1976, Wolfe [21] proposed an algorithm to find the minimum Euclidean norm point in a polytope, and in 1980, Fujishige [3] showed how Wolfe’s algorithm can be used for SFM. For general submodular functions, this FujishigeWolfe minimum norm algorithm seems to have the best empirical performance. Despite its good practical performance, very little is known about Wolfe’s minimum norm algorithm theoretically. To our knowledge, the only result is an exponential time analysis due to Wolfe [21] himself. In this paper we give a maiden convergence analysis of Wolfe’s algorithm. We prove that in t iterations, Wolfe’s algorithm returns an O(1/t)approximate solution to the minnorm point on any polytope. We also prove a robust version of Fujishige’s theorem which shows that anO(1/n2)approximate solution to the minnorm point on the base polytope implies exact submodular minimization. As a corollary, we get the first pseudopolynomial time guarantee for the FujishigeWolfe minimum norm algorithm for unconstrained submodular function minimization. 1
Learning Mixtures of Submodular Functions for Image Collection Summarization
"... We address the problem of image collection summarization by learning mixtures of submodular functions. Submodularity is useful for this problem since it naturally represents characteristics such as fidelity and diversity, desirable for any summary. Several previously proposed image summarization sco ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
We address the problem of image collection summarization by learning mixtures of submodular functions. Submodularity is useful for this problem since it naturally represents characteristics such as fidelity and diversity, desirable for any summary. Several previously proposed image summarization scoring methodologies, in fact, instinctively arrived at submodularity. We provide classes of submodular component functions (including some which are instantiated via a deep neural network) over which mixtures may be learnt. We formulate the learning of such mixtures as a supervised problem via largemargin structured prediction. As a loss function, and for automatic summary scoring, we introduce a novel summary evaluation method called VROUGE, and test both submodular and nonsubmodular optimization (using the submodularsupermodular procedure) to learn a mixture of submodular functions. Interestingly, using nonsubmodular optimization to learn submodular functions provides the best results. We also provide a new data set consisting of 14 realworld image collections along with many humangenerated ground truth summaries collected using Amazon Mechanical Turk. We compare our method with previous work on this problem and show that our learning approach outperforms all competitors on this new data set. This paper provides, to our knowledge, the first systematic approach for quantifying the problem of image collection summarization, along with a new data set of image collections and human summaries. 1
Fast Flux Discriminant for LargeScale Sparse Nonlinear Classification
"... In this paper, we propose a novel supervised learning method, Fast Flux Discriminant (FFD), for largescale nonlinear classification. Compared with other existing methods, FFD has unmatched advantages, as it attains the efficiency and interpretability of linear models as well as the accuracy of no ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
In this paper, we propose a novel supervised learning method, Fast Flux Discriminant (FFD), for largescale nonlinear classification. Compared with other existing methods, FFD has unmatched advantages, as it attains the efficiency and interpretability of linear models as well as the accuracy of nonlinear models. It is also sparse and naturally handles mixed data types. It works by decomposing the kernel density estimation in the entire feature space into selected lowdimensional subspaces. Since there are many possible subspaces, we propose a submodular optimization framework for subspace selection. The selected subspace predictions are then transformed to new features on which a linear model can be learned. Besides, since the transformed features naturally expect nonnegative weights, we only require smooth optimization even with the `1 regularization. Unlike other nonlinear models such as kernel methods, the FFD model is interpretable as it gives importance weights on the original features. Its training and testing are also much faster than traditional kernel models. We carry out extensive empirical studies on realworld datasets and show that the proposed model achieves stateoftheart classification results with sparsity, interpretability, and exceptional scalability. Our model can be learned in minutes on datasets with millions of samples, for which most existing nonlinear methods will be prohibitively expensive in space and time.
On approximate nonsubmodular minimization via treestructured supermodularity
 IN 18TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS2015
, 2015
"... We address the problem of minimizing nonsubmodular functions where the supermodularity is restricted to treestructured pairwise terms. We are motivated by several real world applications, which require submodularity along with structured supermodularity, and this forms a rich class of expressive m ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We address the problem of minimizing nonsubmodular functions where the supermodularity is restricted to treestructured pairwise terms. We are motivated by several real world applications, which require submodularity along with structured supermodularity, and this forms a rich class of expressive models, where the nonsubmodularity is restricted to a tree. While this problem is NP hard (as we show), we develop several practical algorithms to find approximate and nearoptimal solutions for this problem, some of which provide lower and others of which provide upper bounds thereby allowing us to compute a tightness gap. We also show that some of our algorithms can be extended to handle more general forms of supermodularity restricted to arbitrary pairwise terms. We compare our algorithms on synthetic data, and also demonstrate the advantage of the formulation on the real world application of image segmentation, where we incorporate structured supermodularity into higherorder submodular energy minimization.
Efficient Visual Exploration and Coverage with a Micro Aerial Vehicle in Unknown Environments
"... Abstract — In this paper, we propose a novel and computationally efficient algorithm for simultaneous exploration and coverage with a visionguided micro aerial vehicle (MAV) in unknown environments. This algorithm continually plans a path that allows the MAV to fulfil two objectives at the same ti ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract — In this paper, we propose a novel and computationally efficient algorithm for simultaneous exploration and coverage with a visionguided micro aerial vehicle (MAV) in unknown environments. This algorithm continually plans a path that allows the MAV to fulfil two objectives at the same time while avoiding obstacles: observe as much unexplored space as possible, and observe as much of the surface of the environment as possible given viewing angle and distance constraints. The former and latter objectives are known as the exploration and coverage problems respectively. Our algorithm is particularly useful for automated 3D reconstruction at the street level and in indoor environments where obstacles are omnipresent. By solving the exploration problem, we maximize the size of the reconstructed model. By solving the coverage problem, we maximize the completeness of the model. Our algorithm leverages the state lattice concept such that the planned path adheres to specified motion constraints. Furthermore, our algorithm is computationally efficient and able to run onboard the MAV in realtime. We assume that the MAV is equipped with a forwardlooking depthsensing camera in the form of either a stereo camera or RGBD camera. We use simulation experiments to validate our algorithm. In addition, we show that our algorithm achieves a significantly higher level of coverage as compared to an explorationonly approach while still allowing the MAV to fully explore the environment. I.
Superdifferential Cuts for Binary Energies
"... We propose an efficient and general purpose energy optimization method for binary variable energies used in various lowlevel vision tasks. Our method can be used for broad classes of higherorder and pairwise nonsubmodular functions. We first revisit a submodularsupermodular procedure (SSP) [19], ..."
Abstract
 Add to MetaCart
(Show Context)
We propose an efficient and general purpose energy optimization method for binary variable energies used in various lowlevel vision tasks. Our method can be used for broad classes of higherorder and pairwise nonsubmodular functions. We first revisit a submodularsupermodular procedure (SSP) [19], which is previously studied for higherorder energy optimization. We then present our method as generalization of SSP, which is further shown to generalize several stateoftheart techniques for higherorder and pairwise nonsubmodular functions [2, 9, 25]. In the experiments, we apply our method to image segmentation, deconvolution, and binarization, and show improvements over stateoftheart methods. 1.
Streaming Algorithms for Submodular Function Maximization
, 2015
"... We consider the problem of maximizing a nonnegative submodular set function f: 2N → R+ subject to a pmatchoid constraint in the singlepass streaming setting. Previous work in this context has considered streaming algorithms for modular functions and monotone submodular functions. The main result i ..."
Abstract
 Add to MetaCart
We consider the problem of maximizing a nonnegative submodular set function f: 2N → R+ subject to a pmatchoid constraint in the singlepass streaming setting. Previous work in this context has considered streaming algorithms for modular functions and monotone submodular functions. The main result is for submodular functions that are nonmonotone. We describe deterministic and randomized algorithms that obtain a Ω(1p)approximation using O(k log k)space, where k is an upper bound on the cardinality of the desired set. The model assumes value oracle access to f and membership oracles for the matroids defining the pmatchoid constraint.
An Algorithmic Theory of Dependent Regularizers Part 1: Submodular Structure
, 2013
"... We present an exploration of the rich theoretical connections between several classes of regularized models, network flows, and recent results in submodular function theory. This work unifies key aspects of these problems under a common theory, leading to novel methods for working with several impor ..."
Abstract
 Add to MetaCart
We present an exploration of the rich theoretical connections between several classes of regularized models, network flows, and recent results in submodular function theory. This work unifies key aspects of these problems under a common theory, leading to novel methods for working with several important models of interest in statistics, machine learning and computer vision. In Part 1, we review the concepts of network flows and submodular function optimization theory foundational to our results. We then examine the connections between network flows and the minimumnorm algorithm from submodular optimization, extending and improving several current results. This leads to a concise representation of the structure of a large class of pairwise regularized models important in machine learning, statistics and computer vision. In Part 2, we describe the full regularization path of a class of penalized regression problems with dependent variables that includes the graphguided LASSO and total variation constrained models. This description also motivates a practical algorithm. This allows us to efficiently find the regularization path of the discretized version of TV penalized models. Ultimately, our new algorithms scale up to highdimensional problems with millions of variables. 1 ar