Results 1 - 10
of
61
The Burbea-Rao and Bhattacharyya centroids
- IEEE TRANSACTIONS ON INFORMATION THEORY
, 2011
"... We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the nonnegative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are ..."
Abstract
-
Cited by 26 (14 self)
- Add to MetaCart
(Show Context)
We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the nonnegative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called Jensen-Bregman distances yields exactly those Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao divergences, and show that skew Burbea-Rao divergences amount in limit cases to compute Bregman divergences. We then prove that Burbea-Rao centroids can be arbitrarily finely approximated by a generic iterative concave-convex optimization algorithm with guaranteed convergence property. In the second part of the paper, we consider the Bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions. We show that Bhattacharyya distances on members of the same statistical exponential family amount to calculate a Burbea-Rao divergence in disguise. Thus we get an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or “diagonal” multivariate Gaussians. To illustrate the performance of our Bhattacharyya/Burbea-Rao centroid algorithm, we present experimental performance results for k-means and hierarchical clustering methods of Gaussian mixture models.
Coresets and approximate clustering for Bregman divergences
- In Proc. of the 20th ACM-SIAM Symp. on Discrete Algorithms (SODA
, 2009
"... We study the generalized k-median problem with respect to a Bregman divergence Dφ. Given a finite set P ⊆ Rd of size n, our goal is to find a set C of size k such that the sum of errors cost(P, C) = p∈P minc∈C Dφ(p, c) } is minimized. The Bregman k-median problem plays an important role in many appl ..."
Abstract
-
Cited by 15 (2 self)
- Add to MetaCart
(Show Context)
We study the generalized k-median problem with respect to a Bregman divergence Dφ. Given a finite set P ⊆ Rd of size n, our goal is to find a set C of size k such that the sum of errors cost(P, C) = p∈P minc∈C Dφ(p, c) } is minimized. The Bregman k-median problem plays an important role in many applications, e.g. information theory, statistics, text classification, and speech processing. We give the first coreset construction for this problem for a large subclass of Bregman divergences, including important dissimilarity measures such as the Kullback-Leibler divergence and the Itakura-Saito divergence. Using these coresets, we give a (1 + ɛ)-approximation algorithm for( the Bregman k-median problem with running time O dkn + d2 k ( 2 ɛ)Θ(1) log k+2) n. This result improves over the previousely fastest known (1+ɛ)-approximation algorithm from [1]. Unlike the analysis of most coreset constructions our analysis does not rely on the construction of ɛ-nets. Instead, we prove our results by purely combinatorial means. 1
k-MLE: A fast algorithm for learning statistical mixture models
- CoRR
"... We describe k-MLE, a fast and efficient local search algorithm for learning finite statisti-cal mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases th ..."
Abstract
-
Cited by 11 (6 self)
- Add to MetaCart
(Show Context)
We describe k-MLE, a fast and efficient local search algorithm for learning finite statisti-cal mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expected complete) likelihood. Given prescribed mix-ture weights, the hard clustering k-MLE algorithm iteratively assigns data to the most likely weighted component and update the component models using Maximum Likelihood Estimators (MLEs). Using the duality between exponential families and Bregman divergences, we prove that the local convergence of the complete likelihood of k-MLE follows directly from the con-vergence of a dual additively weighted Bregman hard clustering. The inner loop of k-MLE can be implemented using any k-means heuristic like the celebrated Lloyd’s batched or Hartigan’s greedy swap updates. We then show how to update the mixture weights by minimizing a cross-entropy criterion that implies to update weights by taking the relative proportion of cluster points, and reiterate the mixture parameter update and mixture weight update processes until convergence. Hard EM is interpreted as a special case of k-MLE when both the component up-date and the weight update are performed successively in the inner loop. To initialize k-MLE, we propose k-MLE++, a careful initialization of k-MLE guaranteeing probabilistically a global bound on the best possible complete likelihood.
Bregman vantage point trees for efficient nearest neighbor queries
- In IEEE International Conference on Multimedia & Expo
, 2009
"... Nearest Neighbor (NN) retrieval is a crucial tool of many computer vision tasks. Since the brute-force naive search is too time consuming for most applications, several tailored data structures have been proposed to improve the efficiency of NN search. Among these, vantage point tree (vp-tree)was in ..."
Abstract
-
Cited by 9 (3 self)
- Add to MetaCart
(Show Context)
Nearest Neighbor (NN) retrieval is a crucial tool of many computer vision tasks. Since the brute-force naive search is too time consuming for most applications, several tailored data structures have been proposed to improve the efficiency of NN search. Among these, vantage point tree (vp-tree)was introduced for information retrieval in metric spaces. Vptrees have recently shown very good performances for image patch retrieval with respect to the L2 metric. In this paper we generalize the seminal vp-tree construction and search algorithms to the broader class of Bregman divergences. These distorsion measures are preferred in many cases, as they also handle entropic distances (e.g., Kullback-Leibler divergence) besides quadratic distances. We also extend vp-tree to deal with symmetrized Bregman divergences, which are commonplace in applications of content-based multimedia retrieval. We evaluated performances of our Bvp-tree for exact and approximate NN search on two image feature datasets. Our results show good performances of Bvp-tree, specially for symmetrized Bregman NN queries. Index Terms — Nearest neighbor queries, vantage-point
Constructing Two-Dimensional Voronoi Diagrams via Divide-and-Conquer of Envelopes in Space
"... We present a general framework for computing two-dimensional Voronoi diagrams of different site classes under various distance functions. The computation of the diagrams employs the Cgal software for constructing envelopes of surfaces in 3-space, which implements a divide-and-conquer algorithm. A st ..."
Abstract
-
Cited by 9 (4 self)
- Add to MetaCart
(Show Context)
We present a general framework for computing two-dimensional Voronoi diagrams of different site classes under various distance functions. The computation of the diagrams employs the Cgal software for constructing envelopes of surfaces in 3-space, which implements a divide-and-conquer algorithm. A straightforward application of the divide-andconquer approach for Voronoi diagrams yields highly inefficient algorithms. We show that through randomization, the expected running time is near-optimal (in a worst-case sense). We believe this result, which also holds for general envelopes, to be of independent interest. We describe the interface between the construction of the diagrams and the underlying construction of the envelopes, together with methods we have applied to speed up the (exact) computation. We then present results, where a variety of diagrams are constructed with our implementation, including power diagrams, Apollonius diagrams, diagrams of line segments, Voronoi diagrams on a sphere, and more. In all cases the implementation is exact and can handle degenerate input.
Sparse multiscale patches for image processing
- In ETVC, volume 5416/2009 of LNCS
, 2009
"... HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte p ..."
Abstract
-
Cited by 8 (5 self)
- Add to MetaCart
(Show Context)
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Tailored bregman ball trees for effective nearest neighbors
- In European Workshop on Computational Geometry
, 2009
"... Nearest Neighbor (NN) search is a crucial tool that remains critical in many challenging applications of computational geometry (e.g., surface reconstruction, clustering) and computer vision (e.g., image and information retrieval, classification, data mining). We present an effective Bregman ball tr ..."
Abstract
-
Cited by 8 (3 self)
- Add to MetaCart
(Show Context)
Nearest Neighbor (NN) search is a crucial tool that remains critical in many challenging applications of computational geometry (e.g., surface reconstruction, clustering) and computer vision (e.g., image and information retrieval, classification, data mining). We present an effective Bregman ball tree [5] (Bb-tree) construction algorithm that adapts locally its internal node degrees to the inner geometric characteristics of the data-sets. Since symmetric measures are usually preferred for applications in content-based information retrieval, we furthermore extend the Bb-tree to the case of symmetrized Bregman divergences. Exact and approximate NN search experiments using high-dimensional real-world data-sets illustrate that
The dual Voronoi diagrams with respect to representational Bregman divergences
- International Symposium on Voronoi Diagrams (ISVD
, 2009
"... Abstract—We present a generalization of Bregman Voronoi diagrams induced by a Bregman divergence acting on a representation function. Bregman divergences are canonical distortion measures of flat spaces induced by strictly convex and differentiable functions, called Bregman generators. Considering a ..."
Abstract
-
Cited by 8 (6 self)
- Add to MetaCart
(Show Context)
Abstract—We present a generalization of Bregman Voronoi diagrams induced by a Bregman divergence acting on a representation function. Bregman divergences are canonical distortion measures of flat spaces induced by strictly convex and differentiable functions, called Bregman generators. Considering a representation function further allows us to conveniently embed the not necessarily flat source space into a dually flat space for which the dual Voronoi diagrams can be derived from an equivalent power affine diagram. We explain the fundamental dualities induced by the pair of Legendre convex conjugates coupled with a pair of conjugate representations. In particular, we show that Amari’s celebrated family of α-divergences and Eguchi and Copas’s β-divergences are two cases of representational Bregman divergences that are often considered in information geometry. We report closedform formula for their centroids and describe their dual Voronoi diagrams on the induced statistical manifolds. Keywords-Voronoi diagrams, centroids, power diagrams, Bregman divergences, f-divergences, α-divergences, β-divergences.
