Results 1  10
of
61
The BurbeaRao and Bhattacharyya centroids
 IEEE TRANSACTIONS ON INFORMATION THEORY
, 2011
"... We study the centroid with respect to the class of informationtheoretic BurbeaRao divergences that generalize the celebrated JensenShannon divergence by measuring the nonnegative Jensen difference induced by a strictly convex and differentiable function. Although those BurbeaRao divergences are ..."
Abstract

Cited by 26 (14 self)
 Add to MetaCart
(Show Context)
We study the centroid with respect to the class of informationtheoretic BurbeaRao divergences that generalize the celebrated JensenShannon divergence by measuring the nonnegative Jensen difference induced by a strictly convex and differentiable function. Although those BurbeaRao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called JensenBregman distances yields exactly those BurbeaRao divergences. We then proceed by defining skew BurbeaRao divergences, and show that skew BurbeaRao divergences amount in limit cases to compute Bregman divergences. We then prove that BurbeaRao centroids can be arbitrarily finely approximated by a generic iterative concaveconvex optimization algorithm with guaranteed convergence property. In the second part of the paper, we consider the Bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions. We show that Bhattacharyya distances on members of the same statistical exponential family amount to calculate a BurbeaRao divergence in disguise. Thus we get an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or “diagonal” multivariate Gaussians. To illustrate the performance of our Bhattacharyya/BurbeaRao centroid algorithm, we present experimental performance results for kmeans and hierarchical clustering methods of Gaussian mixture models.
Coresets and approximate clustering for Bregman divergences
 In Proc. of the 20th ACMSIAM Symp. on Discrete Algorithms (SODA
, 2009
"... We study the generalized kmedian problem with respect to a Bregman divergence Dφ. Given a finite set P ⊆ Rd of size n, our goal is to find a set C of size k such that the sum of errors cost(P, C) = p∈P minc∈C Dφ(p, c) } is minimized. The Bregman kmedian problem plays an important role in many appl ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
(Show Context)
We study the generalized kmedian problem with respect to a Bregman divergence Dφ. Given a finite set P ⊆ Rd of size n, our goal is to find a set C of size k such that the sum of errors cost(P, C) = p∈P minc∈C Dφ(p, c) } is minimized. The Bregman kmedian problem plays an important role in many applications, e.g. information theory, statistics, text classification, and speech processing. We give the first coreset construction for this problem for a large subclass of Bregman divergences, including important dissimilarity measures such as the KullbackLeibler divergence and the ItakuraSaito divergence. Using these coresets, we give a (1 + ɛ)approximation algorithm for( the Bregman kmedian problem with running time O dkn + d2 k ( 2 ɛ)Θ(1) log k+2) n. This result improves over the previousely fastest known (1+ɛ)approximation algorithm from [1]. Unlike the analysis of most coreset constructions our analysis does not rely on the construction of ɛnets. Instead, we prove our results by purely combinatorial means. 1
kMLE: A fast algorithm for learning statistical mixture models
 CoRR
"... We describe kMLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectationmaximization (EM) soft clustering technique that monotonically increases th ..."
Abstract

Cited by 11 (6 self)
 Add to MetaCart
(Show Context)
We describe kMLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectationmaximization (EM) soft clustering technique that monotonically increases the incomplete (expected complete) likelihood. Given prescribed mixture weights, the hard clustering kMLE algorithm iteratively assigns data to the most likely weighted component and update the component models using Maximum Likelihood Estimators (MLEs). Using the duality between exponential families and Bregman divergences, we prove that the local convergence of the complete likelihood of kMLE follows directly from the convergence of a dual additively weighted Bregman hard clustering. The inner loop of kMLE can be implemented using any kmeans heuristic like the celebrated Lloyd’s batched or Hartigan’s greedy swap updates. We then show how to update the mixture weights by minimizing a crossentropy criterion that implies to update weights by taking the relative proportion of cluster points, and reiterate the mixture parameter update and mixture weight update processes until convergence. Hard EM is interpreted as a special case of kMLE when both the component update and the weight update are performed successively in the inner loop. To initialize kMLE, we propose kMLE++, a careful initialization of kMLE guaranteeing probabilistically a global bound on the best possible complete likelihood.
Bregman vantage point trees for efficient nearest neighbor queries
 In IEEE International Conference on Multimedia & Expo
, 2009
"... Nearest Neighbor (NN) retrieval is a crucial tool of many computer vision tasks. Since the bruteforce naive search is too time consuming for most applications, several tailored data structures have been proposed to improve the efficiency of NN search. Among these, vantage point tree (vptree)was in ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
Nearest Neighbor (NN) retrieval is a crucial tool of many computer vision tasks. Since the bruteforce naive search is too time consuming for most applications, several tailored data structures have been proposed to improve the efficiency of NN search. Among these, vantage point tree (vptree)was introduced for information retrieval in metric spaces. Vptrees have recently shown very good performances for image patch retrieval with respect to the L2 metric. In this paper we generalize the seminal vptree construction and search algorithms to the broader class of Bregman divergences. These distorsion measures are preferred in many cases, as they also handle entropic distances (e.g., KullbackLeibler divergence) besides quadratic distances. We also extend vptree to deal with symmetrized Bregman divergences, which are commonplace in applications of contentbased multimedia retrieval. We evaluated performances of our Bvptree for exact and approximate NN search on two image feature datasets. Our results show good performances of Bvptree, specially for symmetrized Bregman NN queries. Index Terms — Nearest neighbor queries, vantagepoint
Constructing TwoDimensional Voronoi Diagrams via DivideandConquer of Envelopes in Space
"... We present a general framework for computing twodimensional Voronoi diagrams of different site classes under various distance functions. The computation of the diagrams employs the Cgal software for constructing envelopes of surfaces in 3space, which implements a divideandconquer algorithm. A st ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
(Show Context)
We present a general framework for computing twodimensional Voronoi diagrams of different site classes under various distance functions. The computation of the diagrams employs the Cgal software for constructing envelopes of surfaces in 3space, which implements a divideandconquer algorithm. A straightforward application of the divideandconquer approach for Voronoi diagrams yields highly inefficient algorithms. We show that through randomization, the expected running time is nearoptimal (in a worstcase sense). We believe this result, which also holds for general envelopes, to be of independent interest. We describe the interface between the construction of the diagrams and the underlying construction of the envelopes, together with methods we have applied to speed up the (exact) computation. We then present results, where a variety of diagrams are constructed with our implementation, including power diagrams, Apollonius diagrams, diagrams of line segments, Voronoi diagrams on a sphere, and more. In all cases the implementation is exact and can handle degenerate input.
Sparse multiscale patches for image processing
 In ETVC, volume 5416/2009 of LNCS
, 2009
"... HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte p ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
(Show Context)
HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Tailored bregman ball trees for effective nearest neighbors
 In European Workshop on Computational Geometry
, 2009
"... Nearest Neighbor (NN) search is a crucial tool that remains critical in many challenging applications of computational geometry (e.g., surface reconstruction, clustering) and computer vision (e.g., image and information retrieval, classification, data mining). We present an effective Bregman ball tr ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
(Show Context)
Nearest Neighbor (NN) search is a crucial tool that remains critical in many challenging applications of computational geometry (e.g., surface reconstruction, clustering) and computer vision (e.g., image and information retrieval, classification, data mining). We present an effective Bregman ball tree [5] (Bbtree) construction algorithm that adapts locally its internal node degrees to the inner geometric characteristics of the datasets. Since symmetric measures are usually preferred for applications in contentbased information retrieval, we furthermore extend the Bbtree to the case of symmetrized Bregman divergences. Exact and approximate NN search experiments using highdimensional realworld datasets illustrate that
The dual Voronoi diagrams with respect to representational Bregman divergences
 International Symposium on Voronoi Diagrams (ISVD
, 2009
"... Abstract—We present a generalization of Bregman Voronoi diagrams induced by a Bregman divergence acting on a representation function. Bregman divergences are canonical distortion measures of flat spaces induced by strictly convex and differentiable functions, called Bregman generators. Considering a ..."
Abstract

Cited by 8 (6 self)
 Add to MetaCart
(Show Context)
Abstract—We present a generalization of Bregman Voronoi diagrams induced by a Bregman divergence acting on a representation function. Bregman divergences are canonical distortion measures of flat spaces induced by strictly convex and differentiable functions, called Bregman generators. Considering a representation function further allows us to conveniently embed the not necessarily flat source space into a dually flat space for which the dual Voronoi diagrams can be derived from an equivalent power affine diagram. We explain the fundamental dualities induced by the pair of Legendre convex conjugates coupled with a pair of conjugate representations. In particular, we show that Amari’s celebrated family of αdivergences and Eguchi and Copas’s βdivergences are two cases of representational Bregman divergences that are often considered in information geometry. We report closedform formula for their centroids and describe their dual Voronoi diagrams on the induced statistical manifolds. KeywordsVoronoi diagrams, centroids, power diagrams, Bregman divergences, fdivergences, αdivergences, βdivergences.