DMCA
MAD-Bayes: MAP-based Asymptotic Derivations from Bayes
Cached
Download Links
| Citations: | 13 - 2 self |
BibTeX
@MISC{Broderick_mad-bayes:map-based,
author = {Tamara Broderick and Brian Kulis and Michael I. Jordan},
title = {MAD-Bayes: MAP-based Asymptotic Derivations from Bayes},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
The classical mixture of Gaussians model is related to K-means via small-variance asymptotics: as the covariances of the Gaussians tend to zero, the negative log-likelihood of the mixture of Gaussians model approaches the K-means objective, and the EM algorithm approaches the K-means algorithm. Kulis & Jordan (2012) used this observation to obtain a novel K-means-like algorithm from a Gibbs sampler for the Dirichlet process (DP) mixture. We instead consider applying small-variance asymptotics directly to the posterior in Bayesian nonparametric models. This framework is independent of any specific Bayesian inference algorithm, and it has the major advantage that it generalizes immediately to a range of models beyond the DP mixture. To illustrate, we apply our framework to the feature learning setting, where the beta process and Indian buffet process provide an appropriate Bayesian nonparametric prior. We obtain a novel objective function that goes beyond clustering to learn (and penalize new) groupings for which we relax the mutual exclusivity and exhaustivity assumptions of clustering. We demonstrate several other algorithms, all of which are scalable and simple to implement. Empirical results demonstrate the benefits of the new framework. Proceedings of the 30 th
Keyphrases
map-based asymptotic derivation gaussians model small-variance asymptotics classical mixture dp mixture exhaustivity assumption novel objective function beta process k-means objective negative log-likelihood gibbs sampler new framework novel k-means-like algorithm several algorithm kulis jordan indian buffet process k-means algorithm em algorithm mutual exclusivity specific bayesian inference algorithm bayesian nonparametric model major advantage dirichlet process empirical result
