Results 11  20
of
35
Factorized Asymptotic Bayesian Hidden Markov Models
"... This paper addresses the issue of model selection for hidden Markov models (HMMs). We generalize factorized asymptotic Bayesian inference (FAB), which has been recently developed for model selection on independent hidden variables (i.e., mixture models), for timedependent hidden variables. As with ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
This paper addresses the issue of model selection for hidden Markov models (HMMs). We generalize factorized asymptotic Bayesian inference (FAB), which has been recently developed for model selection on independent hidden variables (i.e., mixture models), for timedependent hidden variables. As with FAB in mixture models, FAB for HMMs is derived as an iterative lower bound maximization algorithm of a factorized information criterion (FIC). It inherits, from FAB for mixture models, several desirable properties for learning HMMs, such as asymptotic consistency of FIC with marginal loglikelihood, a shrinkage effect for hidden state selection, monotonic increase of the lower FIC bound through the iterative optimization. Further, it does not have a tunable hyperparameter, and thus its model selection process can be fully automated. Experimental results shows that FAB outperforms statesoftheart variational Bayesian HMM and nonparametric Bayesian HMM in terms of model selection accuracy and computational efficiency. 1.
The Indian Buffet Process: Scalable Inference and Extensions
"... August 2009This dissertation is the result of my own work and includes nothing which is the outcome of work done in collaboration except where specifically indicated in the text. c ○ Copyright by Finale DoshiVelez, 2009. Many unsupervised learning problems seek to identify hidden features from obse ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
August 2009This dissertation is the result of my own work and includes nothing which is the outcome of work done in collaboration except where specifically indicated in the text. c ○ Copyright by Finale DoshiVelez, 2009. Many unsupervised learning problems seek to identify hidden features from observations. In many realworld situations, the number of hidden features is unknown. To avoid specifying the number of hidden features a priori, one can use the Indian Buffet Process (IBP): a nonparametric latent feature model that does not bound the number of active features in a dataset. While elegant, the lack of efficient inference procedures for the IBP has prevented its application in largescale problems. The core contribution of this thesis are three new inference procedures that allow inference in the IBP to be scaled from a few hundred to 100,000 observations. This thesis contains three parts:
Nonparametric Multigroup Membership Model for Dynamic Networks
"... Relational data—like graphs, networks, and matrices—is often dynamic, where the relational structure evolves over time. A fundamental problem in the analysis of timevarying network data is to extract a summary of the common structure and the dynamics of the underlying relations between the entitie ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Relational data—like graphs, networks, and matrices—is often dynamic, where the relational structure evolves over time. A fundamental problem in the analysis of timevarying network data is to extract a summary of the common structure and the dynamics of the underlying relations between the entities. Here we build on the intuition that changes in the network structure are driven by the dynamics at the level of groups of nodes. We propose a nonparametric multigroup membership model for dynamic networks. Our model contains three main components: We model the birth and death of individual groups with respect to the dynamics of the network structure via a distance dependent Indian Buffet Process. We capture the evolution of individual node group memberships via a Factorial Hidden Markov model. And, we explain the dynamics of the network structure by explicitly modeling the connectivity structure of groups. We demonstrate our model’s capability of identifying the dynamics of latent groups in a number of different types of network data. Experimental results show that our model provides improved predictive performance over existing dynamic network models on future network forecasting and missing link prediction. 1
A Discriminative Nonparametric Bayesian Model: Infinite Hidden Conditional Random Fields
"... Nonparametric methods have been successfully applied to many existing graphical models with latent variables [3, 2, 7, 4]. In contrast to all previous work, the infinite Hidden Conditional Random Fields (iHCRF), introduced in this work, is the first, to our knowledge, discriminative bayesian nonpara ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Nonparametric methods have been successfully applied to many existing graphical models with latent variables [3, 2, 7, 4]. In contrast to all previous work, the infinite Hidden Conditional Random Fields (iHCRF), introduced in this work, is the first, to our knowledge, discriminative bayesian nonparametric model.
Modeling Correlated Arrival Events with Latent SemiMarkov Processes
"... The analysis of correlated point process data has wide applications, ranging from biomedical research to network analysis. In this work, we model such data as generated by a latent collection of continuoustime binary semiMarkov processes, corresponding to external events appearing and disappear ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
The analysis of correlated point process data has wide applications, ranging from biomedical research to network analysis. In this work, we model such data as generated by a latent collection of continuoustime binary semiMarkov processes, corresponding to external events appearing and disappearing. A continuoustime modeling framework is more appropriate for multichannel point process data than a binning approach requiring time discretization, and we show connections between our model and recent ideas from the discretetime literature. We describe an efficient MCMC algorithm for posterior inference, and apply our ideas to both synthetic data and a realworld biometrics application. 1.
Infinite structured hidden semiMarkov models. arXiv preprint arXiv:1407.0044
 GPU FOR TIMEVARYING PITMANYOR PROCESSES
, 2014
"... Abstract. This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori statepersistence in particular. This paper als ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Abstract. This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori statepersistence in particular. This paper also introduces a new Bayesian nonparametric framework for generating lefttoright and other structured, explicitduration infinite hidden Markov models that we call the infinite structured hidden semiMarkov model. 1.
JOINT MODELING OF MULTIPLE TIME SERIES VIA THE BETA PROCESS WITH APPLICATION TO MOTION CAPTURE SEGMENTATION
"... We propose a Bayesian nonparametric approach to the problem of jointly modeling multiple related time series. Our model discovers a latent set of dynamical behaviors shared among the sequences, and segments each time series into regions defined by a subset of these behaviors. Using a beta process pr ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We propose a Bayesian nonparametric approach to the problem of jointly modeling multiple related time series. Our model discovers a latent set of dynamical behaviors shared among the sequences, and segments each time series into regions defined by a subset of these behaviors. Using a beta process prior, the size of the behavior set and the sharing pattern are both inferred from data. We develop Markov chain Monte Carlo (MCMC) methods based on the Indian buffet process representation of the predictive distribution of the beta process. Our MCMC inference algorithm efficiently adds and removes behaviors via novel splitmerge moves as well as datadriven birth and death proposals, avoiding the need to consider a truncated model. We demonstrate promising results on unsupervised segmentation of human motion capture data. 1. Introduction. Classical
Bayesian Nonparametric Modeling of Suicide Attempts
"... The National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) database contains a large amount of information, regarding the way of life, medical conditions, etc., of a representative sample of the U.S. population. In this paper, we are interested in seeking the hidden causes behind ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
The National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) database contains a large amount of information, regarding the way of life, medical conditions, etc., of a representative sample of the U.S. population. In this paper, we are interested in seeking the hidden causes behind the suicide attempts, for which we propose to model the subjects using a nonparametric latent model based on the Indian Buffet Process (IBP). Due to the nature of the data, we need to adapt the observation model for discrete random variables. We propose a generative model in which the observations are drawn from a multinomiallogit distribution given the IBP matrix. The implementation of an efficient Gibbs sampler is accomplished using the Laplace approximation, which allows integrating out the weighting factors of the multinomiallogit likelihood model. Finally, the experiments over the NESARC database show that our model properly captures some of the hidden causes that model suicide attempts. 1
Bayesian Nonparametric Hidden Markov Models
, 2011
"... I hereby declare that my dissertation, entitled “Bayesian Nonparametric Hidden Markov Models”, is not substantially the same as any that I have submitted for a degree or diploma or other qualification at any other university. No part of my dissertation has already been, or is concurrently being, sub ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
I hereby declare that my dissertation, entitled “Bayesian Nonparametric Hidden Markov Models”, is not substantially the same as any that I have submitted for a degree or diploma or other qualification at any other university. No part of my dissertation has already been, or is concurrently being, submitted for any degree, diploma, or other qualification. This dissertation is my own work and contains nothing which is the outcome of work done in collaboration with others, except as specified in the text and acknowledgements. This dissertation does not exceed sixty thousand words in length. 2
A Truncated EM Approach for SpikeandSlab Sparse Coding
"... We study inference and learning based on a sparse coding model with ‘spikeandslab ’ prior. As in standard sparse coding, the model used assumes independent latent sources that linearly combine to generate data points. However, instead of using a standard sparse prior such as a Laplace distribution ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We study inference and learning based on a sparse coding model with ‘spikeandslab ’ prior. As in standard sparse coding, the model used assumes independent latent sources that linearly combine to generate data points. However, instead of using a standard sparse prior such as a Laplace distribution, we study the application of a more flexible ‘spikeandslab ’ distribution which models the absence or presence of a source’s contribution independently of its strength if it contributes. We investigate two approaches to optimize the parameters of spikeandslab sparse coding: a novel truncated EM approach and, for comparison, an approach based on standard factored variational distributions. The truncated approach can be regarded as a variational approach with truncated posteriors as variational distributions. In applications to source separation we find that both approaches improve the stateoftheart in a number of standard benchmarks, which argues for the use of ‘spikeandslab ’ priors for the corresponding data domains. Furthermore, we find that the truncated EM approach improves on the standard factored approach in source separation tasks—which hints to biases introduced by assuming posterior independence in the factored variational approach. Likewise, on a standard benchmark for image denoising, we find that the truncated EM approach improves on the factored variational approach. While the performance of the factored approach saturates with increasing numbers of hidden dimensions, the performance of the truncated approach improves the stateoftheart for higher noise levels.