Comparison of a Bayesian SOM with the EM Algorithm for Gaussian Mixtures
|
Abstract:
A Bayesian SOM (BSOM) [8], is proposed and applied to the unsupervised learning of Gaussian mixture distributions and its performance is compared with the expectation-maximisation (EM) algorithm. The BSOM is found to yield as good results as the well-known EM algorithm but with much fewer iterations and, more importantly it can be used as an on-line training method. The neighbourhood function and distance measures of the traditional SOM [3] are replaced by the neuron's on-line estimated posterior probabilities, which can be interpreted as a Bayesian inference of the neuron's opportunity to share in the winning response and so to adapt to the input pattern. Such posteriors starting from uniform priors are gradually sharpened when more and more data samples become available and so improve the estimation of model parameters. Each neuron then converges to one component of the mixture. Experimental results are compared with those of the EM algorithm. 1
Citations
| 4704 | Maximum likelihood from incomplete data via the EM algorithm – Dempster, Laird, et al. - 1977 |
| 3119 | Pattern Classification and Scene Analysis – Duda, Hart - 1973 |
| 390 | Mixture densities, maximum likelihood and the EM algorithm – Redner, Walker - 1984 |
| 95 | Self-organising maps – Kohonen - 1995 |
| 45 | A probabilistic resource allocating network for novelty detection – Roberts, Tarassenko - 1994 |
| 42 | A neural network approach to statistical pattern classification by "semiparametric" estimation of probability density functions – Traven - 1991 |
| 2 | Unsupervised learning by EM algorithm based on finite mixture of Gaussians – Xu, Jordan - 1993 |
| 2 | Bayesian learning for self-organising maps – Yin, Allinson - 1997 |
| 1 | Towards the optimal Bayes classifier using an extended selforganising map – Yin, Allinson - 1995 |
| 1 | Self-Organising Maps: Statistical Analysis, Treatment and Applications – Yin - 1996 |
