Training Products of Experts by Minimizing Contrastive Divergence (2000)
| Venue: | Neural Computation |
| Citations: | 353 - 71 self |
BibTeX
@ARTICLE{Hinton00trainingproducts,
author = {Geoffrey Hinton},
title = {Training Products of Experts by Minimizing Contrastive Divergence},
journal = {Neural Computation},
year = {2000},
volume = {14},
pages = {2002}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
It is possible to combine multiple probabilistic models of the same data by multiplying their probability distributions together and then renormalizing. This is a very ecient way to model high-dimensional data which simultaneously satisfies many different low-dimensional constraints because each individual expert model can focus on giving high probability to data vectors that satisfy just one of the constraints. Data vectors that satisfy this one constraint but violate other constraints will be ruled out by their low probability under the other experts. Training a product of experts appears difficult because, in addition to maximizing the probability that each individual expert assigns to the observed data, it is necessary to make the experts be as different as possible. This ensures that the product of their distributions is small which allows the renormalization to magnify the probability of the data under the product of experts model. Fortunately, if the individual experts are t...
Citations
No citations identified.
