It is possible to combine multiple non-linear probabilistic models of the same data by multiplying the probability distributions together and then renormalizing. A "product of experts " is a very efficient way to model data that simultaneously satisfies many different constraints. It is difficult to fit a product of experts to data using maximum likelihood because the gradient of the log likelihood is intractable, but there is an efficient way of optimizing a different objective function and this produces good models of high-dimensional data. Introduction One way of modeling a complicated, high-dimensional data distribution is to use a large number of relatively simple probabilistic models and to somehow combine the distributions specified by each model. A well-known example of this approach is a mixture of Gaussians in which each simple model is a Gaussian and the combination rule consists of taking a weighted arithmetic mean of the individual distributions. This is equivale...