Abstract — It is shown how steepest descent (or steepest ascent) may be viewed as a message passing algorithm with “local ” message update rules. For example, the well-known backpropagation algorithm for the training of feed-forward neural networks may be viewed as message passing on a factor graph. The factor graph approach with its emphasis on “local ” computations makes it easy to combine steepest descent with other message passing algorithms such as the sum/max-product algorithms, expectation maximization, Kalman filtering/smoothing, and particle filters. As an example, parameter estimation in a state space model is considered. For this example, it is shown how steepest descent can be used for the maximization step in expectation maximization.

Abstract — Based on prior work by Eckford, it is shown how expectation maximization (EM) may be viewed, and used, as a message passing algorithm in factor graphs. I.