T. Minka. Expectation propagation for approximate Bayesian inference. In Uncertainty in Artificial Intelligence (UAI), pages 362--369, 2001. 15  Home/Search   Document Not in Database   Summary   Related Articles   Check

....standard BP algorithm in its various guises as an algorithm that operates on factor graphs. Other formulations of the standard BP algorithm provide different insights, and we refer the interested reader to a number of important recent papers that exploit alternative views of the BP algorithm [21] [22], 23] 24] 25] 26] After our original work which introduced region based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of free energy approximations and GBP algorithms based on junction graphs [27] One of the goals of this ....

T. P. Minka. Expectation propagation for approximate Bayesian inference. UAI-01, 2001.

....posterior is a mixture with jM c j components. In this section we therefore describe a greedy approximate inference strategy for the SLDS. An adaptation for the fine level SLDS from Section II is presented in Section IV B. The approximation is a particular form of expectation propagation [1]. We present it here in the spirit of the sum product algorithm [2] For ease of notation and interpretability we treat u t = fs t ; x t g together as one conditionally Gaussian distributed random variable. Slightly abusing notation, we will use the sum sign for the combined operation of summing ....

T. Minka, "Expectation propagation for approximate Bayesian inference," in Proceedings of UAI-2001.

....out. So the exact posterior is a mixture with [M[ components. In this section we therefore describe a greedy approximate inference strategy for the SLDS. An adaptation for the fine level SLDS from Sect. 2 is presented in Sect. 4.2. The approximation is a particular form of expectation propagation [4]. We present it here in the spirit of the sum product algorithm [3] For ease of notation and interpretability we treat ut = st, xt together as one conditionally Gaussian distributed random variable. Slightly abusing notation, we will use the sum sign for the combined operation of summing out st ....

T. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of UAI-2001, pages 362-369, 2001.

....section we therefore describe a greedy approximate inference strategy for the SLDS. An extension to the model for the third and subsequent levels is presented in Section IV A. The algorithm can be interpreted as approximate belief propagation and is a particular form of expectation propagation [5]. The filtering pass of this approximation has been proposed independently several times. The oldest reference we are aware of is in [6] in the engineering literature it is known as generalized pseudo Bayes 2 (GPB2) 7] The basic idea is to approximate posteriors such as p(xt[st i, y. T) ....

Thomas Minks, "Expectation propagation for approximate Bayesian inference," in Proceedings of the 17th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2001.

....of the IASTED International Conference on Artificial and Computational Intelligence 2002, 251 256. Tokyo, Japan, 25 27 September 2002. application dependent constraints do not permit exact inference. D Ambrosio offers a very informative survey in [3] and some recent developments include [10, 15, 16]. We can classify approximate inference procedures from different perspectives. In terms of how we carry out the approximations, D Ambrosio comes up with two schools of algorithms: approximate inference methods compute distributions with special algorithms using the original network, e.g. 2, ....

Minka, T. P., Expectation propagation for approximate Bayesian inference, Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence, 362--369, 2001.

....the family must correspond to some strongly triangulated tree, which is often exponential in the size of the problem. Thus, the family of representations necessary for Lauritzen s algorithm is often too complex by itself, and there is a need for a different approach. Expectation propagation (EP) Min01] is an alternative approach for finding a family member that is a good approximation to the joint probability distribution. Instead of treating each term t i exactly and then approximating the joint that includes the t i s, EP first approximates t i with some t i and then uses the product ....

....division in line 7 is again simple using canonical forms. Note that, after convergence, we can easily compute other quantities of interest, such as the probability of the evidence or the distribution of the discrete variables A i using our approximation to the joint distribution. It can be shown [Min01] that the EP iterations are guaranteed to have a fixed point when the approximations are in an exponential family. EP often leads to good approximations, especially when the joint distribution can be well approximated by a member of the chosen family. Unfortunately, the algorithm can also behave ....

T. P. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Annual Conference on Uncertainty in AI (UAI), pages 362--369, 2001.

....approximations and their advanced extensions belong to this group. However, it is not clear in general, how to solve these equations efficiently. This latter problem is of central concern to the second class, the Message passing algorithms, like Bayesian online approaches (for references, see e.g. [1]) and belief propagation (BP) which dynamically update approximations to conditional probabilities. Finally, approximations based on Free Energies allow us to derive marginal moments by minimising entropic loss measures. This method introduces new possibilities for algorithms and also gives ....

....graphs where individual dependencies are weak but their overall effect cannot be neglected. A interesting candidate is the adaptive TAP (ADATAP) approach introduced in [3] as a set of self consistency relations. Recently, a message passing algorithm of Minka (termed expectation propagation) [1] was found to solve the ADATAP equations efficiently for models with Gaussian Process (GP) priors. The goal of this paper is three fold. We will add a further derivation of ADATAP using an approximate free energy. A sequential algorithm for minimising the free energy generalises Minka s result. ....

[Article contains additional citation context not shown here]

T.P. Minka. Expectation propagation for approximate Bayesian inference. PhD thesis, Dep. of Electrical Eng. and Comp. Sci.; MIT, 2000.

....for polytree BNs, is applied to an arbitrary DBN until convergence. This approach gives approximate answers, but there are no guarantees as to their quality; once again, no arbitrary improvement of the approximation is possible. Recent generalizations of belief propagation [Yedidia et al. 2001; Minka, 2001] do admit of successively more accurate approximations and may yield a practical filtering algorithm. A third deterministic approximation algorithm can be derived using variational techniques, which use the closest simplified model that is tractable. The original variational algorithms were ....

T. Minka. Expectation propagation for approximate Bayesian inference. In Proc. 17th Annual Conference on Uncertainty in AI, San Francisco, 2001. Morgan Kaufmann Publishers.

....cause the size of the representation to blow up. One approximation is to use weak marginalisation [Lau92] which reduces a mixture of Gaussians to a single Gaussian using moment matching. The implementation of this in [Lau92] is numerically unstable, and has been improved in [LJ99] See also [Min01] 3 If messages are passed sequentially, the scheduling usually uses two passes, often called collect distribute or forwards backwards [PS91] 5 Undirected models are already parameterized in terms of potentials on cliques, so no conversion is necessary. Directed models are parameterized in ....

.... been shown to be using the BP algorithm [MMC98] led to a lot of theoretical analysis, which has shown how BP is closely related to variational methods [YFW01, SO01] Recently this technique has been extended to do approximate Bayesian inference, using a technique called Expectation Propagation [Min01] 4 Learning There are two main kinds of learning: parameter learning (also called model tting) and structure learning (also called model selection) We will discuss each in turn. 4.1 Parameter learning If we adopt a Bayesian approach, parameters are treated just like any other random ....

[Article contains additional citation context not shown here]

T. Minka. Expectation propagation for approximate Bayesian inference. In UAI, 2001.

....field (MF) approximations and their advanced extensions belong to this group. However, it is not clear, how to solve these equations efficiently. This latter problem is of central concern to the second class, the Message passing algorithms, like Bayesian online approaches (for references, see e.g. [1]) and belief propagation (BP) which dynamically update approximations to conditional probabilities. Finally, approximations based on Free Energies allow us to derive marginal moments by a minimising entropic loss measures. This method introduces new possibilities for algorithms and also gives ....

....graphs where individual dependencies are weak but their overall effect cannot be neglected. A interesting candidate is the adaptive TAP (ADATAP) approach introduced in [3] as a set of self consistency relations. Recently, a message passing algorithm of Minka (termed expectation propagation) [1] was found to solve the ADATAP equations efficiently for models with Gaussian Process (GP) priors. The goal of this paper is three fold. We will add a further derivation of ADATAP using an approximate free energy. A sequential algorithm for minimising the free energy generalises Minka s result. ....

[Article contains additional citation context not shown here]

T.P. Minka. Expectation propagation for approximate Bayesian inference. PhD thesis, Dep. of Electrical Eng. and Comp. Sci.; MIT, 2000.

....introduced in [39, 40] and further developed in [41, 42, 43] This technique can be formulated for fairly general model classes but was sofar limited to a single sweep through the data, thereby making the approximation dependent on the ordering of the data sequence. In a recent study by Minka [44] it was shown that by a proper recycling of the data, a convergence to the solutions of the TAP equations for the case of a Gaussian process classi er [8] was achieved. We expect that by a consequent and principled combination of the cavity idea with algorithms that are similar to the on line or ....

T. P. Minka, Expectation Propagation for Approximate Bayesian Inference, Preprint MIT Media Lab (2000). 29

.... and Q) so as to approximately minimize the bound we have derived above. We can choose N = 4 2 ln m D(QjjP ) 1 : 4. Implications We wish to apply the preceding theory to two general learning methods: Maximum Entropy discrimination[11] and Bayes as well as Bayes Point Classi ers [15] [9] We choose these two learning methods because the average in these cases is over many hypotheses, so that the low order terms in the bound are not very signi cant. We begin with a simple toy example that illustrates the bound application. 4.1 Example A quick example will illustrate the ....

Thomas Minka, \Expectation Propagation for approximate Bayesian inference", thesis.

No context found.

Minka, T. P. (2001). Expectation propagation for approximate Bayesian inference. Uncertainty in AI'01.

No context found.

Minka, T. P. (2001b). Expectation propagation for approximate Bayesian inference. UAI (pp.

....complexity. In this paper, we develop an expectation propagation (EP) algorithm for hybrid dynamic systems and apply it to signal detection in flat fading channels. Expectation propagation, a powerful extension of belief propagation, was proposed in the statistical machine learning community [Minka, 2001]. Belief propagation has been widely used in digital communications, such as Turbo decoding [McEliece et al. 1998] However, belief propagation can handle only discrete distributions or continuous distributions in the exponential family. In contrast, expectation propagation not only iteratively ....

....dynamic model, it will be su#cient to infer the exact posterior of any state given the whole observation sequence after propagating all the forward, observation, and backward messages once. However, the hybrid model of our interest is neither linear Gaussian nor completely discrete. As shown in [Minka, 2001], by iterating message propagation, expectation propagation will keep improving the approximation quality until it converges to a local minimum of its energy function; however, it is possible that expectation propagation does not converge as belief propagation, though it is rare in practice. ....

Minka, T. P. (2001). Expectation propagation for approximate Bayesian inference. In Uncertainty in AI'01. http://www.stat.cmu.edu/~minka/papers/ep/.

No context found.

T. Minka. Expectation propagation for approximate Bayesian inference. In Uncertainty in Artificial Intelligence (UAI), pages 362--369, 2001. 15

No context found.

Thomas Minka. Expectation propagation for approximate Bayesian inference. In J. Breese and D. Koller, editors, Uncertainty in Artificial Intelligence 17. Morgan Kaufmann, 2001.

No context found.

T. Minka, Expectation propagation for approximate Bayesian inference, In Proc. UAI, 2001.

No context found.

T. Minka. Expectation propagation for approximate Bayesian inference. In UAI, 2001.

No context found.

T. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, UAI 2001.

No context found.

T. P. Minka. Expectation propagation for approximate bayesian inference. In UAI '01, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc.

No context found.

Thomas Minka. Expectation propagation for approximate Bayesian inference. In J. Breese and D. Koller, editors, Uncertainty in Artificial Intelligence 17. Morgan Kaufmann, 2001.

No context found.

T. Minka. Expectation propagation for approximate Bayesian inference. In Proc. of the Conf. on Uncertainty in AI, 2001.

No context found.

T.P. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings UAI, pages 362--369, 2001.

No context found.

T. Minka. Expectation propagation for approximate Bayesian inference. In Uncertainty in Arti cial Intelligence: Proceedings of the Seventeenth Conference (UAI-2001.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC