Results 11  20
of
51
Exploiting Logical Structure in Lifted Probabilistic Inference
, 2010
"... Representations that combine firstorder logic and probability have been the focus of much recent research. Lifted inference algorithms for them avoid grounding out the domain, bringing benefits analogous to those of resolution theorem proving in firstorder logic. However, all lifted probabilistic ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
Representations that combine firstorder logic and probability have been the focus of much recent research. Lifted inference algorithms for them avoid grounding out the domain, bringing benefits analogous to those of resolution theorem proving in firstorder logic. However, all lifted probabilistic inference algorithms to date treat potentials as black boxes, and do not take advantage of their logical structure. As a result, inference with them is needlessly inefficient compared to the logical case. We overcome this by proposing the first lifted probabilistic inference algorithm that exploits determinism and context specific independence. In particular, we show that AND/OR search can be lifted by introducing POWER nodes in addition to the standard AND and OR nodes. Experimental tests show the benefits of our approach.
Approximate lifted belief propagation
, 2010
"... Lifting can greatly reduce the cost of inference on firstorder probabilistic models, but constructing the lifted network can itself be quite costly. In addition, the minimal lifted network is often very close in size to the fully propositionalized model; lifted inference yields little or no speedup ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
Lifting can greatly reduce the cost of inference on firstorder probabilistic models, but constructing the lifted network can itself be quite costly. In addition, the minimal lifted network is often very close in size to the fully propositionalized model; lifted inference yields little or no speedup in these situations. In this paper, we address both these problems. We propose a compact hypercubebased representation for the lifted network, which can greatly reduce the cost of lifted network construction. We also present two methods for approximate lifted network construction, which groups together similar but distinguishable objects and treats them as if they were identical. This can greatly reduce the size of the lifted network as well as the time required for lifted network construction, but potentially at some cost to accuracy. The coarseness of the approximation can be adjusted depending on the accuracy required, and we can bound the resulting error. Experiments on six domains show great efficiency gains with only minor loss in accuracy.
On the complexity and approximation of binary evidence in lifted inference
 In Advances in Neural Information Processing Systems 26 (NIPS
"... Lifted inference algorithms exploit symmetries in probabilistic models to speed up inference. They show impressive performance when calculating unconditional probabilities in relational models, but often resort to nonlifted inference when computing conditional probabilities. The reason is that cond ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
Lifted inference algorithms exploit symmetries in probabilistic models to speed up inference. They show impressive performance when calculating unconditional probabilities in relational models, but often resort to nonlifted inference when computing conditional probabilities. The reason is that conditioning on evidence breaks many of the model’s symmetries, which can preempt standard lifting techniques. Recent theoretical results show, for example, that conditioning on evidence which corresponds to binary relations is #Phard, suggesting that no lifting is to be expected in the worst case. In this paper, we balance this negative result by identifying the Boolean rank of the evidence as a key parameter for characterizing the complexity of conditioning in lifted inference. In particular, we show that conditioning on binary evidence with bounded Boolean rank is efficient. This opens up the possibility of approximating evidence by a lowrank Boolean matrix factorization, which we investigate both theoretically and empirically. 1
Lifted Online Training of Relational Models with Stochastic Gradient Methods
"... Abstract. Lifted inference approaches have rendered large, previously intractable probabilistic inference problems quickly solvable by employing symmetries to handle whole sets of indistinguishable random variables. Still, in many if not most situations training relational models will not benefit fr ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Lifted inference approaches have rendered large, previously intractable probabilistic inference problems quickly solvable by employing symmetries to handle whole sets of indistinguishable random variables. Still, in many if not most situations training relational models will not benefit from lifting: symmetries within models easily break since variables become correlated by virtue of depending asymmetrically on evidence. An appealing idea for such situations is to train and recombine local models. This breaks longrange dependencies and allows to exploit lifting within and across the local training tasks. Moreover, it naturally paves the way for online training for relational models. Specifically, we develop the first lifted stochastic gradient optimization method with gain vector adaptation, which processes each lifted piece one after the other. On several datasets, the resulting optimizer converges to the same quality solution over an order of magnitude faster, simply because unlike batch training it starts optimizing long before having seen the entire megaexample even once. 1
Exploiting DataIndependence for Fast BeliefPropagation
"... Maximum a posteriori (MAP) inference in graphical models requires that we maximize the sum of two terms: a datadependent term, encoding the conditional likelihood of a certain labeling given an observation, and a dataindependent term, encoding some prior on labelings. Often, datadependent factors ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
Maximum a posteriori (MAP) inference in graphical models requires that we maximize the sum of two terms: a datadependent term, encoding the conditional likelihood of a certain labeling given an observation, and a dataindependent term, encoding some prior on labelings. Often, datadependent factors contain fewer latent variables than dataindependent factors – for instance, many grid and treestructured models contain only firstorder conditionals despite having pairwise priors. In this paper, we note that MAPinference in such models can be made substantially faster by appropriately preprocessing their dataindependent terms. Our main result is to show that messagepassing in any such pairwise model has an expectedcase exponent of only 1.5 on the number of states per node, leading to significant improvements over existing quadratictime solutions. 1.
Lifted Relax, Compensate and then Recover: From Approximate to Exact Lifted Probabilistic Inference
"... We propose an approach to lifted approximate inference for firstorder probabilistic models, such as Markov logic networks. It is based on performing exact lifted inference in a simplified firstorder model, which is found by relaxing firstorder constraints, and then compensating for the relaxation ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
We propose an approach to lifted approximate inference for firstorder probabilistic models, such as Markov logic networks. It is based on performing exact lifted inference in a simplified firstorder model, which is found by relaxing firstorder constraints, and then compensating for the relaxation. These simplified models can be incrementally improved by carefully recovering constraints that have been relaxed, also at the firstorder level. This leads to a spectrum of approximations, with lifted belief propagation on one end, and exact lifted inference on the other. We discuss how relaxation, compensation, and recovery can be performed, all at the firstorder level, and show empirically that our approach substantially improves on the approximations of both propositional solvers and lifted belief propagation. 1
Lifted Belief Propagation: Pairwise Marginals and Beyond
"... Lifted belief propagation (LBP) can be extremely fast at computing approximate marginal probability distributions over single ground atoms and neighboring ones in the underlying graphical model. It does, however, not prescribe a way to compute joint distributions over pairs, triples or ktuples of d ..."
Abstract

Cited by 5 (5 self)
 Add to MetaCart
(Show Context)
Lifted belief propagation (LBP) can be extremely fast at computing approximate marginal probability distributions over single ground atoms and neighboring ones in the underlying graphical model. It does, however, not prescribe a way to compute joint distributions over pairs, triples or ktuples of distant ground atoms. In this paper, we present an algorithm, called conditioned LBP, for approximating these distributions. Essentially, we select variables one at a time for conditioning, running lifted belief propagation after each selection. This naive solution, however, recomputes the lifted network in each step from scratch, therefore often canceling the benefits of lifted inference. We show how to avoid this by efficiently computing the lifted network for each conditioning directly from the one already known for the single node marginals. Our experimental results validate that significant efficiency gains are possible and illustrate the potential for secondorder parameter estimation of Markov logic networks. 1
Symmetryaware marginal density estimation
 In Proceedings of the 27th Conference on Artificial Intelligence (AAAI
, 2013
"... The RaoBlackwell theorem is utilized to analyze and improve the scalability of inference in large probabilistic models that exhibit symmetries. A novel marginal density estimator is introduced and shown both analytically and empirically to outperform standard estimators by several orders of magni ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
The RaoBlackwell theorem is utilized to analyze and improve the scalability of inference in large probabilistic models that exhibit symmetries. A novel marginal density estimator is introduced and shown both analytically and empirically to outperform standard estimators by several orders of magnitude. The developed theory and algorithms apply to a broad class of probabilistic models including statistical relational models considered not susceptible to lifted probabilistic inference.
Lifted Probabilistic Inference: An MCMC Perspective
"... The general consensus seems to be that lifted inference is concerned with exploiting model symmetries and grouping indistinguishable objects at inference time. Since firstorder probabilistic formalisms are essentially template languages providing a more compact representation of a corresponding gro ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
The general consensus seems to be that lifted inference is concerned with exploiting model symmetries and grouping indistinguishable objects at inference time. Since firstorder probabilistic formalisms are essentially template languages providing a more compact representation of a corresponding ground model, lifted inference tends to work especially well in these models. We show that the notion of indistinguishability manifests itself on several different levels – the level of constants, the level of ground atoms (variables), the level of formulas (features), and the level of assignments (possible worlds). We discuss existing work in the MCMC literature on exploiting symmetries on the level of variable assignments and relate it to novel results in lifted MCMC. 1
Efficient Lifting of MAP LP Relaxations Using kLocality
"... Inference in large scale graphical models is an important task in many domains, and in particular for probabilistic relational models (e.g,. Markov logic networks). Such models often exhibit considerable symmetry, and it is a challenge to devise algorithms that exploit this symmetry to speed up infe ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Inference in large scale graphical models is an important task in many domains, and in particular for probabilistic relational models (e.g,. Markov logic networks). Such models often exhibit considerable symmetry, and it is a challenge to devise algorithms that exploit this symmetry to speed up inference. Here we address this task in the context of the MAP inference problem and its linear programming relaxations. We show that symmetry in these problems can be discovered using an elegant algorithm known as the kdimensional WeisfeilerLehman (kWL) algorithm. We run kWL on the original graphical model, and not on the far larger graph of the linear program (LP) as proposed in earlier work in the field. Furthermore, the algorithm is polynomial and thus far more practical than other previous approaches which rely on orbit partitions that are GI complete to find. The fact that kWL can be used in this manner follows from the recently introduced notion of klocal LPs and their relation to Sherali Adams relaxations of graph automorphisms. Finally, for relational models such as Markov logic networks, the benefits of our approach are even more dramatic, as we can discover symmetries in the original domain graph, as opposed to running lifting on the much larger grounded model. 1