Results 1  10
of
62
Markov Logic Networks
 MACHINE LEARNING
, 2006
"... We propose a simple approach to combining firstorder logic and probabilistic graphical models in a single representation. A Markov logic network (MLN) is a firstorder knowledge base with a weight attached to each formula (or clause). Together with a set of constants representing objects in the ..."
Abstract

Cited by 816 (39 self)
 Add to MetaCart
(Show Context)
We propose a simple approach to combining firstorder logic and probabilistic graphical models in a single representation. A Markov logic network (MLN) is a firstorder knowledge base with a weight attached to each formula (or clause). Together with a set of constants representing objects in the domain, it specifies a ground Markov network containing one feature for each possible grounding of a firstorder formula in the KB, with the corresponding weight. Inference in MLNs is performed by MCMC over the minimal subset of the ground network required for answering the query. Weights are efficiently learned from relational databases by iteratively optimizing a pseudolikelihood measure. Optionally, additional clauses are learned using inductive logic programming techniques. Experiments with a realworld database and knowledge base in a university domain illustrate the promise of this approach.
Sound and efficient inference with probabilistic and deterministic dependencies
, 2006
"... Reasoning with both probabilistic and deterministic dependencies is important for many realworld problems, and in particular for the emerging field of statistical relational learning. However, probabilistic inference methods like MCMC or belief propagation tend to give poor results when determin ..."
Abstract

Cited by 133 (17 self)
 Add to MetaCart
(Show Context)
Reasoning with both probabilistic and deterministic dependencies is important for many realworld problems, and in particular for the emerging field of statistical relational learning. However, probabilistic inference methods like MCMC or belief propagation tend to give poor results when deterministic or neardeterministic dependencies are present, and logical ones like satisfiability testing are inapplicable to probabilistic ones. In this paper we propose MCSAT, an inference algorithm that combines ideas from MCMC and satisfiability. MCSAT is based on Markov logic, which defines Markov networks using weighted clauses in firstorder logic. From the point of view of MCMC,MCSAT is a slice sampler with an auxiliary variable per clause, and with a satisfiabilitybased method for sampling the original variables given the auxiliary ones. From the point of view of satisfiability, MCSAT wraps a procedure around the SampleSAT uniform sampler that enables it to sample from highly nonuniform distributions over satisfying assignments. Experiments on entity resolution and collective classification problems show that MCSAT greatly outperforms Gibbs sampling and simulated tempering over a broad range of problem sizes and degrees of determinism.
Discriminative training of markov logic networks
 In Proc. of the Natl. Conf. on Artificial Intelligence
, 2005
"... Many machine learning applications require a combination of probability and firstorder logic. Markov logic networks (MLNs) accomplish this by attaching weights to firstorder clauses, and viewing these as templates for features of Markov networks. Model parameters (i.e., clause weights) can be lear ..."
Abstract

Cited by 107 (19 self)
 Add to MetaCart
Many machine learning applications require a combination of probability and firstorder logic. Markov logic networks (MLNs) accomplish this by attaching weights to firstorder clauses, and viewing these as templates for features of Markov networks. Model parameters (i.e., clause weights) can be learned by maximizing the likelihood of a relational database, but this can be quite costly and lead to suboptimal results for any given prediction task. In this paper we propose a discriminative approach to training MLNs, one which optimizes the conditional likelihood of the query predicates given the evidence ones, rather than the joint likelihood of all predicates. We extend Collins’s (2002) voted perceptron algorithm for HMMs to MLNs by replacing the Viterbi algorithm with a weighted satisfiability solver. Experiments on entity resolution and link prediction tasks show the advantages of this approach compared to generative MLN training, as well as compared to purely probabilistic and purely logical approaches.
Entity Resolution with Markov Logic
 In ICDM
, 2006
"... Entity resolution is the problem of determining which records in a database refer to the same entities, and is a crucial and expensive step in the data mining process. Interest in it has grown rapidly in recent years, and many approaches have been proposed. However, they tend to address only isolate ..."
Abstract

Cited by 105 (10 self)
 Add to MetaCart
(Show Context)
Entity resolution is the problem of determining which records in a database refer to the same entities, and is a crucial and expensive step in the data mining process. Interest in it has grown rapidly in recent years, and many approaches have been proposed. However, they tend to address only isolated aspects of the problem, and are often ad hoc. This paper proposes a wellfounded, integrated solution to the entity resolution problem based on Markov logic. Markov logic combines firstorder logic and probabilistic graphical models by attaching weights to firstorder formulas, and viewing them as templates for features of Markov networks. We show how a number of previous approaches can be formulated and seamlessly combined in Markov logic, and how the resulting learning and inference problems can be solved efficiently. Experiments on two citation databases show the utility of this approach, and evaluate the contribution of the different components. 1
Efficient weight learning for Markov logic networks
 In Proceedings of the Eleventh European Conference on Principles and Practice of Knowledge Discovery in Databases
, 2007
"... Abstract. Markov logic networks (MLNs) combine Markov networks and firstorder logic, and are a powerful and increasingly popular representation for statistical relational learning. The stateoftheart method for discriminative learning of MLN weights is the voted perceptron algorithm, which is ess ..."
Abstract

Cited by 87 (7 self)
 Add to MetaCart
(Show Context)
Abstract. Markov logic networks (MLNs) combine Markov networks and firstorder logic, and are a powerful and increasingly popular representation for statistical relational learning. The stateoftheart method for discriminative learning of MLN weights is the voted perceptron algorithm, which is essentially gradient descent with an MPE approximation to the expected sufficient statistics (true clause counts). Unfortunately, these can vary widely between clauses, causing the learning problem to be highly illconditioned, and making gradient descent very slow. In this paper, we explore several alternatives, from perweight learning rates to secondorder methods. In particular, we focus on two approaches that avoid computing the partition function: diagonal Newton and scaled conjugate gradient. In experiments on standard SRL datasets, we obtain orderofmagnitude speedups, or more accurate models given comparable learning times. 1
Joint Unsupervised Coreference Resolution with Markov Logic
"... Machine learning approaches to coreference resolution are typically supervised, and require expensive labeled data. Some unsupervised approaches have been proposed (e.g., Haghighi and Klein (2007)), but they are less accurate. In this paper, we present the first unsupervised approach that is competi ..."
Abstract

Cited by 87 (6 self)
 Add to MetaCart
(Show Context)
Machine learning approaches to coreference resolution are typically supervised, and require expensive labeled data. Some unsupervised approaches have been proposed (e.g., Haghighi and Klein (2007)), but they are less accurate. In this paper, we present the first unsupervised approach that is competitive with supervised ones. This is made possible by performing joint inference across mentions, in contrast to the pairwise classification typically used in supervised methods, and by using Markov logic as a representation language, which enables us to easily express relations like apposition and predicate nominals. On MUC and ACE datasets, our model outperforms Haghigi and Klein’s one using only a fraction of the training data, and often matches or exceeds the accuracy of stateoftheart supervised models. 1
Tuffy: Scaling up Statistical Inference in Markov Logic Networks using an RDBMS
, 2011
"... Markov Logic Networks (MLNs) have emerged as a powerful framework that combines statistical and logical reasoning; they have been applied to many data intensive problems including information extraction, entity resolution, and text mining. Current implementations of MLNs do not scale to large realw ..."
Abstract

Cited by 55 (9 self)
 Add to MetaCart
Markov Logic Networks (MLNs) have emerged as a powerful framework that combines statistical and logical reasoning; they have been applied to many data intensive problems including information extraction, entity resolution, and text mining. Current implementations of MLNs do not scale to large realworld data sets, which is preventing their widespread adoption. We present Tuffy that achieves scalability via three novel contributions: (1) a bottomup approach to grounding that allows us to leverage the full power of the relational optimizer, (2) a novel hybrid architecture that allows us to perform AIstyle local search efficiently using an RDBMS, and (3) a theoretical insight that shows when one can (exponentially) improve the efficiency of stochastic local search. We leverage (3) to build novel partitioning, loading, and parallel algorithms. We show that our approach outperforms stateoftheart implementations in both quality and speed on several publicly available datasets.
Memoryefficient inference in relational domains
 In Proceedings of the TwentyFirst National Conference on Artificial Intelligence
, 2006
"... Propositionalization of a firstorder theory followed by satisfiability testing has proved to be a remarkably efficient approach to inference in relational domains such as planning (Kautz & Selman 1996) and verification (Jackson 2000). More recently, weighted satisfiability solvers have been use ..."
Abstract

Cited by 48 (9 self)
 Add to MetaCart
Propositionalization of a firstorder theory followed by satisfiability testing has proved to be a remarkably efficient approach to inference in relational domains such as planning (Kautz & Selman 1996) and verification (Jackson 2000). More recently, weighted satisfiability solvers have been used successfully for MPE inference in statistical relational learners (Singla & Domingos 2005). However, fully instantiating a finite firstorder theory requires memory on the order of the number of constants raised to the arity of the clauses, which significantly limits the size of domains it can be applied to. In this paper we propose LazySAT, a variation of the WalkSAT solver that avoids this blowup by taking advantage of the extreme sparseness that is typical of relational domains (i.e., only a small fraction of ground atoms are true, and most clauses are trivially satisfied). Experiments on entity resolution and planning problems show that LazySAT reduces memory usage by orders of magnitude compared to WalkSAT, while taking comparable time to run and producing the same solutions.
Hybrid markov logic networks.
 In Proceedings of the TwentyThird AAAI Conference on Artificial Intelligence (AAAI),
, 2008
"... Abstract Markov logic networks (MLNs) combine firstorder logic and Markov networks, allowing us to handle the complexity and uncertainty of realworld problems in a single consistent framework. However, in MLNs all variables and features are discrete, while most realworld applications also contai ..."
Abstract

Cited by 44 (1 self)
 Add to MetaCart
(Show Context)
Abstract Markov logic networks (MLNs) combine firstorder logic and Markov networks, allowing us to handle the complexity and uncertainty of realworld problems in a single consistent framework. However, in MLNs all variables and features are discrete, while most realworld applications also contain continuous ones. In this paper we introduce hybrid MLNs, in which continuous properties (e.g., the distance between two objects) and functions over them can appear as features. Hybrid MLNs have all distributions in the exponential family as special cases (e.g., multivariate Gaussians), and allow much more compact modeling of noni.i.d. data than propositional representations like hybrid Bayesian networks. We also introduce inference algorithms for hybrid MLNs, by extending the MaxWalkSAT and MCSAT algorithms to continuous domains. Experiments in a mobile robot mapping domaininvolving joint classification, clustering and regressionillustrate the power of hybrid MLNs as a modeling language, and the accuracy and efficiency of the inference algorithms.
A General Method for Reducing the Complexity of Relational Inference And its Application to MCMC
"... Many realworld problems are characterized by complex relational structure, which can be succinctly represented in firstorder logic. However, many relational inference algorithms proceed by first fully instantiating the firstorder theory and then working at the propositional level. The applicabilit ..."
Abstract

Cited by 42 (4 self)
 Add to MetaCart
(Show Context)
Many realworld problems are characterized by complex relational structure, which can be succinctly represented in firstorder logic. However, many relational inference algorithms proceed by first fully instantiating the firstorder theory and then working at the propositional level. The applicability of such approaches is severely limited by the exponential time and memory cost of propositionalization. Singla and Domingos (2006) addressed this by developing a “lazy ” version of the WalkSAT algorithm, which grounds atoms and clauses only as needed. In this paper we generalize their ideas to a much broader class of algorithms, including other types of SAT solvers and probabilistic inference methods like MCMC. Lazy inference is potentially applicable whenever variables and functions have default values (i.e., a value that is much more frequent than the others). In relational domains, the default is false for atoms and true for clauses. We illustrate our framework by applying it to MCSAT, a stateoftheart MCMC algorithm. Experiments on a number of realworld domains show that lazy inference reduces both space and time by several orders of magnitude, making probabilistic relational inference applicable in previously infeasible domains.