Results 1 
8 of
8
Hingeloss Markov random fields and probabilistic soft logic
, 2015
"... A fundamental challenge in developing highimpact machine learning technologies is balancing the ability to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
A fundamental challenge in developing highimpact machine learning technologies is balancing the ability to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge graphs and the Web, to images, video, and natural language. In this paper, we introduce two new formalisms for modeling structured data, distinguished from previous approaches by their ability to both capture rich structure and scale to big data. The first, hingeloss Markov random fields (HLMRFs), is a new kind of probabilistic graphical model that generalizes different approaches to convex inference. We unite three approaches from the randomized algorithms, probabilistic graphical models, and fuzzy logic communities, showing that all three lead to the same inference objective. We then derive HLMRFs by generalizing this unified objective. The second new formalism, probabilistic soft logic (PSL), is a probabilistic programming language that makes HLMRFs easy to define using a syntax based on firstorder logic. We next introduce an algorithm for inferring mostprobable variable assignments (MAP inference) that is much more scalable than generalpurpose convex optimization software, because it uses message passing to take advantage of sparse dependency structures. We then show how to learn the parameters of HLMRFs. The learned HLMRFs are as accurate as analogous discrete models, but much more scalable. Together, these algorithms enable HLMRFs and PSL to model rich, structured data at scales not previously possible.
Inferring user preferences by probabilistic logical reasoning over social networks. arXiv preprint arXiv:1411.2679
, 2014
"... We propose a framework for inferring the latent attitudes or preferences of users by performing probabilistic firstorder logical reasoning over the social network graph. Our method answers questions about Twitter users like Does this user like sushi? or Is this user a New York Knicks fan? by bui ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
We propose a framework for inferring the latent attitudes or preferences of users by performing probabilistic firstorder logical reasoning over the social network graph. Our method answers questions about Twitter users like Does this user like sushi? or Is this user a New York Knicks fan? by building a probabilistic model that reasons over user attributes (the user’s location or gender) and the social network (the user’s friends and spouse), via inferences like homophily (I am more likely to like sushi if spouse or friends like sushi, I am more likely to like the Knicks if I live in New York). The algorithm uses distant supervision, semisupervised data harvesting and vector space models to extract user attributes (e.g. spouse, education, location) and preferences (likes and dislikes) from text. The extracted propositions are then fed into a probabilistic reasoner (we investigate both Markov Logic and Probabilistic Soft Logic). Our experiments show that probabilistic logical reasoning significantly improves the performance on attribute and relation extraction, and also achieves an Fscore of 0.791 at predicting a users likes or dislikes, significantly better than two strong baselines.
Latent topic networks: A versatile probabilistic programming framework for topic models.”
 In International Conference on Machine Learning,
, 2015
"... Abstract Topic models have become increasingly prominent textanalytic machine learning tools for research in the social sciences and the humanities. In particular, custom topic models can be developed to answer specific research questions. The design of these models requires a nontrivial amount of ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract Topic models have become increasingly prominent textanalytic machine learning tools for research in the social sciences and the humanities. In particular, custom topic models can be developed to answer specific research questions. The design of these models requires a nontrivial amount of effort and expertise, motivating generalpurpose topic modeling frameworks. In this paper we introduce latent topic networks, a flexible class of richly structured topic models designed to facilitate applied research. Custom models can straightforwardly be developed in our framework with an intuitive firstorder logical probabilistic programming language. Latent topic networks admit scalable training via a parallelizable EM algorithm which leverages ADMM in the Mstep. We demonstrate the broad applicability of the models with case studies on modeling influence in citation networks, and U.S. Presidential State of the Union addresses.
Paireddual learning for fast training of latent variable hingeloss mrfs
 In Proceedings of the International Conference of Machine Learning
, 2015
"... Latent variables allow probabilistic graphical models to capture nuance and structure in important domains such as network science, natural language processing, and computer vision. Naive approaches to learning such complex models can be prohibitively expensive—because they require repeated inferen ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Latent variables allow probabilistic graphical models to capture nuance and structure in important domains such as network science, natural language processing, and computer vision. Naive approaches to learning such complex models can be prohibitively expensive—because they require repeated inferences to update beliefs about latent variables—so lifting this restriction for useful classes of models is an important problem. Hingeloss Markov random fields (HLMRFs) are graphical models that allow highly scalable inference and learning in structured domains, in part by representing structured problems with continuous variables. However, this representation leads to challenges when learning with latent variables. We introduce paireddual learning, a framework that greatly speeds up training by using tractable entropy surrogates and avoiding repeated inferences. Paireddual learning optimizes an objective with a pair of dual inference problems. This allows fast, joint optimization of parameters and dual variables. We evaluate on socialgroup detection, trust prediction in social networks, and image reconstruction, finding that paireddual learning trains models as accurate as those trained by traditional methods in much less time, often before traditional methods make even a single parameter update. 1.
Injecting Logical Background Knowledge into Embeddings for Relation Extraction
"... Matrix factorization approaches to relation extraction provide several attractive features: they support distant supervision, handle open schemas, and leverage unlabeled data. Unfortunately, these methods share a shortcoming with all other distantly supervised approaches: they cannot learn to extra ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Matrix factorization approaches to relation extraction provide several attractive features: they support distant supervision, handle open schemas, and leverage unlabeled data. Unfortunately, these methods share a shortcoming with all other distantly supervised approaches: they cannot learn to extract target relations without existing data in the knowledge base, and likewise, these models are inaccurate for relations with sparse data. Rulebased extractors, on the other hand, can be easily extended to novel relations and improved for existing but inaccurate relations, through firstorder formulae that capture auxiliary domain knowledge. However, usually a large set of such formulae is necessary to achieve generalization. In this paper, we introduce a paradigm for learning lowdimensional embeddings of entitypairs and relations that combine the advantages of matrix factorization with firstorder logic domain knowledge. We introduce simple approaches for estimating such embeddings, as well as a novel training algorithm to jointly optimize over factual and firstorder logic information. Our results show that this method is able to learn accurate extractors with little or no distant supervision alignments, while at the same time generalizing to textual patterns that do not appear in the formulae. 1
Joint prediction for entity/eventlevel sentiment analysis using probabilistic soft logic models
 In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP
"... Abstract In this work, we build an entity/eventlevel sentiment analysis system, which is able to recognize and infer both explicit and implicit sentiments toward entities and events in the text. We design Probabilistic Soft Logic models that integrate explicit sentiments, inference rules, and +/e ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract In this work, we build an entity/eventlevel sentiment analysis system, which is able to recognize and infer both explicit and implicit sentiments toward entities and events in the text. We design Probabilistic Soft Logic models that integrate explicit sentiments, inference rules, and +/effect event information (events that positively or negatively affect entities). The experiments show that the method is able to greatly improve over baseline accuracies in recognizing entity/eventlevel sentiments.
ENTITY/EVENTLEVEL SENTIMENT DETECTION AND INFERENCE
, 2015
"... This proposal was presented by ..."
Statistical Relational Learning with Soft Quantifiers
"... Abstract. Quantification in statistical relational learning (SRL) is either existential or universal, however humans might be more inclined to express knowledge using soft quantifiers, such as “most ” and “a few”. In this paper, we define the syntax and semantics of PSLQ, a new SRL framework that s ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. Quantification in statistical relational learning (SRL) is either existential or universal, however humans might be more inclined to express knowledge using soft quantifiers, such as “most ” and “a few”. In this paper, we define the syntax and semantics of PSLQ, a new SRL framework that supports reasoning with soft quantifiers, and present its most probable explanation (MPE) inference algorithm. To the best of our knowledge, PSLQ is the first SRL framework that combines soft quantifiers with firstorder logic rules for modeling uncertain relational data. Our experimental results for link prediction in social trust networks demonstrate that the use of soft quantifiers not only allows for a natural and intuitive formulation of domain knowledge, but also improves the accuracy of inferred results. 1