Results 1 
3 of
3
Statistical Mechanics of Online Learning of Drifting Concepts: A Variational Approach
 Machine Learning
, 1998
"... Abstract. We review the application of statistical mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feedforward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best p ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We review the application of statistical mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feedforward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule.
Fast Relational Learning using Bottom Clause Propositionalization with Artificial Neural Networks
, 2013
"... Relational learning can be described as the task of learning firstorder logic rules from examples. It has enabled a number of new machine learning applications, e.g. graph mining and link analysis. Inductive Logic Programming (ILP) performs relational learning either directly by manipulating first ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Relational learning can be described as the task of learning firstorder logic rules from examples. It has enabled a number of new machine learning applications, e.g. graph mining and link analysis. Inductive Logic Programming (ILP) performs relational learning either directly by manipulating firstorder rules or through propositionalization, which translates the relational task into an attributevalue learning task by representing subsets of relations as features. In this paper, we introduce a fast method and system for relational learning based on a novel propositionalization called Bottom Clause Propositionalization (BCP). Bottom clauses are boundaries in the hypothesis search space used by ILP systems Progol and Aleph. Bottom clauses carry semantic meaning and can be mapped directly onto numerical vectors, simplifying the feature extraction process. We have integrated BCP with a wellknown neuralsymbolic system, CIL2P, to perform learning from numerical vectors. CIL2P uses background knowledge in the form of propositional logic programs to build a neural network. The integrated system, which we call CILP++, handles firstorder logic knowledge and is available for download from Sourceforge. We have evaluated CILP++ on seven ILP datasets, comparing results with Aleph and a wellknown propositionalization method, RSD. The results show that CILP++ can achieve accuracy comparable to Aleph, while being generally faster, BCP achieved statistically significant improvement in accuracy in comparison with RSD when running with a neural network, but BCP and RSD perform similarly when running with C4.5. We have also extended CILP++ to include a statistical feature selection method, mRMR, with preliminary results indicating that a reduction of more than 90 % of features can be achieved with a small loss of accuracy.
Statistical Mechanics of Online learning
"... Abstract. We introduce and discuss the application of statistical physics concepts in the context of online machine learning processes. The consideration of typical properties of very large systems allows to perfom averages over the randomness contained in the sequence of training data. It yields a ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. We introduce and discuss the application of statistical physics concepts in the context of online machine learning processes. The consideration of typical properties of very large systems allows to perfom averages over the randomness contained in the sequence of training data. It yields an exact mathematical description of the training dynamics in model scenarios. We present the basic concepts and results of the approach in terms of several examples, including the learning of linear separable rules, the training of multilayer neural networks, and Learning Vector Quantization. 1