Results 1 
9 of
9
Feature Weighting for Lazy Learning Algorithms
, 1998
"... : Learning algorithms differ in the degree to which they process their inputs prior to their use in performance tasks. Many algorithms eagerly compile input samples and use only the compilations to make decisions. Others are lazy: they perform less precompilation and use the input samples to guide ..."
Abstract

Cited by 53 (1 self)
 Add to MetaCart
: Learning algorithms differ in the degree to which they process their inputs prior to their use in performance tasks. Many algorithms eagerly compile input samples and use only the compilations to make decisions. Others are lazy: they perform less precompilation and use the input samples to guide decision making. The performance of many lazy learners significantly degrades when samples are defined by features containing little or misleading information. Distinguishing feature relevance is a critical issue for these algorithms, and many solutions have been developed that assign weights to features. This chapter introduces a categorization framework for feature weighting approaches used in lazy similarity learners and briefly surveys some examples in each category. 1.1 INTRODUCTION Lazy learning algorithms are machine learning algorithms (Mitchell, 1997) that are welcome members of procrastinators anonymous. Purely lazy learners typically display the following characteristics (Aha, 19...
On Bayesian Case Matching
 Advances in CaseBased Reasoning, Proceedings of the 4th European Workshop (EWCBR98), volume 1488 of Lecture Notes in Artificial Intelligence
, 1998
"... . Case retrieval is an important problem in several commercially significant application areas, such as industrial configuration and manufacturing problems. In this paper we extend the Bayesian probability theory based approaches to casebased reasoning, focusing on the case matching task, an essent ..."
Abstract

Cited by 13 (10 self)
 Add to MetaCart
(Show Context)
. Case retrieval is an important problem in several commercially significant application areas, such as industrial configuration and manufacturing problems. In this paper we extend the Bayesian probability theory based approaches to casebased reasoning, focusing on the case matching task, an essential part of any case retrieval system. Traditional approaches to the case matching problem typically rely on some distance measure, e.g., the Euclidean or Hamming distance, although there is no a priori guarantee that such measures really reflect the useful similarities and dissimilarities between the cases. One of the main advantages of the Bayesian framework for solving this problem is that it forces one to explicitly recognize all the assumptions made about the problem domain, which helps in analyzing the performance of the resulting system. As an example of an implementation of the Bayesian case matching approach in practice, we demonstrate how to construct a case retrieval system based ...
Applying General Bayesian Techniques to Improve TAN Induction
 In Proceedings of the International Conference on Knowledge Discovery and Data Mining
, 1999
"... Tree Augmented Naive Bayes (TAN) has shown to be competitive with stateoftheart machine learning algorithms [9]. However, the TAN induction algorithm that appears in [9] can be improved in several ways. In this paper we identify three weak points in it and introduce two ideas to overcome those pro ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
Tree Augmented Naive Bayes (TAN) has shown to be competitive with stateoftheart machine learning algorithms [9]. However, the TAN induction algorithm that appears in [9] can be improved in several ways. In this paper we identify three weak points in it and introduce two ideas to overcome those problems: the multinomial sampling approach to learning bayesian networks and local bayesian model averaging. These ideas are generic and can thus be reused to improve other learning algorithms. We empirically test the new algorithms, and conclude that in many cases they lead to an improvement in accuracy in the classification and in the quality of the probabilities given as predictions.
Tractable Bayesian Learning of Tree Augmented Naive Bayes Classifiers
 In Proceedings of the Twentieth International Conference on Machine Learning
, 2003
"... Bayesian classifiers such as Naive Bayes or Tree Augmented Naive Bayes (TAN) have shown excellent performance given their simplicity and heavy underlying independence assumptions. In this paper we introduce a classifier taking as basis the TAN models and taking into account uncertainty in model sele ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Bayesian classifiers such as Naive Bayes or Tree Augmented Naive Bayes (TAN) have shown excellent performance given their simplicity and heavy underlying independence assumptions. In this paper we introduce a classifier taking as basis the TAN models and taking into account uncertainty in model selection. To do this we introduce decomposable distributions over TANs and show that the expression resulting from the Bayesian model averaging of TAN models can be integrated into closed form if we assume the prior probability distribution to be a decomposable distribution. This result allows for the construction of a classifier with a shorter learning time and a longer classification time than TAN. Empirical results show that the classifier is, most of the cases, more accurate than TAN and approximates better the class probabilities. 1.
TAN classifiers based on decomposable distributions
 Machine Learning
, 2005
"... Abstract. In this paper we present several Bayesian algorithms for learning Tree Augmented Naive Bayes (TAN) models. We extend the results in Meila & Jaakkola (2000a) to TANs by proving that accepting a prior decomposable distribution over TAN’s, we can compute the exact Bayesian model averaging ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract. In this paper we present several Bayesian algorithms for learning Tree Augmented Naive Bayes (TAN) models. We extend the results in Meila & Jaakkola (2000a) to TANs by proving that accepting a prior decomposable distribution over TAN’s, we can compute the exact Bayesian model averaging over TAN structures and parameters in polynomial time. Furthermore, we prove that the kmaximum a posteriori (MAP) TAN structures can also be computed in polynomial time. We use these results to correct minor errors in Meila & Jaakkola (2000a) and to construct several TAN based classifiers. We show that these classifiers provide consistently better predictions over Irvine datasets and artificially generated data than TAN based classifiers proposed in the literature.
An Unsupervised Bayesian Distance Measure
 Proceedings of the Fifth European Workshop on Casebased Reasoning (EWCBR’2000). LNAI1898
, 2000
"... . We introduce a distance measure based on the idea that two vectors are considered similar if they lead to similar predictive probability distributions. The suggested approach avoids the scaling problem inherent to many alternative techniques as the method automatically transforms the original ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
. We introduce a distance measure based on the idea that two vectors are considered similar if they lead to similar predictive probability distributions. The suggested approach avoids the scaling problem inherent to many alternative techniques as the method automatically transforms the original attribute space to a probability space where all the numbers lie between 0 and 1. The method is also flexible in the sense that it allows different attribute types (discrete or continuous) in the same consistent framework. To study the validity of the suggested measure, we ran a series of experiments with publicly available data sets. The empirical results demonstrate that the unsupervised distance measure is sensible in the sense that it can be used for discovering the hidden clustering structure of the data. 1 Introduction Machine learning techniques usually aim at compressing available sample data into more compact representations called models. These models can then be used for ...
The indifferent naive bayes classifier.
, 2003
"... Abstract The Naive Bayes classifier is a simple and accurate classifier. This paper shows that assuming the Naive Bayes classifier model and applying Bayesian model averaging and the principle of indifference, an equally simple, more accurate and theoretically well founded classifier can be obtaine ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract The Naive Bayes classifier is a simple and accurate classifier. This paper shows that assuming the Naive Bayes classifier model and applying Bayesian model averaging and the principle of indifference, an equally simple, more accurate and theoretically well founded classifier can be obtained.
Maximum a Posteriori Tree Augmented Naive Bayes Classifiers
, 2003
"... Bayesian classifiers such as Naive Bayes or Tree Augmented Naive Bayes (TAN) have shown excellent performance given their simplicity and heavy underlying independence assumptions. In this paper we prove that under suitable conditions it is possible to calculate efficiently the maximum a posterior TA ..."
Abstract
 Add to MetaCart
(Show Context)
Bayesian classifiers such as Naive Bayes or Tree Augmented Naive Bayes (TAN) have shown excellent performance given their simplicity and heavy underlying independence assumptions. In this paper we prove that under suitable conditions it is possible to calculate efficiently the maximum a posterior TAN model. Furthermore, we prove that it is also possible to calculate a weighted set with the k maximum a posteriori TAN models. This allows efficient TAN ensemble learning and accounting for model uncertainty. These results can be used to construct two classifiers. Both classifiers have the advantage of allowing the introduction of prior knowledge about structure or parameters into the learning process. Empirical results show that both classifiers lead to an improvement in error rate and accuracy of the predicted class probabilities over established TAN based classifiers with equivalent complexity.
Institut D'investigacio En Intel.ligincia
 Proceedings of the 16th International FLAIRS Conference
, 2003
"... The Naive Bayes classifier is a simple and accurate classifier. This paper shows that assuming the Naive Bayes classifier model and applying bayesian model averaging and the principle of indifference, an equally simple, more accurate and theoretically well founded classifier can be obtained. ..."
Abstract
 Add to MetaCart
The Naive Bayes classifier is a simple and accurate classifier. This paper shows that assuming the Naive Bayes classifier model and applying bayesian model averaging and the principle of indifference, an equally simple, more accurate and theoretically well founded classifier can be obtained.