P. Domingos. Process-oriented estimation of generalization error. In IJCAI-99, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....of hypotheses (e.g. tree pruning) well. Related Work. Average case analyses have been presented for decision stump learners [2] k nearest neighbor [5] naive Bayesian classi ers [4] and some other learners. Analyses in the same spirit but with stronger simpli cations have been presented in [9, 1]. ....

P. Domingos. Process-oriented estimation of generalization error. In Proceedings of the Sixteenth International Joint Conference on Articial Intelligenct, 1999.

....set of relevant attributes) which has to be known before the analysis can be applied. By contrast, our analysis treats hypotheses as black boxes that only possess error rates; our solution depends on a density that, in principle, be estimated from the sample. Sche er and Joachims (1998) and Domingos (1999) discussed almost identical analyses for exhaustive learners under some independence assumptions; Scheffer and Joachims (1999) removed one of these assumptions. These results for empirical error minimizing learners provide the basis of our analysis of cross validation. The fact that we can get ....

Domingos, P. (1999). Process-oriented estimation of generalization error. Proceedings of the Sixteenth International Joint Conference on Articial Intelligenct.

....set of relevant attributes) which has to be known before the analysis can be applied. By contrast, our analysis treats hypotheses as black boxes that only possess error rates; our solution depends on a density that, in principle, can be estimated from the sample. Sche er and Joachims (1998) and Domingos (1999) discussed almost identical analyses for exhaustive learners under some independence assumptions; Scheffer and Joachims (1999) removed one of these assumptions. These results for empirical error minimizing learners provide the basis of our analysis of cross validation. The fact that we can get ....

Domingos, P. (1999). Process-oriented estimation of generalization error. Proceedings of the Sixteenth International Joint Conference on Articial Intelligenct.

....by Sche er and Joachims [18, 17] and later generalized [19] and applied to text categorization and decision tree regularization [15] Independently, Domingos [1] presented a similar analysis which additionally assumes that all hypotheses incur equal error rates. Lifting the latter assumption [2] leads to an analysis that (besides making the additional assumption that the training set error is known) deviates from the rst analysis [18] only in some technical details. The histogram of error rates has been used to improve on worst case error bounds. The idea of a worst case analysis of ....

P. Domingos. Process-oriented estimation of generalization error. In Proceedings of the Sixteenth International Joint Conference on Articial Intelligenct, 1999.

....density that can often be estimated for a given i in O(log m) when a sample is given. Our analysis is an actual case analysis (for a given learner and a given learning problem) rather than a (PAC style) worst case analysis (for the worst possible problem) Compared to earlier actual case analyses [17, 16, 5] our analysis is based on weaker assumptions. Compared to [18] our analysis is considerably simpler and, most importantly, covers greedy learners. An actual case analysis for Naive Bayesian classi ers that is guided by a similar idea has been presented by Langley and Sage [10] an actual case ....

P. Domingos. Process-oriented estimation of generalization error. In IJCAI-99, 1999.

....tempering with the level of pruning, but it happens in a controlled way: the fitting is taken into account already in growing the decision tree, when choosing the attributes and deciding the tree s shape. The combined data fit endorsement and model complexity penalization yields, in Domingos [3, 5] terms, representations oriented evaluation. The rank optimizing algorithm works by backtracking, i.e. the algorithm may choose to revoke decisions it has made previously. The advantage of optimizing a size related parameter is that the learning algorithm cannot suffer from pathology that ....

Domingos, P.: Process-oriented estimation of generalization error. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Francisco, CA (to appear)

....which is based on several strong assumptions (e.g. it is assumed that the hypotheses which are contained in one model have equal true errors. Scheffer and Joachims (1998b, 1998a) have found a solution which is based on two independence assumptions, one of which is still rather strong. Recently, Domingos (1999) has proposed a new analysis, which deviates from the first analysis of Scheffer and Joachims (1998b) only in some technical details. In particular, it exploits the same assumption that hypotheses are drawn at random and independently by the learner and thus have independent true error rates. The ....

....of Scheffer and Joachims (1998b) only in some technical details. In particular, it exploits the same assumption that hypotheses are drawn at random and independently by the learner and thus have independent true error rates. The proximity between the analyses of Scheffer and Joachims (1998a) and Domingos (1999) can best be understood by comparing Equation 10 of Domingos (1999) to Theorem 3 of Scheffer and Joachims (1998a) Note further that Theorem 1 of Scheffer and Joachims (1998a) generalizes Theorem 3 by taking nonzero empirical errors into account. By contrast, in this paper and in its corresponding ....

[Article contains additional citation context not shown here]

Domingos, P. (1999). Process-oriented estimation of generalization error. In IJCAI-99.

No context found.

Domingos, P. (1999a). Process-oriented estimation of generalization error. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. Stockholm, Sweden: Morgan Kaufmann.

....although this may cause underfitting when the hypotheses being tested are not independent. A more general but computationally intensive solution is randomization testing (Edgington, 1980; Jensen, 1992) An approach that explicitly takes dependences into account is proposed in Domingos (1998a) and Domingos (1999). 3.4. Bias Variance Bias and variance are two useful concepts in characterizing the generalization behavior of learning algorithms (Geman et al. 1992) Bias is the systematic component of generalization error, that is incurred even if an infinite sample is available; variance is the additional ....

....that di#er more from the a priori best guess model can be assigned lower priors (Heckerman et al. 1995) Structural risk minimization and prior distributions only take into account the model space searched by the learner, not the way it is searched. Process oriented evaluation (Domingos, 1998a; Domingos, 1999) instead computes how the di#erence between the current best model s training set error and expected generalization error increases as search progresses. Intuitively, the more models have been attempted, the less likely the observed error of the best model found so far is to be close to its true ....

Domingos, P. (1999). Process-oriented estimation of generalization error. Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. Stockholm, Sweden: Morgan Kaufmann.

No context found.

P. Domingos. Process-oriented estimation of generalization error. In IJCAI-99, 1999.

No context found.

P. Domingos. Process-oriented estimation of generalization error. In Proceedings of the Sixteenth International Joint Conference on Arti cial Intelligenct, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC