On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach

DMCA

On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach (1997)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Steven L. Salzberg , Usama Fayyad

Venue:	Data Mining and Knowledge Discovery
Citations:	224 - 0 self

BibTeX

@ARTICLE{Salzberg97oncomparing,
    author = {Steven L. Salzberg and Usama Fayyad},
    title = {On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach},
    journal = {Data Mining and Knowledge Discovery},
    year = {1997},
    pages = {317--328}
}

OpenURL

Abstract

Abstract. An important component of many data mining projects is finding a good classification algorithm, a process that requires very careful thought about experimental design. If not done very carefully, comparative studies of classification and other types of algorithms can easily result in statistically invalid conclusions. This is especially true when one is using data mining techniques to analyze very large databases, which inevitably contain some statistically unlikely data. This paper describes several phenomena that can, if ignored, invalidate an experimental comparison. These phenomena and the conclusions that follow apply not only to classification, but to computational experiments in almost any aspect of data mining. The paper also discusses why comparative analysis is more important in evaluating some types of algorithms than for others, and provides some suggestions about how to avoid the pitfalls suffered by many experimental studies.

DMCA