DMCA
The use of the area under the ROC curve in the evaluation of machine learning algorithms (1997)
| Venue: | PATTERN RECOGNITION |
| Citations: | 684 - 3 self |
BibTeX
@ARTICLE{Bradley97theuse,
author = {Andrew P. Bradley},
title = {The use of the area under the ROC curve in the evaluation of machine learning algorithms},
journal = {PATTERN RECOGNITION},
year = {1997},
volume = {30},
number = {7},
pages = {1145--1159}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
In this paper we investigate the use of the area under the receiver operating characteristic (ROC) curve (AUC) as a performance measure for machine learning algorithms. As a case study we evaluate six machine learning algorithms (C4.5, Multiscale Classifier, Perceptron, Multi-layer Perceptron, k-Nearest Neighbours, and a Quadratic Discriminant Function) on six "real world " medical diagnostics data sets. We compare and discuss the use of AUC to the more conventional overall accuracy and find that AUC exhibits a number of desirable properties when compared to overall accuracy: increased sensitivity in Analysis of Variance (ANOVA) tests; a standard error that decreased as both AUC and the number of test samples increased; decision threshold independent; and it is invariant to a priori class probabilities. The paper concludes with the recommendation that AUC be used in preference to overall accuracy for "single number" evaluation of machine
Keyphrases
machine learning algorithm roc curve overall accuracy increased sensitivity k-nearest neighbour conventional overall accuracy standard error priori class probability multi-layer perceptron multiscale classifier single number quot case study desirable property quadratic discriminant function test sample performance measure medical diagnostics data set real world quot
