Citation Context

...ed and has been shown to provide an adequate and accurate estimate of 4Particularly for the Multi-layer Perceptron. The use of the area under the ROC curve in the evaluation 1149 the true error rate. =-=(27)-=- The cross-validation sampling technique used was random but ensured that the approximate proportions of examples of each class remain 90% in the training set and 10% in the test set. This slight adju...

Citation Context

..., are then applied to these distances, dj, so as to weight the decision function and minimise the Bayes risk of misclassification. For these experiments misclassification costs were used in the range =-=[0,1]-=- in steps of 1/14. k-Nearest Neighbours. For each test example, the five nearest neighbours (calculated in terms of the sum of the squared difference of each input attribute) in the training set are c...

Citation Context

...gration to calculate AUC. The AUC was calculated for each learning algorithm on each of the 10 test partitions. This is in effect using a jackknife estimate to calculate the standard error of the AUC =-=(29)-=- and will be discussed in more detail shortly. Remark. It should be noted that there are two distinct possibilities when it comes to combining the ROC curves from the different test partitions, ~3°) 1...

Citation Context

...rror. However, in general it is not misclassification rate we want to minimise, but rather misclassification cost. Misclassification cost is normally defined as follows: Cost = Fp . CFp -[- F. • CFn. =-=(6)-=- Unfortunately, we rarely know what the individual misclassification costs actually are (here, the cost of a false positive, Cv~ and the cost of a false negative, Cv.) and so system performance is oft...

Citation Context

... blood loss to make the data set consistent with the others used in this paper, lit is also recommended for methods such as k nearest 2They were not highly correlated to the other features neighbours.=-=(16)-=- selected.s1148 A. E BRADLEY and as part of this preliminary study, this simplistic model was thought to be sufficient. However, most of the classification algorithms detailed in Section 4 have been u...

Citation Context

...n the ROC curve [P(Fp) = c~, P(Tp) = 1 -/3] have been obtained the simplest way to calculate the area under the ROC curve is to use trapezoidal integration, AUC=~. {(1-/3i.Ac~)+~[A(1-/~).Ac~]}, where =-=(7)-=- ~(1-~) : (1-/3i)-(l-9i 1), (8) As = c~i - c~i-l. (9) It is also possible to calculate the AUC by assuming that the underlying probabilities of predicting negative or positive are Gaussian. The ROC cu...

Citation Context

...each test example, the five nearest neighbours (calculated in terms of the sum of the squared difference of each input attribute) in the training set are calculated. Then, if greater than L, where L--=-=[0, 1, 2, 3, 4, 5]-=-, if the nearest neighbours are of class 1, the test sample is assigned to class 1; if not, it is assigned to class 0. Release 5 of the C4.5 decision tree generator (2°~ was used with the following mo...

Citation Context

...1]. All three networks were trained using back-propagation with a learning rate of 0.01, and a momentum of 0.2. Initial values for the weights in the networks were set using the Nguyen-Widrow method, =-=(25)-=- and the networks were trained for 20,000 epochs. Again, during the testing phase the output neuron was thresholded at values [0, 0.1, 0.2, 0.3 ..... 1.0] to simulate different misclassification costs...

Citation Context

...Q2-02) cpc. The use of the area under the ROC curve in the evaluation 1147 (10) where, C. and Cp are the number of negative and positive examples respectively and 0 Q1 - (2- 0~' (11) 202 02 - (1 + 0) =-=(12)-=- In this paper we shall calculate AUC using trapezoidal integration and estimate the standard deviation, SD(0), using both SE(W) and cross-validation, details of which are given in Sections 5 and 6. N...

Citation Context

...each test example, the five nearest neighbours (calculated in terms of the sum of the squared difference of each input attribute) in the training set are calculated. Then, if greater than L, where L--=-=[0, 1, 2, 3, 4, 5]-=-, if the nearest neighbours are of class 1, the test sample is assigned to class 1; if not, it is assigned to class 0. Release 5 of the C4.5 decision tree generator (2°~ was used with the following mo...

Citation Context

...Tp) = 1 -/3] have been obtained the simplest way to calculate the area under the ROC curve is to use trapezoidal integration, AUC=~. {(1-/3i.Ac~)+~[A(1-/~).Ac~]}, where (7) ~(1-~) : (1-/3i)-(l-9i 1), =-=(8)-=- As = c~i - c~i-l. (9) It is also possible to calculate the AUC by assuming that the underlying probabilities of predicting negative or positive are Gaussian. The ROC curve will then have an exponenti...

Citation Context

...each test example, the five nearest neighbours (calculated in terms of the sum of the squared difference of each input attribute) in the training set are calculated. Then, if greater than L, where L--=-=[0, 1, 2, 3, 4, 5]-=-, if the nearest neighbours are of class 1, the test sample is assigned to class 1; if not, it is assigned to class 0. Release 5 of the C4.5 decision tree generator (2°~ was used with the following mo...

Citation Context

...each test example, the five nearest neighbours (calculated in terms of the sum of the squared difference of each input attribute) in the training set are calculated. Then, if greater than L, where L--=-=[0, 1, 2, 3, 4, 5]-=-, if the nearest neighbours are of class 1, the test sample is assigned to class 1; if not, it is assigned to class 0. Release 5 of the C4.5 decision tree generator (2°~ was used with the following mo...

Citation Context

... obtained the simplest way to calculate the area under the ROC curve is to use trapezoidal integration, AUC=~. {(1-/3i.Ac~)+~[A(1-/~).Ac~]}, where (7) ~(1-~) : (1-/3i)-(l-9i 1), (8) As = c~i - c~i-l. =-=(9)-=- It is also possible to calculate the AUC by assuming that the underlying probabilities of predicting negative or positive are Gaussian. The ROC curve will then have an exponential form and can be fit...

Citation Context

... xn). In the same situation, given one normal example and one positive example, 6 a classifier with decision threshold t will get both examples correct with a probability, P(C) - P(xp > t)P(x,, < t). =-=(15)-=- P(C) is dependent on the location of the decision threshold t and is therefore not a general measure of classifier performance. 9.3.2. The standard error of AUC. The AUC, 0, is an excellent way to me...

Citation Context

...nd 10% in the test set. This slight adjustment to maintain the prevalence of each class does not bias the error estimates and is supported in the research literature. (26) As pointed out by Friedman, =-=(28)-=- no classification method is universally better than any other, each method having a class of target functions for which it is best suited. These experiments then, are an attempt to investigate which ...

Citation Context

... used in this experiment to minimise any estimation bias. A leave-one-out classification scheme was thought computationally too expensive 4 and so, in accordance with the recommendations in reference =-=(26)-=-, 10-fold crossvalidation was used on all of the data sets. For consistency, exactly the same data were used to train and test all of the nine classification schemes, this is often called a paired exp...

Citation Context

... Gaussian based ROC curve (as in the ML method). The standard error, SE(W), is given by SE(W) = ~O(1-O)+(Cp-1)(QI-02)+(C,-I)(Q2-02) cpc. The use of the area under the ROC curve in the evaluation 1147 =-=(10)-=- where, C. and Cp are the number of negative and positive examples respectively and 0 Q1 - (2- 0~' (11) 202 02 - (1 + 0) (12) In this paper we shall calculate AUC using trapezoidal integration and est...

Citation Context

... P(wj), mean, mj, and covariance, Cj of the two class distributions. The Bayes decision function for class wj of an example x is then given by dj(x) = lnP(wj) -~ 1 In Icjl -~ 1 [(x--mj)Tcj l(x--mj)]. =-=(13)-=- This decision function is then a hyper-quadric, the class of an example being selected as the minimum distance class. Misclassification costs, cj, are then applied to these distances, dj, so as to we...

Citation Context

...s. For the difference between two subsets of means to be significant it must exceed a certain value. This value is called the least significant range for the p means, R m and is given by Rp rpX/s2/r, =-=(14)-=- where the sample variance, s 2, is estimated from the error mean square from the analysis of variance, s 2 3' r the number of observations (rows), and rp the least significant studentized-range for a...

Citation Context

..., 37.5% heart disease present. 4. THE LEARNING ALGORITHMS The learning algorithms chosen for this experimental comparison were: • Quadratic Discriminant Function 08) (Bayes); 3 • k-Nearest Neighbours =-=(19)-=- (KNN); • C4.5 ~2°) (C4.5); • Multiscale Classifier ~15) (MSC); • Perceptron f2~) (PTRON); • and Multi-layer Perceptron <22) (MLP). We chose a cross-section of popular machine learning techniques toge...

Citation Context

...ion function, scaled to give an output in the range [0,1 ]. The output of this linear neuron was then thresholded at values [0, 0.1, 0.2, 0.3 ..... 1.0] to simulate different misclassification costs. =-=(24)-=- The Multi-layer Perceptron. Three network architectures were implemented, each with different numbers of hidden units. Their network architecture was as follows: an input layer consisting of a number...

Citation Context

...) we found that the MSC obtained a higher accuracy (76%) when no pruning was done on the tree. This is an example of a problem domain where the algorithm has been biased by the decision tree pruning. =-=(33)-=- There are three significant subgroups shown for the Breast Cancer data set in Table 4. There is a large amount of overlap in these subgroups and so no real identifiable groups seem to exist. However,...