Peter Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac (editors), Progress in Machine Learning, pages 11--30. Sigma Press, Wilmslow, 1987.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....behavior on these datasets is almost the same. 5.2.3.2 Experiments with Increasing Noise Level In this section, noise tolerance of the FIL algorithms are investigated. There are two major types of noise that can be found in real world datasets: feature (attribute) noise, and classification noise [3, 11, 14, 24, 63]. Feature noise can be defined as incorrect feature value information. Classification noise involves corruption of the class label of an instance. Quinlan demonstrated that feature noise, occurring simultaneously in all features describing the instances, can result in faster degradation in ....

P. Clark and T. Niblett, Induction in Noisy Domains, In I. Bratko and N.Lavrac (Eds.), Progress in Machine Learning, pp:11-30, Wilmslow, England: Sigma Press, 1987.

....affected with the addition of irrelevant features. 5.2.3.2 Experiments with Increasing Noise Level This section investigates the effect of noise in the datasets on the VFI algorithms compared to other algorithms. There are two major types of noise that can be found in real world datasets [3, 11, 15, 27, 69]: 1. Feature (attribute) noise, defined as incorrect feature value. 2. Classification noise, defined as incorrect class label of an instance. Quinlan demonstrated that feature noise, occurring simultaneously in all features describing the instances, can result in faster decrease in ....

....in the figure. The instance is predicted as class 2, which is the actual class predicted by the human expert. But the next highest vote, received by Feature values of test instance 1: F[1] 2 F[2] 3 F[3] 3 F[4] 3 F[5] 3 F[6] 0 F[7] 0 F[8] 0 F[9] 3 F[10] 3 F[11] 0 F[12] 0 F[13] 0 F[14] 0 F[15]:0 F[16] 0 F[17] 3 F[18] 2 F[19] 2 F[20] 3 F[21] 3 F[22] 3 F[23] 1 F[24] 3 F[25] 0 F[26] 0 F[27] 0 F[28] 0 F[29] 0 F[30] 0 F[31] 0 F[32] 1 F[33] 0 F[34] 34 Classes: 1] 2] 3] 4] 5] 6] Votes of Feature[1] 0.16 ....

[Article contains additional citation context not shown here]

P. Clark and T. Niblett, Induction in Noisy Domains, In I. Bratko and N.Lavrac (Eds.), Progress in Machine Learning, 11--30, Wilmslow, England: Sigma Press, 1987.

....to be irrelevant. 2.3 Rule Induction Using the Covering Algorithm Rule learning typically consists of two main procedures: the search procedure that performs search in order to find a single rule and the control procedure that repeatedly executes the search. In the propositional rule learner CN2 [5, 6], for instance, the search procedure performs beam search using classification accuracy of the rule as a heuristic function. The accuracy of rule H B is equal to the conditional probability of head H, given that the body B is satisfied: Acc(H B) p(H B) The accuracy measure can be ....

P. Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Progress in Machine Learning (Proc. of the 2nd European Working Session on Learning), 11--30. Sigma Press, 1987.

....that do not cover any negative cases. Of these expressions it favors the expression that covers the most positive cases. It can be expressed as if jneg(s)j 0 then value(s) Gammajneg(s)j else value(s) jpos(s)j. where s is the state being evaluated. The Laplace preference function [1] trades off accuracy against generality. It is defined as value = jpos(s)j 1 jpos(s)j jneg(s)j no of classes The exact pruning rules that may be employed for a given search problem will depend upon the available means of identifying solutions and of identifying whether the application of ....

Peter Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac (editors), Progress in Machine Learning, pages 11--30. Sigma Press, Wilmslow, 1987.

....data sets, in the medical domains of dermatology and breast cancer. These data sets were obtained from the UCI (University of California at Irvine) Machine Learning Repository [17] These data sets have been used extensively for classification tasks using different paradigms, see, for instance [3] and [5] The main characteristics of each of these domains are described in the next two subsections. 4.1 Dermatology Data Set The differential diagnosis of the disease erythematosquamous is an important problem in dermatology. There are six different diagnoses (six classes) and all of them ....

....with its values expressed in years, and the attribute family history, which takes on the value 1 when the disease has been observed in the family of the patient and 0 otherwise. 4. 2 Breast Cancer Data Set Breast cancer re occurs in up to 30 of the patients that undergo a breast cancer surgery [3]. This data set contains 286 records, each with 9 attributes, and the goal is to determine the patients for whom the cancer will re occur. Hence, there are only two classes, namely no recurrenceevents and recurrence events. All attributes are categorical. More detailed information about this data ....

[Article contains additional citation context not shown here]

Clark, P., Niblett, T. Induction in Noisy Domains. In: Progress in Machine Learning (from the Proceedings of the 2 nd European Working Session on Learning), p. 11-30, Sigma Press, 1987.

.... Comparative review of learning from examples techniques: computational efficiency others Dietterich Michalski 1983 Algorithm complexity Utgoff 1989 Classification accuracy Michalski, Mozetic, Hong Lavrac Mingers Quinlan Utgoff 1986 1989a 1989 1989 Noisy and incomplete data Cestnik Quinlan Clark Niblett 1987 1986 1989 1989 Use of common sets of examples for different studies Cestnik Clark Niblett Michalski 1987 1987 1989 1990 Obtained results legibility Cendrowska Cestnik 1988 1989 Frameworks for tools studies Dhaliwal Benbasat Gams Lavrac 1990 1987 Comparing Symbolic and Neural Learning ....

....getting it on the moon. A tool dealing with cost will solve a problem using the most 10 economic information and will only consider higher cost attributes when they are strictly necessary. Noisy and incomplete data are related to the fact that the normal data source is the real world. These data (Clark 1987; Mingers 1989b) usually come with some mistakes (noise) made during their collection and sometimes include unknown values for some of the collected features, i.e. they are incomplete. Finally, a ML tool is incremental when the learning set does not need to be complete in order to start the ....

Clark, P., and Niblett, T. 1987. Induction in Noisy Domains. In Proceedings of the EWSL 87: Second European Working Session on Learning, 11-30. Bled, Yugoslavia: Sigma Press.

....si les exemples sont d ecrits par le seul masque de l op erateur concern e. Ces incoh erences, qui restent marginales dans la pratique, ne p enalisent cependant pas la d emarche propos ee dans la mesure o u de nombreux algorithmes d apprentissage (dont DiVS) permettent de g erer les incoh erences [50, 9]. 4.3.4 Controle par r egles Les r egles apprises permettent d estimer a priori si un nouvel ev enement (croisement ou mutation de parents donn es) sera bon, mauvais ou inactif ; elles peuvent donc etre utilis ees pour filtrer les ev enements souhaitables pour l evolution. Trois types de ....

P. Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Proc. of European WorkShop on Learning, pages 11--30. Sigma Press, 1987.

....Th j (Ex) tends toward as j increases) includes consistent hypotheses only, and maximally general consistent hypotheses in particular. No doubt this approach is ill suited to real world datasets: when erroneous examples are encountered, strictly consistent hypotheses have few predictive accuracy [4]. And when examples are sparse, maximally general consistent hypotheses are too general: most instances come to be covered by a hypothesis in most Th j (Ex i ) and therefore get unclassified, or classified in the majority class. These limitations were already encountered in the attribute value ....

P. Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Proc. of European WorkShop on Learning, pages 11--30. Sigma Press, 1987.

....keeping simple the used theory. The system is able to use different set ups of these mechanisms which contributes to the good flexibility of the program. The following sections explain these two strategies in more detail. 3. 1 Redundancy Most existing algorithms like for instance, AQ [10] and CN2 [4], use a covering strategy during learning. This means that the algorithm attempts to cover all known examples and that whenever some example has been covered it is removed. These systems would consider a rule useless if it covered examples that are already covered by other rules. AQ16 [15] uses a ....

....and each time an example is covered by some rule the example is removed from the set. Their goal is to make this set empty. In YAILS this is not the case thus enabling the production of redundant rules. The main differences stated between YAILS and AQ type programs also hold in comparison to CN2 [4] with the addition that CN2 is non incremental. In effect CN2 has a search strategy that is similar to AQ with the difference of using ID3 like information measures to find the attributes to use in specialisation. STAGGER [14] system uses weights to characterise its concept descriptions. In ....

Clark, P., Niblett, T. : "Induction in noisy domains", in Proc. of the 2th European Working Session on Learning , Bratko,I. and Lavrac,N. (eds.), Sigma Press, Wilmslow, 1987.

....are capable of generating theories that perform reasonably well on tests. As we had earlier reimplemented ID3 and AQ like systems, we decided to use these as the basic inductive engines in our set up. The reimplementation of ID3 based on earlier work (e.g. Quinlan (1986) Cestnik et al. 1987) Clark and Niblett(1987; 1989) will be referred to as ITL1 (Inductive Tree Learning System) The decision tree generated by this system is automatically converted into a rule form which we find more amenable for further manipulation. The inductive rule learning system IRL1 is an incremental learning program, that was ....

Clark, P. and Niblett, T. (1987): "Induction in Noisy Domains", in Progress in Machine Learning, I.

....at a crossing road, even when roads are sloping provided that a policeman stands there) 4.2 Discussion Validation of the above technique requires both an inconsistent knowledge base and a set of examples. For the sake of convenience, a well studied machine learning problem was considered [9, 4, 3]: we first extracted an inconsistent KB from an example set, then another set of examples is used to refine the inconsistent set of rules [15] This medical problem fits within multi valued propositional logic ; in this case, reduction becomes a straightforward operator since there is no matter of ....

....approximate rules) and then back to induction again (to learn an approximate set of meta rules) and reduction (to build examples about the meta rules behavior) and so forth. This approach was found to achieve accurate prediction, compared to some well known induction algorithms as AQ15 [9] CN2 [4] or Assistant86 [3] Several remarks follow these experiments. First of all, only three iterations of the process inductionreduction were found profitable regarding the predictive accuracy on a test set. Gain from first to second step was significant; this can be explained as more complex rules ....

Clark P. Niblett T. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Progress in machine learning, Proc. EWSL 1987. Sigma Press, 1987.

....affected with the addition of irrelevant features. 5.2.3.2 Experiments with Increasing Noise Level This section investigates the effect of noise in the datasets on the VFI algorithms compared to other algorithms. There are two major types of noise that can be found in real world datasets [3, 11, 15, 27, 69]: 1. Feature (attribute) noise, defined as incorrect feature value. 2. Classification noise, defined as incorrect class label of an instance. Quinlan demonstrated that feature noise, occurring simultaneously in all features describing the instances, can result in faster decrease in classification ....

....which is the actual class predicted by the human expert. But the next highest vote, received by CHAPTER 7. VISUALIZATION OF THE LEARNED CONCEPTS 134 Feature values of test instance 1: F[1] 2 F[2] 3 F[3] 3 F[4] 3 F[5] 3 F[6] 0 F[7] 0 F[8] 0 F[9] 3 F[10] 3 F[11] 0 F[12] 0 F[13] 0 F[14] 0 F[15]:0 F[16] 0 F[17] 3 F[18] 2 F[19] 2 F[20] 3 F[21] 3 F[22] 3 F[23] 1 F[24] 3 F[25] 0 F[26] 0 F[27] 0 F[28] 0 F[29] 0 F[30] 0 F[31] 0 F[32] 1 F[33] 0 F[34] 34 Classes: 1] 2] 3] 4] 5] 6] Votes of Feature[1] 0.16 0.16 ....

[Article contains additional citation context not shown here]

P. Clark and T. Niblett, Induction in Noisy Domains, In I. Bratko and N.Lavrac (Eds.), Progress in Machine Learning, 11--30, Wilmslow, England: Sigma Press, 1987.

....found in (Sebag 1994) 3 TUNING THE CLASSIFICATION This section describes how to adjust the above classification process, in order to account for the rate of noise and the sparseness of the training set. 3. 1 TUNING THE CONSISTENCY Real world datasets always include false examples; as noted by Clark and Niblett (1987), this implies that the set of strictly consistent hypotheses is both large and not of the highest predictive power. Hence, most learners are nowadays concerned with finding hypotheses consistent enough , i.e. admitting a bounded number of inconsistencies within the training examples, rather than ....

Clark, P. and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Proc. of European WorkShop on Learning, pages 11--30. Sigma Press, 1987.

....[39] This data set contains 286 examples each with nine features, some linear and some nominally valued. The learn1 Data provided courtesy of M. Zwitter and M. Soklic. Figure 4.2: Voting data projection ing task is to predict whether cancer will recur following treatment. Previous results [35, 8, 57, 7] have a reported accuracy between 66 and 78 . Using ten five fold cross validation experiments, HDE performs comparably under standard defaults as shown in Table 4.7. Average Minimum Maximum Boundary Points 90 90 90 Candidate Hyperplanes 72.12 67.60 78.80 Hidden Units 4.20 3.20 5.00 Training Set ....

P. Clark and T. Niblett. Induction in noisy domains. In Progress in Machine Learning (from Proceedings of the Second European Working Session on Learning) , pages 11--30, Bled, Yugoslavia, 1987. Sigma Press.

....of the errors (column 6) even though they match only 41 of the examples (column 5) This pattern of errors is not unique to CN2, or to this domain. A similar pattern occurs in the definitions created by ID3 [Qui86] in this domain, and in the definitions created by CN2 in the lymphography domain [CN87] Table 3) Ideally, induced definitions should consist of all, and only, disjuncts that are meaningful (e.g. as measured by a statistical significance test) and have a low error rate. Definitions created by existing methods are ideal with regard to large disjuncts, but far from ideal with ....

.... 13 14 15 default Error Rate 18 0 0 3 2 42 8 0 2 0 5 9 0 0 12 48 Coverage 8 36 58 5 19 5 19 7 5 8 8 5 5 6 5 3 This and other error estimation techniques are described in [Nib87, page 43] 4 Clark and Niblett have observed empirically that rules of low entropy tend to have high significance ( CN87, page 18] mistakenly reports this as a relation between rules of high entropy and high significance) Thus, if error rate is not related to entropy, then neither is it related to significance. Approach 3: Make Small Disjuncts Highly Specific The techniques considered in the previous sections ....

[Article contains additional citation context not shown here]

Peter Clark and Tim Niblett. Induction in noisy domains. In Ivan Bratko and Nada Lavrac, editors, Progress in Machine Learning, pages 11--30. Sigma Press, Wilmslow, England, 1987.

....The consequence of this is that R R 2 2 can learn different rule models for the same data. This form of redundancy is similar to the one used in the classification system Yails [16] In noisy data it has obvious advantages [16] when compared to the more common covering algorithms (like CN 2 [3] or AQ [10] The next step in the algorithm consists of developing a regression model for the chosen condition. R R 2 2 uses a lattice of increasingly complex regression model languages as the basis of this process. The system builds a regression model using each of these languages and chooses ....

. Clark, P., Niblett, T. : Induction in noisy domains, in Proc. of the 2th European Working Session on Learning , Bratko,I. and Lavrac,N. (eds.), Sigma Press, Wilmslow, 1987.

....rule, a decision tree branch) rather than to the model as a whole the latter s generality (usually covering the entire space of examples) typically remains unchanged during search if generality is defined as the proportion of example space covered. or using chi squared tests such as in CN2 2 [14]. The ID3 algorithm, performing a general to specific search already, can be similarly modified to cope with noise by halting branch growth earlier, as illustrated for example by Assistant 86 [15] ffl Rule truncation, or post pruning An alternative to prematurely halting a general to specific ....

Peter Clark and Tim Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Progress in Machine Learning (proceedings of the 2nd European Working Session on Learning), Sigma, Wilmslow, UK, 1987.

....To compare combine this style of image classification with other methods, we use a standard maximum likelihood classification (MLC) algorithm based on application of Bayes rule. This approach has been used successfully in image classification (e.g. 1, 8] and other classification tasks (e.g. [9, 10, 11]) For each prediction class C i , the probability that an example with a feature vector V = v 1 ; v n belongs to C i is: p(C i jV ) p(V jC i ) Theta p(C i ) p(V ) 1) p(V jC i ) Theta p(C i ) P j (p(V jC j ) Theta p(C j ) 2) 2 In fact, the user specifies the percentage of ....

Peter Clark and Tim Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Progress in Machine Learning (proceedings of the 2nd European Working Session on Learning), pages 11--30. Sigma, Wilmslow, UK, 1987.

....to this interpolation. Should the plot not fit well, other plots (e.g. log plots) might be tried. Wells which are poorly fitting may suggest to the expert to search for an explanation for the discrepancy, and hence may have their relevances changed. Note that inductive tools such as ID3 [11] or CN2 [3] are not appropriate to our problem of forming a general theory from examples because we are predicting a numeric value rather than membership of a class, we are working with few ( 10, usually) examples and we wish to give different weights to the examples. In addition, in future we wish to ....

P. Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac, editors, Progress in Machine Learning (proceedings of the 2nd European Working Session on Learning), pages 11--30. Sigma, Wilmslow, UK, 1987.

....[Hayes Michie, 1990] and in process control [Leech, 1986] The continuing development of inductive techniques is thus valuable to pursue. CN2 is an algorithm designed to induce if. then. rules in domains where there might be noise. The algorithm is described in [Clark and Niblett, 1989] and [Clark and Niblett, 1987], and is summarised in this paper. The original algorithm used entropy as its search heuristic, and was only able to generate an ordered list of rules. In this paper, we demonstrate how using the Laplacian error estimate as a heuristic significantly improves the algorithm s performance, and ....

Clark, P. and Niblett, T. (1987). Induction in noisy domains. In Bratko, I. and Lavrac, N., editors, Progress in Machine Learning (proceedings of the 2nd European Working Session on Learning), pages 11--30. Sigma, Wilmslow, UK.

No context found.

Peter Clark and T. Niblett. Induction in noisy domains. In I. Bratko and N. Lavrac (editors), Progress in Machine Learning, pages 11--30. Sigma Press, Wilmslow, 1987.

No context found.

Clark, P and Niblett, T. (1987). Induction in noisy domains. In Bratko and Lavrac (Eds.) Progress in Machine Learning. Wilmslow: Sigma Press.

No context found.

P. Clark and T. Niblett, Induction in Noisy Domains, In I. Bratko and N.Lavrac (Eds.), Progress in Machine Learning, pp:11-30, Wilmslow, England: Sigma Press, 1987.

No context found.

Clark, P. & Niblett, T. (1987). Induction in noisy domains. In Bratko & Lavrac (Ed.), Progress in Machine Learning, Sigma Press.

No context found.

P. Clark and T. Niblett. Induction in noisy domains. In Progress in Machine Learning (from Proceedings of the Second European Working Session on Learning) , pages 11--30, Bled, Slovenia, 1987. Sigma Press.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC