Domingos, P. 1996. Linear-Time Rule Induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 96--101. AAAI Press.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... that there are e examples, a attributes, and (on average) v values for each attribute, even very efficient inductive algorithms based on matching require O(e) matches for each of O(av) potential specializations of each hypothesis for a time complexity of O(eav) as described recently by Domingos (Domingos 1996). Now consider a learner that uses breadth first marker propagation to replace matching. After walking through the examples once, each of the possible specializations will have class counts tallying all the examples that match it. The counts can be retrieved by walking through valueset, which ....

Domingos, P. 1996. Linear-Time Rule Induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 96--101. AAAI Press.

.... search can be surprisingly fast (cf. the demonstrations below) If w beam heuristic pruning is used, rule space search has run time linear in the number of data items (as well as other parameters) Almost all other data mining algorithms have superlinear run time complexity (an exception being CWS (Domingos, 1996)) Let w be the maximum size of the set of most interesting nodes that will be saved at each level; let n be the number of data items. At each level of the search, at most) w nodes are expanded with f operators to produce wf new candidate rules. To gather statistics, each of these candidates ....

.... that there are e examples, a attributes, and (on average) v values for each attribute, even very ecient inductive algorithms based on matching require O(e) matches for each of O(av) potential specializations of each hypothesis for a time complexity of O(eav) as described, for example, by Domingos (Domingos, 1996). Now consider a learner that uses breadth rst marker propagation to replace matching. After walking through the examples once, each of the possible specializations will have class counts tallying all the examples that match it. The counts can be retrieved by walking through valueset, which ....

Domingos, P. (1996). Linear time rule induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 96-101 Menlo Park, CA. AAAI Press.

....iteration. This makes rules discovered at the beginning of the process have a stronger statistical support than the ones discovered later. In turn, inducing rules from small datasets exacerbates the small disjuncts problem [10] KDS uses the conquer without separating approach proposed by Domingos [7] which overcomes such a problem. Thus, all rules in KDS are always mined from the entire dataset. Most induction algorithms use a top down rule generation approach. In such an approach, a for each pos sible selector loop usually takes place at the time candidate rules are generated and ....

....have been proposed in the past. Furnkranz [9] lists and classifies 40 of them. The divide and conquer approach is basically used by all the tree induction algorithms rooted in the work of Quinlan [14] Domingos proposes a conquer without separation approach for his CWS and RISE systems [7, 8]. His without separation approach is oriented to solve the problem of progressive fragmentation of the input dataset. Domingos [6] shows how such a technique achieves substantial improvements in accuracy when mining databases with large number of disjuncts (each one covering few training ....

P. Domingos. Linear-time rule induction. KDD-96, 1996.

....4.3, respectively. After the pruning process, Noah starts a new iteration. This process continues as long as R k contains rules to be refined. Since we never partition the input dataset in Noah, all rules are always induced from the entire training set. This alleviates the small disjunct problem [9, 7]. 4.1 Terms Reordering To optimize rule lookup, all terms from the training set are ordered according to their support. Thus, a new version of the input dataset is created where each term is replaced by a numerical id that corresponds to the sorted order. Such a tokenized representation enables ....

P. Domingos. Linear-time rule induction. In E. Simoudis, J. W. Han, and U. Fayyad, editors, Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), page 96. AAAI Press, 1996.

....to the sequential covering approach eliminates all covered objects, a subset of the association rules is identified that is used as final rule set for classification. Results are reported that show some better classification accuracy than those induced by the sequential covering approach. CWS (Domingos 1996) dynamically interleaves rule induction and performance evaluation of a current rule set. Thus a new rule (which can also be a refinement of an existing rule) is not evaluated independently from the already found rules, but the accuracy of the rule set consisting of the current rules and the new ....

....(Quinlan 1993) generates rules from decision trees (C5.1.3) by statistical postpruning of decision trees. C4.5RULES has empirically been observed to require O(N 3 ) computation time dependent of the number of objects N of the database (Cohen 1995) For CWS, a time complexity of O(N) is claimed (Domingos 1996) and an empirically shown better accuracy on large datasets than C4.5RULES and CN2. RIPPER is competitive with C4.5RULES in accuracy, but has time complexity of O(N log N) But also the number A of attributes is important for analysing time complexity. A worst case bound for CN2 is given by O(N ....

P. Domingos 1996. Linear Time Rule Induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 96--101, Menlo Park, CA, AAAI Press.

....of the power (or impotence) of sampling for data mining, and the dearth of convincing examples of the need to mine massive data sets. SCALING UP INDUCTIVE ALGORITHMS 35 9. Acknowledgements We are indebted to many, including John Aronis, Lars Asker, Bruce Buchanan, Jason Catlett, Pedro Domingos, Phil Chan, Doug Fisher, Dan Hennessy, David Jensen, Ronny Kohavi, Rich Segal, Sal Stolfo, and anonymous referees of previous papers, who have influenced our views of scaling up through many discussions. Thanks also to the many who have pointed us to relevant work. Peter Huber gave an invited talk ....

Domingos, P. (1996b). Linear time rule induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, Menlo Park, CA, pp. 96--101. AAAI Press.

....rules via one algorithm or another, and then prune the rules in order to increase accuracy [61] Unfortunately, reduced error pruning systems generally do not scale well. For example, the rule learning variant of C4.5, C4.5rules, has been empirically observed to sometimes require O(e 3 ) time [26]. Some algorithms effective at finding high accuracy rule sets have O(e 4 ) time complexity in noisy domains [22] F urnkranz and Widmer [35] showed, with their incremental reduced error pruning (IREP) algorithm, that significant speedups can be obtained by pruning each rule as it is learned and ....

....including different rule evaluation criteria, different stopping criteria, and a post processing optimization, producing the algorithm RIPPER. RIPPER is shown to be very competitive with C4.5 rules in terms of error rates, while maintaining the O(e log 2 e) time complexity of IREP. 2 Domingos [26] proposes to improve rule learning efficiency by avoiding growing each rule to its full length in the first place, pointing out that the commonly used separate and conquer methods induce rules by evaluating each rule by itself, without regard to the effect of other rules. To avoid over growth, as ....

[Article contains additional citation context not shown here]

Domingos, P. (1996). Linear Time Rule Induction. In Proc. of the Second Intl.Conf. on Knowledge Discovery and Data Mining (KDD'96), Menlo Park, CA: AAAI Press, pp: 96-101.

....for the largest training set, rule generation consumes considerably larger amounts of time, requiring over 76,000 seconds (over 21 hours) to process the decision tree induced from the largest training set. Similar results have been observed for both artificial and real world data sets (Cohen 1995; Domingos 1996). Clearly, more e#cient implementations of the rule generation strategy of C4.5 are required to allow its use on large, noisy training sets. One approach to enhancing performance is the use of parallel processing; the remainder of this paper describes a parallel implementation of c4.5rules that ....

....The NASA shuttle data set, described in (Catlett 1991) and available from the UCI repository (Merz Murphy 1996) contains 43,500 training instances, each with 9 continuous attributes and assigned to one of seven classes. The data set was altered by introducing 20 classification noise as in (Domingos 1996). sleep A classification task concerning the study of sleep disorders. The training set contains 30,000 examples taken from 38 all night studies of patients admitted to a sleep disorder clinic. Each instance consists of 13 integer valued attributes. There are five class labels, corresponding to ....

Domingos, P. 1996. Linear-time rule induction.

....of the power (or impotence) of sampling for data mining, which we have discussed in depth, and the dearth of convincing examples of the need to mine massive data sets. 9. Acknowledgements We are indebted to many, including John Aronis, Lars Asker, Bruce Buchanan, Jason Catlett, Pedro Domingos, Phil Chan, Doug Fisher, Dan Hennessy, David Jensen, Ronny Kohavi, Rich Segal, Sal Stolfo, and anonymous referees of previous papers, who have influenced our views of scaling up through many discussions. Thanks also to the many who have pointed us to relevant work. Peter Huber gave an invited talk ....

Domingos, P. (1996b). Linear time rule induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, Menlo Park, CA, pp. 96--101. AAAI Press.

No context found.

Domingos, P. (1996a). Linear-time rule induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (p. 96-101). Portland, OR: AAAI Press.

....replicates had been used. m = 25 was used throughout the studies reported below. The number n of examples generated randomly for meta learning was set to 1000. This value reflects the knowledge that C4.5RULES tends to produce rule sets whose size grows approximately linearly with training set size (Domingos, 1996; Oates Jensen, 1997) and thus that using a very large n is likely to lead to unnecessarily complex models. Within this constraint, n was chosen to be larger than any of the dataset sizes present. Given that the randomly generated examples are added to the original ones, this implies that the ....

....obtained by empirically estimating this point for each dataset. 8 More conveniently, CMM may be used with a base learner whose output size is insensitive to training set size, once the accuracy asymptote is reached (or at least less sensitive than C4.5RULES) Experiments with such systems (e.g. (Domingos, 1996; Jensen, 1997) are an area for future research. 4 RELATED WORK The CMM algorithm bears interesting relationships to many pieces of previous research in inductive learning. Apart from its effect on accuracy, pruning of decision trees and rule sets can be viewed as an attempt to extract a ....

Domingos, P. (1996). Linear-time rule induction. Proc.

....replicates had been used. m = 25 was used throughout the studies reported below. The number n of examples generated randomly for meta learning was set to 1000. This value reflects the knowledge that C4.5RULES tends to produce rule sets whose size grows approximately linearly with training set size [13, 25], and thus that using a very large n is likely to lead to unnecessarily complex models. Within this constraint, n was chosen to be larger than any of the dataset sizes present. Given that the randomly generated examples are added to the original ones, this implies that the training set size for ....

....obtained by empirically estimating this point for each dataset. 9 More conveniently, CMM may be used with a base learner whose output size is insensitive to training set size, once the accuracy asymptote is reached (or at least less sensitive than C4.5RULES) Experiments with such systems (e.g. [13, 20]) are an area for future research. 4 Related Work The CMM algorithm bears interesting relationships to many pieces of previous research in inductive learning. Apart from its effect on accuracy, pruning of decision trees and rule sets can be viewed as an attempt to extract a simpler, more ....

P. Domingos. Linear-time rule induction. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pages 96--101, Portland, OR, 1996. AAAI Press.

No context found.

Domingos, P. (1996). Linear-Time Rule Induction. Proceedings of the Second International Conference on Data Mining and Knowledge Discovery, AAAI Press, pp 96-101.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC