Murphy, P.M. and Pazzani, M.J. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the Eighth International Workshop on Machine Learning, pages 183--187, Evanston, IL, June 1991.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....and Merz] and three artificial data sets. Langley and Sage (1994b) indicate that many of the datasets in UCI repository have just a few completely irrelevant features and no complex feature interactions. We also used three artificial data sets that were either M of N concepts or X of N concepts [Murphy and Pazzani, 1991]. The three artificial data sets used for study were at least 3 of 6 among 13 features ( 3 of 6) exactly 3 of 6 among 13 features ( 3 of 6) and exactly 5 of 10 among 13 features ( 5 of 10) These domains have many irrelevant features and the exactly concepts represent a non linearly separable ....

.... artificial data sets used for study were at least 3 of 6 among 13 features ( 3 of 6) exactly 3 of 6 among 13 features ( 3 of 6) and exactly 5 of 10 among 13 features ( 5 of 10) These domains have many irrelevant features and the exactly concepts represent a non linearly separable problems [Murphy and Pazzani, 1991]. Table I, shows the di#erent elements of the SCRAP feature selection algorithm. Ten runs with one third training instance and one third testing set was used to validate the results. It can be seen that real domains have many weakly relevant features rather than strongly relevant or totally ....

Murphy, P.M. and Pazzani, M.J. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the Eighth International Workshop on Machine Learning, pages 183--187, Evanston, IL, June 1991.

....other than the relevant features were randomly generated. 1000 examples were generated from a uniform distribution. The artificial domains were based on m of n and x of y concepts. These concepts are not linearly separable and are known to be di#cult for many classes of learning algorithms [Murphy Pazzani, 1991]. 7.2.1 k Nearest Neighbors SCRAP was designed to improve the performance of the k Nearest Neighbors learners [Hart Cover, 1967] which are non parametric approaches. These are lazy learners and use 11 Dataset Inst. No.of. Abs Ir Relative Ambigous No.of Neigh Space Feat. Rel Rel ....

Murphy, P.M. and Pazzani, M.J. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the Eighth International Workshop on Machine Learning, pages 183--187, Evanston, IL, June 1991.

....constructive induction expands the feature space before building the classification model. There are many ways of creating new features, Zheng et al. [Zij96] presents a discussion of using conjunctive, disjunctive and x of N features. The features of type x of N were first studied by Murphy et al. [MP91] Brodley et al. [BU92] consider composite features which are modeled as linear functions, which operate on different attribute values. In this paper we will be limiting ourselves to the study of conjunctive attribute values, a detailed discussion about the advantages of using conjunctive ....

Patrick M. Murphy and Michael J. Pazzani. Id2-of-3: Constructive induction of m-of-n concepts for discriminators in decision trees. In Proc. of the 8th Int Workshop on Machine Learning, 1991.

....complexity. algorithm internal nodes leaves symbols C4.5 44.3 Sigma 27.8 47.5 Sigma 28.9 44.3 Sigma 27.8 TREPAN 5.8 Sigma 0.9 6.8 Sigma 0.9 20.0 Sigma 4.7 trees than does C4.5. Whereas each split in a C4.5 tree tests only a single attribute, TREPAN s trees can use m of n splits [2]. An m of n split is a Boolean expression that is true if at least m out of a specified set of n conditions are true. In determining the number of splits used in a tree, we count an ordinary, single feature split as one symbol, and we count an m of n split as n symbols, since such a split lists n ....

P. M. Murphy and M. J. Pazzani. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the Eighth International Machine LearningWorkshop,pages 183--187, Evanston, IL, 1991.Morgan Kaufmann.

....case with classical decision trees, and thus with TREPAN. In contrast, fuzzy decision trees may yield continuous as well as fuzzytermed output. 5.3. Conditions at tree nodes Besides the conventional conditions in tree nodes, original TREPAN also allows using more sophisticated M of N condition [18]. This operation may lead to smaller trees but we believe this sacrifices comprehensibility. Justifying why an example reached a particular leaf is more difficult because the example does not have to satisfy all elementary conditions on the path from the root to the leaf. Moreover, M of N ....

P.M. Murphy, M.J. Pazzani. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In: Proc. Eighth Int. Machine Learning Workshop, Morgan Kaufmann 1991.

....decision trees. For example, CART (Breiman et al., 1984) can create one Boolean combination of primitive attributes as a new attribute at each decision node during the process of building decision trees. In addition, CART can also construct linear discriminants as new binary attributes. ID2 of 3 (Murphy and Pazzani, 1991) constructs a new binary attribute directly from training data for each decision node when building decision trees, but it uses at least M of N representations. LFC (Ragavan and Rendell, 1993) uses negation and conjunction as constructive operators. It creates one conjunction for each decision ....

MURPHY, P.M. and Pazzani, M.J. (1991): ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, San Mateo, CA: Morgan Kaufmann, 183--187.

....group are algorithms such as Citre 12 Thanks to Thierry Van de Merckt for the suggestion of presenting the problem in this way. 13 [Matheus and Rendell, 1989] Fringe [Pagallo and Haussler, 1989] and AQ17 hci [Wnek and Michalski, 1994] Systems with the data driven strategy, such as ID2 of 3 [Murphy and Pazzani, 1991] and AQ17 dci [Bloedorn and Michalski, 1991] find relevant patterns from input data directly, while systems with the knowledge driven strategy apply domain knowledge to create new attributes. This strategy needs information additional to what can be obtained from the training data. Example ....

....and disjunction. The two research topics mentioned above explore two novel methods of constructing new attributes for decision tree learning, but they also generate binary attributes by using the existing constructive operators conjunction and negation (implicitly) On the other hand, ID2 of 3 [Murphy and Pazzani, 1991] creates M of N attributes. The M of N representation is more representationally powerful than conjunction and disjunction because the latter are two special cases of the former. Nevertheless, M of N representations still have binary values. Only a few systems explore methods of constructing new ....

[Article contains additional citation context not shown here]

P.M. Murphy and M.J. Pazzani, ID2-of-3: constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, San Mateo, CA: Morgan Kaufmann, 183-187. 227

....partitions is necessarily limited, typically to exploring only splits on the value of a single attribute. There has recently been considerable interest in various ways of expanding the set of candidates so as to permit a bounded look ahead and to explore splits on functions of several attributes [9, 11, 22, 30]. If each leaf of a decision tree is labeled with a predicted class (e.g. the largest class in the leaf) then the accuracy of a tree is the fraction of instances for which the class is correctly predicted. To obtain an unbiased estimate of predictive accuracy, different sets of instances are ....

P. M. Murphy and M. J. Pazzani. ID2-of-3: Constructive induction of m-of-n concepts for discriminators in decision trees. In L. A. Bernbaum and G. C. Collins, editors, Machine Learning: Proceedings of the 8th International Workshop (ML91), pages 183--187, San Mateo, CA, 1991. Morgan Kaufmann.

....LFC [Ragavan and Rendell, 1993] and AQ17 hci [Wnek and Michalski, 1994] use conjunction and or disjunction as constructive operators. That is, the constructed attributes are conjunctions or disjunctions of other attributes. A few systems use other constructive operators, for example, M of N [Murphy and Pazzani, 1991; Ting, 1994] mathematical operators such as multiplication and division [Michalski, 1978; Langley, Simon, Bradshaw, and Zytkow, 1987] attribute counting operators [Michalski, 1978; Bloedorn, Michalski, and Wnek , 1993] and X of N [Zheng, 1995a] A comparison of two decision tree learning ....

.... 1978; Bloedorn, Michalski, and Wnek , 1993] and X of N [Zheng, 1995a] A comparison of two decision tree learning systems showed that the one constructing conjunctions as new attributes outperformed that constructing disjunctions for DNF concepts, and vice versa for CNF concepts [Pagallo, 1990] [Murphy and Pazzani, 1991] demonstrated that constructing M of N representations can improve the performance of decision tree learning. Zheng [1995] shows that X of N representations are more representationally powerful than conjunctive, disjunctive, and M of N representations. Nevertheless, all the comparisons involved in ....

P.M. Murphy and M.J. Pazzani, ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, San Mateo, CA: Morgan Kaufmann, 183-187.

....combinations of primitive attributes as new attributes. During building decision trees, it chooses a split for a node in the form (s 1 s 2 : sn ) or (s 1 s 2 : s n ) that maximizes the decrease in impurity of the node, where s i is a split on a primitive attribute. Like CART, ID2 of 3 [8] constructs new attributes from training data directly during building decision trees, but it uses M of N concepts instead of Boolean combinations. 6. Conclusion and Future Work We have described the approach of attribute construction on decision trees and production rules. As a bias for ....

P.M. Murphy and M.J. Pazzani, ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees, Proceedings of the Eighth International Workshop on Machine Learning, (Morgan Kaufmann, 1991), p. 183-187.

.... for decision tree learning is concerned, related work includes the Fringe family of algorithms such as Fringe, Dual Fringe, Symmetric Fringe [2, 5] SymFringe, DCFringe [6] and SFringe [7] the Citre algorithm [1, 14] the CI algorithms [3, 7] the LFC algorithm [15] the ID2 of 3 algorithm [16], the Lmdt algorithm [17] and the XofN algorithm [10] They use different constructive operators and different strategies to create new attributes. As far as systematic search is concerned, the closest related work is Opus [4] Some ideas in CAT about the systematic search with pruning are from ....

P.M. Murphy and M.J. Pazzani, ID2-of-3: constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning (Morgan Kaufmann, San Mateo, CA, 1991) 183-187.

....decision trees. For example, Cart (Breiman et al. 1984) can create one Boolean combination of primitive attributes as a new attribute at each decision node during the process of building decision trees. In addition, Cart can also construct linear discriminants as new binary attributes. ID2 of 3 (Murphy and Pazzani, 1991) constructs a new binary attribute directly from training data for each decision node when building decision trees, but it uses at least M of N representations. LFC (Ragavan and Rendell, 1993) uses negation and conjunction as constructive operators. It creates one conjunction for each decision ....

Murphy, P.M. and Pazzani, M.J. (1991): ID2-of-3: constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, San Mateo, CA: Morgan Kaufmann, 183-187.

....artificial and real world domains support these statements. 1. Introduction A wide variety of methods has been explored to construct new binary attributes by using, as constructive operators, logical AND, NOT, OR [Matheus and Rendell, 1989; Pagallo, 1990; Ragavan and Rendell, 1993] and M of N [Murphy and Pazzani, 1991]. Some researchers construct new continuous values attributes by using attribute counting operators [Michalski, 1978; Bloedorn et al. 1993] or mathematical operators such as multiplication and division [Michalski, 1978; Langley et al. 1987] We proposed a new approach to constructive induction ....

....such as domains needing X of Ns with only one cut point. These results support our statement that nominal X of N is more representationally powerful than continuous valued X of N but nominal X of N suffers the fragmentation problem. 5. Related Work Like XofN, XofN(c) and XofN(cc) ID2 of 3 [Murphy and Pazzani, 1991] and LFC [Ragavan and Rendell, 1993] are also data driven constructive induction algorithms. They also create one new attribute at each decision node when building a tree. ID2 of 3 uses two different operators to construct M of N representations as binary attributes, while LFC generates ....

P.M. Murphy and M.J. Pazzani, ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, 183-187, Morgan Kaufmann, 1991.

....role in constructive induction. However, other factors also affect the performance of a learning system. Most constructive induction systems such as FRINGE [5] and LFC [7] use conjunction or disjunction as constructive operators. A few systems use other constructive operators, for example, M of N [4], attribute counting operators [2, 1] and X of N [8] To explore the effects on constructive induction, more specifically on decision tree learning, of conjunctive, disjunctive, M of N, and X of N representations as new attributes, this paper compares them by using a single constructive ....

P. M. Murphy and M. J. Pazzani. ID2-of-3: constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the Eighth International Workshop on Machine Learning, pages 183--187. San Mateo, CA: Morgan Kaufmann, 1991.

....may be used, but here we talk about only these three most commonly used types of attributes. constructive induction algorithms such as FRINGE [ Pagallo, 1990 ] and CITRE [ Matheus and Rendell, 1989 ] construct new boolean attributes only by using logical operators such as , and . ID2 of 3 [ Murphy and Pazzani, 1991 ] creates, as new attributes, M of N representations stating whether at least M of N conditions are true. M of N representations are also boolean attributes. A few systems such as BACON [ Langley et al. 1987 ] and INDUCE [Michalski, 1978] explore methods to construct new continuous valued ....

....(M of N concepts) However, on domains requiring X of N representations with more than one cut point, the continuous valued X of N has weaker expressive power than the nominal X of N. For details on this issue, please see [ Zheng, 1995 ] 6 Related Work The closest related work is ID2 of 3 [ Murphy and Pazzani, 1991 ] It constructs new binary attributes in the form of M of N representations, while XofN constructs X of N representations. When building a decision tree, both ID2 of 3 and XofN construct one new attribute for each decision node using the local training set. Instead of building trees, MoN [ ....

P.M. Murphy and M.J. Pazzani, ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Proceedings of the Eighth International Workshop on Machine Learning, 183-187, Morgan Kaufmann, 1991.

....best represented using some form of partial matching or evidence summing, such as M of N concepts, which are true if at least M of a set of N specified features are present in an example. There has been some work on the induction of M of N rules demonstrating the advantages of this representation [17, 9]. Other work has focused on revising rules that have real valued weights [19, 6] However, revising theories with simple M of N rules has not previously been addressed. Since M of N rules are more constrained than rules with real valued weights, they provide a stronger bias and are easier to ....

....made seemed reasonable in light of the alterations made in the modified theories. 4 Related Work Several researchers have developed methods for inducing M of N concepts from scratch. CRLS [17] learns M of N rules and out performed standard rule induction in several medical domains. ID 2 of 3 [9] incorporates M of N tests in decision tree learning and outperformed standard decision tree induction in a number of domains. Both projects clearly demonstrate the advantages of M of N rules. Seek2 [5] includes operators for refining M of N rules; however, its revision process is heuristic and ....

P. M. Murphy and M. J. Pazzani. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the Eighth International Workshop on Machine Learning, pages 183--187, Evanston, IL, June 1991.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC