C. E. Brodley. Addressing the selective superiority problem: Automatic algorithm / model class selection. In International Conference on Machine Learning, pages 17--24, 1993.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....flexible analysis strategies and a variety of representations are more proficient at uncovering structure in data. There are several reasons to suspect this is true. First, in having several techniques to choose from, a system can select the one most suited to the analysis problem at hand (e.g. [2]) Second, integrated systems have the advantage of supplementing one kind of result with others, information which often constitutes an explanation of previous findings (e.g. 7] Finally,interesting structure is often discovered in the process of shifting from one representation to another ....

Carla E. Brodley. Addressing the selective superiority problem: Automatic algorithm /model class selection. In Proceedings of the Tenth International Machine Learning Conference, pages 17--24, 1993.

....the utility of the diversity, the coverage and the class specialty properties of a candidate set of classifiers as alternate metrics that capture additional information for the better analysis and understanding of the characteristics of that set of classifiers. 5.2. 1 Diversity Brodley [ C.Brodley, 1993 ] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers while Chan [ Chan, 1996 ] associates it with the entropy in the set of predictions of the base classifiers. When the predictions ....

C.Brodley. 1993. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proc. 10th Intl. Conf. Machine Learning, 17--24. Morgan Kaufmann.

....error as the fraction of instances for which a pair of base classifiers make the same incorrect predictions and Brodley and Lane [5] measured coverage by computing the fraction of instances for which at least one of the base classifiers produces the correct prediction. 3. 1 Diversity Brodley [6] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers while Chan [7] associates it with the entropy in the set of predictions of the base classifiers. When the predictions of the ....

C.Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proc. 10th Intl. Conf. Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....Similar work is reported by Brazdil (1998) who proposes using meta learning as a pre processing step to model selection; experimentation on accuracy performance is then used to select the best algorithm. Meta rules matching domain characteristics with inductive bias have also been crafted manually (Brodley, 1993; Brodley, 1994) In addition, a domain may be represented by properties of the nal hypothesis rather than the data itself. For example, Bensusan et. al (2000) measure properties of a decision tree, e.g. nodes per feature, maximum tree depth, shape, tree imbalance, etc. and convert them into ....

Brodley Carla (1993). Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. Proceedings of the Tenth International Conference on Machine Learning, 17-24, San Mateo, CA, Morgan Kaufman.

....them after their inclusion in the case base. Stored cases that miss classify to a statistically significant degree are removed. Note that these cases could also be useful exceptions to the class definitions. A number of workers have augmented the IBn algorithms (Cameron Jones, 1992; Zhang, 1992; Brodley, 1993). To summarise, Aha s algorithms offer an incremental approach to filtering, and for this reason offer improved efficiency, but suffer from the order of case presentation. Crucial cases could be rejected early on when the class definitions are poorly defined. Wilson and Martinez present three ....

Brodley, C. (1993). Addressing the selective superiority problem: Automatic algorithm/mode class selection. In Proceedings of the Tenth International Machine Learning Conference, pages 17 -- 24, Amherst, MA.

....to Spearman s correlation coefficient that could be tried is Kendall s tau [16] 6 Related Work Meta knowledge as been used before for the purpose of algorithm selection. This knowledge can be either of theoretical or of experimental origin, or a mixture of both. The rules described by Brodley [6] for instance, captured the knowledge of experts concerning the applicability of certain classification algorithms. The meta knowledge of [1] 4] and [9] was of experimental origin and was obtained by meta learning on past performance information of the algorithms. Its objective is to capture ....

C.E. Brodley. Addressing the selective superiority problem: Automatic Algorithm /Model class selection. In P. Utgoff, editor, Proceedings of the Tenth International Conference on Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....to Spearman s correlation coefficient that could be tried is Kendall s tau [16] 6 Related Work Meta knowledge as been used before for the purpose of algorithm selection. This knowledge can be either of theoretical or of experimental origin, or a mixture of both. The rules described by Brodley [6] for instance, captured the knowledge of experts concerning the applicability of certain classification algorithms. The meta knowledge of [1] 4] and [9] was of experimental origin and was obtained by meta learning on past performance information of the algorithms. Its objective is to capture ....

C.E. Brodley. Addressing the selective superiority problem: Automatic Algorithm /Model class selection. In P. Utgoff, editor, Proceedings of the Tenth International Conference on Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....in the problem of algorithm selection based on past performance is growing 6 . Most recent approaches exploited Meta knowledge concerning the performance of algorithms. This knowledge can be either theoretical or of experimental origin, or a mixture of both. The rules described by Brodley [5] captured the knowledge of experts concerning the applicability of certain classification algorithms. Most often, the meta knowledge is of experimental origin [1, 4, 10, 11, 18] In the analysis of the results of project StatLog [12] the objective of the meta knowledge is to capture certain ....

C.E. Brodley. Addressing the selective superiority problem: Automatic Algorithm /Model class selection. In P. Utgoff, editor, Proceedings of the 10th International Conference on Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....to Spearman s correlation coefficient that could be tried is Kendall s tau [16] 6 Related Work Meta knowledge as been used before for the purpose of algorithm selection. This knowledge can be either of theoretical or of experimental origin, or a mixture of both. The rules described by Brodley [6] for instance, captured the knowledge of experts concerning the applicability of certain classification algorithms. The meta knowledge of [1] 4] and [9] was of experimental origin and was obtained by meta learning on past performance information of the algorithms. Its objective is to capture ....

C.E. Brodley. Addressing the selective superiority problem: Automatic Algorithm /Model class selection. In P. Utgoff, editor, Proceedings of the 10th International Conference on Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....difficulties are compensated for by a number of important advances in areas relevant to KDD. Here we list only a few: Multistrategy systems: Several recent comparisons of different learning and discovery algorithms have showed that different methods are superior for different types of problems (Brodley 1993); no single method is best across a range of problems. As a result, there is a movement toward multistrategy learning methods, especially for classification, that apply a number of different methods to the same task and select rules from the best method. Multistrategy systems is an area of active ....

Brodley, C. 1993. Addressing the Selective Superiority Problem: Automatic Algorithm -Model Class Selection. In Proceedings of the Tenth Machine Learning Conference, 17--24. San Mateo, Calif.: Morgan Kaufmann.

.... of these other steps is a matter of art rather than a routine allowing automation [8] Lately some e#orts in the KDD field have been directed towards intelligent support to the data mining process, in particular, assistance in the selection of an analysis method depending on data characteristics [2,4]. A particular case of KDD is knowledge extraction from spatially referenced data, i.e. data referring to geographic objects or locations or parts of a territory division. In analysis of such data it is very important to account for the spatial component (relative positions, adjacency, distances, ....

Brodley, C.: Addressing the Selective Superiority Problem: Automatic Algorithm / Model Class Selection. In: Machine Learning: Proceedings of the 10th International Conference, University of Massachusetts, Amherst, June 27-29, 1993. San Mateo, Calif.: Morgan Kaufmann (1993) 17-24

....expectations of the modern cognitivists. Concurrently to these cognitive motivations, many works in the field of inductive learning have highlighted the relation between biases implemented in the learning algorithms and their efficiency to infer correct hypothesis [Utgoff, 1986; Schaffer, 1993, Brodley, 1993]. For example, it has been shown that the orthogonality of class boundaries is often a too strong constraint that might deeply affect the accuracy and the simplicity of the decision surfaces drawn in the description space. Techniques like oblique decision trees [Murthy and al. 1994] or perceptron ....

.... It is the deep knowledge level which results from an inductive learning process whose biases focus on accuracy of the concept representation. These biases may include any kind of Background Knowledge that helps the system to choose a specific knowledge model regarding the domain (as done by [Brodley, 1993]) or that encodes a priori domain theory that generates constraints on the possible concept representations. The resulting representation, called the Internal Concept Representation (ICR) is further used by a classifying function which aims to recognize instances from non instances of the ....

Brodley Carla E. (1993) Addressing the Selective Superiority Problem: Automatic Algorithm/ Model Class Selection, Proceedings of the Tenth International Conference on Machine Learning ML'93, Morgan Kaufmann, pp. 17-24.

....measure coverage by computing the fraction of instances for which at least one of the classifiers produces the correct prediction. 1 TP stands for True Positive, i.e. percentage of actual fraud that is caught, FP stands for False Positive, i.e. percentage of false alarms 2. 1 Diversity Brodley [5] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers, while Chan [6] associates it with the entropy in the predictions of the base classifiers. When the predictions of the classifiers ....

C.Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. 10th Intl. Conf. Mach. Learning, 1993.

....in Table 3. The highest accuracy achieved for each dataset is shown in bold type. The average over all datasets is shown near the bottom of the table. As can be seen from the results in the table, no algorithm had the highest accuracy on all of the datasets, due to the selective superiority (Brodley 1993) of each algorithm, i.e. the degree to which each bias (Mitchell 1980) is appropriately matched for each dataset (Dietterich 1989; Wolpert 1993; Schaffer 1994; Wilson Martinez 1997c) However, IDIBL had the highest accuracy of any of the algorithms for more of the datasets than any of the other ....

Brodley, Carla E. 1993. Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. Proceedings of the Tenth International Machine Learning Conference, Amherst, MA, pp. 17-24.

....into different categories or classes. Classification has been the subject of much study in the past, and has proved a useful tool in real world applications. Nevertheless, the reasons explaining why an algorithm is more successful than others in giving good approximations to C remain elusive (Brodley, 1993). To elucidate the problem above, experiments employing a broad spectrum of learning algorithms have been sought to find the class of domains for which each algorithm works best (Michie, 1994; Weiss Kulikowski, 1990) The idea is to trace a link between a learning algorithm and the domains on ....

Brodley, C. E. (1993). Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proceedings of the Tenth International Conference on Machine Learning, pp.

....several other meta learning strategies. 1.1 Introduction Determining the conditions for which a given learning algorithm is appropriate is an open problem in machine learning. Methods for selecting a learning algorithm for a given domain (e.g. Aha92, Breiman84] or for a portion of the domain ([Brodley93, Brodley94]) have met with limited success. This paper proposes a new approach that dynamically selects a learning algorithm for each example by locating it in the example space and then choosing the best learner(s) for prediction in that part of the example space. The regions of the example space are ....

....is often difficult to obtain. DS DW emphasizes the selection of a learning algorithm at the example level rather than at the domain level since it avoids the need for characterization rules by instead considering each learning algorithm s past performance on similar examples from that domain. [Brodley93] proposes a knowledge based approach to building hybrid decision structures by using heuristics to select the best algorithm at a given stage of learning. These heuristics are created from practical knowledge about how to detect when a generalization is a good fit or when to switch to a different ....

Brodley, C. E. (1993). Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. Proceedings of the Tenth International Machine Learning Conference (pp. 17-24). San Mateo, CA: Morgan Kaufmann.

....error the fraction of instances for which a pair of base classifiers make the same incorrect predictions and Brodley and Lane [4] measured coverage by computing the fraction of instances for which at least one of the base classifiers produces the correct prediction. 2. 1 Diversity Brodley [5] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers while Chan [6] associates it with the entropy in the predictions of the base classifiers. When the predictions of the classifiers ....

C.Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proc. 10th Intl. Conf. Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....classifier can be used, and we are currently developing recognition graphs that use artificial neural networks and multivariate decision trees instead. One could also build a hybrid system that selected the best classification induction algorithm for each level of abstraction, a la Brodley [7]. It is important, however, that the classifier have a very low false negative rate. At all but the highest level of abstraction, if a classifier verifies a false instance the mistake will most likely be corrected when the instance is transformed to the next level of representation. If a ....

C.E. Brodley. "Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection," Proc. of Tenth International Machine Learning Conference, June 27-29, Amherst, MA., pp. 17-24.

....error as the fraction of instances for which a pair of base classifiers make the same incorrect predictions and Brodley and Lane [5] measured coverage by computing the fraction of instances for which at least one of the base classifiers produces the correct prediction. 3. 1 Diversity Brodley [6] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers while Chan [7] associates it with the entropy in the set of predictions of the base classifiers. When the predictions of the ....

C.Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proc. 10th Intl. Conf. Machine Learning, pages 17--24. Morgan Kaufmann, 1993.

....several other meta learning strategies. 27.1 Introduction Determining the conditions for which a given learning algorithm is appropriate is an open problem in machine learning. Methods for selecting a learning algorithm for a given domain (e.g. Aha92, Breiman84] or for a portion of the domain ([Brodley93, Brodley94]) have met with limited success. This paper proposes a new approach that dynamically selects a learning algorithm for each example by locating it in the example space and then choosing the best learner(s) for prediction in that part of the example space. The regions of the example space are ....

....is often difficult to obtain. DS DW emphasizes the selection of a learning algorithm at the example level rather than at the domain level since it avoids the need for characterization rules by instead considering each learning algorithm s past performance on similar examples from that domain. [Brodley93] proposes a knowledge based approach to building hybrid decision structures by using heuristics to select the best algorithm at a given stage of learning. These heuristics are created from practical knowledge about how to detect when a generalization is a good fit or when to switch to a different ....

Brodley, C. E. (1993). Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. Proceedings of the Tenth International Machine Learning Conference (pp. 17-24). San Mateo, CA: Morgan Kaufmann.

....measure coverage by computing the fraction of instances for which at least one of the classifiers produces the correct prediction. 1 TP stands for True Positive, i.e. percentage of actual fraud that is caught, FP stands for False Positive, i.e. percentage of false alarms 2. 1 Diversity Brodley [5] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers, while Chan [6] associates it with the entropy in the predictions of the base classifiers. When the predictions of the classifiers ....

C.Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. 10th Intl. Conf. Mach. Learning, 1993.

....in many practical domains is still far from 100 , it is unclear how much, if any, improvement is still possible with current methods. Empirical studies have also shown repeatedly that each approach works best in some, but not all, domains; this has been termed the selective superiority problem [ Brodley, 1993 ] Ideally, we would like to have an algorithm that in each domain of interest performs as well as the best of the algorithms above, or better. While it is now clearly understood that induction is a zero sum game , and thus this goal is unachievable for the set of all mathematically possible ....

....never occur in practice. One way to attempt this is by combining two or more of the basic approaches into an algorithm that will behave as the most appropriate of them in each situation. This line of research may be termed empirical multi strategy learning [ Michalski and Tecuci, 1994 ] MCS [ Brodley, 1993 ] KBNGE [ Wettschereck, 1994 ] and ITRULE [ Smyth et al. 1990 ] are examples of systems of this type. Two induction paradigms with largely complementary strengths and weaknesses are rule induction and instance based learning (IBL) Rule induction systems often succeed in identifying small sets ....

[Article contains additional citation context not shown here]

C. E. Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proc. 10th Machine Learning Conf., pages 17--24, 1993.

.... 1991; Chan and Stolfo 1995) Heath, Kasif, and Salzberg 1993) and counting operations (Murphy and Pazzani 1991; Zheng 1995) ffl Use of error correcting codes when there are more than two classes (Dietterich and Bakiri 1995) ffl Decision trees that incorporate classifiers of other kinds (Brodley 1993; Ting 1994) ffl Automatic methods for setting learning system parameters (Kohavi and John 1995) On typical datasets, all have been shown to lead to more accurate classifiers at the cost of additional computation that ranges from modest to substantial. There has recently been renewed interest ....

Brodley, C. E. 1993. Addressing the selective superiority problem: automatic algorithm/model class selection.

....back propagation (Rumelhart et al., 1986) and genetic algorithms (Booker et al., 1989) Empirical comparison of these different approaches in a variety of application domains has shown that each performs best in some, but not all, domains. This has been termed the selective superiority problem (Brodley, 1993), and presents a dilemma to the knowledge engineer approaching a new task: which induction paradigm should be used One solution is to try each one in turn, and use cross validation to choose the one that appears to perform best. This is a long and tedious process, especially considering the large ....

....it can be improved, and if so how, is an open and important one. Unfortunately, the success of this approach has so far been moderate. The resulting algorithms are prone to be cumbersome, and often achieve accuracies that lie between those of their parents, instead of matching the highest (e.g. Brodley, 1993). Here a theoretical question arises. It is well known that no induction algorithm can be the best in all possible domains; each algorithm contains an explicit or implicit bias (Mitchell, 1980) that leads it to prefer certain generalizations over others, and it will be successful only insofar as ....

[Article contains additional citation context not shown here]

Brodley, C. E. (1993). Addressing the selective superiority problem: Automatic algorithm/model class selection. Proceedings of the Tenth International Conference on Machine Learning (pp. 17--24). San Mateo, CA: Morgan Kaufmann.

....well when irrelevant attributes are present in a dataset. IB5 [Aha, 1992] extends IB4 to handle the addition of new attributes to the problem after training has already begun. These extensions of IB3 address issues that are beyond the scope of this paper, and are thus only mentioned here. MCS. Brodley [1993] introduced a Model Class Selection (MCS) system that uses an instance based learning algorithm (which claims to be based loosely on IB3 ) as part of a larger hybrid learning algorithm. Her algorithm for reducing the size of the training set is to keep track of how many times each instance was ....

Brodley, Carla E. (1993). Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. Proceedings of the Tenth International Machine Learning Conference, Amherst, MA, pp. 17-24.

....in machine learning, theory revision (Flann Dietterich 1989) Mooney 1993) Ourston 1991) Baffes Mooney 1993) Richards Mooney 1995) Towell, Shavlik, Noordewier 1990) R. S. Michalski 1993) Cohen 1992) Bergadano Giordana 1988) and bias selection (Merz 1995) Ho, Hull, Srihari 1994) (Brodley 1993) (Schaffer 1993) are viewed from a single perspective. Theory revision systems make use of two sources of knowledge: an existing imperfect model of a domain and a set of available data. Bias selection systems, on the other hand, make use of available data for a domain and several empirical ....

Brodley, C. E. 1993. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proceedings of the Tenth International Conference on Machine Learning, 17--24.

....on which algorithm is most appropriate. For example, it cannot currently select between competing algorithms that both perform generalization. Meta AQUA does not reason at the micro level, as do systems that address the selective superiority problem 6 in inductive learning (see, for instance, Brodley, 1993; Provost Buchanan, 1992; Schaffer, 1993) although the scope of learning problems solved by Meta AQUA is greater than these other systems. Another limitation of the Meta AQUA implementation is that learning self evaluation (step 3 of Figure 1) does not exist. Thus, Meta AQUA cannot ....

....inductive algorithms are better at classifying specific classes or particular distributions of data than others. Each algorithm is good at some but not all learning tasks. The selective superiority problem is to choose the most appropriate inductive algorithm, given a particular set of data (Brodley, 1993). 9 ....

Brodley, C. E. (1993). Addressing the selective superiority problem: Automatic algorithm / model class selection. Machine Learning: Proc. of the Tenth International Conference (pp. 17-24). San Mateo, CA: Morgan Kaufmann.

....error as the fraction of instances for which a pair of classifiers make the same incorrect predictions and Brodley and Lane (Brodley Lane 1996) measure coverage by computing the fraction of instances for which at least one of the classifiers produces the correct prediction. Diversity Brodley (C.Brodley 1993) defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers, while Chan (Chan 1996) associates it with the entropy in the predictions of the base classifiers. When the predictions of the ....

C.Brodley. 1993. Addressing the selective superiority problem: Automatic algorithm/model class selection.

....2. Related Work The goal of constructive induction is to find simple, accurate descriptions of the concepts to be learned. One approach to finding such descriptions is to search for a induction method that was well suited to the distribution of examples. This approach is taken by Brodley (Brodley, 1993) in which a number of different methods, each with different type of constructs for building descriptions, are combined. The selection of an appropriate induction method is based on a number of heuristic rules. Work on the automated selection an appropriate bias is also closely related to this ....

Brodley, C., E., "Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection," Proceedings of the Tenth International Conference on Machine Learning, pp. 1724, 1993.

....cmerz ics.uci.edu (714)725 3491 1 Introduction Determining the conditions for which a given learning algorithm is appropriate is an open problem in machine learning. Methods for selecting a learning algorithm for a given domain (e.g. Aha, 1992; Breiman, et al., 1984) or for a portion of the domain (Brodley, 1993) have met with limited success. This paper proposes a new approach which dynamically selects a learning algorithm for each test example by observing the prediction patterns of a suite of algorithms given in a crossvalidation history. This dynamic selection of a learning algorithm, DS, frequently ....

....to obtain. Once again, DS emphasizes the selection of a learning algorithm at the example level rather than at 2 the domain level, thus avoiding the derivation of characterization rules by instead considering each learning algorithm s past performance on similar examples from that domain. Brodley (1993) proposes a knowledge based approach to building hybrid decision structures by using heuristics to select the best algorithm at a given stage of learning. These heuristics are created from practical knowledge about how to detect when a generalization is a good fit or when to switch to a different ....

Brodley, C. E. (1993). Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. Machine Learning: Proceedings of the Tenth International Conference (pp. 17-24). San Mateo, CA: Morgan Kaufmann.

....in the first matrix to identify the rows closest to the row vector of the new instance. The accuracies for each induced theory are computed from the corresponding rows (by averaging) in the second matrix. The induced theory having the highest accuracy is selected for final prediction. MCS (Brodley, 1993) uses a hand crafted rule to guide the construction of three different models at the nodes of a decision tree, and each model is trained on part of the training set. In contrast, the two algorithms in the composite learner are trained independently on the total training data and work cooperatively ....

Brodley, C.E. (1993), Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection, in Proceedings of the Tenth International Conference on Machine Learning, pp. 17-24.

....predictions. We call the former data combination and the latter model combination . Figure 1 shows these two types of combination at the data and model levels. While there has been a considerable amount of research on methods to combine multiple models reported in the literature (e.g. Brodley, 1993; Breiman, Data Data 1 Data 2 Learner Learner Learner Source Data 1 Data 2 Data Combination Model Combination Model 1 Model 2 Model 1 2 Fig. 1. Combination at different levels data or model. 1996a,1996b; Freund Schapire, 1996; Perrone Cooper, 1993; Krogh Vedelsby, 1995) investigation ....

....partitions of data) is better than a single model learned from the entire dataset. Some methods provide guidance as to how to partition the description space. Some use the information gain criterion (Utgoff, 1989) user provided information (Tcheng, Lambert Rendell, 1989) or hand crafted rules (Brodley, 1993) to guide the recursive partitioning process in a tree structure; and others (Ting, 1994; Wettschereck, 1994) employ a confidence measure provided from one particular learned model, during classification, to decide which one of the two different models shall be used for final prediction. The ....

Brodley, C.E. (1993), Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection, in Proceedings of the Tenth International Conference on Machine Learning, pp. 17-24.

....Shaffer [10] described a method which involves testing competing models using cross validation. It was demonstrated that the truly best model is selected with high probability. The disadvantage of this approach is that we have to do quite a lot of testing before a decision can be made. As Brodley [4] has shown the decision as to which is the best model can be guided by rules and can thus be potentially quicker. The rules used in [4] incorporate the knowledge of domain experts and hence are not easy to update when new algorithms, or test results, become available. Various attempts have been ....

....model is selected with high probability. The disadvantage of this approach is that we have to do quite a lot of testing before a decision can be made. As Brodley [4] has shown the decision as to which is the best model can be guided by rules and can thus be potentially quicker. The rules used in [4] incorporate the knowledge of domain experts and hence are not easy to update when new algorithms, or test results, become available. Various attempts have been made to automate the generation of such rules ( 2] 3] The method in [3] incorporated the process of learning, using decision ....

Brodley C. (1993): "Addressing the Selective Superiority Problem: Automatic Algorithm / Model Class Selection Problem", in ML93, Machine Learning, Proceedings of the 10th International Conference, P. Utgoff (ed.), Morgan Kaufmann.

.... The research described in this paper falls in the general area of empirical multi strategy learning, which attempts to produce more accurate learners by combining two or more individual algorithms [9] The MCS system combines decision trees with the nearestneighbor algorithm and linear machines [2]. FCLS combines rules with specific examples in a bestmatch framework [20] Quinlan has combined decision trees and other methods with nearest neighbor in regression tasks [15] The post pruning step used in many rule and decision tree learners can be seen as a form of specific to general ....

C. E. Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proceedings of the Tenth International Conference on Machine Learning, pages 17--24, Amherst, MA, 1993. Morgan Kaufmann.

....error the fraction of instances for which a pair of base classifiers make the same incorrect predictions and Brodley and Lane [3] measured coverage by computing the fraction of instances for which at least one of the base classifiers produces the correct prediction. 2. 1 Diversity Brodley [4] defines diversity by measuring the classification overlap of a pair of classifiers, i.e. the percentage of the instances classified the same way by two classifiers while Chan [5] associates it with the entropy in the predictions of the base classifiers. When the predictions of the classifiers ....

C.Brodley. Addressing the selective superiority problem: Automatic algorithm /model class selection. In Proc. 10th Intl. Conf. Machine Learning, pages 17-- 24. Morgan Kaufmann, 1993.

....predictions. We call the former data combination and the latter theory combination . Figure 1 shows these two types of combination at the data and theory levels. While there has been a considerable amount of research on methods to combine multiple models reported in the literature (e.g. Brodley, 1993; Breiman, 1996a,1996b; Freund Schapire, 1996; Perrone Cooper, 1993; Krogh Vedelsby, 1995) investigation into combining theories from a single learning algorithm induced using completely disjoint sets of data has been limited. Most work shows that combining multiple models induced from ....

....where p j is the probability that the switch will select the output from expert j. Some methods provide guidance as to how to partition the description space. Some use information gain criterion (Utgoff, 1989) user provided information (Tcheng, Lambert Rendell, 1989) or hand crafted rules (Brodley, 1993) to guide the recursive partitioning process in a tree structure; and others (Ting, 1994; Wettschereck, 1994) employ a confidence measure provided from one particular learned theory, during classification, to decide which one of the two different theories shall be used for final prediction. The ....

Brodley, C.E. (1993), Addressing the Selective Superiority Problem: Automatic Algorithm /Model Class Selection, in Proceedings of the Tenth International Conference on Machine Learning, pp. 17-24.

No context found.

C. Brodley. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proceedings of the Tenth International Conference on Machine Learning, pages 17--24, 1993.

.... decision tree (Quinlan 1986) instance based (Duda Hart 1973) and neural net al..gorithms (Rumelhart McClelland 1986) The results of empirical comparisons of existing algorithms illustrate that each algorithm has a selective superiority: it is best for some but not all classification tasks (Brodley 1993). Selective superiority arises because each learning algorithm searches within a restricted hypothesis space defined by its class of models. For example, the class of first order linear regression models is not appropriate when the data is best fit by a second order linear regression model. In ....

Brodley, C. E. 1993. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Machine Learning: Proceedings of the Tenth International Conference, 17--24, Morgan Kaufmann.

....We illustrate empirically that straightforward integration methods fail to utilize the coverage of the base level classifiers. Measuring Coverage and Diversity One metric for for determining the similarity of the classification decisions of a set of learning algorithms is classification overlap (Brodley, 1993). To compute the overlap among a set of classifiers requires counting the number of instances that were classified the same way by each of the classifiers, including instances that were classified incorrectly by all algorithms. To measure the diversity of a set of classifiers one can look at their ....

Brodley, C. E. (1993). Addressing the selective superiority problem: Automatic algorithm/model class selection. Machine Learning: Proceedings of the Tenth International Conference (pp. 17-24).

No context found.

C. E. Brodley. Addressing the selective superiority problem: Automatic algorithm / model class selection. In International Conference on Machine Learning, pages 17--24, 1993.

No context found.

C.E. Brodley. Addressing the selective superiority problem: Automatic algorithm for model class selection. In Proceedings 10th Machine Learning Conference, pages 17--24, 1993.

No context found.

C. E. Brodley. Addressing the selective superiority problem: Automatic algorithm /model class selection. In Proceedings of the Tenth International Conference on Machine Learning, pages 17--24, Amherst, MA, 1993.

No context found.

C. E. Brodley. Addressing the Selective Superiority Problem: Automatic Algorithm/Model Class Selection. In Proceedings of the Tenth International Machine Learning Conference, pages 17--24. Amherst, MA, 1993.

No context found.

Brodley C. (1993): Addressing the Selective Superiority Problem: Automatic Algorithm / Model Class Selection Problem, in Machine Learning, Proceedings of 10th Machine Learning Conference, Morgan Kaufmann.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC