Markovitch, S. and Scott, P. D. 1993. Information filtering: selection mechanisms in learning systems. Machine Learning 10:113--151.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....with good success [Quinlan, 1993, Rosenblatt, 1962, Aha et al., 1991, Michalski, 1983] Many of the existing learning approaches consider the learning algorithm as a passive entity that makes use of the information presented to them. Such schemes are called Passive Learners by Cohn et al. (1995) Markovitch (1989) identifies irrelevant, noisy and redundant information as the harmful elements of knowledge. The passive learning schemes will degrade performance on domains with these harmful elements. In this paper, we focus on the issue of selecting relevant information. This involves solving two problems ....

....passive learning schemes will degrade performance on domains with these harmful elements. In this paper, we focus on the issue of selecting relevant information. This involves solving two problems namely, the problem of selecting relevant features and the problem of selecting relevant examples. Markovitch (1989) defines these components of selective learning as Selective Attention and Selective Utilization respectively. We present the benefits of selective attention and selective utilization independently and as a combined strategy. c #2003 Baranidharan Raman and Thomas.R.Ioerger. Figure 1: ....

[Article contains additional citation context not shown here]

Shaul Markovitch. Information Filtering: Selection Mechanisms in Learning Systems. PhD thesis, EECS Department, University of Michigan, 1989

....new case if only a small part of its solution is novel. If the entire solution is learned then significant redundancy can be introduced into the case base. This in turn can have an adverse affect on problem solving performance and cause the early onset of the utility problem (see, e.g. 9] 10] [20], 23] 31] 32] 38] 39] 42] HCBR avoids this type of problem because learning operates at a finer level of granularity. Complex problems and their solutions are treated as composite entities and, so, parts of these problems can be learned as separate cases at various levels of ....

....in terms of competence and efficiency. However, the so called utility problem tells us that since case base size increases lead to retrieval time increases, problem solving performance will eventually degrade when rising retrieval costs outweigh adaptation savings (see, for example, 9] 10] [20], 31] 32] In practice then, this places a performance related limit on the size of a case base that a given system can support. The limit depends on the characteristics of the retrieval and adaptation processes used by the system, and will vary from one system to the next. It means that it ....

S. Markovitch and P.D. Scott, "Information Filtering. Selection Mechanisms in Learning Systems," Machine Learning, vol. 10, pp. 113-151, 1993.

....we must be clear about the degree to which we are willing to let the original classification accuracy depreciate. For example, if we have a fixed storage limit then the number of cases we are forced to remove might be too large, and unavoidably result in a degradation of classification accuracy (Markovitch and Scott, 1993; Smyth and Keane, 1995) Usually, the principle objective of an instance selection scheme is unintrusive storage reduction. Here, classification accuracy is primary: we desire the same (or higher) classification accuracy but we require it faster and taking up less space. Ideally, accuracy should ....

....of class definitions does not lead to a drop in classification accuracy as discrimination is still possible. a) The two spirals dataset, an example of a problem space not defined by homogeneous collections of cases. b) Chang s prototype creation algorithm retains the class structure well 1988; Markovitch and Scott, 1993; Smyth and Keane, 1995; Aha et al. 1991) We argue, as others have (Swonger, 1972; Wilson and Martinez, 1997) that instances which lie on borders between classes are almost always critical to the classification process. The instances located at the interior of class regions are superfluous as ....

Markovitch, S. and Scott, P. D. (1993). Information filtering: Selection mechanisms in learning systems. Machine Learning, 10(2):113--151.

....base we must be clear about the degree to which we are willing to let the original classification competence depreciate. For example, if we have a fixed storage limit then the number of cases we need to remove might be too large, and unavoidably result in a degradation of classification competence [6, 9]. Typically, the principle objective of a filtering algorithm is unintrusive storage reduction. Here, classification competence is primary: we desire the same (or higher) learning competence but we require it faster and taking up less space. Ideally, competence should not (a) b) Fig. 1. a) The ....

S. Markovitch and P. D. Scott. Information filtering: Selection mechanisms in learning systems. Machine Learning, 10(2):113--151, 1993.

.... a constant size of the case base, then the calculation of the Phi( B [fc 0 g) Gamma fc i g) for c i 2 B[fc 0 g enables us to make the best choice (i.e. the argument of Phi which gives a minimum value) This forgetting criterion belongs to the general framework of information ltering as dened in (Markovitch Scott, 1993); more precisely it is an approach of selective retention. Among these two problems, we have more studied the rst one. Before dening this criterion, we give a model of the case based reasoning process oriented to our purpose. An introducing example close to problems we want to solve follows. In ....

Markovitch, S., & Scott, P. D. (1993). Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, 10, 113151.

....that a case base evolves in a predictable and characteristic way with respect to competence. In fact, we show that there is a characteristic competence curve for CBR systems, just as many machine learning systems have a characteristic efficiency curve associated with the utility problem [2,4,5]. We demonstrate how a case base moves predictably through four distinct developmental phases as it evolves and we argue that this is an important step forward when it comes to understanding the true nature of competence in CBR systems. 2 AModel of Case Competence Competence is all about the ....

Markovitch, S. and Scott, P.D. (1993) Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, 10,: 113-151.

....a number of competence categories to permit a coarse grained competence assessment. Alternatively Smyth McKenna [17] focus on the competence of groups of cases. We are interested in developing a more fine grained measure that is similar in spirit to efficiency models such as the utility metric [11, 12]. To measure the competence of an individual case one must take into account the local coverage of the case as well as the degree to which this coverage is duplicated by nearby cases. To do this we define a measure called relative coverage (RC) which estimates the unique competence contribution ....

Marckovitch, S. & Scott, P.D.: Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, 10 (1993) 113-151

....these methods. Unfortunately, learned knowledge can hurt performance [22] this is known as the utility problem. Some reports showed that in some systems, learning degrades problem solving performance [10,35] One approach to this problem is to use some form of selective learning or forgetting. [20] provides a general framework for analyzing this approach. Examples include discarding learned rules if they turn out to cause overall system slowdown [22] disabling the learning component after some desired or peak performance level has been reached [16] learning only certain types of rules ....

- S. Markovitch, P. D. Scott, Information Filtering: Selection Mechanisms in Learning Systems, Machine Learning 10, pp. 113-151, 1993.

....about knowledge and process. For example, Meta XPs can represent the difference between remembering and forgetting [Cox and Ram, 1992a] Since forgetting is the absence of a successful outcome, this is difficult to capture in most frameworks. Forgetting is an important issue in machine learning [Markovitch and Scott, 1993] and in metamemory [Spear, 1978; Wellman and Johnson, 1979] Meta TS implements a simple form of forgetting in which obsolete knowledge is deleted once it is identified. Third, because the approach taken by the introspective learning paradigm clearly addresses the issue of memory organization ....

S. Markovitch and P. D. Scott. Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, 10:113--151, 1993.

.... by being selective about when to learn or which rules or types of rules to learn, or by forgetting previously learned rules if they slow down the matcher enough to cause an overall system slowdown (Minton, 1988; Etzioni, 1990b; Holder, 1992; Gratch and DeJong, 1992; Greiner and Jurisica, 1992; Markovitch and Scott, 1993). Unfortunately, this approach is inadequate for the long term goals of AI because, given the current state of match technology, it precludes learning a vast amount of knowledge. Moreover, it is intuitively desirable to have AI systems that take advantage of every opportunity for learning, rather ....

Markovitch, S. and Scott, P. D. (1993). Information filtering: Selection mechanisms in learning systems. Machine Learning, 10(2):113--151.

....suffers from the utility problem [19] since CABINS requires more time for case matching and retrieval with increase in case base size. Although we can define the optimal case base size by monitoring the performance of CABINS for the problems in the domain [22] some knowledge filtering techniques [17] might be useful for improving efficiency of CABINS by dynamically eliminating redundant or incorrect cases in the case base. Acknowledgment The authors are indebted to Dr. Johan Vanwelkenhuysen at Osaka University (currently at INRIA, France) for insightful discussions on knowledge modeling. ....

S. Markovitch and P.D. Scott, Information Filtering: Selection Mechanisms in Learning Systems, Machine Learning 10 (1993) pp.113-151

....of their application. An uncontrolled use of all generated lemmas (although useful lemmas are generated which can lead to proof length reductions) normally increases the branching rate of the search tree in such a way that the advantages like proof length reduction are outweighed (see e.g. Min90, MS93, Fuc98c, Fuc98b] Criteria are needed in order to select some relevant lemmas. Thus, in this report we will develop methods for selecting lemmas which appear to be relevant for a given proof task. In the past criteria based on syntactic properties or the derivation tree of a possible lemma have ....

....properties or the derivation tree of a possible lemma have been used for this purpose. e.g. in [Sch94, AS92] short clauses are favored in order to increase the probability that the lemmas can be applied during the proof process. Additionally, lemmas with large derivations are favored (e.g. in [MS93, FW98] because they can possibly provide large search reductions. These criteria, however, work rather uninformedly since a lemma is judged without consideration of the concrete proof task. It may be better to try to estimate the relevancy of a lemma in order to contribute to a refutation of the ....

[Article contains additional citation context not shown here]

S. Markovitch and P.D. Scott. Information filtering: Selection mechanisms in learning systems. Machine Learning, 10(2):113--151, 1993.

....chunk problem. Previous work on the expensive chunk problem has investigated how to produce cheaper rules (Prieditis Mostow 1987; Minton 1988; Shell Carbonell 1991; Shavlik 1990; Etzioni 1990) and how to filter out expensive rules (Minton 1988; Greiner Jurisica 1992; Gratch Dejong 1992; Markovitch Scott 1993). However, none of these approaches can generally guarantee that the cost of using the learned rules will always be bounded by the cost of the problem solving episode from which they are learned. That is, the cost of a learned rule can be greater than the cost of solving the problem with the ....

Markovitch, S., and Scott, P. D. 1993. Information filtering : Selection mechanism in learning systems. Machine Learning 10(2):113--151.

....shown that it is possible to learn over one million rules while still allowing their efficient use [14, 15] This research focuses on the expensive chunk problem in EBL. There have been approaches which are useful for producing cheaper rules [16, 3, 12, 17, 11, 10] or filtering out expensive rules [3, 18, 19, 20]. However, these approaches cannot generally guarantee that the cost of using the learned rules will always be bounded by the cost of the planning episode from which they are learned. That is, the cost of a learned rule can be greater than the cost of planning with the original set of rules. One ....

S. Markovitch and P. D. Scott. Information filtering : Selection mechanism in learning systems. Machine Learning, 10(2):113--151, 1993.

....target problems. Unfortunately, one of the side effects of this simplistic approach to learning is that it can cause system performance to degrade rather than improve. This effect is known as the utility problem and has been the subject of considerable research by the machine learning community ([18], 21] 35] 36] 39] and more recently, by case based reasoning researchers ( 9] 10] 30] In general, the utility problem describes how the performance of knowledge based systems can degrade if the knowledge base becomes populated with harmful or redundant knowledge items. In case based ....

S. Markovitch, and P. D. Scott, "Information Filtering. Selection mechanisms in Learning Systems," Machine Learning, Vol. 10, pp 113-151, 1993.

.... Leake, 1992; Leake, this volume; Ram Hunter, 1992) An analysis of the utility of learning can help in determining the target of learning (desJardins, 1992) in guiding learning processes (Gratch, DeJong, Chien, this volume; Provost, this volume) and also in deciding whether to learn at all (Markovitch Scott, 1993; Minton, 1990) ffl Facilitating the use of opportunities to learn: If a reasoner does not have sufficient resources at the time it realizes it has a need to learn, or if the requisite knowledge is not available at that time, the reasoner can suspend its learning goals in memory so that they ....

.... include goal concepts, target concepts, purposes, operationality criteria, bias, policies, quality metrics, and utility metrics (desJardins, 1992; Gordon Perlis, 1989; Gratch, DeJong, Chien, this volume; Kedar Cabelli, 1987; Keller, 1988; Laird, Rosenbloom, Newell, 1986; Leake, 1991, 1992; Markovitch Scott, 1993; Martin, this volume; Michalski, 1983; Minton, 1990; Mitchell, 1982; Mitchell, Keller, KedarCabelli, 1986; Perez, this volume; Provost, this volume; Utgoff, 1986) Policies and constraints are not learning goals in the sense that the learner does not actively seek to satisfy them; instead, they ....

Markovitch, S. & Scott, P.D. (1993). Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, 10:113--151.

....is possible to learn over one million rules while still allowing their efficient use [10, 11] In this article we focus on the expensive chunk problem. Previous work on the expensive chunk problem has investigated how to produce cheaper rules [12, 5, 8, 13, 7] and how to filter out expensive rules [5, 14, 15, 16]. However, none of these approaches can generally guarantee that the cost of using the learned rules will always be bounded by the cost of the problem solving episode from which they are learned. That is, the cost of a learned rule can be greater than the cost of solving the problem with the ....

....involves placing tiles 1 through 9 in empty squares one at a time. If the sums of horizontal, vertical, and diagonal lines are different with the current tile placement, 4] goal g2 operator x4 ) 4] g2 state s26 ) 4] s26 length 4) 4] x4 at l13 ) 4] s26 at l13 ) [ 16] ( g2 operator x1 ) 64] g2 operator o7 ) 256] g2 operator o8 ) 256] g2 problem space p15 ) 256] p15 name path) 256] x4 to l1 ) 256] x4 at l2 ) 64] l2 right l1 ) 64] x1 to l3 ) 64] x1 at l4 ) 48] l4 conn right a2 ....

S. Markovitch and P. D. Scott. Information filtering : Selection mechanism in learning systems. Machine Learning, 10(2):113--151, 1993.

....be incremental would be difficult due to its semantic disjointness requirement on discovered concepts. As pointed out by Bisson (1992b) KBG would be fairly easy to coax into an incremental mode. Finally, the Sa column of Table 3 refers to the manner in which concepts are selectively acquired (Markovitch and Scott, 1993) or discovered. LABYRINTH is derived from COBWEB and uses a predictiveness measure to judge the expected worth of a potential concept. Concepts formed by the CLUSTER S system wholly depend on the initial seed objects selected from the pool of examples. Levinson (1985) employs a sophisticated ....

Markovitch, S. and Scott, P. D. 1993. Information filtering: selection mechanisms in learning systems. Machine Learning 10:113--151.

....of success is low. The filtering strategies were experimentally evaluated in the context of job shop scheduling, a well known ill structured problem. 1 Introduction Recently, there has been increased interest in the issue of the utility of knowledge in knowledge based systems. Several studies [ Markovitch and Scott, 1993; Minton, 1988 ] have defied the traditional belief that increasing a problem solver s knowledge is a monotonically beneficial process. Utility of knowledge depends on the difference between its cost (e.g. cost of knowledge acquisition, storage, matching) and its benefits (e.g. increased problem ....

....irrelevant or redundant) Hence it is very important for machine learning systems to consider the quality of the knowledge they employ and develop heuristic strategies to eliminate harmful knowledge, i.e. knowledge whose elimination from consideration would improve problem solving performance. Markovitch and Scott, 1993 ] has termed such strategies information filters because they filter out harmful knowledge from being used in problem solving. Examples of information filters include discarding of least useful board positions in the Checkers Player [ Samuel, 1959 ] selecting only misclassified instances in ID3 ....

Shaul Markovitch and Paul D. Scott. Information filtering: Selection mechanisms in learning systems. Machine Learning, 10:113--151, 1993.

....cases from a fixed distribution. This is similar to the approach taken in Composer [11] which adds a control rule to the system only if it shows incremental utility. The incremental utility is evaluated by expected problem solving cost in a sequence of problems. The information filtering model [19] proposes a more general framework of selective learning, and defines various methods for eliminating harmful knowledge from the learning system. Selection processes, called filters, may be inserted to remove such knowledge. The filters include selective experience, selective attention, selective ....

S. MarkovitchandP. D. Scott. Information filtering : Selection mechanism in learning systems. Machine Learning, 10(2):113--151, 1993.

....of run time after learning being greater than run time before learning. This utility problem has been a particular focus of research in explanation based learning (EBL) There have been approaches which are useful for producing cheaper rules [1, 2, 3, 4, 5, 6] or filtering out expensive rules [2, 7, 8, 9]. However, these approaches cannot generally guarantee that the cost of using the learned rules will always be bounded by the cost of the problem solving from which they are learned, given the same situation. One way of finding a solution which can guarantee such cost boundness is to analyze all ....

S. Markovitch and P. D. Scott. Information filtering : Selection mechanism in learning systems. Machine Learning, 10(2):113--151, 1993.

.... 20; Ram Hunter, 1992 chapter 4) An analysis of the utility of learning can help in determining the target of learning (desJardins, 1992 chapter 8) in guiding learning processes (Gratch DeJong, 1993; Gratch, DeJong, Chien, 1994; Provost, 1994) and also in deciding whether to learn at all (Markovitch Scott, 1993; Minton, 1990 chapter 3) ffl Facilitating the use of opportunities to learn: If a reasoner does not have sufficient resources at the time it realizes it has a need to learn, or if the requisite knowledge is not available at that time, the reasoner can suspend its learning goals in memory so ....

....the effects of that rule on performance are monitored; rules that do not improve performance are removed from the rule library. This kind of selective storage and retention of learned rules is an instance of a more general kind of goal directed control of learning called information filtering (Markovitch Scott, 1993). Information filters can be used to decide which learned items to store, which to retain in memory over time, which to apply in a given situation, and even which training experiences it should seek out and which it should learn from. The reasoner s task goals guide the filters in selecting what ....

Markovitch, S. & Scott, P.D. (1993). Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, 10:113--151.

....the research described in this paper is to design algorithms that automatically find efficient orderings of subgoal sequences. Several researchers have explored the problem of automatic reordering of subgoals in logic programs (Warren, 1981; Naish, 1985b; Smith Genesereth, 1985; Natarajan, 1987; Markovitch Scott, 1989). The general subgoal ordering problem is known to be NP hard (Ullman, 1982; Ullman Vardi, 1988) Smith and Genesereth (1985) and Markovitch and Scott (1989) present search algorithms for finding optimal orderings. These algorithms are general and carry exponential costs for non trivial sets of ....

.... generators (Naish, 1985a) ffl Prefer calls that fail quickly (Naish, 1985b) The heuristic methods usually execute quickly, but may yield suboptimal orderings. The second approach, which is adopted in this paper, aims at finding optimal orderings (Smith Genesereth, 1985; Natarajan, 1987; Markovitch Scott, 1989). Natarajan proposed an efficient way to order a special sort of subgoal set (where all subgoals are independent) while Smith and Genesereth proposed a general, but inefficient algorithm. In the following section we build a unifying framework for dealing with subgoal ordering and describe ....

[Article contains additional citation context not shown here]

Markovitch, S. (1989). Information Filtering: Selection Mechanisms in Learning Systems.

....(Tadepalli Natarajan 1996) Several works in speedup learning concentrated on acquiring control knowledge for controlling the search performed by the problem solver. When the cost of using the acquired control knowledge outweighs its benefits, we face the so called Utility Problem (Minton 1988; Markovitch Scott 1993). Existing works dealing with the utility of control knowledge are based on a model of control Copyright c fl 1998, American Association for Artificial Intelligence (www.aaai.org) All rights reserved. rules whose main associated cost is the time it takes to match their preconditions. Most of ....

....for Artificial Intelligence (www.aaai.org) All rights reserved. rules whose main associated cost is the time it takes to match their preconditions. Most of the existing solutions for this problem involve filtering out control rules that are estimated to be of low utility (Minton 1988; Markovitch Scott 1993; Gratch DeJong 1992) Others try to restrict the complexity of the preconditions (Tambe, Newell, Rosenbloom 1990) In this work we deal with a different setup where the control procedure has potentially very high complexity regardless of the specific control knowledge acquired. In this setup, ....

Markovitch, S., and Scott, P. D. 1993. Information filtering: Selection mechanisms in learning systems.

....macros is the increased branching factor of the search space. When c fl1998 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved. Finkelstein Markovitch the costs outweigh the benefits, we face a phenomenon called the utility problem (Minton, 1988; Gratch DeJong, 1992; Markovitch Scott, 1993; Mooney, 1989) Due to the very large number of macros available for acquisition, a learning program must be selective in order to obtain a macro set with high utility. The goal of this research is to demonstrate that a simple macro learning technique, combined with the right selection ....

....combined with the right selection mechanisms, can lead to a speedup learning algorithm that is powerful and general, yet simple as well. We start by defining a framework for selective macro learning and describe the general architecture of a macro learner. The information filtering framework (Markovitch Scott, 1993) is used to describe the various logical components of a macro learner. In particular, the framework emphasizes the important role of the selection mechanisms used during the learning process. We continue by describing the Micro Hillary algorithm. To make the presentation and the experiments ....

[Article contains additional citation context not shown here]

Markovitch, S., & Scott, P. D. (1993). Information filtering: Selection mechanisms in learning systems. Machine Learning, 10 (2), 113--151.

.... recent years, the problem of irrelevant features in inductive learning received more attention, and some feature selection algorithms emerged (such algorithms per Learning of Resource Allocation Strategies for Game Playing 12 form attention filtering according to the framework described in [ Markovitch and Scott, 1993 ] In the experiments described in section 5, we used a feature selection algorithm called RELIEF [ Kira and Rendell, 1992 ] that attempts to eliminate features statistically irrelevant to the class. 3.4. Using Context sensitive Classifiers in Resource Allocation Context As noted earlier, many ....

Shaul Markovitch and Paul D. Scott. Information filtering: Selection mechanisms in learning systems. Machhine Learning, 10:113--151, 1993.

....features it is important that such features will be filtered out. In recent years the problem of irrelevant features in inductive learning received more attention, and some feature selection algorithms emerged (such algorithms perform attention filtering according to the framework described in [ Markovitch and Scott, 1993 ] In the experiments described in section 5, we used a feature selection algorithm called RELIEF [ Kira and Rendell, 1992 ] that attempts to eliminate features statistically irrelevant to the class. 3.4 Using Classifiers in Resource Allocation Context As noted earlier, many classification ....

Shaul Markovitch and Paul D. Scott. Information filtering: Selection mechanisms in learning systems. Machhine Learning, 10:113--151, 1993.

....advantages of the derivative crossover in evaluation function learning in the domain of Checkers. Section six concludes. Learning As A Search Process Learning is the process of using experience in order to modify some knowledge base with the goal of improving it with respect to certain criteria [11]. The state of the knowledge base at any moment can be viewed as the current hypothesis, and the process of learning can be viewed as exploring the state space of all plausible hypotheses for the best hypothesis under the constraints imposed by experience [12] There are three types of processes ....

S. Markovitch and P. D. Scott. Information filtering: Selection mechanisms in learning systems. Machine Learning, 10:113--151, 1993.

No context found.

Markovitch, S. and Scott, P. D. 1993. Information filtering: selection mechanisms in learning systems. Machine Learning 10:113--151.

No context found.

Shaul Markovitch and Paul D. Scott. Information Filtering: Selection Mechanisms in Learning Systems. Machine Learning, Volume 10, number 2, pages 113--151, February 1993.

No context found.

Markovitch S., Scott P. D., Information Filtering: Selection Mechanisms in Learning Systems, Machine Learning 10, pp. 113-151, 1993.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC