B. Eisenberg and R. Rivest. On the sample complexity of PAC learning using random and chosen examples. In M. Fulk and J. Case, editors, Proceedings of the Third Annual ACM Workshop on Computational Learning Theory, pages 154--162, San Mateo, CA, 1990. Kaufmann. V. V. Fedorov. Theory of Optimal Experiments. Academic Press, New York, 1972.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....batch mode since the choice of which examples to query for is often based on the performance of the learner on the examples obtained so far. Some previous work in this direction include querying for correct classification labels of points in the feature space (cf. Angluin [1] Rivest and Eisenberg [13]) the notion of selective sampling (cf. Cohn, et al. 4] Cohn [3] where regions of high classification uncertainty are identified and from which labeled samples are randomly drawn. In pattern classification problems, it is often not useful or possible to query for the correct classification of ....

# R.L. Rivest, B. Eisenberg, "On the Sample Complexity of PacLearning Using Random and Chosen Examples," Proc. 1990.

....assets and drawbacks of both methods. An important disadvantage of the heuristic, constructive algorithms is that they cannot utilize information about the input distribution to the student. It is known from learning theory that access to the input distribution indeed is important for the student [42, 43]. In this respect, the constructive active student is at a disadvantage in comparison to the passive student, who receives this distributional information along with the randomly drawn training set. A related problem, especially in high dimensional input spaces, is that the constructed samples ....

....as is done in passive learning with the expectation that the active student consequently needs fewer training samples than a passive student to achieve a xed generalization ability. The properties of active learning have been rigorously analyzed in the PAC framework for concept learning [42, 43] with the result that the set of PAC learnable concept classes is not enlarged by active learning but that in general fewer training samples are needed, i.e. the sample complexity can be reduced by active learning. This last point is con rmed by studies on active learning in more realistic ....

B. Eisenberg and R. L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In M. Fulk and J. Case, editors, Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 154-162, San Mateo, CA, August 1990. Univ. of Rochester NY, Morgan Kaufmann.

....both the fat shattering and the co fat shattering have a polynomial upper bound (in the learning parameters) which enables efficient learning of the dual problem. 3 Dense Learning Problems A learning problem is dense if every hypothesis has many hypotheses which are close but not equal to it [4]: Definition 1 A learning problem h Omega ; Hi is dense with respect to the probability measure D if for every h 2 H and every ffl 0 there exists h 0 2 H such that 0 Pr x2D [h(x) 6= h 0 (x) ffl. The density property is distribution dependent: for every learning problem there exists ....

B. Eisenberg and R. L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In Proc. of the 3'rd Ann. Conference on Computational Learning Theory, pages 154--162, 1990.

....are a natural candidate for selective sampling strategies. If one knew a priori the identity of the support patterns in a data set, it would be possible to discard all the other patterns, and still recover the same final hypothesis. Theoretical results and artificial examples discussed in Rivest and Eisenberg (1990) show that it is possible to invent malicious distributions for which the number of queries is comparable to the sample size, hence removing any advantage. In practice, such adversarial distributions may not occur frequently. Indeed we find that the sparsity of the solution is the most important ....

Rivest R.L. & Eisenberg, B. (1990). On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational LearningTheory (pp. 154--162). San Mateo, CA: Morgan Kaufmann.

....are a natural candidate for instance selection strategies. If one knew a priori the identity of the support patterns in a dataset, it would be possible to discard all the other patterns, and still recover the same final hypothesis. Theoretical results and artificial examples discussed in [Rivest Eisenberg, 1990] show that it is possible to invent malicious distributions for which the number of queries is comparable to the sample size, hence removing any advantage. In practice, such adversarial distributions may not occur frequently. Indeed we find that the sparsity of the solution is the most important ....

R. L. Rivest and B. Eisenberg. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational LearningTheory, pages 154--162, Morgan Kaufmann, San Mateo, CA, 1990.

....We show that for classes which consist of su ciently smooth functions, both the fat shattering and the co fatshattering have an upper bound which is polynomial in the learning parameters and enables e cient learning of the dual problem. 3 Dense Learning Problems A learning problem is dense [7] if every hypothesis has many hypotheses which are close but not equal to it. De nition 1. A learning problem h ; Hi is dense with respect to the probability measure D if for every h 2 H and every 0 there exists h 0 2 H such that 0 Pr x2D [h(x) 6= h 0 (x) The density property ....

B. Eisenberg and R. L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In In the Proceedings of the Third Annual Conference on Computational Learning Theory, pages 154-162. Morgan-Kaufmann, 1990.

....even by relaxing to weak learning, restricting the target distribution to be uniform, providing membership queries, and allowing the learner to play a significant role in the choice of the target concept. Similar issues have recently been investigated in Euclidean domains by Eisenberg and Rivest [4]. 11 Toward a Characterization of Weak Sample Complexity As we have mentioned, it is well known that the sample size required for strong learning is characterized by the Vapnik Chervonenkis dimension. In Section 6, we saw that this same measure fails to characterize weak sample complexity for ....

Bonnie Eisenberg and Ronald L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 154--162. Morgan Kaufmann, August 1990.

....have access to membership queries, i.e. algorithms that can ask for the label of any particular instance, can reduce the number of training examples by querying only on this small set of instances whose labels are the most informative. This question was previously studied by Eisenberg and Rivest [Eisenberg and Rivest, 1990] in the PAC learning framework. They give a negative result, and show that for a natural set of concept classes, which they call dense in themselves , queries can not decrease the number of labels that the learner has to observe before it can generate an accurate hypothesis. Intuitively, the ....

....assumptions. We assume that both the examples and the target concept are chosen randomly. In particular, we show that queries can help accelerate learning of concept classes that are already learnable from just unlabeled data. This question was previously studied by Eisenberg and Rivest [Eisenberg and Rivest, 1990] in the PAC learning framework. They give a negative result, and show that, for a natural set of concept classes, which they call dense in themselves , queries are essentially useless. They show that giving the learner the ability to ask membership queries ( questions of the type what is the ....

Bonnie Eisenberg and Ronald L. Rivest. On the sample complexity of paclearning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational Learning Theory, pages 154--162, 1990.

....assumptions. We assume that both the examples and the target concept are chosen randomly. In particular, we show that queries can help accelerate learning of concept classes that are already learnable from just unlabeled data. This question was previously studied by Eisenberg and Rivest [ER90] in the PAC learning framework. They give a negative result, and show that, for a natural set of concept classes, which they call dense in themselves , queries are essentially useless. They show that giving the learner the ability to ask membership queries (questions of the type what is the ....

B. Eisenberg and R. L. Rivest. On the sample complexity of PAC-learning using random and chosen examples. In Proc. 3rd Annu. Workshop on Comput. Learning Theory, pages 154--162, San Mateo, CA, 1990. Morgan Kaufmann.

.... using similar techniques as above, together with by now standard Chernov bound techniques from [6, 5] one can prove that opt P (F; ffl; ffi) O( opt PB (F; ffl=2; ffi=2) log(1=ffi) ffl) 6) If opt PM is defined similarly where membership queries replace boolean queries, Eisenberg and Rivest [11] showed that for all F with a certain property, opt PM (F; ffl; ffi) Omega Gamma12816 =ffi) ffl) PAC learning using only boolean queries was studied by Kulkarni, Mitter and Tsitsiklis [13] 6 Boolean queries and computationally efficient algorithms To discuss issues of computational ....

B. Eisenberg and R.L. Rivest. On the sample complexity of PAC-learning using random and chosen examples. The 1990 Workshop on Computational Learning Theory, pages 154--162, 1990.

....assumptions. We assume that both the examples and the target concept are chosen randomly. In particular, we show that queries can help accelerate learning of concept classes that are already learnable from just unlabeled data. This question was previously studied by Eisenberg and Rivest [ER90] in the PAC learning framework. They give a negative result, and show that, for a natural set of concept classes, which they call dense in themselves , queries are essentially useless. They show that giving the learner the ability to ask membership queries (questions of the type what is the ....

Bonnie Eisenberg and Ronald L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational Learning Theory, pages 154--162, 1990.

....were not polynomially learnable (in the Valiant sense) from examples alone, they could be learned using a polynomial number of queries to an oracle that provides counter examples. Valiant (1984) considers various classes that are learnable using a variety of forms of active learning. Work by Eisenberg and Rivest (1990) puts bounds on the degree to which membership queries examples can help generalization when the underlying distribution is unknown. Additionally, given certain smoothness constraints on the distribution, they describe how queries may be used to learn the class of initial segments on the unit ....

B. Eisenberg and R. Rivest. (1990) On the sample complexity of pac-learning using random and chosen examples. In M. Fulk and J. Case, eds., ACM 3rd Annual Workshop on Computational Learning Theory, Morgan Kaufmann.

....we have some indication of where this important part is, we may have no chance of locating it. To obviate this problem, most work on such 11 domains, including the work described later in this dissertation, uses the paradigm of learning from both membership queries and random examples [BL91, Eis91] 1.3 Two Learning Systems In this section, we discuss two learning systems with which we will be concerned in the following chapters: feedforward neural networks, and vector quantizers. Although they have many other applications, we will only be concerned with neural networks as binary concept ....

....domain and an oracle returns the classification of that point. Much work in formal learning theory has been directed to the study of queries (see e.g. Ang86, Val84, Ams88] but only very recently have queries been examined with respect to their role in improving generalization behavior [CAL90, Eis91] Although in the worst case, self directed learning may do no better than random sampling [AK91] in many formal problems self directed learning is provably more 89 powerful than passively learning from randomly given examples. A simple example is that of locating a boundary on the unit line ....

[Article contains additional citation context not shown here]

B. Eisenberg. On the sample complexity of pac-learning using random and chosen examples. Master's thesis, Massachusetts Institute of Technology, 1991.

No context found.

B. Eisenberg and R. Rivest. On the sample complexity of PAC learning using random and chosen examples. In M. Fulk and J. Case, editors, Proceedings of the Third Annual ACM Workshop on Computational Learning Theory, pages 154--162, San Mateo, CA, 1990. Kaufmann. V. V. Fedorov. Theory of Optimal Experiments. Academic Press, New York, 1972.

No context found.

R.L. Rivest B. Eisenberg. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational LearningTheory, pages 154--162, San Mateo, CA, 1990. Morgan Kaufmann.

No context found.

Bonnie Eisenberg and Ronald L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational Learning Theory, pages 154--162, 1990.

No context found.

R.L. Rivest B. Eisenberg. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational LearningTheory, pages 154--162, San Mateo, CA, 1990. Morgan Kaufmann.

No context found.

Bonnie Eisenberg and Ronald L. Rivest. On the sample complexity of pac-learning using random and chosen examples. In Proceedings of the 1990 Workshop on Computational Learning Theory, pages 154--162, 1990.

No context found.

B. Eisenberg and R. Rivest. (1990) On the sample complexity of pac-learning using random and chosen examples.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC