D Haussler. Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results. In Proceedings of the 30th IEEE Symposium on the Foundations of Computer Science, pages 40--45, 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and labels are real valued and P = P the set of all probability distributions on X Theta Y . Building on the work of Dudley ( 16] and [18] and Pollard ( 23] and [24] Haussler has made much progress in finding conditions that are sufficient for femp to be a simultaneous estimator (see [25] and [14] One of these conditions is that a certain pseudodimension be finite. This pseudodimension generalizes the VC dimension to classes of real valued functions. Vapnik generalizes the VC dimension in a different fashion in [4] 14 For r 2 IR, let sign[r] be 1 if r 0 and 0 otherwise. ....

D. Haussler, "Generalizing the PAC model: Sample size bounds from metric dimensionbased uniform convergence results," in 30th Annual IEEE Symposium on Foundations of Computer Science, pp. 40--45, 1989.

....learning. We show that for C = 2 X the compression parameter and the order independent compression parameter differ by at least a logarithmic factor. Finally we show that in both cases there are classes where d(C) VCdim(C) A further extension of the model allows robust learning as defined in [5, 7]. Here the hypothesis class may be much weaker than the target concept class. The objective is to find a hypothesis which is almost the best approximation possible in this hypothesis class. The modifications necessary consist of a second counter and one more memory cell capable of storing an ....

David Haussler. Generalizing the pac model: Sample size bounds from metric-- dimension based uniform convergence results. In Proceedings of the 30'th Annual Symposium on the Foundations of Computer Science, pages 40--46, 1989.

....D, P [sup #R( X n # ) #R( # ] # 8E[# # (L; 8;X n # ) exp( #n # 128M # ) where M =sup ;x L( x) # inf ;x L( x) and X n # = #x # ; x n # are independently drawn from D. The constants in the above theorem can be improved for certain problems: see [6, 13, 35, 36] for related results. However, they yield very similar bounds. The result most relevant for this paper is a lemma in [3] where the 1 norm covering numberisreplacedbythe# norm covering number. The latter can be bounded by a scale sensitivecombinatorial dimension [1] which can be bounded from the ....

D. Haussler. Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results. In Proc. 30th IEEE Symposium on Foundations of Computer Science, pages 40-45, 1989.

....and use it to analyze several methods ffl covering, hashing, clustering, tree structured clustering, and receptive fields for learning smooth functions. The sample size and system complexity are derived for each method. Our model is built upon the generalized PAC learning model of Haussler [16] and is closely related to the method of vector quantization in data compression. Our main result is that we can build memory based learning systems using new clustering algorithms (Lin and Vitter [24] to PAC learn in polynomial time using only polynomial storage in typical situations. Keywords: ....

....computational complexities of training That is, how much time does it require to determine system parameters from the sample In Section 2 we outline a theoretical framework for answering these problems. Our memory based learning model is built upon the generalized PAC learning model of Haussler [16] and is closely related to the method of vector quantization in data compression (Gersho [13] Gray [15] Riskin [38] Gersho and Gray [14] Section 3 introduces the notion of quantization number , which is intended to capture the optimal memory requirement of memory based learning systems for a ....

[Article contains additional citation context not shown here]

D. Haussler, "Generalizing the PAC Model: Sample Size Bounds from Metric Dimension-Based Uniform Convergence Results," in Proceedings of the 30th Annual IEEE Symposium on Foundations of Computer Science, 1989, 40--45.

....and labels are real valued and P = P , the set of all probability distributions on X Theta Y . Building on the work of Dudley ( 16] and [18] and Pollard ( 23] and [24] Haussler has made much progress in finding conditions that are sufficient for femp to be a simultaneous estimator (see [25] and [14] One of these conditions is that a certain pseudodimension be finite. This pseudodimension generalizes the VC dimension to classes of real valued functions. Vapnik generalizes the VC dimension in a different fashion in [4] For r 2 IR, let sign[r] be 1 if r 0 and 0 otherwise. ....

D. Haussler, "Generalizing the PAC model: Sample size bounds from metric dimensionbased uniform convergence results," in 30th Annual IEEE Symposium on Foundations of Computer Science, pp. 40--45, 1989.

....based on standard circuit complexity, one is quickly led to study very powerful and apparently difficult classes such as disjunctive normal form Boolean expressions. Another contribution of this research is in demonstrating the feasibility and practicality of the approach suggested by Haussler [11]. His work addressed the issue of sample complexity upper bounds in great generality, even encompassing the case where the input output relation to be learned has no prescribed functional form. This generality prevents Haussler from obtaining either good sample size lower bounds or efficient ....

David Haussler. Generalizing the PAC model: Sample size bounds from metric dimensionbased uniform convergence results. In 30th Annual Symposium on Foundations of Computer Science, pages 40--45, October 1989.

.... scales with the parameters # and # (which specify the demanded accuracy and confidence of learning) The distribution free learning model has been transferred to probabilistic concept classes (also called p concept classes) by Kearns and Schapire (see [6] and to function classes by Haussler (see [5]) In this paper, we restrict ourselves to functions with real values in the range [0, 1] hereafter simply called functions) 1 The main di#erence between p concept learning and function learning lies in the kind of feedback. A labeled example for a function f has the form (x, f(x) i.e. the ....

D. Haussler, Generalizing the pac model: Sample size bounds from metric--dimension based uniform convergence results, in Proc. 30th Annual Symposium on the Foundations of Computer Science, IEEE Computer Society Press, Los Alamitos, CA, 1989, pp. 40--46.

....into Bayes decision theory and its importance for pattern recognition see [DH73] Our aim is to put the elements of Bayes decision theory into the framework of pac learning. An extension of pac learning to real valued functions in a probabilistic setting has been investigated by Haussler [Hau89]. Also Kearns and Schapire [KS90] have recently considered the learning of probabilistic concepts (p concepts) In this paper we investigate a case that cannot be (obviously) treated with their methods. We assume that D is fixed but unknown and that all predictions are only based on sample ....

D. Haussler. Generalizing the pac model: Sample size bounds from metric dimension-based uniform convergence results. In colt89. Morgan Kaufmann, San Mateo, CA, 1989.

....et al. 9] give a similar result. Both algorithms return hypotheses containing O(s lg m) halfspaces where m is the size of the sample. Baum gives efficient algorithms for learning several classes with infinite VC dimension (such as convex polyhedral sets) under uniform distributions [6] Haussler [23] also gives distribution specific algorithms for several classes of functions. Research has also been done on the learnability of unions of axis parallel boxes. Blumer et al. present an algorithm to PAC learn an s fold union of boxes in E d by drawing a sufficiently large sample of size m = ....

David Haussler. Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results. In 30th Ann. Symp. on Foundations of Comp. Sci., pages 40--45, October 1989.

....statistical parameter estimations. We applied this framework to classification problems with 2 states and Boolean or real valued features. All our results are easily generalized to the case of a constant number of states whereby the order of the sample size remains the same. The papers of Haussler [Hau89] and Kearns, Schapire [KS90] contain quite general results concerning learning of probabilistic concepts. In their terminology, the a posteriori probability c(x) D[ 1 jx] appears as the probabilistic target concept whose decision rule must be learned. They describe several sufficient ....

D. Haussler. Generalizing the pac model: Sample size bounds from metric dimensionbased uniform convergence results. In 30th Annual Symposium on Foundations of Computer Science, pages 40--45, 1989.

....correctly. ffl A fully connected feedforward network with one hidden layer, trained on fewer than Omega Gamma W ffl Delta examples will, for a dichotomy realizable by the network, fail to find the requisite set of weights for more than a fraction (1 Gamma ffl) of future examples. Haussler [12] shows that, for it to be likely that feedforward networks with sigmoidal units obtain a low estimation error, the number of examples must grow linearly with both the number of modifiable weights and the number of hidden layers. That is, either of the following desiderata demands a larger training ....

D. Haussler. Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results. In Proc. 30th Annual Symp. Foundations of Computer Science, pages 40--45. Computer Society Press, IEEE, 1989.

....and Goodrich [14] present a set covering algorithm that allows them to return a hypothesis containing O(ds lg(ds) halfspaces. Baum gives efficient algorithms for learning several classes with infinite VC dimension (such as convex polyhedral sets) under uniform distributions [8] Haussler [26] also gives distribution specific algorithms for several classes of functions. Bshouty, Goldman and Mathias [17] have given noise tolerant algorithms for several geometric classes. In particular, they studied C d s against the product distribution and a restricted version of this class, in which ....

David Haussler. Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results. In 30th Annual Symposium on Foundations of Computer Science, pages 40--45, October 1989.

....examples correctly. ffl A fully connected feedforward network with one hidden layer, trained on fewer than Omega i W ffl j examples will, for a dichotomy realizable by the network, fail to find the requisite set of weights for more than a fraction (1 Gamma ffl) of future examples. Haussler [12] shows that, for it to be likely that feedforward networks with sigmoidal units obtain a low estimation error, the number of examples must grow linearly with both the number of modifiable weights and the number of hidden layers. That is, either of the following desiderata demands a larger training ....

D. Haussler. Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results. In Proc. 30th Annual Symp. Foundations of Computer Science, pages 40--45. Computer Society Press, IEEE, 1989.

....with a bounded number of such erroneous responses, and Frazier and Pitt [23] consider learning when such incorrect responses occur randomly with probability at most 1 2 . In other related work, Kearns and Schapire [32] generalized the PAC setting to non binary values using Haussler s framework [28]. They define a p concept in which each instance x 2 X has some probability p(x) of being classified as positive. In their model, the goal of the learner is to make optimal predictions, or more commonly, to accurately predict p(x) for all x 2 X . One way to compare our model to theirs is to ....

David Haussler. Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results. In 30th Annual Symposium on Foundations of Computer Science, pages 40--45, October 1989.

....maps them to an outcome. Classical supervised learning is an example of such a learning system. 2. 1 Direct Inverse Modeling of a Convex Solution Space Solution regions and learnability are well studied characteristics in the machine learning community; see for example, Anthony and Biggs, 1992, Haussler, 1989, Minsky and Papert, 1969] A solution set X is said to be convex if and only if there exist three co linear points p; q; r such that, if p 2 X and r 2 X , then q 2 X ; otherwise the region is non convex. See Figure 2. Figure 2: Non Convexity of One to Many Mappings Intentions Actions The point ....

Haussler, D. "Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results." In Proceedings of the 30th Annual Symposium on Foundations of Computer Science. IEEE Computer Society Press.

.... one as m increases (regardless of the particular value of P 2 P) Most existing results on learning in frameworks other than the PAC model fix attention only on pairs (P; H) for which femp is a simultaneous estimator, and thus involve finding a hypothesis that minimizes femp (see [23] [15], 16] 21] 24] and [22] Such results are based on, or heavily influenced by, the pioneering work of Vapnik and Chervonenkis ( 25] 26] and [23] which provides necessary and sufficient conditions for femp to simultaneously estimate (P; H) In addition, Dudley ( 27] 28] Pollard ....

.... results are based on, or heavily influenced by, the pioneering work of Vapnik and Chervonenkis ( 25] 26] and [23] which provides necessary and sufficient conditions for femp to simultaneously estimate (P; H) In addition, Dudley ( 27] 28] Pollard ( 29] 30] Talagrand ( 31] Haussler ([15], 21] and others ( 32] 33] have found useful conditions that are sufficient for femp to simultaneously estimate (P; H) This paper goes beyond such empirical error minimization and investigates simultaneous estimation problems that can be solved by estimators satisfying only a ....

[Article contains additional citation context not shown here]

D. Haussler, "Generalizing the PAC model: Sample size bounds from metric dimensionbased uniform convergence results," in 30th Annual IEEE Symposium on Foundations of Computer Science, pp. 40--45, 1989.

No context found.

D Haussler. Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results. In Proceedings of the 30th IEEE Symposium on the Foundations of Computer Science, pages 40--45, 1989.

No context found.

David Haussler. Generalizing the PAC model: Sample size bounds from metric dimensionbased uniform convergence results. In 30th Annual Symposium on Foundations of Computer Science, pages 40--45, October 1989.

No context found.

David Haussler. Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results. In 30th Annual Symposium on Foundations of Computer Science, pages 40--45, October 1989.

No context found.

Haussler D., "Generalizing the PAC Model: Sample Size Bounds From Metric Dimension-Based Uniform Convergence Results", Proc. of 29th FOCS pp. 40-45, 1988.

No context found.

David Haussler. Generalizing the pac model: Sample size bounds from metric--dimension based uniform convergence results. In Proceedings of the 30'th Annual Symposium on the Foundations of Computer Science, pages 40--46, 1989.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC