D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36(2):177--221, 1988.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....can be considered as a specific computable approximation to the KC based Occam s razor. As one of the examples, we will demonstrate that the standard trivial learning algorithm for monomials actually often has a better sample complexity than the more sophisticated Haussler s greedy algorithm [7]. This is contrary to the commen, but mistaken, belief that Haussler s algorithm is better in all cases (to be sure, Haussler s method is superior for target monimials of small length) Another issue related to Occam s razor theorem is the status of the reverse assertion. Although a partial ....

....set the concept representation M : x 1 x 1 . x n x n (a conjunction of all literals of n variables which contradicts every example) ii) For each positive example, delete from the current M the literals that contradict the example. iii) Return the resulting monomial M . Haussler [7] proposed a more sophisticated algorithm based on set cover approximation as follows. Let k be the number of variables in the target monomial, and m be the number of examples used. Haussler s Algorithm. 11 (i) Use only negative examples. For each literal x, define S x to be the set of negative ....

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:2(1988), 177-222.

.... Set covering techniques have been used in several similarity based learning systems, such as the AQ family of algorithms [Dietterich and Michalski, 1983; Michalski et al. 1986] the CN2 learning algorithm [Clark and Niblett, 1989] and Haussler s algorithm for learning pure disjunctive concepts [Haussler, 1988]. In all of these algorithms, the basic procedure is to start with an empty cover and repeatedly add to it some conjunctive set or complex ; often this complex is one that is chosen to cover as many positive examples as possible. A EBL differs from the algorithms above in the nature of ....

David Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36, 1988.

....over the natural distribution of samples (queries, tests, problems, that the system will encounter. There are (at least) two potential problems with implementing such a learning system: First, the task of identifying the globally optimal element is intractable for many spaces; cf. Hau88] Gre91] A common solution to this problem is to use a hill climbing approach to find a locally optimal solution. Two well known inductive learning systems that use this approach are id3 [Qui86] which uses a greedy technique to reduce the expected entropy of a decision tree, and backprop ....

.... set of performance elements PE, where each PE 2 PE is a function that returns an answer to each given query (or problem or goal or : For example, in context of seeking a good classification function, each PE 2 PE may be a particular decision tree [Qui86] or a specific boolean formula [Hau88] or a credulous prioritized default theory [Gre92] Within the context of speed up learning, GJ92] views each PE 2 PE as a particular prolog program, where all of the programs in PE include exactly the same clauses, but differ in the order of these clauses. Let Q = fq 1 ; q 2 ; g be the ....

David Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, pages 177--221, 1988.

....processed training data minus i. For the learning process version spaces have to be represented. The standard representation is by boundary sets [5 7, 12] They are correct for the class of admissible concept languages [5, 12] but their size can grow exponentially in the size of training data [1]. To overcome this problem alternative version space representations were introduced in [2 4, 8 15] They extended the classes of concept languages for which version spaces are e#ciently computable. A shortcoming of most version space representations is that they are ine# cient for instance ....

Haussler, D.: Quantifying Inductive Bias: AI Learning Algorithms and Valiants Learning Framework. Artificial Intelligence 36 (1988) 177-221

.... the dimension [13] This dimension has been related, for specific concept classes, to Littlestone and Warmuth s compressibility notion [5] Furthermore, relations between dimension and classes of functions show that the dimension increases as the complexity of the function does (see [7] ) The generality of theoretical bounds, such as the ones mentioned above, constitutes at the same time their beauty and their weakness; in fact, they are often too far from the actual values they are intended to bound to be useful in practice [11] In this work we have taken, instead, an ....

David Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Artificial Intelligence, 36(2):177--221, 1988.

....each j and any h 2 H , i) ae x (a n; x (h) h) ffl and (ii) a n; x (h) 2 H . Using a large M allows the elements of the cover to be less simple. In some cases, finding the simplest hypothesis consistent with a labeling is much harder than finding one that is only reasonably simple (see [42]) This would dictate using M 1. A key observation is that we can construct finite M simple empirical coverings. Lemma 4.1. Under Assumption 2.2, for any M 1 we can construct an M simple empirical ffl covering for H based on x(n) that has at most [K(ffl=2) elements. Proof: See the ....

D. Haussler, "Quantifying inductive bias: AI learning algorithms and Valiant's learning framework," Artificial Intelligence, vol. 36, no. 2, pp. 117--222, 1988.

....learning algorithm for MN;k (with respect to sample complexity) compared to a consistent learning algorithm which can represent only the functions MN;k . We suggest that this is a natural corollary of the generality of CB1(oeH ) this seems a clear example of the concept of inductive bias [Hau88] [Sch94] Bias refers to any prior information or knowledge that might be encoded in a learning algorithm that defines a preference for choosing a hypothesis from the many that might be available to account for the training data. If a bias is strong and correct, then the concept learning task ....

D Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

....standard representation is by boundary sets [5, 7, 10] Boundary sets are the minimal and maximal sets of version spaces. They are correct for the class of admissible concept languages [5, 10] An analysis of boundary sets shows that their size can grow exponentially in the size of training data [1]. To overcome this problem alternative versionspace representations were introduced in [2 4, 8 13] They extended the classes of concept languages for which version spaces are eciently computable. A shortcoming of most version space representations is that they are not ecient for instance ....

D. Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Arti cial Intelligence, 36(2):177-221, 1988.

....in the case where l k a proof (or a counterexample ) eludes the author. Therefore we give up on trying to reuse the reduction from lemma 10. Fortunately, we can do something else instead. Using a reduction by Haussler we will reduce SET COVERING to the hA; Bi version of minimum length DNF. In [7] Haussler used the reduction to prove that nding a consistent monomial is NP hard; in our terminology is a consistent monomial a DNF, consisting of a single term, that realizes hA; Bi. But its applicability does not stop here Indeed the reduction is the closest we get to a panacea for our ....

Haussler, D., \Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework", in: Arti cial Intelligence 36 No. 2 (1988) 177-222.

....for inferring hidden causation especially by constructing Bayesian or belief networks and are often semantically better understood than the latter. Our first issue is how to develop a formal, prescriptive framework (cf. Valiant s quantitative PAC analysis for bias in inductive learning) [7] The position we defend below is that the milieu described by a random sampling problem is a determinant of computational learning complexity, as are the concept and hypothesis languages in the PAC framework. Our second issue is the problem, in uncertain reasoning, of statistical learning with ....

D. Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Artficial Intelligence, 36:177--221, 1988.

....lists. With this in mind we set N = N fi 0 for some 0 fi 1. The above estimate of jRj assumed there were only two classes. Theoretical results using the Vapnik Chervonenkis dimension to bound the difference between empirical error and actual error [1] applied to conjunctive concepts [4], suggest that this difference does not depend on the number of classes. Thus we compute N as described above even when there are more than two classes. The reader may be concerned at the unrealistic assumptions and rough approximations used in the derivation of estErr( In fact, our ....

Haussler, D. (1988.) Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence 36, 177--221.

....k log 1 factor better than results from additive error SQ) Proof: We construct a PAC algorithm which tolerates this malicious error by simulating an efficient relative error SQ algorithm for the problem. The SQ algorithm uses the set cover approach for learning conjunctions of k literals [12]. The additive error SQ version of this algorithm is a conversion from elimination and covering examples to elimination and 15 1. Learn k Conjunction: 2. V : fx 1 ; x n ; x 1 ; x n g 3. h : TRUE 4. r : 2k ln(2= 5. foreach v i 2 V do begin 6. P : STAT (f; D) v i = 1 ....

David Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177--221, 1988. 38

....can be represented by boundary sets. Boundary sets are sets of minimal and maximal descriptions in version spaces. It was proven that they correctly represent version spaces [5, 11] An analysis of boundary sets shows that their size can grow exponentially in the number of training instances [1]. To overcome this problem alternative version space representations were introduced in [2, 3, 4, 8, 9, 10, 11, 12, 13, 14] They extend the scope of concept languages for which version spaces are eciently applicable. One of the main diculties with the alternative version space representations is ....

D. Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Articial Intelligence, 36(2):177-221, 1988.

....When concept languages are partially ordered, version spaces can be represented by boundary sets. Boundary sets are de ned as sets of minimal and maximal descriptions in version spaces. It was proven that they correctly represent version spaces [6, 11] An analysis of boundary sets, given in [1], shows that their size can grow exponentially in the number of training instances. To overcome this problem alternative version space representations were introduced in [3, 4, 5, 8, 9, 10, 11, 12, 13] They extend the scope of concept languages for which version spaces can be eciently ....

D. Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Articial Intelligence, 36(2):177-221, 1988.

....than any more precise answer refining it. Monotone probability approximations have been pursued actively in recent work on learning, specifically on so called probably approximately correct (PAC) algorithms that produce approximately correct answers with high probability (see (Valiant, 1984; Haussler, 1988)) These algorithms combine the earlier notions of probabilistic algorithms, including Monte Carlo algorithms (which quickly produce an answer that is usually right) and Las Vegas algorithms (which produce a correct answer, usually quickly) PAC algorithms produce definitions of concepts from ....

Haussler, D. 1988. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36(2):177--221.

....rays are highly correlated and because not all parts of the molecular surface are likely to be involved in binding. The obvious next step is to choose a subset of the bounds of this APR that are sufficient to exclude all of the negative instances. This is analogous to the method described by Haussler (1988, 1989) The process of removing bounds from the APR is best organized as a process of adding bounds to an APR that covers the entire feature space. A greedy algorithm considers each bound from the all positive APR and chooses the bound that eliminates the most negative instances. This bound is ....

Haussler, D., (1988). Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36 (2), 177--222.

....proofs are indeed de ned. See [13, 8] for details. 3 Distribution independent sample sizes The Vapnik Chervonenkis dimension (or VC dimension) 16] has been widely used in order to obtain some measure of the degree of expressibility of a hypothesis space, and hence to obtain learnability results [9, 8, 4]. Given a hypothesis space H, de ne, for each x = x 1 ; x m ) 2 X m , a function x : H f0; 1g m by x (h) h(x 1 ) h(x m ) The growth function, H from the set of integers to itself is de ned by H (m) maxfjfx (h) h 2 Hgj : x 2 X m g 2 m : 3 If jfx ....

David Haussler, Quantifying inductive bias: AI learning algorithms and Valiant's learning framework, Articial Intelligence, 36, 1988: 177-221.

....given set of (data independent) features, Valiant (1984) has proposed a verysimx# learning algorithm for building a conjunctionfrom positive examqb6 only (or building a disjunctionfrom negativeexamx86 only) However, the obtained function mnc t contain a lot of features. To reduce their num ber, Haussler (1988) has proposed to use the classical greedy set coveringalgorithm (see Kearns Vazirani (1994) for a clear exposition of thesealgorithm6 on the remqx;x# examx;x that are not used by the Valiant algorithm It was shown by Haussler (1988) that good generalization of thisalgorithm is expected whenever ....

....mnc t contain a lot of features. To reduce their num ber, Haussler (1988) has proposed to use the classical greedy set coveringalgorithm (see Kearns Vazirani (1994) for a clear exposition of thesealgorithm6 on the remqx;x# examx;x that are not used by the Valiant algorithm It was shown by Haussler (1988) that good generalization of thisalgorithm is expected whenever there exists asm#x conjunction (or disjunction) of few relevant (data independent) features thatma es zero error with the training examing6 Because conjunctions and disjunctions are (just) subsets of the set of linearly separable ....

Haussler D. (1988). Quantifying Inductive Bias: AI LearningAlgorithm and Valiant's Learning Fram( work. Artificial Intelligence Vol. 36, (pp. 177--221).

....be vacuous conditions like anycolor(X) For the remainder of the paper, we will write the nonvacuous conditions in boldface. Since the final stage is to form a disjunction of these rules, for this domain theory, ELGIN[k tip] operates much like Haussler s greedy algorithm for learning k DNF [ Haussler, 1988 ] The actual implementation of ELGIN produces (in 1.2 seconds on a Sparc 1 ) the following specialization from the examples of Figure 2. xmas present(A) anysize(A) red(A) rectangle(A) xmas present(A) anysize(A) green(A) rectangle(A) With this domain theory, there is only one proof for ....

David Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36, 1988.

....In this paper we give a simple combinatorial proof of a bound, due to Vapnik, on the probability of relative deviation of frequencies from probabilities for a class of events. This result has applications in the theory of PAC learning, introduced by Valiant [13] and developed by many researchers [4, 5]. We discuss these applications, and describe how the theory of learnability may be extended from learning sets to learning functions, following work of Haussler [6] and Natarajan [8] We consider functions with nite or countably in nite range, generalising the de nition of the VC dimension [15] ....

....to be one of the functions from H. In the simplest form of the standard framework, it is shown that if H has nite VC dimension, then there is a sample size such that any hypothesis from H consistent with the target concept on that many examples is likely to be a good approximation to the target [5, 7, 14, 12, 1]. However, in any real learning situation, where there is a learning algorithm for producing the hypothesis supposed to approximate the target, it is unrealistic to assume that the hypothesis produced is consistent with the target on all of the training sample. It is more reasonable to assume only ....

David Haussler, Quantifying inductive bias: AI learning algorithms and Valiant's learning framework, Articial Intelligence, 36 (1988) 177-221. 12

....to the existence of a large number of features describing the data. Large feature sets increase the size of the space of models; they increase the likelihood that, by chance, a learning program will find a model that fits the data well, and thereby increase the size of the example sets required (Haussler 1988). Scaling up is also an issue in applications not concerned with predictive modeling, but with the discovery of interesting knowledge from large databases. For example, the ability to learn small disjuncts well is often of interest to scientists and business analysts, because small disjuncts ....

Haussler, D. 1988. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36, 177-221.

No context found.

Haussler, D., "Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Model", Artificial Intelligence, 36, 1988, pp. 177-221.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36(2):177--221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial intelligence, 36:177 -- 221, 1988.

No context found.

Haussler, D.: Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework. Artificial Intelligence, 36. (1988)

No context found.

Haussler, D.: Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence 36 (1988) 177--221

No context found.

D. Haussler, "Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework". Artificial Intelligence 36, pp177-221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial intelligence, 36:177 -- 221, 1988.

No context found.

Haussler, D. #1988#. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Arti#cial Intelligence, 32 #2#, 177#222.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Arti cial Intelligence, 36(2):177-221, 1988.

No context found.

Haussler, D.: Quantifying Inductive Bias: AI Learning Algorithms and Valiants Learning Framework. Arti cial Intelligence 36 (1988) 177-221

No context found.

Haussler, D.: Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial intelligence 36 (1988) 177 -- 221

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial intelligence, 36:177 -- 221, 1988.

No context found.

D Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: Ai learning algorithms and valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36(2):177--221, Sept. 1988.

No context found.

D. Haussler, `Quantifying inductive bias: AI learning algorithms and Valiant's learning framework', J. Artificial Intelligence, 36, 177-221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: Ai learning algorithms and valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

No context found.

D. Haussler. Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework. Artificial Intelligence, 36(2): 177-221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and valiant's learning framework. Artificial Intelligence, 36(2):177--221, 1988.

No context found.

D. Haussler. Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework. Artificial Intelligence, 36(2): 177-221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Arti cial Intelligence, 36:177-221, 1988.

No context found.

Haussler D. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artif Intell 1988;36:177--221.

No context found.

David Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177--221, 1988.

No context found.

D. Haussler. Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework. Artificial Intelligence, 36(2): 177-221, 1988.

No context found.

D. Haussler. Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework. Artificial Intelligence, 36(2): 177-221, 1988.

No context found.

D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177-221, 1988.

No context found.

David Haussler. Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework. Articial Intelligence, 36(2), 177-222, 1988.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC