B. Natarajan. Machine Learning: a Theoretical Approach. Morgan Kaufmann, San Mateo, CA, 1991.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....report introduces a simple model that allows the analysis of the learning behaviour of a case based reasoning system. In essence, we apply recent formalisations of the knowledge content of a case memory system from a functional point of view [Jan92] WG94] within the PAC learning model [Hau90] [Nat91] [AB92] KV94] due originally to Valiant [Val84b] The functional viewpoint sees the case base as a representation of a mapping between input and output values. Adopting the PAC learning framework, we contrast this with the true relation that holds in the application domain between input and ....

B K Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, 1991.

....(LimInf H ) learns L. In addition, we write L 2 LimTxt (L 2 LimInf ) provided there are a hypothesis space H and an IIM M that LimTxtH (LimInf H ) learns L. Next, we focus our attention on Valiant s [23] model of probably approximately correct learning (PAC model, for short; see also the textbook [16] for further details) In contrast to Gold s [9] model, the focus is now on learning algorithms that, based on randomly chosen positive and negative examples, nd, fast and with high probability, a suciently good approximation of the target language. To give a precise de nition of the PAC model, ....

B.K. Natarajan, Machine Learning - A Theoretical Approach, Morgan Kaufmann Publ., 1991.

....implementing them listed below. 3.1 Decision Trees Decision trees are perhaps the most widely studied inductive learning models in the machine learning community. The literature abounds with papers proposing new models or variations of existing models and case studies using decision trees ([14, 21, 22, 25, 30, 34, 22 40, 43, 49, 50, 51, 53, 89, 93, 98, 99, 100, 101, 102, 104, 105, 106, 107, 109, 110, 111, 112, 113, 114, 118, 120, 123, 126, 129, 130, 131, 133, 134, 136]) For this case study, we use decision tree software from Quinlan and Buntine. Quinlan introduces decision trees and illustrates the use of his C4.5 software for decision trees (c4.5tree) and production rules derived therefrom (c4.5rule) in [105] Several decision tree algorithms (cart, id3, c4, ....

.... test file name xecute all quit quit 33 33 73 Default values were used for all other parameters. The parameters are explained in depth in [23, 24] D. 5 Neural Networks For this study, the perceptron always used the logistic activation function and the delta learning rule (cf. [56, 84, 93]) During training, weights were adjusted after every instance. The net was trained either until the sum of errors squared or the normalized error reached a specified value. The sum or errors squared is, as usual, defined to be while the normalized error is defined to be where o and o are the ....

Balas K. Natarajan (1991). Machine Learning: A Theoretical Approach. Morgan Kaufmann Publishers, San Mateo, CA.

.... and Definitions Much work has recently been carried out on probabilistic models of machine learning such as the probably approximately correct (or pac) model due to Valiant [26] In particular, the pac learning of f0; 1g valued functions (equivalently, sets) has been studied in great depth; see [12, 5, 18], for example. More recently, attention has been focussed on the extension of the pac model to classes of real valued functions; see, for example, 14, 1, 9] The problem studied in this paper is a problem in probability theory which is motivated by, and has applications to, the learnability of ....

Natarajan, B.K. (1991). Machine Learning: A Theoretical Approach, Morgan Kaufmann, San Mateo, CA.

....networks on binary inputs, is due to Natarajan [79] Theorem 7.6 Let N be a linear threshold network with N neurons (including the input nodes) n input nodes and a total of W variable weights and thresholds. Then V Cdim(N ; f0; 1g n ) WN log N: Proof The proof uses a result due to Hong (see [80] for details) which states that a linear threshold network with N neurons and f0; 1g inputs need only use integer weights which can be encoded using N log N binary bits. It follows that the number of distinct functions computable by the network is at most Gamma 2 N logN Delta W . Since, for ....

....be the error sets. Details may be found in [5, 18, 35] Graph Dimension and Multiple Output Nets 44 dimension is a necessary condition for learnability in this generalized model. Natarajan showed that it is not: there are PAC learnable function spaces with infinite graph dimension (see Natarajan [80]) Natarajan finds a weaker necessary condition for learnability, showing that a certain measure, now known as the Natarajan dimension, must be finite for H to be PAC learnable. More recently, Ben David, Cesa Bianchi and Long [29, 28] have shown that when Y is finite, the finiteness of the graph ....

B. K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, California, 1991.

....d dimensional patterns. 3 as negative examples. The idea is that we can learn a hypothesis from these examples that can accurately predict whether a new pattern came from near or not near the landmark. Chapters 3 and 4 work within the PAC (probably approximately correct) model of learning theory [79, 80, 54, 67, 3]. It is a batch model in that it takes a set of examples, processes them, and then outputs a hypothesis. A PAC algorithm is given inputs ffl and ffi and a set of examples of the target concept randomly drawn according to a fixed, arbitrary, and unknown probability distribution D. It outputs a ....

....Theory Background In this part of the thesis we work within the PAC (probably approximately correct) model and the on line (or mistake bound) model. The PAC model was introduced by Valiant [79, 80] and details of it may be found in such textbooks as Kearns and 14 Vazirani [54] Natarajan [67], and Anthony and Biggs [3] Sections 2.2.1 2.2.5 review the basic definitions and results of the PAC model that are used here. Details on the on line learning model can be found in Angluin [2] and Littlestone [61] Section 2.2.6 reviews the details of this model that are germane to this part of ....

B. K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, CA, 1991.

....with respect to a probability distribution. We obtain a sucient condition for feasible (polynomially bounded) sample size bounds for distributionspeci c (solid) learnability. 1 1 Introduction There have been extensive studies of probabilitic models of machine learning; see the books [3, 11, 12], for example. In the standard PAC model of learning, the de nition of successful learning is distribution free . A number of researchers have examined learning where the probability distribution generating the examples is known; see [6, 5] for example. In this paper we seek conditions under ....

Balas K. Natarajan, Machine Learning: A Theoretical Approach, Morgan Kaufmann, San Mateo, California, 1991.

....was placed on the computational complexity of learning algorithms, which is not something we shall address here. The main probabilistic tools which have become useful for the analysis of this model and its variants have their roots in the work of Vapnik and others (see [21, 23, 22] The books [4, 14, 16] contain general discussions of PAC learning. In its simplest form, the PAC model of learning may be described as follows. There is a set of examples X, and a target function t : X f0; 1g. It is known that t belongs to some set C of functions, but that is all that is known about it. There is ....

Natarajan, B.K. (1991). Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, California, 1991.

....then H is pac learnable by any consistent algorithm. It is natural to ask whether nite graph dimension is a necessary condition for learnability in this generalised model. However, Natarajan showed this not to be the case: there are pac learnable function spaces with in nite graph dimension (see Natarajan (1991)) Natarajan nds a weaker necessary condition for learnability, showing a certain measure, now known as the Natarajan dimension, must be nite for H to be pac learnable. More recently, Ben David, Cesa Bianchi and Long (1992) have shown that when Y is nite, the niteness of the graph dimension is ....

Natarajan (1991): B.K. Natarajan, Machine Learning: A Theoretical Approach, Morgan Kaufmann.

....for identi cation from positive data. See also (Sakakibara, 1992) Yokomori, 1995) de Jongh and Kanazawa, 1996) Kanazawa, 1996) T. Koshiba, 1997) Head et al. 1998) Due to the free distribution and polynomial running time requirements, results are still weaker in PAC learning framework (Natarajan, 1991b) Shvayster, 1990), Yokomori, 1995) Denis, 1998) Many very valuable heuristics and learning algorithms from positive examples alone have been proposed yet and many of them have been used quite succesfully in some practical situations such as speech recognition, and natural language learning. For example, see ....

....learnable from positive data (as the class of k CNF) the output hypothesis must be included in the target concept. But as it is impossible from positive data to di erentiate a negative example from an absent positive one, even very simple classes cannot be PAC learnable from positive data. See (Natarajan, 1991b) Shvayster, 1990), Denis, 1998) for a detailed study. See also (Sakakibara, 1992) for results on grammatical inference about learning from structured positive examples, Kanazawa, 1996) for identi cation in the limit of categorial grammars and (Tellier, 1998) for syntactico semantic searning of categorical ....

Natarajan, B. (1991a). Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, CA.

....one obvious grammar for a document. This de facto grammar, however, is often quite large and, moreover, it is overly restrictive with respect to updates. Thus, one should be able to generalize the productions in some meaningful way. For the generalization, we use techniques from machine learning [5, 6] and formulate the problem as a grammatical inference problem (see Section 3) The method we have developed proceeds as follows. 1. The example productions are transformed to a set of finite automata, one for each nonterminal. These automata accept exactly the right hand sides of the example ....

Balas K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann Publishers, May 1991.

....magnitude by (n 1) n 1) 2 . Thus we deduce that j Delta i j (n 1) n 1) 2 =2 n . 2 Theorem 5. 1 is due to Muroga, Toda, and Takasu [23] and appears in more detail in Muroga [22] Weaker versions of this result were more recently rediscovered by Hong [15] Raghavan [34] and Natarajan [25]. It is unknown whether the upper bound of Theorem 5.1 is tight. For obtaining lower bounds, it is useful to count the number of n input weighted threshold functions. Theorem 5.2 There are at least 2 n(n Gamma1) 2 weighted threshold functions with n inputs. Proof: Let C(n) be the number of ....

B. K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, 1991.

....set of training examples E of the target policy. These examples are drawn from some space of instances I according to some unknown but fixed probability distribution P . Note that this formulation of the inductive concept learning problem is based on the Probably Approximately Correct (PAC) model [27, 37]. The goal of the robot is to determine a policy in H that minimizes the expected error X i2I P rob( i) 6= i) Many algorithms have been developed for solving the inductive concept learning problem, but two of the most well known algorithms are decision trees [31] and neural nets [21] A ....

B. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, 1991.

No context found.

B. Natarajan. Machine Learning: a Theoretical Approach. Morgan Kaufmann, San Mateo, CA, 1991.

No context found.

B.K. Natarajan. Machine Learning: A Theoretical Approach, Morgan Kaufmann, 1991.

No context found.

B K Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, 1991.

No context found.

B. K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, CA, May 1991.

No context found.

B.K. Natarajan, Machine Learning - A Theoretical Approach, Morgan Kaufmann Publ., 1991.

No context found.

B. K. Natarajan. Machine learning: A theoretical approach. Morgan Kaufmann, 1991.

No context found.

B. K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kauffman, 1991.

No context found.

B. K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, CA, 1991.

No context found.

) Natarajan, B., Machine Learning- A Theoretical Approach, Morgan Kaufmann

No context found.

B.K. Natarajan. Machine Learning: A Theoretical Approach. Morgan Kaufmann, 1991.

No context found.

Balas K. Natarajan. Machine Learning: a Theoretical Approach. Morgan Kaufmann, 1991.

No context found.

Natarajan, B. K. 1991. Machine Learning: A Theoretical Approach. Morgan Kaufmann, San Mateo, CA. 39

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC