J. Barzdin and R. Freivald. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....we use j x to denote the last hypothesis output by M , if any, on successive input d 0 ; d x . We say that M changes its mind, or synonymously, M performs a mind change, iff j x 6= j x 1 . The number of mind changes is a measure of efficiency and has been introduced by Barzdin and Freivalds [8]. Subsequently, this measure of efficiency has been intensively studied (cf. e.g. Barzdin et al. 9] Case and Smith [13] Wiehagen et al. 41] Gasarch and Velauthapillai [16] However, all the mentioned papers considered the learnability of recursive functions. Hence, it is only natural to ....

....particular, the number of allowed mind changes is not required to be universally bounded for all L 2 range(L) Within the next definition we consider the special case that the number of allowed mind changes is universally bounded by an a priori fixed number. Definition 2.2. Barzdin and Freivalds [8]) Let L be an indexed family, let L be a language, let G = G j ) j2IN be a hypothesis space, and let n 2 IN [ f3g. An IIM CLIMn identifies L from text with respect to G iff (1) M CLIM identifies L from text with respect to G, 2) for every text t for L the IIM M performs, when fed t, at most ....

Barzdin, Ya.M., and Freivalds, R.V.: On the prediction of general recursive functions. Sov. Math. Dokl. 13 (1972), 1224 -- 1228.

....in the worst case against a computationally unlimited adversary. This is the set up commonly assumed in the analysis of algorithms for finitely many consultants. The PM (Pascal Matrix) algorithms can be seen as a generalization of the Halving algorithm (Angluin [4] Barzdin and Freivalds [8]) to the case when the candidate predictors are allowed multiple errors. A zero sum multi stage game is a competition between two players. Popular examples include chess, checkers, backgammon. An on line algorithm, one performing a multi round conversation with the user (as opposed to an ....

J.M. Barzdin and R.V. Freivalds. On the prediction of general recursive functions. Soviet. Math. Dokl., 13:1224--1228, 1972.

....a machine model for the learning algorithm, the source of information, the hypotheses spaces used, and the criteria of success. A learning algorithm will simply be called learner. The first learning model we are going to deal with is the on line prediction model going back to Barzdin and Freivald [1] and Littlestone [13] In this setting the source of information is specified as follows. The learner is given a sequence of labeled examples d = d j ) j2N = hb 1 ; c(b 1 ) b 2 ; c(b 2 ) b 3 ; c(b 3 ) i from the concept c , where the b j 2 X n , and c(b j ) 1 if b j 2 c and c(b j ) ....

....k = k(m) the probability to generate a positive example is 2 . Hence, the probability to draw a negative example is 1 0 2 . Consequently, Pr[3 j = 1] Therefore, E[CONV] E[3 1 3 2 1 1 1 3CONV ] E[3 1 3 2 1 1 1 3 i j CONV = i] 1 Pr[CONV = i] i 1 E[3 1 ] 1 Pr[CONV = i] E[CONV ] 1 E[3 1 ] By Theorem 6, we have E[CONV ] dlog 2 ke 3 , and thus it remains to estimate E[3 1 ] A simple calculation shows Lemma 2. For every 0 a 1 holds: 1) 1 a = 1 0 a) 02 Using this estimation we can conclude E[3 1 ] 1) 1 Pr[3 1 ....

[Article contains additional citation context not shown here]

J.M. Barzdin and R.V. Freivald, On the prediction of general recursive functions. Soviet Math. Doklady 13:1224-1228, 1972.

....a machine model for the learning algorithm, the source of information, the hypotheses spaces used, and the criteria of success. A learning algorithm will simply be called learner. The first learning model we are going to deal with is the on line prediction model going back to Barzdin and Freivald [1] and Littlestone [13] In this setting the source of information is specified as follows. The learner is given a sequence of labeled examples d = d j ) j2N = hb 1 ; c(b 1 ) b 2 ; c(b 2 ) b 3 ; c(b 3 ) i from the concept c , where the b j 2 X n , and c(b j ) 1 if b j 2 c and c(b j ) ....

....a positive example is 2 . Hence, the probability to draw a negative example is 1 Gamma 2 . Consequently, Pr[ j = 1] Therefore, E[CONV] E [ 1 2 Delta Delta Delta CONV ] E [ 1 2 Delta Delta Delta i j CONV = i] Delta Pr[CONV = i] i Delta E [ 1 ] Delta Pr[CONV = i] E[CONV ] Delta E [ 1 ] By Theorem 6, we have E[CONV ] dlog 2 ke 3 , and thus it remains to estimate E [ 1 ] A simple calculation shows Lemma 2. For every 0 a 1 holds: 1) Delta a = 1 Gamma a) Using this estimation we can conclude E [ 1 ] ....

[Article contains additional citation context not shown here]

J.M. Barzdin and R.V. Freivald, On the prediction of general recursive functions. Soviet Math. Doklady 13:1224-1228, 1972.

....mistakes in the worst case for any fixed 0 fi 1 against any query sequence 1 . So for fi = 1=2, we get a lower bound of km 2 (n Gamma k 2 )blg k Gamma 1c on the number of mistakes made by any prediction algorithm. If computational efficiency is not a concern, the halving algorithm [4, 18] makes at most km (n Gamma k) lg k mistakes against any query sequence. The halving algorithm predicts according to the majority of the feasible relations (or concepts) and thus each mistake halves the number of remaining relations. We present an efficient algorithm making at most km (n ....

....binary relation on a set where the predicate induces a total order on the set. For example the predicate may be . In the second half of this paper we study the case in which the learner has a priori knowledge that the relation forms a total order. Once again, we see that the halving algorithm [4, 18] yields a good mistake bound against any query sequence. This motivates a second goal of this research: to develop efficient implementations of the halving algorithm. We uncover an interesting application of randomized approximation schemes to computational learning theory. Namely, we describe a ....

[Article contains additional citation context not shown here]

J. Barzdin and R. Freivald. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.

....as a realization of Bayes Rule, with a universally applicable apriori distribution. Since the distribution M is incomputable, we view the main open problem of inductive inference to find maximally efficient approximations to it. Sometimes, even a simple approximation gives nontrivial results (see [1]) Information theory. Since with large probability, H( is close to Gamma log ( the entropy Gamma P ( log ( of the distribution is close to the average complexity P ( H( The complexity H(x) of an object x can indeed be interpreted as the distribution free definition of ....

Ya. M. Barzdin' and R. Freivald. On the prediction of general recursive functions. Soviet Math. Doklady, 206:1224--28, 1972.

....inference machine M is successively fed, then we use h 1 ; h 2 ; to denote the corresponding hypotheses produced by M . We say that M changes its mind, or synonymously, M performs a mind change, iff h i 6= h i 1 . The number of mind changes is a measure of efficiency and has been introduced by Barzdin and Freivalds (1972). Subsequently, this measure has been studied intensively. Barzdin and Freivalds (1974) proved the following remarkable result concerning inductive inference of enumerable classes of recursive functions. Gold s (1967) identification by enumeration technique yields successful inference within the ....

....some problems remained open. It would be very interesting to know how many mind changes are necessary to learn indexed families that cannot be inferred by class preserving conservatively working IIMs. Moreover, it seems to be very challenging to study the question whether or not the results of Barzdin and Freivalds (1972, 1974) as well as of Barzdin, Kinber and Podnieks (1974) may be extended to language learning from positive data. Acknowledgement The authors gratefully acknowledge many valuable comments on the preparation of the paper by Yasuhito Mukouchi. 6. ....

Barzdin, Ya.M., and Freivalds, R.V. (1972), On the prediction of general recursive functions, Sov. Math. Dokl. 13, 1224 - 1228.

....languages that have xed length substitutions then we can learn it eciently in the on line model with the presence of attribute noise. We note that for the case of a single pattern with xed length substitutions without any attribute noise, one can use a direct application of the halving algorithm [14, 28] to obtain an algorithm with a polynomial mistake bound. Along with the restrictions mentioned above, when directly using the halving algorithm exponential time is required to make each prediction. The algorithm we present handles a union of a constant number of patterns, is robust against ....

J. M. Barzdin and R. V. Frievald. On the prediction of general recursive functions. Soviet Math. Doklady, 13:1224-1228, 1972.

....teachable. In fact, Bshouty s [7] result that arbitrary decision trees are learnable with membership and equivalence queries implies that a much broader class than 1 decision lists is T L teachable with a polynomial time learner. Letting A in the proof of Theorem 2 be the halving algorithm [5, 18], we immediately get the following corollary. Corollary 4 Any representation class C is T L teachable (by a computationally unbounded teacher and learner) with a teaching set of length at most log jCj. Because our model incorporates a very powerful set of queries, classes that may not be ....

J. Barzdin and R. Freivald. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.

....r.e. sequence of tell tale sets S 0 ; S 1 ; as featured in Equation (2) above. The additional input information of a program for generating the tell tales therefore makes a huge difference in the mind change complexity of the synthesized learning machine 4 Future Directions Barzdin [BF72] first considered improvements of archetypal enumeration techniques, involving a majority vote strategy which has vastly better mind change complexity. See also [LM89] It would be interesting to look into variants of our algorithms above for synthesizing learning machines with improved ....

J. Barzdin and R. Freivalds. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.

....machine model for the learning algorithm, the source of information, the hypotheses spaces used, and the criteria of success. A learning algorithm will simply be called learner. The first learning model we are going to deal with is the on line prediction model going back to Barzdin and Freivald [1] and Littlestone [13] In this setting the source of information is specified as follows. The learner is given a sequence of labeled examples d = hb 1 ; c(b 1 ) b 2 ; c(b 2 ) b 3 ; c(b 3 ) i from the concept c , where the b j 2 X n , and c(b j ) 1 if b j 2 c and c(b j ) 0 otherwise. ....

....example is 1 Gamma 2 k Gamman . Consequently, Pr[ j = 1] 1 Gamma 2 k Gamman Delta 2 k Gamman : Therefore, E[CONV] E [ 1 2 Delta Delta Delta CONV ] 1 X i=0 E [ 1 2 Delta Delta Delta i j CONV = i] Delta Pr[CONV = i] 1 X i=0 i Delta E [ 1 ] Delta Pr[CONV = i] E[CONV ] Delta E [ 1 ] By Theorem 6, we have E[CONV ] dlog 2 ke 3 , and thus it remains to estimate E [ 1 ] A simple calculation shows Lemma 2. For every 0 a 1 it holds: 1 X =0 ( 1) Delta a = 1 Gamma a) Gamma2 : Using this estimation we can ....

[Article contains additional citation context not shown here]

J.M. Barzdin and R.V. Freivald, On the prediction of general recursive functions. Soviet Math. Doklady 13:1224-1228, 1972.

....machine model for the learning algorithm, the source of information, the hypotheses spaces used, and the criteria of success. A learning algorithm will simply be called learner. The first learning model we are going to deal with is the on line prediction model going back to Barzdin and Freivald [1] and Littlestone [13] In this setting the source of information is specified as follows. The learner is given a sequence of labeled examples d = d j ) j2N = hb 1 ; c(b 1 ) b 2 ; c(b 2 ) b 3 ; c(b 3 ) i from the concept c , where the b j 2 X n , and c(b j ) 1 if b j 2 c and c(b j ) ....

....example is 1 Gamma 2 k Gamman . Consequently, Pr[ j = 1] 1 Gamma 2 k Gamman Delta 2 k Gamman : Therefore, E[CONV] E [ 1 2 Delta Delta Delta CONV ] 1 X i=0 E [ 1 2 Delta Delta Delta i j CONV = i] Delta Pr[CONV = i] 1 X i=0 i Delta E [ 1 ] Delta Pr[CONV = i] E[CONV ] Delta E [ 1 ] By Theorem 6, we have E[CONV ] dlog 2 ke 3 , and thus it remains to estimate E [ 1 ] A simple calculation shows Lemma 2. For every 0 a 1 holds: 1 X =0 ( 1) Delta a = 1 Gamma a) Gamma2 : Using this estimation we can ....

[Article contains additional citation context not shown here]

J.M. Barzdin and R.V. Freivald, On the prediction of general recursive functions. Soviet Math. Doklady 13:1224-1228, 1972.

....the probability of mistake (known as the 0 1 loss in decision theory) for an optimal learning algorithm, and the Shannon information gain from the labels of the instance sequence. In doing so, we borrow from and contribute to the work on weighted majority and aggregating learning strategies [18,20,36,11,2,19], as well as to the VC dimension and statistical physics work. This study leads to a new understanding of the sample complexity of learning in several existing models. 1 More general Bayesian approaches to learning in neural networks are described in the recent papers [21,6] One of our main ....

J. M. Barzdin and R. V. Freivald. On the prediction of general recursive functions. Soviet Mathematics-Doklady, 13:1224--1228, 1972.

....machine M is successively fed with, then we use h 1 ; h 2 ; to denote the corresponding hypotheses produced by M . We say that M changes its mind, or synonymously, M performs a mind change, iff h i 6= h i 1 . The number of mind changes is a measure of efficiency and has been introduced by Barzdin and Freivalds (1972). Subsequently, this measure has been intensively studied. Barzdin and Freivalds (1972) proved the following remarkable result concerning inductive inference of enumerable classes of recursive functions. Gold s (1967) identification by enumeration technique yields successful inference within the ....

....hypotheses produced by M . We say that M changes its mind, or synonymously, M performs a mind change, iff h i 6= h i 1 . The number of mind changes is a measure of efficiency and has been introduced by Barzdin and Freivalds (1972) Subsequently, this measure has been intensively studied. Barzdin and Freivalds (1972) proved the following remarkable result concerning inductive inference of enumerable classes of recursive functions. Gold s (1967) identification by enumeration technique yields successful inference within the enumeration but n 0 1 mind changes may be necessary to learn the nth function. On the ....

[Article contains additional citation context not shown here]

Barzdin, Ya.M., and Freivalds, R.V. (1972), On the prediction of general recursive functions, Sov. Math. Dokl. 13, 1224 - 1228.

....of those concepts that can be described by monotone monomials, i.e. by monomials containing positive literals only. It holds jMC n j = 2 n . 3. Learning Models and Complexity Measures The first learning model we are dealing with is the on line prediction model going back to Barzdin, Freivald [1] and Littlestone [10] In this setting the source of information is specified as follows. The learner is given a sequence of labeled examples d = hd j i j2N = hb 1 ; c(b 1 ) b 2 ; c(b 2 ) b 3 ; c(b 3 ) i from the concept c , where the b j 2 Xn , and c(b j ) 1 if b j 2 c and c(b j ) ....

J.M. Barzdin, R.V. Freivald, On the prediction of general recursive functions. Soviet Math. Doklady 13:1224-1228, 1972.

....machine M is successively fed with, then we use h 1 ; h 2 ; to denote the corresponding hypotheses produced by M . We say that M changes its mind, or synonymously, M performs a mind change, iff h i 6= h i 1 . The number of mind changes is a measure of efficiency and has been introduced by Barzdin and Freivalds (1972). Subsequently, this measure has been studied intensively. Barzdin and Freivalds (1974) proved the following remarkable result concerning inductive inference of enumerable classes of recursive functions. Gold s (1967) identification by enumeration technique yields successful inference within the ....

....changes are necessary to learn indexed families that cannot be inferred by class preserving conservatively working IIMs. Furthermore, we did not fully succeed in separating ELIM k 0 INF and LIM k 0 INF . Moreover, it seems to be very challenging to study the question whether or not the results of Barzdin and Freivalds (1972, 1974) as well as of Barzdin, Kinber and Podnieks (1974) may be extended to language learning from positive data. Acknowledgement The authors gratefully acknowledge valuable comments on the preparation of the paper by Yasuhito Mukouchi. 6. ....

Barzdin, Ya.M., and Freivalds, R.V. (1972), On the prediction of general recursive functions, Sov. Math. Dokl. 13, 1224 - 1228.

....successively fed, then we use j x to denote the last hypothesis output by M , if any, on successive input d 0 ; d x . We say that M changes its mind, or synonymously, M performs a mind change, iff j x 6= j x 1 . The number of mind changes is a measure of efficiency and has been introduced by Barzdin and Freivalds (1972). Subsequently, this measure of efficiency has been intensively studied (cf. e.g. Barzdin, Kinber and Podnieks (1974) Case and Smith (1983) Wiehagen, Freivalds and Kinber (1984) Gasarch and Velauthapillai (1992) However, all the mentioned papers considered the learnability of recursive ....

....L 2 range(L) In particular, the number of allowed mind changes is not required to be universally bounded for all L 2 range(L) Within the next definition we consider the special case that the number of allowed mind changes is universally bounded by an a priori fixed number. Definition 2. (Barzdin and Freivalds, 1972) Let L be an indexed family, let L be a language, let G = G j ) j2IN be a hypothesis space, and let k 2 IN[ f3g. An IIM CLIM k identifies L from text with respect to G iff (1) M CLIM identifies L from text with respect to G, 2) for every text t for L the IIM M performs, when fed t, at ....

Barzdin, Ya.M., and Freivalds, R.V. (1972), On the prediction of general recursive functions, Sov. Math. Dokl. 13, 1224 -- 1228.

....r.e. sequence of tell tale sets S 0 ; S 1 ; as featured in Equation (1) above. The additional input information of a program for generating the tell tales therefore makes a huge difference in the mind change complexity of the synthesized learning machine 4 Future Directions Barzdin [BF72] first considered improvements of archetypal enumeration techniques, involving a majority vote strategy which has vastly better mind change complexity. See also [LW89] It would be interesting to look into variants of our algorithms above for synthesizing learning machines with improved ....

J. Barzdin and R. Freivalds. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.

....machine model for the learning algorithm, the source of information, the hypotheses spaces used, and the criteria of success. A learning algorithm will simply be called learner. The first learning model we are going to deal with is the on line prediction model going back to Barzdin and Freivald [1] and Littlestone [13] In this setting the source of information is specified as follows. The learner is given a sequence of labeled examples d = d j ) j2N = hb 1 ; c(b 1 ) b 2 ; c(b 2 ) b 3 ; c(b 3 ) i from the concept c , where the b j 2 X n , and c(b j ) 1 if b j 2 c and c(b j ) ....

....example is 2 k0n . Hence, the probability to draw a negative example is 1 0 2 k0n . Consequently, Pr[3 j = 1] 1 0 2 k0n 1 2 k0n : Therefore, E[CONV] E[3 1 3 2 1 1 1 3CONV ] 1 X i=0 E[3 1 3 2 1 1 1 3 i j CONV = i] 1 Pr[CONV = i] 1 X i=0 i 1 E[3 1 ] 1 Pr[CONV = i] E[CONV ] 1 E[3 1 ] By Theorem 6, we have E[CONV ] dlog 2 ke 3 , and thus it remains to estimate E[3 1 ] A simple calculation shows Lemma 2. For every 0 a 1 holds: 1 X =0 ( 1) 1 a = 1 0 a) 02 : Using this estimation we can conclude E[3 1 ] 1 X =0 ....

[Article contains additional citation context not shown here]

J.M. Barzdin and R.V. Freivald, On the prediction of general recursive functions. Soviet Math. Doklady 13:1224-1228, 1972.

....of efficiency we deal with is the number of mind changes an IIM M is allowed to perform. We say that M changes its mind, or synonymously, M performs a mind change iff two consecutively hypotheses output by M are different (cf. Definition 3) This measure of efficiency has been introduced by Barzdin and Freivalds (1972). Subsequently, various authors used the number of mind changes to characterize the complexity of learning (cf. e.g. Barzdin and Freivalds (1974) Barzdin, Kinber and Podnieks (1974) Case and Smith (1983) Wiehagen, Freivalds and Kinber (1984) Mukouchi (1992, 1994) Gasarch and Velauthapillai ....

....by iterative IIMs without limiting learning power. 7. Trading Monotonicity Constraints Versus Efficiency This section deals with the efficiency of learning. The measure of efficiency we use is the number of mind changes an IIM is allowed to perform. Starting with the pioneering paper by Barzdin and Freivalds (1972) this measure of efficiency has been intensively studied (cf. e.g. Barzdin, Kinber and Podnieks (1974) Barzdin and Freivalds (1974) Case and Smith (1983) Wiehagen, Freivalds and Kinber (1984) However, all the mentioned papers considered the learnability of recursive functions. Hence, it is ....

Barzdin, Ya.M., and Freivalds, R.V. (1972), On the prediction of general recursive functions, Sov. Math. Dokl. 13, 1224 -- 1228.

....the algorithm receives an instance from some fixed domain and is to produce a binary prediction. At the end of the trial the algorithm receives a binary label, which can be viewed as the correct prediction for the instance. We evaluate such algorithms according to how many mistakes they make [Ang88,BF72,Lit88,Lit89b]. A mistake occurs if the prediction and the label disagree. We also briefly discuss the case in which predictions and labels are chosen from the interval [0; 1] In this paper 1 we investigate the situation where we are given a pool of prediction algorithms that make varying numbers of ....

....of the pool that computes f ; at each trial the prediction made for an instance x by that pool member is just f(x) such an algorithm pays no attention to the labels) We will refer to pools of this type interchangeably as pools of algorithms and as pools of functions. The Halving Algorithm [Ang88,BF72] (it is given this name in [Lit88] can be interpreted as a master algorithm that learns a target class using such a pool. For each instance, the Halving Algorithm predicts according to the majority of all consistent functions of the pool. A function is consistent if its values agree with the ....

[Article contains additional citation context not shown here]

J. M. Barzdin and R. V. Freivalds. On the prediction of general recursive functions. Sov. Math. Dokl., 13:1224--1228, 1972.

....is necessary and sufficient if k is unknown. For Horn sentences the relationship is more complex. In this paper log denotes the logarithm base 2, and ln denotes the natural logarithm. 4 A Generalization of the Halving Algorithm In this section we consider a generalization of the halving algorithm [BF72, Lit88] in which we can reduce the number of equivalence queries required by allowing the learner to make membership queries. In the section we do not bound the computation time of the learner, and the hypotheses proposed need not come from C. However, the learner is still limited to make a polynomial ....

IAn Barzdin and R¯usi¸ns Freivald. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.

No context found.

J. Barzdin and R. Freivald. On the prediction of general recursive functions. Soviet Mathematics Doklady, 13:1224--1228, 1972.

No context found.

Y Barzdin and R Freivalds. On the prediction of general recursive functions. Soviet Mathematics (Doklady), 13:1224--1228, 1972.

No context found.

J. M. Barzdin and R. V. Frievald. On the prediction of general recursive functions. Soviet Math. Doklady, 13:1224-1228, 1972.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC