W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Technical Report 89-35, UC Irvine, 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....real valued attributes are infinite, so only the first bound is applicable. 6 Criticisms of the PAC Model The two criticisms most often leveled at the PAC model by AI researchers interested in empirical machine learning are 1. the worst case emphasis in the model makes it unusable in practice [13,39] and 2. the notions of target concepts and noise free training data are too restrictive in practice [1,9] We take these in turn. There are two aspects of the worst case nature of the PAC model that are at issue. One is the use of the worst case model to measure the computational complexity of ....

....learning curves. Some variants of the PAC model come closer, however. One simple variant is to make it distribution specific, i.e. define and analyze the sample complexity of a learning algorithm for a specific distribution on the instance space, e.g. the uniform distribution on a Boolean space [8,39]. There are two potential problems with this. The first is finding distributions that are both analyzable and indicative of the distributions that arise in practice. The second is that the bounds obtained may be very sensitive to the particular distribution analyzed, and not be very reliable if ....

[Article contains additional citation context not shown here]

W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Technical Report 89-35, UC Irvine, 1989.

....weights (presumed to be roughly the same as the VC dimension) Martin Pittman, 1991) though the VC bounds are vacuous for m smaller than d. Discrepancies between the VC bounds and actual learning curve behavior have also been pointed out and analyzed in other machine learning work (Oblow, 1992; Sarrett Pazzani, 1992). Of course, the VC bounds might simply be inapplicable to these experiments, because backpropagation is not equivalent to empirical error minimization. It has been conjectured that backpropagation can access only a limited portion of the function space, so that the effective dimension is much ....

....LEARNING CURVE BOUNDS 213 x 1 , x N , along with the empty (always 0) function # and the universal (always 1) function 0, 1 N . The input distribution D N is uniform over 0, 1 N . A similar scenario has also been analyzed in the machine learning literature (Oblow, 1992; Sarrett Pazzani, 1992). We will examine the thermodynamic limit for two different choices of target functions f N . We begin with the target function f = 0, 1 N , in which every input is a positive example. Any conjunction h of exactly i variables from x 1 , x N has generalization error # gen (h) Pr ....

Sarrett, W., & Pazzani, M. (1992). Average case analysis of empirical and explanation-based learning algorithms.

....described in the second part of this chapter, and in the chapter by Vapnik. Criticisms of the PAC Model The two criticisms most often leveled at the PAC model by AI researchers interested in empirical machine learning are 1. the worst case emphasis in the model makes it unusable in practice (e.g. [26, 114]) and 2. the notions of target concepts and noise free training data are unrealistic in practice (e.g. 4, 19] We take these in turn. There are two aspects of the worst case nature of the PAC model that are at issue. One is the use of the worst case model to measure the computational complexity ....

....learning curves. Some variants of the PAC model come closer, however. One simple variant is to make it distribution specific, i.e. define and analyze the sample complexity of a learning algorithm for a specific distribution on the instance space, e.g. the uniform distribution on a Boolean space [18, 114]. There are two potential problems with this. The first is finding distributions that are both analyzable and indicative of the distributions that arise in practice. The second is that the bounds obtained may be very sensitive to the particular distribution analyzed, and not be very reliable if ....

[Article contains additional citation context not shown here]

W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Machine Learning, 9(4):349--372, 1992.

....smaller than the number of weights (presumed to be roughly the same as the VC dimension) 16] though the VC bounds are vacuous for m smaller than d. Discrepancies between the VC bounds and actual learning curve behavior have also been pointed out and analyzed in other machine learning work [19, 17]. Of course, the VC bounds might simply be inapplicable to these experiments, because backpropagation is not equivalent to empirical error minimization. Vapnik has conjectured that backpropagation can access only a limited por 1 Here for simplicity we are using the O( Delta) notation, which ....

....the conjunction of a subset of the input variables x1 ; xN , along with the empty (always 0) function ; and the universal (always 1) function f0; 1g N . The input distribution DN is uniform over f0; 1g N . A similar scenario has also been analyzed in the machine learning literature [19, 17]. We will examine the thermodynamic limit for two different choices of target functions fN . We begin with the target function f = f0; 1g N , in which every input is a positive example. Any conjunction h of exactly i variables from x1 ; xN has generalization error ffl gen (h) Pr x2D ....

W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Machine Learning, 9(4):349--372, 1992.

.... approaches to learning in neural networks are described in the recent papers [21,6] One of our main motivations for this research arises from the frequent claims of machine learning practitioners that sample complexity bounds derived via the VC dimension are overly pessimistic in practice [5,26]. This pessimism can be traced to three assumptions that are implicit in results that are based on the VC dimension. The first pessimistic assumption is that only the worst case performance over possible target concepts counts. This is the minimax pessimism. We may think of an adversary choosing ....

W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Technical Report 89-35, UC Irvine, 1989. To appear in Machine Learning.

....smaller than the number of weights (presumed to be roughly the same as the VC dimension) 20] though the VC bounds are vacuous for m smaller than d. Discrepancies between the VC bounds and actual learning curve behavior have also been pointed out and analyzed in other machine learning work [23, 21]. Of course, the VC bounds might simply be inapplicable to these experiments, because backpropagation is not equivalent to empirical error minimization. Vapnik has conjectured that backpropagation can access only a limited portion of the function space, so that the effective dimension is much ....

....the conjunction of a subset of the input variables x1 ; xN , along with the empty (always 0) function ; and the universal (always 1) function f0; 1g N . The input distribution DN is uniform over f0; 1g N . A similar scenario has also been analyzed in the machine learning literature [23, 21]. We will examine the thermodynamic limit for two different choices of target functions fN . We begin with the target function f = f0; 1g N , in which every input is a positive example. Any conjunction h of exactly i variables from x1 ; xN has generalization error ffl gen (h) Pr x2D ....

W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Machine Learning, 9(4):349--372, 1992.

....to a new understanding of the sample complexity of learning in several existing models. One of our main motivations for this research arises from the frequent claims of machine learning practitioners that sample complexity bounds derived via the VC dimension are overly pessimistic in practice [5, 26]. This pessimism can be traced to three assumptions that are implicit in results that are based on the VC dimension. The first pessimistic assumption is that only the worst case performance over possible target concepts counts. This is the minimax pessimism. We may think of an adversary choosing ....

W. Sarrett and M. Pazzani. Average case analysis of empirical and explanation-based learning algorithms. Technical Report 89-35, UC Irvine, 1989. To appear in Machine Learning.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC